From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953644; cv=none; d=zohomail.com; s=zohoarc; b=VCjgK5s6yfwY4qqK50LXqSdHGNsdbpWN1DqZLji0dfqhpJvjEsCbG7H3Yuo4F7+Aqj/63zo6n0eT7vXznLPjj6udkCdRpW9niOdETnrSuZ2PCMfB1COKYCla/zTFNPfO5Zsx7tQU2ZQMYd2b8JFKTU2s4e1CBopdXQD3+sZMmmE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953644; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7YiqykGqMXYvPkgLKlv59B2ChlcQqmrofuwgStJpbrQ=; b=JqIh9RqQU0RF0uv6TgHFdhCXItIeXyj7cDK5ZaGB3ZtXnGfFyzurgCXV1lT73puBRL3CA5JObgDImI55pClLhDNh28EMF7pFUFHi8NK/T2Ro16CEEMWZXVCCd0vJhH6x77TFNKKqd2pu82tBsT+8BHBkT62WEcy1ULujCtk84nA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 166695364413097.19839484128488; Fri, 28 Oct 2022 03:40:44 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMlp-0002CM-5g; Fri, 28 Oct 2022 06:39:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMlm-0002Ar-N0 for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:22 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlk-00020B-7K for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:22 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 279AA219CF; Fri, 28 Oct 2022 10:39:17 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B6A4213A6E; Fri, 28 Oct 2022 10:39:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6FqxKVSxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:16 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7YiqykGqMXYvPkgLKlv59B2ChlcQqmrofuwgStJpbrQ=; b=FnGUpU9OjYRhvr+R3vmaSOMDktNPC306zWbOIcAkrPuvYYrh/7FynxlxOk72NrNc2433qB 5sQXzhCzq4bsxVeL3L98HftQJnl5Balhn2PtO77h2bJs2KIRTv9nEhqkFpAldVsYdOQ2c+ CwISALoQOP2943wR8Ee6SXfyzZriERY= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 01/14] migration: support file: uri for source migration Date: Fri, 28 Oct 2022 13:39:01 +0300 Message-Id: <20221028103914.908728-2-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953645722100001 Implement support for a "file:" uri so that a migration can be initiated directly to a file from QEMU. Unlike other migration protocol backends, the 'file' protocol cannot honour non-blocking mode. POSIX file/block storage will always report ready to read/write, regardless of how slow the underlying storage will be at servicing the request. For outgoing migration this limitation is not a serious problem as the migration data transfer always happens in a dedicated thread. It may, however, result in delays in honouring a request to cancel the migration operation. Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: Nikolay Borisov --- migration/file.c | 23 +++++++++++++++++++++++ migration/file.h | 9 +++++++++ migration/meson.build | 1 + migration/migration.c | 3 +++ 4 files changed, 36 insertions(+) create mode 100644 migration/file.c create mode 100644 migration/file.h diff --git a/migration/file.c b/migration/file.c new file mode 100644 index 000000000000..02896a7cab99 --- /dev/null +++ b/migration/file.c @@ -0,0 +1,23 @@ +#include "qemu/osdep.h" +#include "channel.h" +#include "io/channel-file.h" +#include "file.h" +#include "qemu/error-report.h" + + +void file_start_outgoing_migration(MigrationState *s, const char *fname, E= rror **errp) +{ + QIOChannelFile *ioc; + + ioc =3D qio_channel_file_new_path(fname, O_CREAT|O_TRUNC|O_WRONLY, 0660, = errp); + if (!ioc) { + error_report("Error creating a channel"); + return; + } + + qio_channel_set_name(QIO_CHANNEL(ioc), "migration-file-outgoing"); + migration_channel_connect(s, QIO_CHANNEL(ioc), NULL, NULL); + object_unref(OBJECT(ioc)); +} + + diff --git a/migration/file.h b/migration/file.h new file mode 100644 index 000000000000..d476eb1157f9 --- /dev/null +++ b/migration/file.h @@ -0,0 +1,9 @@ +#ifndef QEMU_MIGRATION_FILE_H +#define QEMU_MIGRATION_FILE_H + +void file_start_outgoing_migration(MigrationState *s, + const char *filename, + Error **errp); + +#endif + diff --git a/migration/meson.build b/migration/meson.build index 690487cf1a81..30a8392701c3 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -17,6 +17,7 @@ softmmu_ss.add(files( 'colo.c', 'exec.c', 'fd.c', + 'file.c', 'global_state.c', 'migration.c', 'multifd.c', diff --git a/migration/migration.c b/migration/migration.c index 739bb683f34f..b5373db38535 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -20,6 +20,7 @@ #include "migration/blocker.h" #include "exec.h" #include "fd.h" +#include "file.h" #include "socket.h" #include "sysemu/runstate.h" #include "sysemu/sysemu.h" @@ -2415,6 +2416,8 @@ void qmp_migrate(const char *uri, bool has_blk, bool = blk, exec_start_outgoing_migration(s, p, &local_err); } else if (strstart(uri, "fd:", &p)) { fd_start_outgoing_migration(s, p, &local_err); + } else if (strstart(uri, "file:", &p)) { + file_start_outgoing_migration(s, p, &local_err); } else { if (!(has_resume && resume)) { yank_unregister_instance(MIGRATION_YANK_INSTANCE); --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953742; cv=none; d=zohomail.com; s=zohoarc; b=byoE/gLax0AUz1zOOOcoMhky3u97WauRP1dCXWRbm6szSKTfZQt6cYpfUQvQ94Z7C5Lx9dA6cpl2NIy0gnd4xCFdyE7nfHjPfGBwQKDAjYSFvyBDlDbUyJ384n2OQGWCPfp0sC1vNKh1Z6udRE6SjAxjJFvnqUmKHoyueOXC62c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953742; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mdOQNpWIlx2kBMOLoxmwwEZavWGpfaqIsHexGxEafQk=; b=UnxVknxkImzNHMLTMq7eIvXUwug7FQct5+yzwqTSZXEkIG8p7oCX+2GPSjBdjkbDajZ1SiU3dnz1uNM+N2ebZGAjzOfTyvWXxTpaWUHN5vEd1bjmIDEHNTuFbwE9ArKpI8dGtYKXYqRD/hAegAhwMdQ7f8BXYVkZ6ms8m/kzHTk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953742507817.5285949451984; Fri, 28 Oct 2022 03:42:22 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMln-0002B9-F8; Fri, 28 Oct 2022 06:39:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMll-0002Ah-P6 for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:21 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlk-000205-37 for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:21 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A313A219DD; Fri, 28 Oct 2022 10:39:17 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 394BC13A6E; Fri, 28 Oct 2022 10:39:17 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id qFRSC1WxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mdOQNpWIlx2kBMOLoxmwwEZavWGpfaqIsHexGxEafQk=; b=Utma2M0qscCDeOYPSYKlJCYogKlnK9Z0uqWUHzo1n+j1M/GhHYWhbaaYv13h3RBR7GrvaS aBE0A+xHaPKuRuVzNMFALsWBsYmNZfWiu2uziu6ynZQwqAySA6oCo+DmAH7Ah0kSupCOnS l/HdKL5DMz39r1wd89UKXF5oTDU2Nwo= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 02/14] migration: Add support for 'file:' uri for incoming migration Date: Fri, 28 Oct 2022 13:39:02 +0300 Message-Id: <20221028103914.908728-3-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.28; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953744202100001 This is a counterpart to the 'file:' uri support for source migration, now a file can also serve as the source of an incoming migration. Unlike other migration protocol backends, the 'file' protocol cannot honour non-blocking mode. POSIX file/block storage will always report ready to read/write, regardless of how slow the underlying storage will be at servicing the request. For incoming migration this limitation may result in the main event loop not being fully responsive while loading the VM state. This won't impact the VM since it is not running at this phase, however, it may impact management applications. Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: Nikolay Borisov --- migration/file.c | 15 +++++++++++++++ migration/file.h | 1 + migration/migration.c | 2 ++ 3 files changed, 18 insertions(+) diff --git a/migration/file.c b/migration/file.c index 02896a7cab99..93eb718aa0f4 100644 --- a/migration/file.c +++ b/migration/file.c @@ -21,3 +21,18 @@ void file_start_outgoing_migration(MigrationState *s, co= nst char *fname, Error * } =20 =20 +void file_start_incoming_migration(const char *fname, Error **errp) +{ + QIOChannelFile *ioc; + + ioc =3D qio_channel_file_new_path(fname, O_RDONLY, 0, errp); + if (!ioc) { + error_report("Error creating a channel"); + return; + } + + qio_channel_set_name(QIO_CHANNEL(ioc), "migration-file-incoming"); + migration_channel_process_incoming(QIO_CHANNEL(ioc)); + object_unref(OBJECT(ioc)); +} + diff --git a/migration/file.h b/migration/file.h index d476eb1157f9..cdbd291322d4 100644 --- a/migration/file.h +++ b/migration/file.h @@ -5,5 +5,6 @@ void file_start_outgoing_migration(MigrationState *s, const char *filename, Error **errp); =20 +void file_start_incoming_migration(const char *fname, Error **errp); #endif =20 diff --git a/migration/migration.c b/migration/migration.c index b5373db38535..eafd887254dd 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -506,6 +506,8 @@ static void qemu_start_incoming_migration(const char *u= ri, Error **errp) exec_start_incoming_migration(p, errp); } else if (strstart(uri, "fd:", &p)) { fd_start_incoming_migration(p, errp); + } else if (strstart(uri, "file:", &p)) { + file_start_incoming_migration(p, errp); } else { error_setg(errp, "unknown migration protocol: %s", uri); } --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953611; cv=none; d=zohomail.com; s=zohoarc; b=OFSlFBIDAFeec5ckr8owg+sm0Cy74j0K+EqifkOU/ha7b8RSBAK7JFYb2r7+llKq3x75FQzgy/M+XLlbV4R0S4eJafLCRgQ32AdR3UU5SIi2KsZN41nRUkx7W38xFDvCbCgIWAmq1bgEKBRChxCeu9pvxff9rbUMKdHw5kVW+7I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953611; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9C8PwvMRsKBk9X4GsBVDxHxTD4+h+6VD+DfPcJyROrs=; b=mz8uY3Jcn2XwrLfICEAPZxqmfIW3bMa2rB1g4wE+vITJlsXA2H7XKh0HeWJdQ+IRGf38Xs+xUTydQluOXR0OjNq6VQKOpTJSKGO1UOuC16LOr5pZZpPuEM5Pb40tC3gk11SsyuVOsnTedJiEiFVnid3CD1VnFHTfR+vIbxOUQ3w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953611482216.10851289306163; Fri, 28 Oct 2022 03:40:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMlq-0002DJ-Il; Fri, 28 Oct 2022 06:39:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMln-0002B6-80 for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlk-000209-7H for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 27803219E7; Fri, 28 Oct 2022 10:39:18 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B3A0513A6E; Fri, 28 Oct 2022 10:39:17 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 4LlVKVWxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9C8PwvMRsKBk9X4GsBVDxHxTD4+h+6VD+DfPcJyROrs=; b=RIJRSs4smUf8oWEBlbh7mC+40jN5y8ULQxTekA5fCyV0DSUTGmUdYLh1KvoDEMj5xh4sSx iht9hn/VfH6C6BLx+Y+T4585OuR58vg0fp9wQzLb58nuRNRJ4jEoOvKPO/sUAE33PosIsN fSeHSALefeTvceyxghJZWGRFymsVwc0= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 03/14] migration: Initial support of fixed-ram feature for analyze-migration.py Date: Fri, 28 Oct 2022 13:39:03 +0300 Message-Id: <20221028103914.908728-4-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.28; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953613672100003 Content-Type: text/plain; charset="utf-8" In order to allow analyze-migration.py script to work with migration streams that have the 'fixed-ram' capability set it's required to have access to the stream's configuration object. This commit enables this by making migration json writer part of MigrationState struct, allowing the configuration object be serialized to json. Signed-off-by: Nikolay Borisov --- migration/migration.c | 5 ++++ migration/migration.h | 3 +++ migration/savevm.c | 47 ++++++++++++++++++++++------------ scripts/analyze-migration.py | 49 +++++++++++++++++++++++++++++++++--- 4 files changed, 85 insertions(+), 19 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index eafd887254dd..11ceea340702 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1897,6 +1897,8 @@ static void migrate_fd_cleanup(MigrationState *s) g_free(s->hostname); s->hostname =3D NULL; =20 + json_writer_free(s->vmdesc); + qemu_savevm_state_cleanup(); =20 if (s->to_dst_file) { @@ -2155,6 +2157,7 @@ void migrate_init(MigrationState *s) error_free(s->error); s->error =3D NULL; s->hostname =3D NULL; + s->vmdesc =3D NULL; =20 migrate_set_state(&s->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_S= ETUP); =20 @@ -4270,6 +4273,8 @@ void migrate_fd_connect(MigrationState *s, Error *err= or_in) return; } =20 + s->vmdesc =3D json_writer_new(false); + if (multifd_save_setup(&local_err) !=3D 0) { error_report_err(local_err); migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, diff --git a/migration/migration.h b/migration/migration.h index cdad8aceaaab..96f27aba2210 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -17,6 +17,7 @@ #include "exec/cpu-common.h" #include "hw/qdev-core.h" #include "qapi/qapi-types-migration.h" +#include "qapi/qmp/json-writer.h" #include "qemu/thread.h" #include "qemu/coroutine_int.h" #include "io/channel.h" @@ -261,6 +262,8 @@ struct MigrationState { =20 int state; =20 + JSONWriter *vmdesc; + /* State related to return path */ struct { /* Protected by qemu_file_lock */ diff --git a/migration/savevm.c b/migration/savevm.c index 48e85c052c2c..44a222888306 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1137,13 +1137,23 @@ void qemu_savevm_non_migratable_list(strList **reas= ons) =20 void qemu_savevm_state_header(QEMUFile *f) { + MigrationState *s =3D migrate_get_current(); trace_savevm_state_header(); qemu_put_be32(f, QEMU_VM_FILE_MAGIC); qemu_put_be32(f, QEMU_VM_FILE_VERSION); =20 - if (migrate_get_current()->send_configuration) { + if (s->send_configuration) { qemu_put_byte(f, QEMU_VM_CONFIGURATION); - vmstate_save_state(f, &vmstate_configuration, &savevm_state, 0); + /* + * This starts the main json object and is paired with the + * json_writer_end_object in + * qemu_savevm_state_complete_precopy_non_iterable + */ + json_writer_start_object(s->vmdesc, NULL); + json_writer_start_object(s->vmdesc, "configuration"); + vmstate_save_state(f, &vmstate_configuration, &savevm_state, s->vm= desc); + json_writer_end_object(s->vmdesc); + } } =20 @@ -1364,15 +1374,16 @@ int qemu_savevm_state_complete_precopy_non_iterable= (QEMUFile *f, bool in_postcopy, bool inactivate_disks) { - g_autoptr(JSONWriter) vmdesc =3D NULL; + MigrationState *s =3D migrate_get_current(); int vmdesc_len; SaveStateEntry *se; int ret; =20 - vmdesc =3D json_writer_new(false); - json_writer_start_object(vmdesc, NULL); - json_writer_int64(vmdesc, "page_size", qemu_target_page_size()); - json_writer_start_array(vmdesc, "devices"); + if (!s->send_configuration) { + json_writer_start_object(s->vmdesc, NULL); + } + json_writer_int64(s->vmdesc, "page_size", qemu_target_page_size()); + json_writer_start_array(s->vmdesc, "devices"); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { =20 if ((!se->ops || !se->ops->save_state) && !se->vmsd) { @@ -1385,12 +1396,12 @@ int qemu_savevm_state_complete_precopy_non_iterable= (QEMUFile *f, =20 trace_savevm_section_start(se->idstr, se->section_id); =20 - json_writer_start_object(vmdesc, NULL); - json_writer_str(vmdesc, "name", se->idstr); - json_writer_int64(vmdesc, "instance_id", se->instance_id); + json_writer_start_object(s->vmdesc, NULL); + json_writer_str(s->vmdesc, "name", se->idstr); + json_writer_int64(s->vmdesc, "instance_id", se->instance_id); =20 save_section_header(f, se, QEMU_VM_SECTION_FULL); - ret =3D vmstate_save(f, se, vmdesc); + ret =3D vmstate_save(f, se, s->vmdesc); if (ret) { qemu_file_set_error(f, ret); return ret; @@ -1398,7 +1409,7 @@ int qemu_savevm_state_complete_precopy_non_iterable(Q= EMUFile *f, trace_savevm_section_end(se->idstr, se->section_id, 0); save_section_footer(f, se); =20 - json_writer_end_object(vmdesc); + json_writer_end_object(s->vmdesc); } =20 if (inactivate_disks) { @@ -1417,14 +1428,18 @@ int qemu_savevm_state_complete_precopy_non_iterable= (QEMUFile *f, qemu_put_byte(f, QEMU_VM_EOF); } =20 - json_writer_end_array(vmdesc); - json_writer_end_object(vmdesc); - vmdesc_len =3D strlen(json_writer_get(vmdesc)); + json_writer_end_array(s->vmdesc); + /* + * This finishes the top level json object, its opoening counter part + * is either in this function or in qemu_savevm_state_header + */ + json_writer_end_object(s->vmdesc); + vmdesc_len =3D strlen(json_writer_get(s->vmdesc)); =20 if (should_send_vmdesc()) { qemu_put_byte(f, QEMU_VM_VMDESCRIPTION); qemu_put_be32(f, vmdesc_len); - qemu_put_buffer(f, (uint8_t *)json_writer_get(vmdesc), vmdesc_len); + qemu_put_buffer(f, (uint8_t *)json_writer_get(s->vmdesc), vmdesc_l= en); } =20 return 0; diff --git a/scripts/analyze-migration.py b/scripts/analyze-migration.py index b82a1b0c58c4..9785a640fbf8 100755 --- a/scripts/analyze-migration.py +++ b/scripts/analyze-migration.py @@ -23,6 +23,7 @@ import collections import struct import sys +import math =20 =20 def mkdir_p(path): @@ -119,11 +120,16 @@ def __init__(self, file, version_id, ramargs, section= _key): self.file =3D file self.section_key =3D section_key self.TARGET_PAGE_SIZE =3D ramargs['page_size'] + self.TARGET_PAGE_BITS =3D math.log2(self.TARGET_PAGE_SIZE) self.dump_memory =3D ramargs['dump_memory'] self.write_memory =3D ramargs['write_memory'] + self.fixed_ram =3D ramargs['fixed-ram'] self.sizeinfo =3D collections.OrderedDict() + self.bitmap_offset =3D collections.OrderedDict() + self.pages_offset =3D collections.OrderedDict() self.data =3D collections.OrderedDict() self.data['section sizes'] =3D self.sizeinfo + self.ram_read =3D False self.name =3D '' if self.write_memory: self.files =3D { } @@ -140,7 +146,13 @@ def __str__(self): def getDict(self): return self.data =20 + def write_or_dump_fixed_ram(self): + pass + def read(self): + if self.fixed_ram and self.ram_read: + return + # Read all RAM sections while True: addr =3D self.file.read64() @@ -167,7 +179,25 @@ def read(self): f.truncate(0) f.truncate(len) self.files[self.name] =3D f + + if self.fixed_ram: + bitmap_len =3D self.file.read32() + # skip the pages_offset which we don't need + offset =3D self.file.tell() + 8 + self.bitmap_offset[self.name] =3D offset + offset =3D ((offset + bitmap_len + self.TARGET_PAG= E_SIZE - 1) // self.TARGET_PAGE_SIZE) * self.TARGET_PAGE_SIZE + self.pages_offset[self.name] =3D offset + self.file.file.seek(offset + len) + flags &=3D ~self.RAM_SAVE_FLAG_MEM_SIZE + if self.fixed_ram: + self.ram_read =3D True + # now we should rewind to the ram page offset of the first + # ram section + if self.fixed_ram: + if self.write_memory or self.dump_memory: + self.write_or_dump_fixed_ram() + return =20 if flags & self.RAM_SAVE_FLAG_COMPRESS: if flags & self.RAM_SAVE_FLAG_CONTINUE: @@ -208,7 +238,7 @@ def read(self): =20 # End of RAM section if flags & self.RAM_SAVE_FLAG_EOS: - break + return =20 if flags !=3D 0: raise Exception("Unknown RAM flags: %x" % flags) @@ -521,6 +551,7 @@ def read(self, desc_only =3D False, dump_memory =3D Fal= se, write_memory =3D False): ramargs['page_size'] =3D self.vmsd_desc['page_size'] ramargs['dump_memory'] =3D dump_memory ramargs['write_memory'] =3D write_memory + ramargs['fixed-ram'] =3D False self.section_classes[('ram',0)][1] =3D ramargs =20 while True: @@ -528,8 +559,20 @@ def read(self, desc_only =3D False, dump_memory =3D Fa= lse, write_memory =3D False): if section_type =3D=3D self.QEMU_VM_EOF: break elif section_type =3D=3D self.QEMU_VM_CONFIGURATION: - section =3D ConfigurationSection(file) - section.read() + config_desc =3D self.vmsd_desc.get('configuration') + if config_desc is not None: + config =3D VMSDSection(file, 1, config_desc, 'configur= ation') + config.read() + caps =3D config.data.get("configuration/capabilities") + if caps is not None: + caps =3D caps.data["capabilities"] + if type(caps) !=3D list: + caps =3D [caps] + for i in caps: + # chomp out string length + cap =3D i.data[1:].decode("utf8") + if cap =3D=3D "fixed-ram": + ramargs['fixed-ram'] =3D True elif section_type =3D=3D self.QEMU_VM_SECTION_START or section= _type =3D=3D self.QEMU_VM_SECTION_FULL: section_id =3D file.read32() name =3D file.readstr() --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953581; cv=none; d=zohomail.com; s=zohoarc; b=nRE1qyLk5rABkJsFvp4Xqo7LdUwHHRR/EX33gGkGzM7nhpFEN9rPiBl1bMT1PhAqSWkE0hfXOmSaKOuDWSb1NV85FHiLjvBRVBMArReMM7v+Ygj7KTGRCTvp7L1ddVyh0qfqSgbt6TupCWyhnFs3R6QN6UMPlKEdnmqfGQr1xfs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953581; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=k2f9WwEcMIdwR9UDR42xe8YzlVcRbtdm1BcDgGbCpfs=; b=N7vsGCtrwFh/Wi511bMsX+pBybv57E3V0wNLNIZvCbyMt8z1bZqYMrOtzvESvMdu9FAwMNHclxzNs5nS5oPBkvFeYeyTRGPJMTqofbAWDZBBFeFUcqhuOQd035YjCuNITbDjAtraNF51BpO7xfbmVuxUla6EOrYTMQiPb98RFB0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 166695358178291.2647500670464; Fri, 28 Oct 2022 03:39:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMlp-0002Ci-PL; Fri, 28 Oct 2022 06:39:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMlm-0002At-QB for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:22 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlk-00020D-Ha for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:22 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9A873219B7; Fri, 28 Oct 2022 10:39:18 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3699B13A6E; Fri, 28 Oct 2022 10:39:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id +AzDClaxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:18 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k2f9WwEcMIdwR9UDR42xe8YzlVcRbtdm1BcDgGbCpfs=; b=PrZUhkAX0c6AsQY+6Gw7EwfVbKUXEEvony0QYSu3XiTQ9TUr2adQWzKMVHQ0PaeHmsZ0pp TmYqeLugAF+Xxp7mxiAf0eOOasHtp8tlymCbLY9VxS2gB9rP2qs2l5/GmXkrdDMjJugEOf zOuKrwzCAjbE7BWalBmlyXq1P6TcgKQ= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 04/14] io: Add generic pwritev/preadv interface Date: Fri, 28 Oct 2022 13:39:04 +0300 Message-Id: <20221028103914.908728-5-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953583591100003 Content-Type: text/plain; charset="utf-8" Introduce basic pwriteve/preadv support in the generic channel layer. SPecific implementation will follow for the file channel as this is required in order to support migration streams with fixed location of each ram page. Signed-off-by: Nikolay Borisov --- include/io/channel.h | 49 ++++++++++++++++++++++++++++++++++++++++++++ io/channel.c | 26 +++++++++++++++++++++++ 2 files changed, 75 insertions(+) diff --git a/include/io/channel.h b/include/io/channel.h index c680ee748021..6b10bce8bbdf 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -124,6 +124,16 @@ struct QIOChannelClass { Error **errp); =20 /* Optional callbacks */ + ssize_t (*io_pwritev)(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + off_t offset, + Error **errp); + ssize_t (*io_preadv)(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + off_t offset, + Error **errp); int (*io_shutdown)(QIOChannel *ioc, QIOChannelShutdown how, Error **errp); @@ -504,6 +514,45 @@ int qio_channel_set_blocking(QIOChannel *ioc, int qio_channel_close(QIOChannel *ioc, Error **errp); =20 + +/** + * qio_channel_io_pwritev + * @ioc: the channel object + * @iov: the array of memory regions to write data from + * @niov: the length of the @iov array + * @offset: offset in the channel where writes should begin + * @errp: pointer to a NULL-initialized error object + * + * Not all implementations will support this facility, so may report an er= ror. + * To avoid errors, the caller may check for the feature flag + * QIO_CHANNEL_FEATURE_SEEKABLE prior to calling this method. + * + * Behaves as qio_channel_writev_full, apart from not supporting sending o= f file + * handles as well as beginning the write at the passed @offset + * + */ +ssize_t qio_channel_io_pwritev(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, Error **errp); + + +/** + * qio_channel_io_preadv + * @ioc: the channel object + * @iov: the array of memory regions to read data into + * @niov: the length of the @iov array + * @offset: offset in the channel where writes should begin + * @errp: pointer to a NULL-initialized error object + * + * Not all implementations will support this facility, so may report an er= ror. + * To avoid errors, the caller may check for the feature flag + * QIO_CHANNEL_FEATURE_SEEKABLE prior to calling this method. + * + * Behaves as qio_channel_readv_full, apart from not supporting receiving = of file + * handles as well as beginning the read at the passed @offset + * + */ +ssize_t qio_channel_io_preadv(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, Error **errp); /** * qio_channel_shutdown: * @ioc: the channel object diff --git a/io/channel.c b/io/channel.c index 0640941ac573..f5ac9499a7ad 100644 --- a/io/channel.c +++ b/io/channel.c @@ -437,6 +437,32 @@ GSource *qio_channel_add_watch_source(QIOChannel *ioc, } =20 =20 +ssize_t qio_channel_io_pwritev(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, Error **errp) +{ + QIOChannelClass *klass =3D QIO_CHANNEL_GET_CLASS(ioc); + + if (!klass->io_pwritev) { + error_setg(errp, "Channel does not support pwritev"); + return -1; + } + + return klass->io_pwritev(ioc, iov, niov, offset, errp); +} + +ssize_t qio_channel_io_preadv(QIOChannel *ioc, const struct iovec *iov, + size_t niov, off_t offset, Error **errp) +{ + QIOChannelClass *klass =3D QIO_CHANNEL_GET_CLASS(ioc); + + if (!klass->io_preadv) { + error_setg(errp, "Channel does not support preadv"); + return -1; + } + + return klass->io_preadv(ioc, iov, niov, offset, errp); +} + int qio_channel_shutdown(QIOChannel *ioc, QIOChannelShutdown how, Error **errp) --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953717; cv=none; d=zohomail.com; s=zohoarc; b=icne9iUO4r056H3rWX6Zplx1Zd7QsRmNx6h1PtnZ7bdoRBaPO1vOdgOTm+KmnQR9J61Afcu4V2qLQsIVG03vrK04HQFBUfgNTA7DaoYg989vEGp6p1TJ9ff2dJbZRhsP5feaRyaSeCgP0juMPbIpOs1QtfblRRdMUqtXxMl8a5s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953717; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=sigvptjhrAQET0tYrU2QZiRhDwRPChxgNGt6Rss/X6s=; b=T2ELyRt9aKZQwpN5FkUSlb6cpABcgTC9qu3hWLsoywtnkjrq7IS0hYCrY1lBtKNPJojX75WSb8/FlO7msBRtk30Xxs2ImW5HCjV5yyaBa7ieThaVehiVVnkbB8U3gRQu37In7aM3fpT8qI0m+ffWEBoDp4qqKlUDFxWdPuXwouE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953717246818.7919176783797; Fri, 28 Oct 2022 03:41:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMlt-0002Ej-2S; Fri, 28 Oct 2022 06:39:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMln-0002Be-Uz for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlm-00020i-AU for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1BC7C219F4; Fri, 28 Oct 2022 10:39:19 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id ABB8813A6E; Fri, 28 Oct 2022 10:39:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id IOVfJ1axW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:18 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sigvptjhrAQET0tYrU2QZiRhDwRPChxgNGt6Rss/X6s=; b=WZvJLDUN/Y67ZWq5wv6/032NX3ItKgITTj7yuhrpsYqC0895xicitOdsPJv0kVkv5bwMDi tA2OSDcPCxcIGdFRYFvQN14H2oB2qfPO2inG6b2aANxWdQt6/B+pBppg5SN6QFxOLKvINh l3n6QqiWFI/79powClwCdDPlKGkS6l8= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 05/14] io: implement io_pwritev for QIOChannelFile Date: Fri, 28 Oct 2022 13:39:05 +0300 Message-Id: <20221028103914.908728-6-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953718136100005 Content-Type: text/plain; charset="utf-8" The upcoming 'fixed-ram' feature would require qemu to write data at specific offsets of the file. Add a minimal implementation of pwritev and expose it via the io_pwritev interface. Signed-off-by: Nikolay Borisov Reviewed-by: Daniel P. Berrang=C3=A9 --- io/channel-file.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/io/channel-file.c b/io/channel-file.c index b67687c2aa64..a7a90c12dc2b 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -136,6 +136,30 @@ static ssize_t qio_channel_file_writev(QIOChannel *ioc, return ret; } =20 +static ssize_t qio_channel_file_pwritev(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + off_t offset, + Error **errp) +{ + QIOChannelFile *fioc =3D QIO_CHANNEL_FILE(ioc); + ssize_t ret; + + retry: + ret =3D pwritev(fioc->fd, iov, niov, offset); + if (ret <=3D 0) { + if (errno =3D=3D EAGAIN) { + return QIO_CHANNEL_ERR_BLOCK; + } + if (errno =3D=3D EINTR) { + goto retry; + } + error_setg_errno(errp, errno, "Unable to write to file"); + return -1; + } + return ret; +} + static int qio_channel_file_set_blocking(QIOChannel *ioc, bool enabled, Error **errp) @@ -218,6 +242,7 @@ static void qio_channel_file_class_init(ObjectClass *kl= ass, ioc_klass->io_writev =3D qio_channel_file_writev; ioc_klass->io_readv =3D qio_channel_file_readv; ioc_klass->io_set_blocking =3D qio_channel_file_set_blocking; + ioc_klass->io_pwritev =3D qio_channel_file_pwritev; ioc_klass->io_seek =3D qio_channel_file_seek; ioc_klass->io_close =3D qio_channel_file_close; ioc_klass->io_create_watch =3D qio_channel_file_create_watch; --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953752; cv=none; d=zohomail.com; s=zohoarc; b=eEWHqedDLbFgygVfS+D0XOV1iw+3HGtEUJFyD5Q8L8/eIVA+hrH4PwiPUT8kSHeHc+D+oHKCfO4vW44G7ejS9HCwmtQ7JmH/3PDCNf0AIiQywkhmqff3LyetwGznE4IeBvo2NE5zzr7GWE/D/xdB5S9P6drzP3Nvf3DmN3oNR1I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953752; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hxNN+bHmh99vHxaMD5hTuWnC6gOpUab965bxfRxyBnw=; b=Y3ZYDzCWsG1td/ab9XvPb+7FHAe80P/i29yt+z2wIQtO4JoNp5tUySg0QQsuSliB5TvJMsz3nehs7a9sg3jeTvKUMs5/AYhP7+8XA4bX04zuOSdQTuCI5ZxG2F++naUOsFbbguByTWB8Nl4hfV/yw1IdNFk4aoLNQqg0tFxorgw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953752270445.28085260146145; Fri, 28 Oct 2022 03:42:32 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMls-0002Dt-DN; Fri, 28 Oct 2022 06:39:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMln-0002Bc-TK for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlm-00020h-A4 for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8FE1D219F7; Fri, 28 Oct 2022 10:39:19 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2C85013A6E; Fri, 28 Oct 2022 10:39:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id yENfCFexW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:19 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hxNN+bHmh99vHxaMD5hTuWnC6gOpUab965bxfRxyBnw=; b=Cof0Z8KRyM/YiZ3Kf63K98qmdkDbJR+X8Gc/IW+z1LIPfJKnw75BvwT+EV+kjc7vqUPLs7 L3BpPjDl0+E5oyifLyCjkfNtwLPhAAzqlK8KJOsAUlk7hCrbGPVy0DvGdU7iaoS1m/EoPO G7OSW0clNqv69M17YgJaqeJSwIff5cM= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 06/14] io: add and implement QIO_CHANNEL_FEATURE_SEEKABLE for channel file Date: Fri, 28 Oct 2022 13:39:06 +0300 Message-Id: <20221028103914.908728-7-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953754238100007 Content-Type: text/plain; charset="utf-8" Add a generic QIOChannel feature SEEKABLE which would be used by the qemu_file* apis. For the time being this will be only implemented for file channels. Signed-off-by: Nikolay Borisov --- include/io/channel.h | 1 + io/channel-file.c | 9 +++++++++ 2 files changed, 10 insertions(+) diff --git a/include/io/channel.h b/include/io/channel.h index 6b10bce8bbdf..b645989e467c 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -41,6 +41,7 @@ enum QIOChannelFeature { QIO_CHANNEL_FEATURE_SHUTDOWN, QIO_CHANNEL_FEATURE_LISTEN, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY, + QIO_CHANNEL_FEATURE_SEEKABLE, }; =20 =20 diff --git a/io/channel-file.c b/io/channel-file.c index a7a90c12dc2b..e213a0fd7cd2 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -35,6 +35,11 @@ qio_channel_file_new_fd(int fd) =20 ioc->fd =3D fd; =20 + if (lseek(fd, 0, SEEK_CUR) !=3D (off_t)-1) { + qio_channel_set_feature(QIO_CHANNEL(ioc), QIO_CHANNEL_FEATURE_SEEK= ABLE); + } + + qio_channel_set_feature(QIO_CHANNEL(ioc), QIO_CHANNEL_FEATURE_SEEKABLE= ); trace_qio_channel_file_new_fd(ioc, fd); =20 return ioc; @@ -59,6 +64,10 @@ qio_channel_file_new_path(const char *path, return NULL; } =20 + if (lseek(ioc->fd, 0, SEEK_CUR) !=3D (off_t)-1) { + qio_channel_set_feature(QIO_CHANNEL(ioc), QIO_CHANNEL_FEATURE_SEEK= ABLE); + } + trace_qio_channel_file_new_path(ioc, path, flags, mode, ioc->fd); =20 return ioc; --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953717; cv=none; d=zohomail.com; s=zohoarc; b=jYQH3R1skL/wJbT+fxFtAzu7AHLHvUnpCzy0EWVAePzjbGvcO5kUms0V0sBlIosns/sSs/IFzO8JzQxcatyGWZCOOi2YzJ6TccDdi/Fzmp7Bwei2PRRp8MkfCmH4f5FB1wrlENb8bRjCu7ccsenVdnHvIEQ5j3JqQEibR/GPpzs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953717; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=s9bKrkZ1F6VuY8XjmMNCciNuAxzPpDYI0h/IH99bPc4=; b=UiqAlgDo28c+nDQY3XTYiHOcoBplg2P3ubNqXxLyCnEZ5CPrXBYu5XaNKYjY/IG7sLC9jMp2JrVamQ4O5Uhxw1JTw5wD3bFtrX8k7FseEJnD5jzmozhlOsmvJH3DIU45oA/SL8tWlb5ALOgdSIrPX923YnfkwrtdKX/xW9JXsUc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953717577622.1510165736762; Fri, 28 Oct 2022 03:41:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMlr-0002DQ-9C; Fri, 28 Oct 2022 06:39:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMln-0002BB-Ea for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMll-00020N-GB for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 113241F38D; Fri, 28 Oct 2022 10:39:20 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A0C5113A6E; Fri, 28 Oct 2022 10:39:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 8BmpJFexW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:19 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s9bKrkZ1F6VuY8XjmMNCciNuAxzPpDYI0h/IH99bPc4=; b=fDXHPm8FeCHETy50gdObRrA34tESnRSuPHYB0DODTbhNc46z2P9FBz+yAsf3l+6k5y6Q0N M2L4WZREhB118LuaS0d1cVxjrrJFtmvOf45KXC/N8OioRMQqyvrI+DB44Nn8RyHNyGX7gb oMKpghm6+Rk2SDCpAA485KMqLRwDd0o= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 07/14] migration/qemu-file: add utility methods for working with seekable channels Date: Fri, 28 Oct 2022 13:39:07 +0300 Message-Id: <20221028103914.908728-8-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.29; envelope-from=nborisov@suse.com; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953718148100007 Content-Type: text/plain; charset="utf-8" Add qemu_file_is_seekable/qemu_put_buffer_at/qemu_set_offset/qemu_get_offset as those utility methods will be needed when implementing 'fixed-ram' migration capability. Signed-off-by: Nikolay Borisov --- include/migration/qemu-file-types.h | 2 + migration/qemu-file.c | 59 +++++++++++++++++++++++++++++ migration/qemu-file.h | 3 ++ 3 files changed, 64 insertions(+) diff --git a/include/migration/qemu-file-types.h b/include/migration/qemu-f= ile-types.h index 2867e3da84ab..eb0325ee8687 100644 --- a/include/migration/qemu-file-types.h +++ b/include/migration/qemu-file-types.h @@ -50,6 +50,8 @@ unsigned int qemu_get_be16(QEMUFile *f); unsigned int qemu_get_be32(QEMUFile *f); uint64_t qemu_get_be64(QEMUFile *f); =20 +bool qemu_file_is_seekable(QEMUFile *f); + static inline void qemu_put_be64s(QEMUFile *f, const uint64_t *pv) { qemu_put_be64(f, *pv); diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 4f400c2e5265..d0e0ba6150f7 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -30,6 +30,7 @@ #include "qemu-file.h" #include "trace.h" #include "qapi/error.h" +#include "io/channel-file.h" =20 #define IO_BUF_SIZE 32768 #define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64) @@ -260,6 +261,10 @@ static void qemu_iovec_release_ram(QEMUFile *f) memset(f->may_free, 0, sizeof(f->may_free)); } =20 +bool qemu_file_is_seekable(QEMUFile *f) +{ + return qio_channel_has_feature(f->ioc, QIO_CHANNEL_FEATURE_SEEKABLE); +} =20 /** * Flushes QEMUFile buffer @@ -538,6 +543,60 @@ void qemu_put_buffer(QEMUFile *f, const uint8_t *buf, = size_t size) } } =20 +void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, of= f_t pos) +{ + Error *err =3D NULL; + struct iovec iov =3D { .iov_base =3D (char *)buf, .iov_len =3D buflen = }; + + if (f->last_error) { + return; + } + + qemu_fflush(f); + + if (qio_channel_io_pwritev(f->ioc, &iov, 1, pos, &err) =3D=3D (off_t)-= 1) + goto error; + + return; + + error: + qemu_file_set_error_obj(f, -EIO, err); + return; +} + +void qemu_set_offset(QEMUFile *f, off_t off, int whence) +{ + Error *err =3D NULL; + off_t ret; + + qemu_fflush(f); + + if (!qemu_file_is_writable(f)) { + f->buf_index =3D 0; + f->buf_size =3D 0; + } + + ret =3D qio_channel_io_seek(f->ioc, off, whence, &err); + if (ret =3D=3D (off_t)-1) { + qemu_file_set_error_obj(f, -EIO, err); + } +} + +off_t qemu_get_offset(QEMUFile *f) +{ + Error *err =3D NULL; + off_t ret; + + qemu_fflush(f); + + ret =3D qio_channel_io_seek(f->ioc, 0, SEEK_CUR, &err); + if (ret =3D=3D (off_t)-1) { + qemu_file_set_error_obj(f, -EIO, err); + } + return ret; +} + + void qemu_put_byte(QEMUFile *f, int v) { if (f->last_error) { diff --git a/migration/qemu-file.h b/migration/qemu-file.h index fa13d04d787c..33cfc07b81d1 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -148,6 +148,9 @@ int qemu_file_shutdown(QEMUFile *f); QEMUFile *qemu_file_get_return_path(QEMUFile *f); void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); +void qemu_set_offset(QEMUFile *f, off_t off, int whence); +off_t qemu_get_offset(QEMUFile *f); +void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, of= f_t pos); =20 void ram_control_before_iterate(QEMUFile *f, uint64_t flags); void ram_control_after_iterate(QEMUFile *f, uint64_t flags); --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953607; cv=none; d=zohomail.com; s=zohoarc; b=cJvEulvKrAYS4m+kx6037MyPEIRsJj9WECHNj5SptzljMqyzPi8YUajTyy0Sw1JKhS5yGlGoxjIbbnE/HYnaR5iZrLCJL/tEg7ZmCbUNKyBRHbgQfCkuDg+RNppjK577M7CRgRdl4BoKIvcFBVLJwraqdkYgpw6j19u6aYg/eJA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953607; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Bp6aukOzDziruSd2yqnmr/ESExWo5n/x8u4ppxqz4rw=; b=R+pgHUy/fX+R1jxIEhTt2ZS9sYggbqFYDAKzoHHnGKoaD0sp16eWytQ/9j4QWHO5jJkR2IF/Du8sYFhEvFb0CuYB0U09zsZwHJq2YdJenNuyA7lgE2lTP4cSzNhIbV+vgadgZreg9b/TiKcHh/kRTKSyEI2CIq9kljHbV+06wGw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953607402211.97623043836745; Fri, 28 Oct 2022 03:40:07 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMlw-0002Fx-TN; Fri, 28 Oct 2022 06:39:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMlo-0002C9-FB for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:24 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlm-00020q-KB for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:24 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 86CD621A1B; Fri, 28 Oct 2022 10:39:20 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 214B713A6E; Fri, 28 Oct 2022 10:39:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id UBK7BVixW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Bp6aukOzDziruSd2yqnmr/ESExWo5n/x8u4ppxqz4rw=; b=DxH93/OGtwGjfe29eYcHvGGZsTNuMbC/cx+FTcS4G019fHz11aM3q7eUvOUgkvwmf8MOpz Y+ViEZYO9+exkkEoieEXyNtFZ5nRiCHo+TpEzl1ch92wQWqwnVAPlCjWbZPss9go53s4c4 RQSEwFmja+9Zx2HVuHEqgA4DNqRsFuI= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 08/14] io: Add preadv support to QIOChannelFile Date: Fri, 28 Oct 2022 13:39:08 +0300 Message-Id: <20221028103914.908728-9-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953609597100003 Content-Type: text/plain; charset="utf-8" preadv is going to be needed when 'fixed-ram'-enabled stream are to be restored. Add a minimal implementation of preadv for file channels and expose it via the generic io_preadv interface. Signed-off-by: Nikolay Borisov --- io/channel-file.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/io/channel-file.c b/io/channel-file.c index e213a0fd7cd2..d2f4706b7f6d 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -145,6 +145,32 @@ static ssize_t qio_channel_file_writev(QIOChannel *ioc, return ret; } =20 +static ssize_t qio_channel_file_preadv(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + off_t offset, + Error **errp) +{ + QIOChannelFile *fioc =3D QIO_CHANNEL_FILE(ioc); + ssize_t ret; + + retry: + ret =3D preadv(fioc->fd, iov, niov, offset); + if (ret < 0) { + if (errno =3D=3D EAGAIN) { + return QIO_CHANNEL_ERR_BLOCK; + } + if (errno =3D=3D EINTR) { + goto retry; + } + + error_setg_errno(errp, errno, "Unable to read from file"); + return -1; + } + + return ret; +} + static ssize_t qio_channel_file_pwritev(QIOChannel *ioc, const struct iovec *iov, size_t niov, @@ -252,6 +278,7 @@ static void qio_channel_file_class_init(ObjectClass *kl= ass, ioc_klass->io_readv =3D qio_channel_file_readv; ioc_klass->io_set_blocking =3D qio_channel_file_set_blocking; ioc_klass->io_pwritev =3D qio_channel_file_pwritev; + ioc_klass->io_preadv =3D qio_channel_file_preadv; ioc_klass->io_seek =3D qio_channel_file_seek; ioc_klass->io_close =3D qio_channel_file_close; ioc_klass->io_create_watch =3D qio_channel_file_create_watch; --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953630; cv=none; d=zohomail.com; s=zohoarc; b=JNlclItfHA4S6DPAoTocppcMuAylbPSDmZHhbeea5bZfRk7JPoRIiWIZ1NkH+KjYeZcK0i+JQv8TbW4q0ttMl3Kcg0LWjb2sjbxFoKTEPibrayFOPy98G7IQA3yvzpZ4l4nEyIgFdpKnugGs/KHaxn4qHAGXaOI/dvukeWViZcA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953630; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=flRvW1C3PsOyCNEvmdUpkIVVzW2jum1+2m8RaNbI9YI=; b=gWMuGsVnmqHU8uu4fsy1lAjXoO9Cc1u4YS31K4R3uhoWN0Pt0pUcBF5Qu3ohvBKRrz8vOx0tM8uHWzT/wOBEXct6H1LoYCvyLf+3TFayDnWrBu22VU3ZzTb2wvjZiFlh9wXsME2JclNy+7Ojeja3L7iG4mRHkc+tmM+c8xIAfH0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953630313161.64553753098835; Fri, 28 Oct 2022 03:40:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMlu-0002FW-3D; Fri, 28 Oct 2022 06:39:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMlo-0002Bp-1l for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:24 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlm-00020p-Ip for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0748F21A1C; Fri, 28 Oct 2022 10:39:21 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 96ACA13A6E; Fri, 28 Oct 2022 10:39:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id OBo2IlixW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953561; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=flRvW1C3PsOyCNEvmdUpkIVVzW2jum1+2m8RaNbI9YI=; b=LlNScKW5X7vscEWtaAWim0GRFOAe9XzLZNm0LOlCihJqWDPmqp91rujyGmXSnQdZ1IyuxC G13Kig9kMswsX1S+A0HkzCjAdChpxgP67GTGfhCMMS7ZEwpsBXl0b/68rA9zK/AolIvdhi AY8Y/JSAxMMFpMLHU24UYJWY/0AnI6Y= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 09/14] migration: add qemu_get_buffer_at Date: Fri, 28 Oct 2022 13:39:09 +0300 Message-Id: <20221028103914.908728-10-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.28; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953631682100001 Content-Type: text/plain; charset="utf-8" Restoring a 'fixed-ram' enabled migration stream would require reading from specific offsets in the file so add a helper to QEMUFile that uses the newly introduced qio_channel_file_preadv. Signed-off-by: Nikolay Borisov --- migration/qemu-file.c | 23 +++++++++++++++++++++++ migration/qemu-file.h | 1 + 2 files changed, 24 insertions(+) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index d0e0ba6150f7..b24972d5728d 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -564,6 +564,29 @@ void qemu_put_buffer_at(QEMUFile *f, const uint8_t *bu= f, size_t buflen, off_t po return; } =20 + +size_t qemu_get_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, = off_t pos) +{ + Error *err =3D NULL; + struct iovec iov =3D { .iov_base =3D (char *)buf, .iov_len =3D buflen = }; + ssize_t ret; + + if (f->last_error) { + return 0; + } + + ret =3D qio_channel_io_preadv(f->ioc, &iov, 1, pos, &err); + if (ret =3D=3D -1) { + goto error; + } + + return (size_t)ret; + + error: + qemu_file_set_error_obj(f, -EIO, err); + return 0; +} + void qemu_set_offset(QEMUFile *f, off_t off, int whence) { Error *err =3D NULL; diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 33cfc07b81d1..ab10c3ad7e42 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -151,6 +151,7 @@ void qemu_file_set_blocking(QEMUFile *f, bool block); void qemu_set_offset(QEMUFile *f, off_t off, int whence); off_t qemu_get_offset(QEMUFile *f); void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, of= f_t pos); +size_t qemu_get_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, = off_t pos); =20 void ram_control_before_iterate(QEMUFile *f, uint64_t flags); void ram_control_after_iterate(QEMUFile *f, uint64_t flags); --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953626; cv=none; d=zohomail.com; s=zohoarc; b=fvoKU24WE5mXVCf7MV+ZtoMhX/0iUiuUsjSlKk71O628/Gbtu0l+dxY7HdyTJ0+aMLwFC4aG9DCOstwXy3xSOyrPxwbq3DmuyVLUblb64J9w64wgULj1FgCd5XgDFYZlqC/jkZqoDrN0xvdHYxMNE6LMqB3Qw4eWBzIKW/UxecY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953626; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lw1CYlXqaU6ky2DXH+PId9OjLvIAefaqgXhuB7q+oUQ=; b=e87cjFpOcASP1/8QcetvBFNlPDXpIPzIroDJNm3Ww9XcpVmxVvWfC5vR2iyh5oFwRMdfx+JMYQZeN3GqUhdGcPgzEKkeCIttfWUVZPoVNgZmUiOWDphTIZNgYS6RbTyyMIE9lqcJnjz7Aysl3xJkNa8HeGFiGKULnWtGmqKUOiM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953626575141.12054632710328; Fri, 28 Oct 2022 03:40:26 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMm1-0002Hn-WF; Fri, 28 Oct 2022 06:39:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMlp-0002CW-G3 for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:25 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlm-00020t-Qz for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:25 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 81CF71F390; Fri, 28 Oct 2022 10:39:21 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1885213A6E; Fri, 28 Oct 2022 10:39:21 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id yCWCA1mxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953561; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lw1CYlXqaU6ky2DXH+PId9OjLvIAefaqgXhuB7q+oUQ=; b=ljEgslXFdmqvClSSH0bYHeS19fE9DojSu1u3r3GZr05hnE0J1WFKHepWHQyWLB9lvp3teg x1vz5gGZFBd2m553z5IiJ/AFhZsW3RK3OQoT1ZVIjsXg8FzkDZ72JL2stmhEwtTAbANUYs MHeCitKoGr2aziSfrmNNHzIPqTtBsmU= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 10/14] migration/ram: Introduce 'fixed-ram' migration stream capability Date: Fri, 28 Oct 2022 13:39:10 +0300 Message-Id: <20221028103914.908728-11-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=nborisov@suse.com; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953627716100003 Content-Type: text/plain; charset="utf-8" Implement 'fixed-ram' feature. The core of the feature is to ensure that each ram page of the migration stream has a specific offset in the resulting migration stream. The reason why we'd want such behavior are two fold: - When doing a 'fixed-ram' migration the resulting file will have a bounded size, since pages which are dirtied multiple times will always go to a fixed location in the file, rather than constantly being added to a sequential stream. This eliminates cases where a vm with, say, 1g of ram can result in a migration file that's 10s of Gbs, provided that the workload constantly redirties memory. - It paves the way to implement DIO-enabled save/restore of the migration stream as the pages are ensured to be written at aligned offsets. The features requires changing the format. First, a bitmap is introduced which tracks which pages have been written (i.e are dirtied) during migration and subsequently it's being written in the resultin file, again at a fixed location for every ramblock. Zero pages are ignored as they'd be zero in the destination migration as well. With the changed format data would look like the following: |name len|name|used_len|pc*|bitmap_size|pages_offset|bitmap|pages| * pc - refers to the page_size/mr->addr members, so newly added members begin from "bitmap_size". This layout is initialized during ram_save_setup so instead of having a sequential stream of pages that follow the ramblock headers the dirty pages for a ramblock follow its header. Since all pages have a fixed location RAM_SAVE_FLAG_EOS is no longer generated on every migration iteration but there is effectively a single RAM_SAVE_FLAG_EOS right at the end. Signed-off-by: Nikolay Borisov --- include/exec/ramblock.h | 7 +++ migration/migration.c | 51 +++++++++++++++++++++- migration/migration.h | 1 + migration/ram.c | 97 +++++++++++++++++++++++++++++++++-------- migration/savevm.c | 1 + qapi/migration.json | 2 +- 6 files changed, 138 insertions(+), 21 deletions(-) diff --git a/include/exec/ramblock.h b/include/exec/ramblock.h index 6cbedf9e0c9a..30216c1a41d3 100644 --- a/include/exec/ramblock.h +++ b/include/exec/ramblock.h @@ -43,6 +43,13 @@ struct RAMBlock { size_t page_size; /* dirty bitmap used during migration */ unsigned long *bmap; + /* shadow dirty bitmap used when migrating to a file */ + unsigned long *shadow_bmap; + /* offset in the file pages belonging to this ramblock are saved, used + * only during migration to a file + */ + off_t bitmap_offset; + uint64_t pages_offset; /* bitmap of already received pages in postcopy */ unsigned long *receivedmap; =20 diff --git a/migration/migration.c b/migration/migration.c index 11ceea340702..c7383845a5b4 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -165,7 +165,8 @@ INITIALIZE_MIGRATE_CAPS_SET(check_caps_background_snaps= hot, MIGRATION_CAPABILITY_XBZRLE, MIGRATION_CAPABILITY_X_COLO, MIGRATION_CAPABILITY_VALIDATE_UUID, - MIGRATION_CAPABILITY_ZERO_COPY_SEND); + MIGRATION_CAPABILITY_ZERO_COPY_SEND, + MIGRATION_CAPABILITY_FIXED_RAM); =20 /* When we add fault tolerance, we could have several migrations at once. For now we don't need to add @@ -1326,6 +1327,27 @@ static bool migrate_caps_check(bool *cap_list, } #endif =20 + if (cap_list[MIGRATION_CAPABILITY_FIXED_RAM]) { + if (cap_list[MIGRATION_CAPABILITY_MULTIFD]) { + error_setg(errp, "Directly mapped memory incompatible with multifd"); + return false; + } + + if (cap_list[MIGRATION_CAPABILITY_XBZRLE]) { + error_setg(errp, "Directly mapped memory incompatible with xbzrle"); + return false; + } + + if (cap_list[MIGRATION_CAPABILITY_COMPRESS]) { + error_setg(errp, "Directly mapped memory incompatible with compressi= on"); + return false; + } + + if (cap_list[MIGRATION_CAPABILITY_POSTCOPY_RAM]) { + error_setg(errp, "Directly mapped memory incompatible with postcopy = ram"); + return false; + } + } =20 /* incoming side only */ if (runstate_check(RUN_STATE_INMIGRATE) && @@ -2630,6 +2652,11 @@ MultiFDCompression migrate_multifd_compression(void) return s->parameters.multifd_compression; } =20 +int migrate_fixed_ram(void) +{ + return migrate_get_current()->enabled_capabilities[MIGRATION_CAPABILIT= Y_FIXED_RAM]; +} + int migrate_multifd_zlib_level(void) { MigrationState *s; @@ -4190,6 +4217,21 @@ static void *bg_migration_thread(void *opaque) return NULL; } =20 +static int +migrate_check_fixed_ram(MigrationState *s, Error **errp) +{ + if (!s->enabled_capabilities[MIGRATION_CAPABILITY_FIXED_RAM]) + return 0; + + if (!qemu_file_is_seekable(s->to_dst_file)) { + error_setg(errp, "Directly mapped memory requires a seekable trans= port"); + return -1; + } + + return 0; +} + + void migrate_fd_connect(MigrationState *s, Error *error_in) { Error *local_err =3D NULL; @@ -4265,6 +4307,12 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) return; } =20 + if (migrate_check_fixed_ram(s, &local_err) < 0) { + migrate_fd_cleanup(s); + migrate_fd_error(s, local_err); + return; + } + if (resume) { /* Wakeup the main migration thread to do the recovery */ migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_PAUSED, @@ -4398,6 +4446,7 @@ static Property migration_properties[] =3D { DEFINE_PROP_STRING("tls-authz", MigrationState, parameters.tls_authz), =20 /* Migration capabilities */ + DEFINE_PROP_MIG_CAP("x-fixed-ram", MIGRATION_CAPABILITY_FIXED_RAM), DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), DEFINE_PROP_MIG_CAP("x-rdma-pin-all", MIGRATION_CAPABILITY_RDMA_PIN_AL= L), DEFINE_PROP_MIG_CAP("x-auto-converge", MIGRATION_CAPABILITY_AUTO_CONVE= RGE), diff --git a/migration/migration.h b/migration/migration.h index 96f27aba2210..9aab1b16f407 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -410,6 +410,7 @@ bool migrate_zero_blocks(void); bool migrate_dirty_bitmaps(void); bool migrate_ignore_shared(void); bool migrate_validate_uuid(void); +int migrate_fixed_ram(void); =20 bool migrate_auto_converge(void); bool migrate_use_multifd(void); diff --git a/migration/ram.c b/migration/ram.c index dc1de9ddbc68..4f5ddaff356b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1261,9 +1261,14 @@ static int save_zero_page_to_file(RAMState *rs, QEMU= File *file, int len =3D 0; =20 if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { - len +=3D save_page_header(rs, file, block, offset | RAM_SAVE_FLAG_= ZERO); - qemu_put_byte(file, 0); - len +=3D 1; + if (migrate_fixed_ram()) { + /* for zero pages we don't need to do anything */ + len =3D 1; + } else { + len +=3D save_page_header(rs, file, block, offset | RAM_SAVE_F= LAG_ZERO); + qemu_put_byte(file, 0); + len +=3D 1; + } ram_release_page(block->idstr, offset); } return len; @@ -1342,15 +1347,22 @@ static bool control_save_page(RAMState *rs, RAMBloc= k *block, ram_addr_t offset, static int save_normal_page(RAMState *rs, RAMBlock *block, ram_addr_t offs= et, uint8_t *buf, bool async) { - ram_transferred_add(save_page_header(rs, rs->f, block, - offset | RAM_SAVE_FLAG_PAGE)); - if (async) { - qemu_put_buffer_async(rs->f, buf, TARGET_PAGE_SIZE, - migrate_release_ram() && - migration_in_postcopy()); - } else { - qemu_put_buffer(rs->f, buf, TARGET_PAGE_SIZE); - } + + if (migrate_fixed_ram()) { + qemu_put_buffer_at(rs->f, buf, TARGET_PAGE_SIZE, + block->pages_offset + offset); + set_bit(offset >> TARGET_PAGE_BITS, block->shadow_bmap); + } else { + ram_transferred_add(save_page_header(rs, rs->f, block, + offset | RAM_SAVE_FLAG_PAGE)); + if (async) { + qemu_put_buffer_async(rs->f, buf, TARGET_PAGE_SIZE, + migrate_release_ram() && + migration_in_postcopy()); + } else { + qemu_put_buffer(rs->f, buf, TARGET_PAGE_SIZE); + } + } ram_transferred_add(TARGET_PAGE_SIZE); ram_counters.normal++; return 1; @@ -2683,6 +2695,8 @@ static void ram_save_cleanup(void *opaque) block->clear_bmap =3D NULL; g_free(block->bmap); block->bmap =3D NULL; + g_free(block->shadow_bmap); + block->shadow_bmap =3D NULL; } =20 xbzrle_cleanup(); @@ -3044,6 +3058,7 @@ static void ram_list_init_bitmaps(void) */ block->bmap =3D bitmap_new(pages); bitmap_set(block->bmap, 0, pages); + block->shadow_bmap =3D bitmap_new(block->used_length >> TARGET= _PAGE_BITS); block->clear_bmap_shift =3D shift; block->clear_bmap =3D bitmap_new(clear_bmap_size(pages, shift)= ); } @@ -3226,12 +3241,34 @@ static int ram_save_setup(QEMUFile *f, void *opaque) qemu_put_buffer(f, (uint8_t *)block->idstr, strlen(block->idst= r)); qemu_put_be64(f, block->used_length); if (migrate_postcopy_ram() && block->page_size !=3D - qemu_host_page_size) { + qemu_host_page_size) { qemu_put_be64(f, block->page_size); } if (migrate_ignore_shared()) { qemu_put_be64(f, block->mr->addr); } + + if (migrate_fixed_ram()) { + long num_pages =3D block->used_length >> TARGET_PAGE_BITS; + long bitmap_size =3D BITS_TO_LONGS(num_pages) * sizeof(uns= igned long); + + + /* Needed for external programs (think analyze-migration.p= y) */ + qemu_put_be32(f, bitmap_size); + + /* + * Make pages offset aligned to TARGET_PAGE_SIZE to enable + * DIO in the future. Also add 8 to account for the page o= ffset + * itself + */ + block->bitmap_offset =3D qemu_get_offset(f) + 8; + block->pages_offset =3D ROUND_UP(block->bitmap_offset + + bitmap_size, TARGET_PAGE_SI= ZE); + qemu_put_be64(f, block->pages_offset); + + /* Now prepare offset for next ramblock */ + qemu_set_offset(f, block->pages_offset + block->used_lengt= h, SEEK_SET); + } } } =20 @@ -3249,6 +3286,17 @@ static int ram_save_setup(QEMUFile *f, void *opaque) return 0; } =20 +static void ram_save_shadow_bmap(QEMUFile *f) +{ + RAMBlock *block; + + RAMBLOCK_FOREACH_MIGRATABLE(block) { + long num_pages =3D block->used_length >> TARGET_PAGE_BITS; + long bitmap_size =3D BITS_TO_LONGS(num_pages) * sizeof(unsigned lo= ng); + qemu_put_buffer_at(f, (uint8_t *)block->shadow_bmap, bitmap_size, = block->bitmap_offset); + } +} + /** * ram_save_iterate: iterative stage for migration * @@ -3358,9 +3406,15 @@ static int ram_save_iterate(QEMUFile *f, void *opaqu= e) return ret; } =20 - qemu_put_be64(f, RAM_SAVE_FLAG_EOS); - qemu_fflush(f); - ram_transferred_add(8); + /* + * For fixed ram we don't want to pollute the migration stream with + * EOS flags. + */ + if (!migrate_fixed_ram()) { + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); + ram_transferred_add(8); + } =20 ret =3D qemu_file_get_error(f); } @@ -3405,7 +3459,10 @@ static int ram_save_complete(QEMUFile *f, void *opaq= ue) pages =3D ram_find_and_save_block(rs); /* no more blocks to sent */ if (pages =3D=3D 0) { - break; + if (migrate_fixed_ram()) { + ram_save_shadow_bmap(f); + } + break; } if (pages < 0) { ret =3D pages; @@ -3428,8 +3485,10 @@ static int ram_save_complete(QEMUFile *f, void *opaq= ue) return ret; } =20 - qemu_put_be64(f, RAM_SAVE_FLAG_EOS); - qemu_fflush(f); + if (!migrate_fixed_ram()) { + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); + } =20 return 0; } diff --git a/migration/savevm.c b/migration/savevm.c index 44a222888306..847a8bdfb6ce 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -240,6 +240,7 @@ static bool should_validate_capability(int capability) /* Validate only new capabilities to keep compatibility. */ switch (capability) { case MIGRATION_CAPABILITY_X_IGNORE_SHARED: + case MIGRATION_CAPABILITY_FIXED_RAM: return true; default: return false; diff --git a/qapi/migration.json b/qapi/migration.json index 88ecf86ac876..6196617171e8 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -485,7 +485,7 @@ ## { 'enum': 'MigrationCapability', 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks', - 'compress', 'events', 'postcopy-ram', + 'compress', 'events', 'postcopy-ram', 'fixed-ram', { 'name': 'x-colo', 'features': [ 'unstable' ] }, 'release-ram', 'block', 'return-path', 'pause-before-switchover', 'multifd', --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953616; cv=none; d=zohomail.com; s=zohoarc; b=SHYQzEHxcTnptGCNhcGGDnRpdfNYjiSQYu2ehbBRWVg3h5BtqRJFBy0AipjbWKkPTrkEncCE3CiOD4hDVzpskLZCc220EZXCibU7l6RMIsG3jaFISVY4YMN3xO8g1Mizji9O5EB2hzqgXkwG30lR1g0a68rpmD5WEOqtPlxuF7w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953616; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/URCrYomxDofh61RepDYdpcp3wDN5Mzs6KVh2CPI3d0=; b=WW1/PLpcCry1nfBGvbN/vg1FKQgMKaPzfYaDESJohrC48meODcTQOgGhlA0cPgheSrSe92gUeBrjb3J9JOTWKmipOoNp4R5N3ICjLaRE31EyQzmKWqK3tOpPnOnFa39FvO3/8opSxWcd+ugktakc/K31EO2dEiSnOjk0q+RI1V0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953616934258.3805831714121; Fri, 28 Oct 2022 03:40:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMlz-0002Gh-AG; Fri, 28 Oct 2022 06:39:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMlp-0002CN-7j for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:25 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMln-000213-5X for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:25 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 06B891F897; Fri, 28 Oct 2022 10:39:22 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 92A1713A6E; Fri, 28 Oct 2022 10:39:21 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 0CUvIVmxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953562; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/URCrYomxDofh61RepDYdpcp3wDN5Mzs6KVh2CPI3d0=; b=IEL2N4vmZtbHNsTyHSwyNpFMs19iBEP00G1RPjE3V6uyAbWwq08GbVG+Ipr1fzeG0OUM16 FoezOqEaSge7SEd4FAwEgOSaOSFU0kFD8VdyFU5Z/wxUVCyC6Wr6HoPC1Gj/6l3ATw38DF QClSyNluvjgazvBNPm5O29S2qBXZLpQ= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 11/14] migration: Refactor precopy ram loading code Date: Fri, 28 Oct 2022 13:39:11 +0300 Message-Id: <20221028103914.908728-12-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.29; envelope-from=nborisov@suse.com; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953617715100001 Content-Type: text/plain; charset="utf-8" To facilitate easier implementaiton of the 'fixed-ram' migration restore factor out the code responsible for parsing the ramblocks headers. This also makes ram_load_precopy easier to comprehend. Signed-off-by: Nikolay Borisov --- migration/ram.c | 142 +++++++++++++++++++++++++++--------------------- 1 file changed, 80 insertions(+), 62 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 4f5ddaff356b..1dd68c221667 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4253,6 +4253,83 @@ void colo_flush_ram_cache(void) trace_colo_flush_ram_cache_end(); } =20 +static int parse_ramblock(QEMUFile *f, RAMBlock *block, ram_addr_t length) +{ + int ret =3D 0; + /* ADVISE is earlier, it shows the source has the postcopy capability = on */ + bool postcopy_advised =3D postcopy_is_advised(); + + assert(block); + + if (!qemu_ram_is_migratable(block)) { + error_report("block %s should not be migrated !", block->idstr); + ret =3D -EINVAL; + } + + if (length !=3D block->used_length) { + Error *local_err =3D NULL; + + ret =3D qemu_ram_resize(block, length, &local_err); + if (local_err) { + error_report_err(local_err); + } + } + /* For postcopy we need to check hugepage sizes match */ + if (postcopy_advised && migrate_postcopy_ram() && + block->page_size !=3D qemu_host_page_size) { + uint64_t remote_page_size =3D qemu_get_be64(f); + if (remote_page_size !=3D block->page_size) { + error_report("Mismatched RAM page size %s " + "(local) %zd !=3D %" PRId64, block->idstr, + block->page_size, remote_page_size); + ret =3D -EINVAL; + } + } + if (migrate_ignore_shared()) { + hwaddr addr =3D qemu_get_be64(f); + if (ramblock_is_ignored(block) && + block->mr->addr !=3D addr) { + error_report("Mismatched GPAs for block %s " + "%" PRId64 "!=3D %" PRId64, block->idstr, + (uint64_t)addr, + (uint64_t)block->mr->addr); + ret =3D -EINVAL; + } + } + ram_control_load_hook(f, RAM_CONTROL_BLOCK_REG, block->idstr); + + return ret; +} + +static int parse_ramblocks(QEMUFile *f, ram_addr_t total_ram_bytes) +{ + int ret =3D 0; + + /* Synchronize RAM block list */ + while (!ret && total_ram_bytes) { + char id[256]; + RAMBlock *block; + ram_addr_t length; + int len =3D qemu_get_byte(f); + + qemu_get_buffer(f, (uint8_t *)id, len); + id[len] =3D 0; + length =3D qemu_get_be64(f); + + block =3D qemu_ram_block_by_name(id); + if (block) { + ret =3D parse_ramblock(f, block, length); + } else { + error_report("Unknown ramblock \"%s\", cannot accept " + "migration", id); + ret =3D -EINVAL; + } + total_ram_bytes -=3D length; + } + + return ret; +} + /** * ram_load_precopy: load pages in precopy case * @@ -4267,14 +4344,13 @@ static int ram_load_precopy(QEMUFile *f) { MigrationIncomingState *mis =3D migration_incoming_get_current(); int flags =3D 0, ret =3D 0, invalid_flags =3D 0, len =3D 0, i =3D 0; - /* ADVISE is earlier, it shows the source has the postcopy capability = on */ - bool postcopy_advised =3D postcopy_is_advised(); + if (!migrate_use_compression()) { invalid_flags |=3D RAM_SAVE_FLAG_COMPRESS_PAGE; } =20 while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { - ram_addr_t addr, total_ram_bytes; + ram_addr_t addr; void *host =3D NULL, *host_bak =3D NULL; uint8_t ch; =20 @@ -4345,65 +4421,7 @@ static int ram_load_precopy(QEMUFile *f) =20 switch (flags & ~RAM_SAVE_FLAG_CONTINUE) { case RAM_SAVE_FLAG_MEM_SIZE: - /* Synchronize RAM block list */ - total_ram_bytes =3D addr; - while (!ret && total_ram_bytes) { - RAMBlock *block; - char id[256]; - ram_addr_t length; - - len =3D qemu_get_byte(f); - qemu_get_buffer(f, (uint8_t *)id, len); - id[len] =3D 0; - length =3D qemu_get_be64(f); - - block =3D qemu_ram_block_by_name(id); - if (block && !qemu_ram_is_migratable(block)) { - error_report("block %s should not be migrated !", id); - ret =3D -EINVAL; - } else if (block) { - if (length !=3D block->used_length) { - Error *local_err =3D NULL; - - ret =3D qemu_ram_resize(block, length, - &local_err); - if (local_err) { - error_report_err(local_err); - } - } - /* For postcopy we need to check hugepage sizes match = */ - if (postcopy_advised && migrate_postcopy_ram() && - block->page_size !=3D qemu_host_page_size) { - uint64_t remote_page_size =3D qemu_get_be64(f); - if (remote_page_size !=3D block->page_size) { - error_report("Mismatched RAM page size %s " - "(local) %zd !=3D %" PRId64, - id, block->page_size, - remote_page_size); - ret =3D -EINVAL; - } - } - if (migrate_ignore_shared()) { - hwaddr addr =3D qemu_get_be64(f); - if (ramblock_is_ignored(block) && - block->mr->addr !=3D addr) { - error_report("Mismatched GPAs for block %s " - "%" PRId64 "!=3D %" PRId64, - id, (uint64_t)addr, - (uint64_t)block->mr->addr); - ret =3D -EINVAL; - } - } - ram_control_load_hook(f, RAM_CONTROL_BLOCK_REG, - block->idstr); - } else { - error_report("Unknown ramblock \"%s\", cannot " - "accept migration", id); - ret =3D -EINVAL; - } - - total_ram_bytes -=3D length; - } + ret =3D parse_ramblocks(f, addr); break; =20 case RAM_SAVE_FLAG_ZERO: --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953717; cv=none; d=zohomail.com; s=zohoarc; b=EMWkYcTfH8vPkQm3cqJig35CnwV0cjD21dt2jJuzP/vKErTam9xxONX6fzK/PEX7v2hNYJ3bBM0kr1o5/OHPzVRTCDACULzAwa1SaaynJA1lvzteVWDWcHT2ZkLg4RgnYYC/VW4xMvvePsTtuiF7M1MXyoxQtsFSUFp8WbmvN5Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953717; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cn0gocB5kw66POTC/oATjK5+Jyeej+1QhPCIMPRKVeo=; b=PKwYJd/s1CiDUlaNTzudS1DhOd4t1RlItU9vmwaqd4DU2gSIj8BCax8FkPpYeZLeMh/r3sojGbCZyBlzZg35rZCoONCKCxiiezrcxBNUOneeqx5hg2W6Jk4ePGrxMyPxCDrb5qcIUwXC9YlLYqvexrirQCrEM9/+gD0qHhEQaJc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953717270383.8436470809005; Fri, 28 Oct 2022 03:41:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMm0-0002HH-TR; Fri, 28 Oct 2022 06:39:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMlp-0002CO-97 for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:25 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMln-00021H-Ju for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:25 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 7A3A621982; Fri, 28 Oct 2022 10:39:22 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1652813A6E; Fri, 28 Oct 2022 10:39:22 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id IK/JAlqxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953562; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cn0gocB5kw66POTC/oATjK5+Jyeej+1QhPCIMPRKVeo=; b=eyHxWG6f/G4w/xkFb2vpMpLE3urMPClupGV+i1teNCwHJBAo0RqV+IfaTPujRHDLmsv0VU fVFBBdYc2TPvVbJ63nzM3KI3YcNNdKGFNsoWoE5CofzIWyG2PJSfoM1dH+UUUGNFUf/c37 Lcv+nKoExZBM15fXLA1t1oE3rAq5p4Y= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 12/14] migration: Add support for 'fixed-ram' migration restore Date: Fri, 28 Oct 2022 13:39:12 +0300 Message-Id: <20221028103914.908728-13-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.28; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953718164100008 Content-Type: text/plain; charset="utf-8" Add the necessary code to parse the format changes for 'fixed-ram' capability. One of the more notable changes in behavior is that in the 'fixed-ram' case ram pages are restored in one go rather than constantly looping through the migration stream. Also due to idiosyncrasies of the format I have added the 'ram_migrated' since it was easier to simply return directly from ->load_state rather than introducing more conditionals around the code to prevent ->load_state being called multiple times (from qemu_loadvm_section_start_full/qemu_loadvm_section_part_end i.e. from multiple QEMU_VM_SECTION_(PART|END) flags). Signed-off-by: Nikolay Borisov --- migration/migration.h | 2 + migration/ram.c | 95 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 95 insertions(+), 2 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 9aab1b16f407..7a832d072415 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -96,6 +96,8 @@ struct MigrationIncomingState { bool have_listen_thread; QemuThread listen_thread; =20 + bool ram_migrated; + /* For the kernel to send us notifications */ int userfault_fd; /* To notify the fault_thread to wake, e.g., when need to quit */ diff --git a/migration/ram.c b/migration/ram.c index 1dd68c221667..e085a2431f88 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4330,6 +4330,90 @@ static int parse_ramblocks(QEMUFile *f, ram_addr_t t= otal_ram_bytes) return ret; } =20 + +static int parse_ramblocks_fixed_ram(QEMUFile *f) +{ + int ret =3D 0; + + while (!ret) { + char id[256]; + RAMBlock *block; + ram_addr_t length; + unsigned long clear_bit_idx; + long num_pages, bitmap_size; + int len =3D qemu_get_byte(f); + g_autofree unsigned long *dirty_bitmap =3D NULL; + + qemu_get_buffer(f, (uint8_t *)id, len); + id[len] =3D 0; + length =3D qemu_get_be64(f); + + block =3D qemu_ram_block_by_name(id); + if (block) { + ret =3D parse_ramblock(f, block, length); + if (ret < 0) { + return ret; + } + } else { + error_report("Unknown ramblock \"%s\", cannot accept " + "migration", id); + ret =3D -EINVAL; + continue; + } + + /* 1. read the bitmap size */ + num_pages =3D length >> TARGET_PAGE_BITS; + bitmap_size =3D qemu_get_be32(f); + + assert(bitmap_size =3D=3D BITS_TO_LONGS(num_pages)*sizeof(unsigned= long)); + + block->pages_offset =3D qemu_get_be64(f); + + /* 2. read the actual bitmap */ + dirty_bitmap =3D g_malloc0(bitmap_size); + if (qemu_get_buffer(f, (uint8_t *)dirty_bitmap, bitmap_size) !=3D = bitmap_size) { + error_report("Error parsing dirty bitmap"); + return -EINVAL; + } + +#define BUFSIZE (4*1024*1024) + for (unsigned long set_bit_idx =3D find_first_bit(dirty_bitmap, nu= m_pages); + set_bit_idx < num_pages; + set_bit_idx =3D find_next_bit(dirty_bitmap, num_pages, clear_= bit_idx + 1)) { + + clear_bit_idx =3D find_next_zero_bit(dirty_bitmap, num_pages, = set_bit_idx + 1); + unsigned long len =3D TARGET_PAGE_SIZE * (clear_bit_idx - set_= bit_idx); + ram_addr_t offset =3D set_bit_idx << TARGET_PAGE_BITS; + + for (size_t read =3D 0, completed =3D 0; completed < len; offs= et +=3D read) { + void *host =3D host_from_ram_block_offset(block, offset); + size_t read_len =3D MIN(len, BUFSIZE); + + read =3D qemu_get_buffer_at(f, host, read_len, + block->pages_offset + offset); + completed +=3D read; + } + } + + /* Skip pages array */ + qemu_set_offset(f, block->pages_offset + length, SEEK_SET); + + /* Check if this is the last ramblock */ + if (qemu_get_be64(f) =3D=3D RAM_SAVE_FLAG_EOS) { + ret =3D 1; + } else { + /* + * If not, adjust the internal file index to account for the + * previous 64 bit read + */ + qemu_file_skip(f, -8); + ret =3D 0; + } + } + + return ret; +} + /** * ram_load_precopy: load pages in precopy case * @@ -4349,7 +4433,7 @@ static int ram_load_precopy(QEMUFile *f) invalid_flags |=3D RAM_SAVE_FLAG_COMPRESS_PAGE; } =20 - while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { + while (!ret && !(flags & RAM_SAVE_FLAG_EOS) && !mis->ram_migrated) { ram_addr_t addr; void *host =3D NULL, *host_bak =3D NULL; uint8_t ch; @@ -4421,7 +4505,14 @@ static int ram_load_precopy(QEMUFile *f) =20 switch (flags & ~RAM_SAVE_FLAG_CONTINUE) { case RAM_SAVE_FLAG_MEM_SIZE: - ret =3D parse_ramblocks(f, addr); + if (migrate_fixed_ram()) { + ret =3D parse_ramblocks_fixed_ram(f); + if (ret =3D=3D 1) { + mis->ram_migrated =3D true; + } + } else { + ret =3D parse_ramblocks(f, addr); + } break; =20 case RAM_SAVE_FLAG_ZERO: --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953717; cv=none; d=zohomail.com; s=zohoarc; b=FE6uQJvIFagzvHnnMAh/QehqxeYkNVyb4wdEVtT2r8GDwjsTpcuLa3nOrEh3zeF0Bw88mHktnGVmw9GzzVTD1RYbNS7y7TE5Mx0NLt4R2Zin3zJPZexU7Kne4xPeVSimcJyA1S8/TXYcBs5ofY8oW6n189coYQmH6ZGLNF3Qf3s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953717; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=i0Vu3pJYv0nEyhI+zxj/E1vjt3aFiQmNWq0u9oAS678=; b=HTONE3ur3boVR4xpcZdJnXxf5zz3aVGj0WFPEM8Y5KtsAbYY6/4UJWPE7v/BDa6clNVnd+dqNqM0Og+sJrUkXFOUuqztTOzQbZTfyEgVCP/e9YtyZKwnJpjsBDP/b1N+IC3kgU4MgBEjaf47zNDwKvqU/tr1Pd7H7hMRt/dwQ5A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953717262355.02974846652535; Fri, 28 Oct 2022 03:41:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMm3-0002Hz-Gb; Fri, 28 Oct 2022 06:39:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMlp-0002Ce-La for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:25 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlo-00021R-4E for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:25 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id EE8E6219CF; Fri, 28 Oct 2022 10:39:22 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8B50813A6E; Fri, 28 Oct 2022 10:39:22 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 2PF6H1qxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953562; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i0Vu3pJYv0nEyhI+zxj/E1vjt3aFiQmNWq0u9oAS678=; b=StBpJEi4UquhEOcL7EATwCcSeU9i2/PgxF2d1LwvZZbMXvExBqZFJbXv7YQSiMS4Yg2DSM eCbmtJwGQag81181RnEgvFSyogVue2D6VdWExFwbW09bAKjty2ynyExDtmrbwajrM9EEp9 R9IFdd6amisFcWtMkg3VeQhRQnWB8Cg= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 13/14] tests: Add migrate_incoming_qmp helper Date: Fri, 28 Oct 2022 13:39:13 +0300 Message-Id: <20221028103914.908728-14-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953718147100006 Content-Type: text/plain; charset="utf-8" file-based migration requires the target to initiate its migration after the source has finished writing out the data in the file. Currently there's no easy way to initiate 'migrate-incoming', allow this by introducing migrate_incoming_qmp helper, similarly to migrate_qmp. Signed-off-by: Nikolay Borisov Reviewed-by: Daniel P. Berrang=C3=A9 --- tests/qtest/migration-helpers.c | 19 +++++++++++++++++++ tests/qtest/migration-helpers.h | 4 ++++ 2 files changed, 23 insertions(+) diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helper= s.c index f6f3c6680f57..8161495c2764 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -130,6 +130,25 @@ void migrate_qmp(QTestState *who, const char *uri, con= st char *fmt, ...) qobject_unref(rsp); } =20 + +void migrate_incoming_qmp(QTestState *who, const char *uri, const char *fm= t, ...) +{ + va_list ap; + QDict *args, *rsp; + + va_start(ap, fmt); + args =3D qdict_from_vjsonf_nofail(fmt, ap); + va_end(ap); + + g_assert(!qdict_haskey(args, "uri")); + qdict_put_str(args, "uri", uri); + + rsp =3D qtest_qmp(who, "{ 'execute': 'migrate-incoming', 'arguments': = %p}", args); + + g_assert(qdict_haskey(rsp, "return")); + qobject_unref(rsp); +} + /* * Note: caller is responsible to free the returned object via * qobject_unref() after use diff --git a/tests/qtest/migration-helpers.h b/tests/qtest/migration-helper= s.h index db0684de48b2..c0385aa27e11 100644 --- a/tests/qtest/migration-helpers.h +++ b/tests/qtest/migration-helpers.h @@ -30,6 +30,10 @@ QDict *qmp_command(QTestState *who, const char *command,= ...); G_GNUC_PRINTF(3, 4) void migrate_qmp(QTestState *who, const char *uri, const char *fmt, ...); =20 +G_GNUC_PRINTF(3, 4) +void migrate_incoming_qmp(QTestState *who, const char *uri, + const char *fmt, ...); + QDict *migrate_query(QTestState *who); QDict *migrate_query_not_failed(QTestState *who); =20 --=20 2.34.1 From nobody Sat May 18 07:09:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953715; cv=none; d=zohomail.com; s=zohoarc; b=hiip1GUvCt4mVV4xnf0DTCEK0G+r1RJZl0ISOCiSjC43RCGTKZ4elP7gWq8Jh70/zHo3vU8dGzEBkNpXmjbyoPAqU+T0MX6PLH6a+q6UNgkQLeEqO+dtFBaXckiBKAwzV5Qu7x0lcsLAOSg3n7K7EXpMk+Nmxpz9kiP+PCpWlvo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953715; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ifcleT7pBfKzPGQ0kfkVrXFW2VaaxuRXmqqwY5FzkBw=; b=Y/dmQ1xvYaxw4f27Qd7PTfqVt+a0ppOKubcVQ1T7vxP5GeOr1a2xPjgrFLnoH81eyjsV9vbpNtGP5QzN4cXP1SFg3PJfj/s7upnVd+arEcT47tajI9Vi97ymDZ9G+wbS+4oV6RUkdjq1oZtmvU9U6g6IXi6JtlkEFrBM67Hrpro= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953715309578.3201133062648; Fri, 28 Oct 2022 03:41:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMm4-0002Jb-CY; Fri, 28 Oct 2022 06:39:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMlq-0002DG-Dh for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:26 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlo-00021e-MK for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:26 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 705F11F38D; Fri, 28 Oct 2022 10:39:23 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0BC9C13A6E; Fri, 28 Oct 2022 10:39:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id wFo/AFuxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:23 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ifcleT7pBfKzPGQ0kfkVrXFW2VaaxuRXmqqwY5FzkBw=; b=jeqKnnp6o7Q4C5THJsAZqVB4Vmfsx7MpGuvPof/AsFf+WBTfQva+oINiwXpPP5XMJaWQSk ZpTyvlr74OxD46e2Zpt/5m0Jv9PSyUHmOBT0NWI2OkYBRMc619qYM+5Liow8kgpgEALmwr 0Js2VvXNs+vPSweqvygO+2zJDJmkBrE= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 14/14] tests/qtest: migration-test: Add tests for file-based migration Date: Fri, 28 Oct 2022 13:39:14 +0300 Message-Id: <20221028103914.908728-15-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.29; envelope-from=nborisov@suse.com; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953716348100001 Content-Type: text/plain; charset="utf-8" Add basic tests for file-based migration as well as for the 'fixed-ram' feature. Signed-off-by: Nikolay Borisov --- tests/qtest/migration-test.c | 46 ++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index ef4427ff4d41..de877473f193 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -748,6 +748,7 @@ static void test_migrate_end(QTestState *from, QTestSta= te *to, bool test_dest) cleanup("migsocket"); cleanup("src_serial"); cleanup("dest_serial"); + cleanup("migfile"); } =20 #ifdef CONFIG_GNUTLS @@ -1359,6 +1360,14 @@ static void test_precopy_common(MigrateCommon *args) * hanging forever if migration didn't converge */ wait_for_migration_complete(from); =20 + /* + * For file based migration the target must begin its migration af= ter + * the source has finished + */ + if (strstr(args->connect_uri, "file:")) { + migrate_incoming_qmp(to, args->connect_uri, "{}"); + } + if (!got_stop) { qtest_qmp_eventwait(from, "STOP"); } @@ -1514,6 +1523,39 @@ static void test_precopy_unix_xbzrle(void) test_precopy_common(&args); } =20 +static void test_precopy_unix_file(void) +{ + g_autofree char *uri =3D g_strdup_printf("file:%s/migfile", tmpfs); + MigrateCommon args =3D { + .connect_uri =3D uri, + .listen_uri =3D "defer", + }; + + test_precopy_common(&args); +} + +static void * +test_migrate_fixed_ram_start(QTestState *from, + QTestState *to) +{ + migrate_set_capability(from, "fixed-ram", true); + migrate_set_capability(to, "fixed-ram", true); + + return NULL; +} + +static void test_precopy_unix_fixed_ram(void) +{ + g_autofree char *uri =3D g_strdup_printf("file:%s/migfile", tmpfs); + MigrateCommon args =3D { + .connect_uri =3D uri, + .listen_uri =3D "defer", + .start_hook =3D test_migrate_fixed_ram_start, + }; + + test_precopy_common(&args); +} + static void test_precopy_tcp_plain(void) { MigrateCommon args =3D { @@ -2506,6 +2548,10 @@ int main(int argc, char **argv) test_precopy_unix_tls_psk); #endif =20 + qtest_add_func("/migration/precopy/unix/file", test_precopy_unix_file); + qtest_add_func("/migration/precopy/unix/fixed-ram", + test_precopy_unix_fixed_ram); + if (has_uffd) { /* * NOTE: psk test is enough for postcopy, as other types of TLS --=20 2.34.1