From nobody Tue Feb 10 01:34:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1666953611; cv=none; d=zohomail.com; s=zohoarc; b=OFSlFBIDAFeec5ckr8owg+sm0Cy74j0K+EqifkOU/ha7b8RSBAK7JFYb2r7+llKq3x75FQzgy/M+XLlbV4R0S4eJafLCRgQ32AdR3UU5SIi2KsZN41nRUkx7W38xFDvCbCgIWAmq1bgEKBRChxCeu9pvxff9rbUMKdHw5kVW+7I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1666953611; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9C8PwvMRsKBk9X4GsBVDxHxTD4+h+6VD+DfPcJyROrs=; b=mz8uY3Jcn2XwrLfICEAPZxqmfIW3bMa2rB1g4wE+vITJlsXA2H7XKh0HeWJdQ+IRGf38Xs+xUTydQluOXR0OjNq6VQKOpTJSKGO1UOuC16LOr5pZZpPuEM5Pb40tC3gk11SsyuVOsnTedJiEiFVnid3CD1VnFHTfR+vIbxOUQ3w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1666953611482216.10851289306163; Fri, 28 Oct 2022 03:40:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooMlq-0002DJ-Il; Fri, 28 Oct 2022 06:39:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooMln-0002B6-80 for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ooMlk-000209-7H for qemu-devel@nongnu.org; Fri, 28 Oct 2022 06:39:23 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 27803219E7; Fri, 28 Oct 2022 10:39:18 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B3A0513A6E; Fri, 28 Oct 2022 10:39:17 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 4LlVKVWxW2PVPwAAMHmgww (envelope-from ); Fri, 28 Oct 2022 10:39:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1666953558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9C8PwvMRsKBk9X4GsBVDxHxTD4+h+6VD+DfPcJyROrs=; b=RIJRSs4smUf8oWEBlbh7mC+40jN5y8ULQxTekA5fCyV0DSUTGmUdYLh1KvoDEMj5xh4sSx iht9hn/VfH6C6BLx+Y+T4585OuR58vg0fp9wQzLb58nuRNRJ4jEoOvKPO/sUAE33PosIsN fSeHSALefeTvceyxghJZWGRFymsVwc0= From: Nikolay Borisov To: dgilbert@redhat.com, berrange@redhat.com Cc: qemu-devel@nongnu.org, jfehlig@suse.com, Claudio.Fontana@suse.com, dfaggioli@suse.com, Nikolay Borisov Subject: [PATCH v3 03/14] migration: Initial support of fixed-ram feature for analyze-migration.py Date: Fri, 28 Oct 2022 13:39:03 +0300 Message-Id: <20221028103914.908728-4-nborisov@suse.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221028103914.908728-1-nborisov@suse.com> References: <20221028103914.908728-1-nborisov@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.28; envelope-from=nborisov@suse.com; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1666953613672100003 Content-Type: text/plain; charset="utf-8" In order to allow analyze-migration.py script to work with migration streams that have the 'fixed-ram' capability set it's required to have access to the stream's configuration object. This commit enables this by making migration json writer part of MigrationState struct, allowing the configuration object be serialized to json. Signed-off-by: Nikolay Borisov --- migration/migration.c | 5 ++++ migration/migration.h | 3 +++ migration/savevm.c | 47 ++++++++++++++++++++++------------ scripts/analyze-migration.py | 49 +++++++++++++++++++++++++++++++++--- 4 files changed, 85 insertions(+), 19 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index eafd887254dd..11ceea340702 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1897,6 +1897,8 @@ static void migrate_fd_cleanup(MigrationState *s) g_free(s->hostname); s->hostname =3D NULL; =20 + json_writer_free(s->vmdesc); + qemu_savevm_state_cleanup(); =20 if (s->to_dst_file) { @@ -2155,6 +2157,7 @@ void migrate_init(MigrationState *s) error_free(s->error); s->error =3D NULL; s->hostname =3D NULL; + s->vmdesc =3D NULL; =20 migrate_set_state(&s->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_S= ETUP); =20 @@ -4270,6 +4273,8 @@ void migrate_fd_connect(MigrationState *s, Error *err= or_in) return; } =20 + s->vmdesc =3D json_writer_new(false); + if (multifd_save_setup(&local_err) !=3D 0) { error_report_err(local_err); migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, diff --git a/migration/migration.h b/migration/migration.h index cdad8aceaaab..96f27aba2210 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -17,6 +17,7 @@ #include "exec/cpu-common.h" #include "hw/qdev-core.h" #include "qapi/qapi-types-migration.h" +#include "qapi/qmp/json-writer.h" #include "qemu/thread.h" #include "qemu/coroutine_int.h" #include "io/channel.h" @@ -261,6 +262,8 @@ struct MigrationState { =20 int state; =20 + JSONWriter *vmdesc; + /* State related to return path */ struct { /* Protected by qemu_file_lock */ diff --git a/migration/savevm.c b/migration/savevm.c index 48e85c052c2c..44a222888306 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1137,13 +1137,23 @@ void qemu_savevm_non_migratable_list(strList **reas= ons) =20 void qemu_savevm_state_header(QEMUFile *f) { + MigrationState *s =3D migrate_get_current(); trace_savevm_state_header(); qemu_put_be32(f, QEMU_VM_FILE_MAGIC); qemu_put_be32(f, QEMU_VM_FILE_VERSION); =20 - if (migrate_get_current()->send_configuration) { + if (s->send_configuration) { qemu_put_byte(f, QEMU_VM_CONFIGURATION); - vmstate_save_state(f, &vmstate_configuration, &savevm_state, 0); + /* + * This starts the main json object and is paired with the + * json_writer_end_object in + * qemu_savevm_state_complete_precopy_non_iterable + */ + json_writer_start_object(s->vmdesc, NULL); + json_writer_start_object(s->vmdesc, "configuration"); + vmstate_save_state(f, &vmstate_configuration, &savevm_state, s->vm= desc); + json_writer_end_object(s->vmdesc); + } } =20 @@ -1364,15 +1374,16 @@ int qemu_savevm_state_complete_precopy_non_iterable= (QEMUFile *f, bool in_postcopy, bool inactivate_disks) { - g_autoptr(JSONWriter) vmdesc =3D NULL; + MigrationState *s =3D migrate_get_current(); int vmdesc_len; SaveStateEntry *se; int ret; =20 - vmdesc =3D json_writer_new(false); - json_writer_start_object(vmdesc, NULL); - json_writer_int64(vmdesc, "page_size", qemu_target_page_size()); - json_writer_start_array(vmdesc, "devices"); + if (!s->send_configuration) { + json_writer_start_object(s->vmdesc, NULL); + } + json_writer_int64(s->vmdesc, "page_size", qemu_target_page_size()); + json_writer_start_array(s->vmdesc, "devices"); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { =20 if ((!se->ops || !se->ops->save_state) && !se->vmsd) { @@ -1385,12 +1396,12 @@ int qemu_savevm_state_complete_precopy_non_iterable= (QEMUFile *f, =20 trace_savevm_section_start(se->idstr, se->section_id); =20 - json_writer_start_object(vmdesc, NULL); - json_writer_str(vmdesc, "name", se->idstr); - json_writer_int64(vmdesc, "instance_id", se->instance_id); + json_writer_start_object(s->vmdesc, NULL); + json_writer_str(s->vmdesc, "name", se->idstr); + json_writer_int64(s->vmdesc, "instance_id", se->instance_id); =20 save_section_header(f, se, QEMU_VM_SECTION_FULL); - ret =3D vmstate_save(f, se, vmdesc); + ret =3D vmstate_save(f, se, s->vmdesc); if (ret) { qemu_file_set_error(f, ret); return ret; @@ -1398,7 +1409,7 @@ int qemu_savevm_state_complete_precopy_non_iterable(Q= EMUFile *f, trace_savevm_section_end(se->idstr, se->section_id, 0); save_section_footer(f, se); =20 - json_writer_end_object(vmdesc); + json_writer_end_object(s->vmdesc); } =20 if (inactivate_disks) { @@ -1417,14 +1428,18 @@ int qemu_savevm_state_complete_precopy_non_iterable= (QEMUFile *f, qemu_put_byte(f, QEMU_VM_EOF); } =20 - json_writer_end_array(vmdesc); - json_writer_end_object(vmdesc); - vmdesc_len =3D strlen(json_writer_get(vmdesc)); + json_writer_end_array(s->vmdesc); + /* + * This finishes the top level json object, its opoening counter part + * is either in this function or in qemu_savevm_state_header + */ + json_writer_end_object(s->vmdesc); + vmdesc_len =3D strlen(json_writer_get(s->vmdesc)); =20 if (should_send_vmdesc()) { qemu_put_byte(f, QEMU_VM_VMDESCRIPTION); qemu_put_be32(f, vmdesc_len); - qemu_put_buffer(f, (uint8_t *)json_writer_get(vmdesc), vmdesc_len); + qemu_put_buffer(f, (uint8_t *)json_writer_get(s->vmdesc), vmdesc_l= en); } =20 return 0; diff --git a/scripts/analyze-migration.py b/scripts/analyze-migration.py index b82a1b0c58c4..9785a640fbf8 100755 --- a/scripts/analyze-migration.py +++ b/scripts/analyze-migration.py @@ -23,6 +23,7 @@ import collections import struct import sys +import math =20 =20 def mkdir_p(path): @@ -119,11 +120,16 @@ def __init__(self, file, version_id, ramargs, section= _key): self.file =3D file self.section_key =3D section_key self.TARGET_PAGE_SIZE =3D ramargs['page_size'] + self.TARGET_PAGE_BITS =3D math.log2(self.TARGET_PAGE_SIZE) self.dump_memory =3D ramargs['dump_memory'] self.write_memory =3D ramargs['write_memory'] + self.fixed_ram =3D ramargs['fixed-ram'] self.sizeinfo =3D collections.OrderedDict() + self.bitmap_offset =3D collections.OrderedDict() + self.pages_offset =3D collections.OrderedDict() self.data =3D collections.OrderedDict() self.data['section sizes'] =3D self.sizeinfo + self.ram_read =3D False self.name =3D '' if self.write_memory: self.files =3D { } @@ -140,7 +146,13 @@ def __str__(self): def getDict(self): return self.data =20 + def write_or_dump_fixed_ram(self): + pass + def read(self): + if self.fixed_ram and self.ram_read: + return + # Read all RAM sections while True: addr =3D self.file.read64() @@ -167,7 +179,25 @@ def read(self): f.truncate(0) f.truncate(len) self.files[self.name] =3D f + + if self.fixed_ram: + bitmap_len =3D self.file.read32() + # skip the pages_offset which we don't need + offset =3D self.file.tell() + 8 + self.bitmap_offset[self.name] =3D offset + offset =3D ((offset + bitmap_len + self.TARGET_PAG= E_SIZE - 1) // self.TARGET_PAGE_SIZE) * self.TARGET_PAGE_SIZE + self.pages_offset[self.name] =3D offset + self.file.file.seek(offset + len) + flags &=3D ~self.RAM_SAVE_FLAG_MEM_SIZE + if self.fixed_ram: + self.ram_read =3D True + # now we should rewind to the ram page offset of the first + # ram section + if self.fixed_ram: + if self.write_memory or self.dump_memory: + self.write_or_dump_fixed_ram() + return =20 if flags & self.RAM_SAVE_FLAG_COMPRESS: if flags & self.RAM_SAVE_FLAG_CONTINUE: @@ -208,7 +238,7 @@ def read(self): =20 # End of RAM section if flags & self.RAM_SAVE_FLAG_EOS: - break + return =20 if flags !=3D 0: raise Exception("Unknown RAM flags: %x" % flags) @@ -521,6 +551,7 @@ def read(self, desc_only =3D False, dump_memory =3D Fal= se, write_memory =3D False): ramargs['page_size'] =3D self.vmsd_desc['page_size'] ramargs['dump_memory'] =3D dump_memory ramargs['write_memory'] =3D write_memory + ramargs['fixed-ram'] =3D False self.section_classes[('ram',0)][1] =3D ramargs =20 while True: @@ -528,8 +559,20 @@ def read(self, desc_only =3D False, dump_memory =3D Fa= lse, write_memory =3D False): if section_type =3D=3D self.QEMU_VM_EOF: break elif section_type =3D=3D self.QEMU_VM_CONFIGURATION: - section =3D ConfigurationSection(file) - section.read() + config_desc =3D self.vmsd_desc.get('configuration') + if config_desc is not None: + config =3D VMSDSection(file, 1, config_desc, 'configur= ation') + config.read() + caps =3D config.data.get("configuration/capabilities") + if caps is not None: + caps =3D caps.data["capabilities"] + if type(caps) !=3D list: + caps =3D [caps] + for i in caps: + # chomp out string length + cap =3D i.data[1:].decode("utf8") + if cap =3D=3D "fixed-ram": + ramargs['fixed-ram'] =3D True elif section_type =3D=3D self.QEMU_VM_SECTION_START or section= _type =3D=3D self.QEMU_VM_SECTION_FULL: section_id =3D file.read32() name =3D file.readstr() --=20 2.34.1