From nobody Thu May 2 10:52:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1547122056749848.3406359493158; Thu, 10 Jan 2019 04:07:36 -0800 (PST) Received: from localhost ([127.0.0.1]:53269 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghZ7X-0005cq-2z for importer@patchew.org; Thu, 10 Jan 2019 07:07:35 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49407) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghZ1m-0001Cn-Cf for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:43 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ghZ1l-00005q-2p for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:38 -0500 Received: from forwardcorp1g.cmail.yandex.net ([87.250.241.190]:46363) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ghZ1k-0008SH-97 for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:36 -0500 Received: from mxbackcorp2j.mail.yandex.net (mxbackcorp2j.mail.yandex.net [IPv6:2a02:6b8:0:1619::119]) by forwardcorp1g.cmail.yandex.net (Yandex) with ESMTP id ECC6221871; Thu, 10 Jan 2019 15:01:27 +0300 (MSK) Received: from smtpcorp1j.mail.yandex.net (smtpcorp1j.mail.yandex.net [2a02:6b8:0:1619::137]) by mxbackcorp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id WviXPHoP7T-1RJWaiA6; Thu, 10 Jan 2019 15:01:27 +0300 Received: from dynamic-red.dhcp.yndx.net (dynamic-red.dhcp.yndx.net [2a02:6b8:0:40c:e1bb:a1a7:a235:d6b4]) by smtpcorp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id v4PGSBu40N-1Rhu3Msu; Thu, 10 Jan 2019 15:01:27 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1547121687; bh=1oiXi9X65LRpMZZGO8d9RrL5iqqnzvQMgCubrNWRDuE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Til0eaesXMH1MIET36mE2ll84tfMzrcqjTmzfFGA1XcWey/C5DFJwrAkqfjiu2bDu lp1P6Mc6ZRlXu1y2aKvYtKKS0Q3LIPdaVfpJrpoFtvFUf0sbLc/rjNHdGaZzDEdSFe 3/mAx6qoYi3gaHECGMndGiV+7BviIQ4JcBBOffAE= Authentication-Results: mxbackcorp2j.mail.yandex.net; dkim=pass header.i=@yandex-team.ru From: Yury Kotov To: qemu-devel@nongnu.org, Eduardo Habkost , Igor Mammedov , Paolo Bonzini , Peter Crosthwaite , Richard Henderson , Juan Quintela , "Dr. David Alan Gilbert" , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier Date: Thu, 10 Jan 2019 15:01:17 +0300 Message-Id: <20190110120120.9943-2-yury-kotov@yandex-team.ru> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190110120120.9943-1-yury-kotov@yandex-team.ru> References: <20190110120120.9943-1-yury-kotov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 87.250.241.190 Subject: [Qemu-devel] [PATCH 1/4] migration: add RAMBlock's offset validation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wrfsh@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" RAM migration has a RAMBlock validation stage (flag RAM_SAVE_FLAG_MEM_SIZE). In this stage QEMU checks further information about RAMBlock: 1. Presence (by idstr), 2. Length (trying to resize, when differs), 3. Optional page size. This patch adds a check for RAMBlock's offset. Currently we check it during RAM pages loading - every RAM page has an offset in its header. But there i= s a case when we don't send RAM pages (see below). The following commits introduce a capability (ignore-external) to skip some RAM blocks from migration. In such case the migration stream contains only meta information about RAM blocks to validate them. So, the only way to che= ck block's offset is to send it explicitly. Signed-off-by: Yury Kotov --- migration/ram.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 7e7deec4d8..39629254e1 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3171,6 +3171,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) if (migrate_postcopy_ram() && block->page_size !=3D qemu_host_page= _size) { qemu_put_be64(f, block->page_size); } + qemu_put_be64(f, block->offset); } =20 rcu_read_unlock(); @@ -4031,7 +4032,7 @@ static int ram_load(QEMUFile *f, void *opaque, int ve= rsion_id) =20 seq_iter++; =20 - if (version_id !=3D 4) { + if (version_id < 4) { ret =3D -EINVAL; } =20 @@ -4132,6 +4133,16 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) ret =3D -EINVAL; } } + if (version_id >=3D 5) { + ram_addr_t offset; + offset =3D qemu_get_be64(f); + if (block->offset !=3D offset) { + error_report("Mismatched RAM block offset %s " + "%" PRId64 "!=3D %" PRId64, + id, offset, (uint64_t)block->offs= et); + ret =3D -EINVAL; + } + } ram_control_load_hook(f, RAM_CONTROL_BLOCK_REG, block->idstr); } else { @@ -4363,5 +4374,5 @@ static SaveVMHandlers savevm_ram_handlers =3D { void ram_mig_init(void) { qemu_mutex_init(&XBZRLE.lock); - register_savevm_live(NULL, "ram", 0, 4, &savevm_ram_handlers, &ram_sta= te); + register_savevm_live(NULL, "ram", 0, 5, &savevm_ram_handlers, &ram_sta= te); } --=20 2.20.1 From nobody Thu May 2 10:52:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1547122127902708.5478739244862; Thu, 10 Jan 2019 04:08:47 -0800 (PST) Received: from localhost ([127.0.0.1]:53606 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghZ8g-0006XA-Sj for importer@patchew.org; Thu, 10 Jan 2019 07:08:46 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49400) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghZ1m-0001CP-38 for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ghZ1l-00005l-23 for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:38 -0500 Received: from forwardcorp1j.cmail.yandex.net ([2a02:6b8:0:1630::190]:46728) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ghZ1k-0008SW-AC for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:36 -0500 Received: from mxbackcorp1o.mail.yandex.net (mxbackcorp1o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::301]) by forwardcorp1j.cmail.yandex.net (Yandex) with ESMTP id DD704214C0; Thu, 10 Jan 2019 15:01:28 +0300 (MSK) Received: from smtpcorp1j.mail.yandex.net (smtpcorp1j.mail.yandex.net [2a02:6b8:0:1619::137]) by mxbackcorp1o.mail.yandex.net (nwsmtp/Yandex) with ESMTP id d2V7HgmQQs-1ShCtckd; Thu, 10 Jan 2019 15:01:28 +0300 Received: from dynamic-red.dhcp.yndx.net (dynamic-red.dhcp.yndx.net [2a02:6b8:0:40c:e1bb:a1a7:a235:d6b4]) by smtpcorp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id v4PGSBu40N-1ShWTwNP; Thu, 10 Jan 2019 15:01:28 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1547121688; bh=yvT5e3Voe+f4VgnYTnsxIaLwwaR8Q/7pKbPJONY6WzM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=nuQh95ymu6QPoIGvJeRL7xf11k5GvYZ1RSNlJ9wuxCf+j6SWJ5/wpocIz3bDmSvMw yqzLVk2q7GcTTwkH62bLpspD4nlXYcvhV7Q2g5HvniQcgQS+r2UvKj3VZm0EASIikV DDbNOYoSfa+h9HAOpupp55O+3OKlLfrA3nfJ7/ks= Authentication-Results: mxbackcorp1o.mail.yandex.net; dkim=pass header.i=@yandex-team.ru From: Yury Kotov To: qemu-devel@nongnu.org, Eduardo Habkost , Igor Mammedov , Paolo Bonzini , Peter Crosthwaite , Richard Henderson , Juan Quintela , "Dr. David Alan Gilbert" , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier Date: Thu, 10 Jan 2019 15:01:18 +0300 Message-Id: <20190110120120.9943-3-yury-kotov@yandex-team.ru> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190110120120.9943-1-yury-kotov@yandex-team.ru> References: <20190110120120.9943-1-yury-kotov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:6b8:0:1630::190 Subject: [Qemu-devel] [PATCH 2/4] exec: add RAM_EXTERNAL flag to mark non-QEMU allocated blocks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wrfsh@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This flag allows to determine whether RAM block is available from the outsi= de. E.g. when we use -object memory-backend-file or -mem-path options we have a RAM block which is mapped to shared file. We need this flag in the following commits. Signed-off-by: Yury Kotov --- backends/hostmem-file.c | 3 ++- exec.c | 2 +- include/exec/memory.h | 3 +++ numa.c | 4 ++-- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/backends/hostmem-file.c b/backends/hostmem-file.c index 7a34e25c43..37fe28f2ac 100644 --- a/backends/hostmem-file.c +++ b/backends/hostmem-file.c @@ -63,7 +63,8 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Err= or **errp) name, backend->size, fb->align, (backend->share ? RAM_SHARED : 0) | - (fb->is_pmem ? RAM_PMEM : 0), + (fb->is_pmem ? RAM_PMEM : 0) | + RAM_EXTERNAL, fb->mem_path, errp); g_free(name); #endif diff --git a/exec.c b/exec.c index 6e875f0640..ef2f29d7cb 100644 --- a/exec.c +++ b/exec.c @@ -2241,7 +2241,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, Mem= oryRegion *mr, int64_t file_size; =20 /* Just support these ram flags by now. */ - assert((ram_flags & ~(RAM_SHARED | RAM_PMEM)) =3D=3D 0); + assert((ram_flags & ~(RAM_SHARED | RAM_PMEM | RAM_EXTERNAL)) =3D=3D 0); =20 if (xen_enabled()) { error_setg(errp, "-mem-path not supported with Xen"); diff --git a/include/exec/memory.h b/include/exec/memory.h index ffd23ed8d8..3a9cb34f1e 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -126,6 +126,9 @@ typedef struct IOMMUNotifier IOMMUNotifier; /* RAM is a persistent kind memory */ #define RAM_PMEM (1 << 5) =20 +/* RAM is from external source (e.g. from file) */ +#define RAM_EXTERNAL (1 << 6) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end, diff --git a/numa.c b/numa.c index 50ec016013..653c5a08de 100644 --- a/numa.c +++ b/numa.c @@ -482,8 +482,8 @@ static void allocate_system_memory_nonnuma(MemoryRegion= *mr, Object *owner, if (mem_path) { #ifdef __linux__ Error *err =3D NULL; - memory_region_init_ram_from_file(mr, owner, name, ram_size, 0, 0, - mem_path, &err); + memory_region_init_ram_from_file(mr, owner, name, ram_size, 0, + RAM_EXTERNAL, mem_path, &err); if (err) { error_report_err(err); if (mem_prealloc) { --=20 2.20.1 From nobody Thu May 2 10:52:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1547121982110259.08969778055985; Thu, 10 Jan 2019 04:06:22 -0800 (PST) Received: from localhost ([127.0.0.1]:52921 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghZ6J-0004dl-1W for importer@patchew.org; Thu, 10 Jan 2019 07:06:19 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49451) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghZ1p-0001Fa-I1 for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ghZ1k-00005T-Uy for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:41 -0500 Received: from forwardcorp1o.cmail.yandex.net ([37.9.109.47]:48490) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ghZ1k-0008Sj-AB for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:36 -0500 Received: from mxbackcorp1g.mail.yandex.net (mxbackcorp1g.mail.yandex.net [IPv6:2a02:6b8:0:1402::301]) by forwardcorp1o.cmail.yandex.net (Yandex) with ESMTP id 89093216A4; Thu, 10 Jan 2019 15:01:29 +0300 (MSK) Received: from smtpcorp1j.mail.yandex.net (smtpcorp1j.mail.yandex.net [2a02:6b8:0:1619::137]) by mxbackcorp1g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id 98J7RAFVgx-1TRiGr7p; Thu, 10 Jan 2019 15:01:29 +0300 Received: from dynamic-red.dhcp.yndx.net (dynamic-red.dhcp.yndx.net [2a02:6b8:0:40c:e1bb:a1a7:a235:d6b4]) by smtpcorp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id v4PGSBu40N-1ThufcP8; Thu, 10 Jan 2019 15:01:29 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1547121689; bh=53jqSK6+8ALofi9OqTD429HSosnRtdYKeWmtqXFsYT0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=DBhkgi1cDQh+pXzWBZiUx9uk3CjK2iDFLNh/OL63kHBNPBtRZ7clLM+DUnxSvUDXU MemLSsjhp7Rbj2s3evYyZEQYQzYktTKTkL/wNTxpkAxSHjeknrsrnOYx3+Y+GdiO7T JyD7peI9sTqfI8m/rgzueM38uVnQhflNhfJjweSg= Authentication-Results: mxbackcorp1g.mail.yandex.net; dkim=pass header.i=@yandex-team.ru From: Yury Kotov To: qemu-devel@nongnu.org, Eduardo Habkost , Igor Mammedov , Paolo Bonzini , Peter Crosthwaite , Richard Henderson , Juan Quintela , "Dr. David Alan Gilbert" , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier Date: Thu, 10 Jan 2019 15:01:19 +0300 Message-Id: <20190110120120.9943-4-yury-kotov@yandex-team.ru> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190110120120.9943-1-yury-kotov@yandex-team.ru> References: <20190110120120.9943-1-yury-kotov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 37.9.109.47 Subject: [Qemu-devel] [PATCH 3/4] migration: introduce ignore-external capability X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wrfsh@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We want to use local migration to update QEMU for running guests. In this case we don't need to migrate external RAM. So, add a capability to ignore such blocks during live migration. Signed-off-by: Yury Kotov --- exec.c | 5 +++++ include/exec/cpu-common.h | 1 + migration/migration.c | 9 +++++++++ migration/migration.h | 1 + migration/ram.c | 37 ++++++++++++++++++++++++++++++++++--- qapi/migration.json | 6 +++++- 6 files changed, 55 insertions(+), 4 deletions(-) diff --git a/exec.c b/exec.c index ef2f29d7cb..3c3e42993f 100644 --- a/exec.c +++ b/exec.c @@ -2000,6 +2000,11 @@ void qemu_ram_unset_migratable(RAMBlock *rb) rb->flags &=3D ~RAM_MIGRATABLE; } =20 +bool qemu_ram_is_external(RAMBlock *rb) +{ + return rb->flags & RAM_EXTERNAL; +} + /* Called with iothread lock held. */ void qemu_ram_set_idstr(RAMBlock *new_block, const char *name, DeviceState= *dev) { diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index 2ad2d6d86b..57e84e5aa4 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -78,6 +78,7 @@ void qemu_ram_set_uf_zeroable(RAMBlock *rb); bool qemu_ram_is_migratable(RAMBlock *rb); void qemu_ram_set_migratable(RAMBlock *rb); void qemu_ram_unset_migratable(RAMBlock *rb); +bool qemu_ram_is_external(RAMBlock *rb); =20 size_t qemu_ram_pagesize(RAMBlock *block); size_t qemu_ram_pagesize_largest(void); diff --git a/migration/migration.c b/migration/migration.c index ffc4d9e556..9b789d3535 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1979,6 +1979,15 @@ bool migrate_dirty_bitmaps(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_DIRTY_BITMAPS]; } =20 +bool migrate_ignore_external(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_X_IGNORE_EXTERNAL]; +} + bool migrate_use_events(void) { MigrationState *s; diff --git a/migration/migration.h b/migration/migration.h index e413d4d8b6..06691242f3 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -255,6 +255,7 @@ bool migrate_release_ram(void); bool migrate_postcopy_ram(void); bool migrate_zero_blocks(void); bool migrate_dirty_bitmaps(void); +bool migrate_ignore_external(void); =20 bool migrate_auto_converge(void); bool migrate_use_multifd(void); diff --git a/migration/ram.c b/migration/ram.c index 39629254e1..0091a8ae3a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -166,6 +166,11 @@ out: =20 #undef RAMBLOCK_FOREACH =20 +static bool is_ignored_block(RAMBlock *block) +{ + return migrate_ignore_external() && qemu_ram_is_external(block); +} + static void ramblock_recv_map_init(void) { RAMBlock *rb; @@ -1537,7 +1542,7 @@ unsigned long migration_bitmap_find_dirty(RAMState *r= s, RAMBlock *rb, unsigned long *bitmap =3D rb->bmap; unsigned long next; =20 - if (!qemu_ram_is_migratable(rb)) { + if (!qemu_ram_is_migratable(rb) || is_ignored_block(rb)) { return size; } =20 @@ -1651,6 +1656,9 @@ static void migration_bitmap_sync(RAMState *rs) qemu_mutex_lock(&rs->bitmap_mutex); rcu_read_lock(); RAMBLOCK_FOREACH_MIGRATABLE(block) { + if (is_ignored_block(block)) { + continue; + } migration_bitmap_sync_range(rs, block, 0, block->used_length); } ram_counters.remaining =3D ram_bytes_remaining(); @@ -2472,19 +2480,27 @@ void acct_update_position(QEMUFile *f, size_t size,= bool zero) } } =20 -uint64_t ram_bytes_total(void) +static uint64_t ram_bytes_total_common(bool skip_ignored) { RAMBlock *block; uint64_t total =3D 0; =20 rcu_read_lock(); RAMBLOCK_FOREACH_MIGRATABLE(block) { + if (skip_ignored && is_ignored_block(block)) { + continue; + } total +=3D block->used_length; } rcu_read_unlock(); return total; } =20 +uint64_t ram_bytes_total(void) +{ + return ram_bytes_total_common(true); +} + static void xbzrle_load_setup(void) { XBZRLE.decoded_buf =3D g_malloc(TARGET_PAGE_SIZE); @@ -3162,7 +3178,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) =20 rcu_read_lock(); =20 - qemu_put_be64(f, ram_bytes_total() | RAM_SAVE_FLAG_MEM_SIZE); + qemu_put_be64(f, ram_bytes_total_common(false) | RAM_SAVE_FLAG_MEM_SIZ= E); =20 RAMBLOCK_FOREACH_MIGRATABLE(block) { qemu_put_byte(f, strlen(block->idstr)); @@ -3172,6 +3188,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) qemu_put_be64(f, block->page_size); } qemu_put_be64(f, block->offset); + qemu_put_byte(f, is_ignored_block(block) ? 1 : 0); } =20 rcu_read_unlock(); @@ -4135,13 +4152,27 @@ static int ram_load(QEMUFile *f, void *opaque, int = version_id) } if (version_id >=3D 5) { ram_addr_t offset; + bool ignored; offset =3D qemu_get_be64(f); + ignored =3D qemu_get_byte(f); if (block->offset !=3D offset) { error_report("Mismatched RAM block offset %s " "%" PRId64 "!=3D %" PRId64, id, offset, (uint64_t)block->offs= et); ret =3D -EINVAL; } + if (ignored) { + if (!migrate_ignore_external()) { + error_report("Unexpected ignored RAM block= %s: " + "ignore-external capability i= s " + "disabled", id); + ret =3D -EINVAL; + } else if (!qemu_ram_is_external(block)) { + error_report("Only external RAM block %s " + "can be ignored", id); + ret =3D -EINVAL; + } + } } ram_control_load_hook(f, RAM_CONTROL_BLOCK_REG, block->idstr); diff --git a/qapi/migration.json b/qapi/migration.json index 31b589ec26..10bbac87f8 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -406,13 +406,17 @@ # devices (and thus take locks) immediately at the end of migrat= ion. # (since 3.0) # +# @x-ignore-external: If enabled, QEMU will not migrate shared memory +# (since 4.0) +# # Since: 1.2 ## { 'enum': 'MigrationCapability', 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks', 'compress', 'events', 'postcopy-ram', 'x-colo', 'release-ram', 'block', 'return-path', 'pause-before-switchover', 'x-multifd', - 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate' ] } + 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', + 'x-ignore-external' ] } =20 ## # @MigrationCapabilityStatus: --=20 2.20.1 From nobody Thu May 2 10:52:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=yandex-team.ru Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1547121990406139.81580864122395; Thu, 10 Jan 2019 04:06:30 -0800 (PST) Received: from localhost ([127.0.0.1]:52959 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghZ6P-0004jk-71 for importer@patchew.org; Thu, 10 Jan 2019 07:06:25 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49405) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ghZ1m-0001Cj-AH for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ghZ1k-00005d-VP for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:38 -0500 Received: from forwardcorp1j.cmail.yandex.net ([2a02:6b8:0:1630::190]:46736) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ghZ1k-0008Sl-AP for qemu-devel@nongnu.org; Thu, 10 Jan 2019 07:01:36 -0500 Received: from mxbackcorp1j.mail.yandex.net (mxbackcorp1j.mail.yandex.net [IPv6:2a02:6b8:0:1619::162]) by forwardcorp1j.cmail.yandex.net (Yandex) with ESMTP id 273F3214B9; Thu, 10 Jan 2019 15:01:30 +0300 (MSK) Received: from smtpcorp1j.mail.yandex.net (smtpcorp1j.mail.yandex.net [2a02:6b8:0:1619::137]) by mxbackcorp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id AGMArRxFNa-1T1GVYBJ; Thu, 10 Jan 2019 15:01:30 +0300 Received: from dynamic-red.dhcp.yndx.net (dynamic-red.dhcp.yndx.net [2a02:6b8:0:40c:e1bb:a1a7:a235:d6b4]) by smtpcorp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id v4PGSBu40N-1Th8DPpv; Thu, 10 Jan 2019 15:01:29 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1547121690; bh=DiBlv69i97f+QL72KGV3VS4MTr69CbzO0aBQ/18BdrM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Xy2W9jlE69JC1dA/Bmow+fYeUsiVBgUO8uJ12n/TrLIkXq7SlxqXATPc6DQXDTWnx NIOtNgHA8ue8mVi1SPY5WZlhoGDaoL/NfyUZBkb3MSePjctTiIv5DbRXHOOESPedwJ 2O1jO1DYuwFnrCRKfZLY54qVqqwxFpAhxj+WE7rw= Authentication-Results: mxbackcorp1j.mail.yandex.net; dkim=pass header.i=@yandex-team.ru From: Yury Kotov To: qemu-devel@nongnu.org, Eduardo Habkost , Igor Mammedov , Paolo Bonzini , Peter Crosthwaite , Richard Henderson , Juan Quintela , "Dr. David Alan Gilbert" , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier Date: Thu, 10 Jan 2019 15:01:20 +0300 Message-Id: <20190110120120.9943-5-yury-kotov@yandex-team.ru> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190110120120.9943-1-yury-kotov@yandex-team.ru> References: <20190110120120.9943-1-yury-kotov@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:6b8:0:1630::190 Subject: [Qemu-devel] [PATCH 4/4] tests/migration-test: Add a test for ignore-external capability X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wrfsh@yandex-team.ru Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Signed-off-by: Yury Kotov --- tests/migration-test.c | 109 +++++++++++++++++++++++++++++++++-------- 1 file changed, 89 insertions(+), 20 deletions(-) diff --git a/tests/migration-test.c b/tests/migration-test.c index 06ca5068d8..67e6d6dad2 100644 --- a/tests/migration-test.c +++ b/tests/migration-test.c @@ -332,6 +332,13 @@ static void cleanup(const char *filename) g_free(path); } =20 +static char *get_shmem_opts(const char *mem_size, const char *shmem_path) +{ + return g_strdup_printf("-object memory-backend-file,id=3Dmem0,size=3D%= s" + ",mem-path=3D%s,share=3Don -numa node,memdev=3D= mem0", + mem_size, shmem_path); +} + static void migrate_check_parameter(QTestState *who, const char *parameter, long long value) { @@ -430,73 +437,91 @@ static void migrate_postcopy_start(QTestState *from, = QTestState *to) } =20 static int test_migrate_start(QTestState **from, QTestState **to, - const char *uri, bool hide_stderr) + const char *uri, bool hide_stderr, + bool use_shmem) { gchar *cmd_src, *cmd_dst; char *bootpath =3D g_strdup_printf("%s/bootsect", tmpfs); + char *extra_opts =3D NULL; + char *shmem_path =3D NULL; const char *arch =3D qtest_get_arch(); const char *accel =3D "kvm:tcg"; =20 got_stop =3D false; =20 + if (use_shmem) { + shmem_path =3D g_strdup_printf("/dev/shm/qemu-%d", getpid()); + } + if (strcmp(arch, "i386") =3D=3D 0 || strcmp(arch, "x86_64") =3D=3D 0) { init_bootfile(bootpath, x86_bootsect); + extra_opts =3D use_shmem ? get_shmem_opts("150M", shmem_path) : NU= LL; cmd_src =3D g_strdup_printf("-machine accel=3D%s -m 150M" " -name source,debug-threads=3Don" " -serial file:%s/src_serial" - " -drive file=3D%s,format=3Draw", - accel, tmpfs, bootpath); + " -drive file=3D%s,format=3Draw %s", + accel, tmpfs, bootpath, + extra_opts ? extra_opts : ""); cmd_dst =3D g_strdup_printf("-machine accel=3D%s -m 150M" " -name target,debug-threads=3Don" " -serial file:%s/dest_serial" " -drive file=3D%s,format=3Draw" - " -incoming %s", - accel, tmpfs, bootpath, uri); + " -incoming %s %s", + accel, tmpfs, bootpath, uri, + extra_opts ? extra_opts : ""); start_address =3D X86_TEST_MEM_START; end_address =3D X86_TEST_MEM_END; } else if (g_str_equal(arch, "s390x")) { init_bootfile_s390x(bootpath); + extra_opts =3D use_shmem ? get_shmem_opts("128M", shmem_path) : NU= LL; cmd_src =3D g_strdup_printf("-machine accel=3D%s -m 128M" " -name source,debug-threads=3Don" - " -serial file:%s/src_serial -bios %s", - accel, tmpfs, bootpath); + " -serial file:%s/src_serial -bios %s %s= ", + accel, tmpfs, bootpath, + extra_opts ? extra_opts : ""); cmd_dst =3D g_strdup_printf("-machine accel=3D%s -m 128M" " -name target,debug-threads=3Don" " -serial file:%s/dest_serial -bios %s" - " -incoming %s", - accel, tmpfs, bootpath, uri); + " -incoming %s %s", + accel, tmpfs, bootpath, uri, + extra_opts ? extra_opts : ""); start_address =3D S390_TEST_MEM_START; end_address =3D S390_TEST_MEM_END; } else if (strcmp(arch, "ppc64") =3D=3D 0) { + extra_opts =3D use_shmem ? get_shmem_opts("256M", shmem_path) : NU= LL; cmd_src =3D g_strdup_printf("-machine accel=3D%s -m 256M -nodefaul= ts" " -name source,debug-threads=3Don" " -serial file:%s/src_serial" " -prom-env 'use-nvramrc?=3Dtrue' -prom-= env " "'nvramrc=3Dhex .\" _\" begin %x %x " "do i c@ 1 + i c! 1000 +loop .\" B\" 0 " - "until'", accel, tmpfs, end_address, - start_address); + "until' %s", accel, tmpfs, end_address, + start_address, extra_opts ? extra_opts := ""); cmd_dst =3D g_strdup_printf("-machine accel=3D%s -m 256M" " -name target,debug-threads=3Don" " -serial file:%s/dest_serial" - " -incoming %s", - accel, tmpfs, uri); + " -incoming %s %s", + accel, tmpfs, uri, + extra_opts ? extra_opts : ""); =20 start_address =3D PPC_TEST_MEM_START; end_address =3D PPC_TEST_MEM_END; } else if (strcmp(arch, "aarch64") =3D=3D 0) { init_bootfile(bootpath, aarch64_kernel); + extra_opts =3D use_shmem ? get_shmem_opts("150M", shmem_path) : NU= LL; cmd_src =3D g_strdup_printf("-machine virt,accel=3D%s,gic-version= =3Dmax " "-name vmsource,debug-threads=3Don -cpu = max " "-m 150M -serial file:%s/src_serial " - "-kernel %s ", - accel, tmpfs, bootpath); + "-kernel %s %s", + accel, tmpfs, bootpath, + extra_opts ? extra_opts : ""); cmd_dst =3D g_strdup_printf("-machine virt,accel=3D%s,gic-version= =3Dmax " "-name vmdest,debug-threads=3Don -cpu ma= x " "-m 150M -serial file:%s/dest_serial " "-kernel %s " - "-incoming %s ", - accel, tmpfs, bootpath, uri); + "-incoming %s %s", + accel, tmpfs, bootpath, uri, + extra_opts ? extra_opts : ""); =20 start_address =3D ARM_TEST_MEM_START; end_address =3D ARM_TEST_MEM_END; @@ -507,6 +532,7 @@ static int test_migrate_start(QTestState **from, QTestS= tate **to, } =20 g_free(bootpath); + g_free(extra_opts); =20 if (hide_stderr) { gchar *tmp; @@ -524,6 +550,16 @@ static int test_migrate_start(QTestState **from, QTest= State **to, =20 *to =3D qtest_init(cmd_dst); g_free(cmd_dst); + + /* + * Remove shmem file immediately to avoid memory leak in test failed c= ase. + * It's valid becase QEMU has already opened this file + */ + if (use_shmem) { + unlink(shmem_path); + g_free(shmem_path); + } + return 0; } =20 @@ -603,7 +639,7 @@ static int migrate_postcopy_prepare(QTestState **from_p= tr, char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); QTestState *from, *to; =20 - if (test_migrate_start(&from, &to, uri, hide_error)) { + if (test_migrate_start(&from, &to, uri, hide_error, false)) { return -1; } =20 @@ -720,7 +756,7 @@ static void test_baddest(void) char *status; bool failed; =20 - if (test_migrate_start(&from, &to, "tcp:0:0", true)) { + if (test_migrate_start(&from, &to, "tcp:0:0", true, false)) { return; } migrate(from, "tcp:0:0", "{}"); @@ -745,7 +781,7 @@ static void test_precopy_unix(void) char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); QTestState *from, *to; =20 - if (test_migrate_start(&from, &to, uri, false)) { + if (test_migrate_start(&from, &to, uri, false, false)) { return; } =20 @@ -781,6 +817,38 @@ static void test_precopy_unix(void) g_free(uri); } =20 +static void test_ignore_external(void) +{ + char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + QTestState *from, *to; + + if (test_migrate_start(&from, &to, uri, false, true)) { + return; + } + + migrate_set_capability(from, "x-ignore-external", true); + migrate_set_capability(to, "x-ignore-external", true); + + /* Wait for the first serial output from the source */ + wait_for_serial("src_serial"); + + migrate(from, uri, "{}"); + + wait_for_migration_pass(from); + + if (!got_stop) { + qtest_qmp_eventwait(from, "STOP"); + } + + qtest_qmp_eventwait(to, "RESUME"); + + wait_for_serial("dest_serial"); + wait_for_migration_complete(from); + + test_migrate_end(from, to, true); + g_free(uri); +} + int main(int argc, char **argv) { char template[] =3D "/tmp/migration-test-XXXXXX"; @@ -832,6 +900,7 @@ int main(int argc, char **argv) qtest_add_func("/migration/deprecated", test_deprecated); qtest_add_func("/migration/bad_dest", test_baddest); qtest_add_func("/migration/precopy/unix", test_precopy_unix); + qtest_add_func("/migration/ignore_external", test_ignore_external); =20 ret =3D g_test_run(); =20 --=20 2.20.1