From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488285779819521.4375919122962; Tue, 28 Feb 2017 04:42:59 -0800 (PST) Received: from localhost ([::1]:60712 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih7K-0000ri-2m for importer@patchew.org; Tue, 28 Feb 2017 07:42:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39869) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5U-0007v9-BF for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5S-00007C-Jx for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:04 -0500 Received: from mx1.redhat.com ([209.132.183.28]:42846) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5S-00005D-BZ for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:02 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 66C9480463; Tue, 28 Feb 2017 12:41:02 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewt1019858; Tue, 28 Feb 2017 07:41:00 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:30 +0000 Message-Id: <20170228124056.5074-2-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 28 Feb 2017 12:41:02 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 01/27] Changing error message of QMP 'migrate_set_downtime' to seconds X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Daniel Henrique Barboza Using QMP, the error message of 'migrate_set_downtime' was displaying the values in milliseconds, being misleading with the command that accepts the value in seconds: { "execute": "migrate_set_downtime", "arguments": {"value": 3000}} {"error": {"class": "GenericError", "desc": "Parameter 'downtime_limit' expects an integer in the range of 0 to 2000000 milliseconds"}} This message is also seen in HMP when trying to set the same parameter: (qemu) migrate_set_parameter downtime-limit 3000000 Parameter 'downtime_limit' expects an integer in the range of 0 to 2000000 milliseconds To allow for a proper error message when using QMP, a validation of the user input was added in 'qmp_migrate_set_downtime'. Signed-off-by: Daniel Henrique Barboza Message-Id: <20170222151729.5812-1-danielhb@linux.vnet.ibm.com> Reviewed-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index c6ae69d..4fab538 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -49,6 +49,10 @@ * for sending the last part */ #define DEFAULT_MIGRATE_SET_DOWNTIME 300 =20 +/* Maximum migrate downtime set to 2000 seconds */ +#define MAX_MIGRATE_DOWNTIME_SECONDS 2000 +#define MAX_MIGRATE_DOWNTIME (MAX_MIGRATE_DOWNTIME_SECONDS * 1000) + /* Default compression thread count */ #define DEFAULT_MIGRATE_COMPRESS_THREAD_COUNT 8 /* Default decompression thread count, usually decompression is at @@ -843,10 +847,11 @@ void qmp_migrate_set_parameters(MigrationParameters *= params, Error **errp) return; } if (params->has_downtime_limit && - (params->downtime_limit < 0 || params->downtime_limit > 2000000)) { - error_setg(errp, QERR_INVALID_PARAMETER_VALUE, - "downtime_limit", - "an integer in the range of 0 to 2000000 milliseconds"); + (params->downtime_limit < 0 || + params->downtime_limit > MAX_MIGRATE_DOWNTIME)) { + error_setg(errp, "Parameter 'downtime_limit' expects an integer in= " + "the range of 0 to %d milliseconds", + MAX_MIGRATE_DOWNTIME); return; } if (params->has_x_checkpoint_delay && (params->x_checkpoint_delay < 0)= ) { @@ -1289,6 +1294,13 @@ void qmp_migrate_set_speed(int64_t value, Error **er= rp) =20 void qmp_migrate_set_downtime(double value, Error **errp) { + if (value < 0 || value > MAX_MIGRATE_DOWNTIME_SECONDS) { + error_setg(errp, "Parameter 'downtime_limit' expects an integer in= " + "the range of 0 to %d seconds", + MAX_MIGRATE_DOWNTIME_SECONDS); + return; + } + value *=3D 1000; /* Convert to milliseconds */ value =3D MAX(0, MIN(INT64_MAX, value)); =20 --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488285954977620.7749106125434; Tue, 28 Feb 2017 04:45:54 -0800 (PST) Received: from localhost ([::1]:60730 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihA9-0003pl-O1 for importer@patchew.org; Tue, 28 Feb 2017 07:45:53 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39887) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5V-0007wF-IM for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5U-0000A8-I7 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:05 -0500 Received: from mx1.redhat.com ([209.132.183.28]:34248) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5U-00007R-A5 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:04 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 635E761BBC; Tue, 28 Feb 2017 12:41:04 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewt2019858; Tue, 28 Feb 2017 07:41:02 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:31 +0000 Message-Id: <20170228124056.5074-3-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 28 Feb 2017 12:41:04 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 02/27] migration/vmstate: renames in (load|save)_state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Halil Pasic The vmstate_(load|save)_state start out with an a void *opaque pointing to some struct, and manipulate one or more elements of one field within that struct. First the field within the struct is pinpointed as opaque + offset, then if this is a pointer the pointer is dereferenced to obtain a pointer to the first element of the vmstate field. Pointers to further elements if any are calculated as first_element + i * element_size (where i is the zero based index of the element in question). Currently base_addr and addr is used as a variable name for the pointer to the first element and the pointer to the current element being processed. This is suboptimal because base_addr is somewhat counter-intuitive (because obtained as base + offset) and both base_addr and addr not very descriptive (that we have a pointer should be clear from the fact that it is declared as a pointer). Let make things easier to understand by renaming base_addr to first_elem and addr to curr_elem. This has the additional benefit of harmonizing with other names within the scope (n_elems, vmstate_n_elems). Signed-off-by: Halil Pasic Reviewed-by: Dr. David Alan Gilbert Message-Id: <20170222160119.52771-2-pasic@linux.vnet.ibm.com> Signed-off-by: Dr. David Alan Gilbert --- migration/vmstate.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/migration/vmstate.c b/migration/vmstate.c index b4d8ae9..36efa80 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -116,21 +116,21 @@ int vmstate_load_state(QEMUFile *f, const VMStateDesc= ription *vmsd, field->field_exists(opaque, version_id)) || (!field->field_exists && field->version_id <=3D version_id)) { - void *base_addr =3D vmstate_base_addr(opaque, field, true); + void *first_elem =3D vmstate_base_addr(opaque, field, true); int i, n_elems =3D vmstate_n_elems(opaque, field); int size =3D vmstate_size(opaque, field); =20 for (i =3D 0; i < n_elems; i++) { - void *addr =3D base_addr + size * i; + void *curr_elem =3D first_elem + size * i; =20 if (field->flags & VMS_ARRAY_OF_POINTER) { - addr =3D *(void **)addr; + curr_elem =3D *(void **)curr_elem; } if (field->flags & VMS_STRUCT) { - ret =3D vmstate_load_state(f, field->vmsd, addr, + ret =3D vmstate_load_state(f, field->vmsd, curr_elem, field->vmsd->version_id); } else { - ret =3D field->info->get(f, addr, size, field); + ret =3D field->info->get(f, curr_elem, size, field); } if (ret >=3D 0) { ret =3D qemu_file_get_error(f); @@ -321,7 +321,7 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescr= iption *vmsd, while (field->name) { if (!field->field_exists || field->field_exists(opaque, vmsd->version_id)) { - void *base_addr =3D vmstate_base_addr(opaque, field, false); + void *first_elem =3D vmstate_base_addr(opaque, field, false); int i, n_elems =3D vmstate_n_elems(opaque, field); int size =3D vmstate_size(opaque, field); int64_t old_offset, written_bytes; @@ -329,18 +329,18 @@ void vmstate_save_state(QEMUFile *f, const VMStateDes= cription *vmsd, =20 trace_vmstate_save_state_loop(vmsd->name, field->name, n_elems= ); for (i =3D 0; i < n_elems; i++) { - void *addr =3D base_addr + size * i; + void *curr_elem =3D first_elem + size * i; =20 vmsd_desc_field_start(vmsd, vmdesc_loop, field, i, n_elems= ); old_offset =3D qemu_ftell_fast(f); - if (field->flags & VMS_ARRAY_OF_POINTER) { - addr =3D *(void **)addr; + assert(curr_elem); + curr_elem =3D *(void **)curr_elem; } if (field->flags & VMS_STRUCT) { - vmstate_save_state(f, field->vmsd, addr, vmdesc_loop); + vmstate_save_state(f, field->vmsd, curr_elem, vmdesc_l= oop); } else { - field->info->put(f, addr, size, field, vmdesc_loop); + field->info->put(f, curr_elem, size, field, vmdesc_loo= p); } =20 written_bytes =3D qemu_ftell_fast(f) - old_offset; --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286044015141.92654713390766; Tue, 28 Feb 2017 04:47:24 -0800 (PST) Received: from localhost ([::1]:60736 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihBa-00055L-3G for importer@patchew.org; Tue, 28 Feb 2017 07:47:22 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39905) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5X-0007xr-SY for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5W-0000CZ-HT for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:07 -0500 Received: from mx1.redhat.com ([209.132.183.28]:42666) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5W-0000CL-8g for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:06 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 64151D1FDF; Tue, 28 Feb 2017 12:41:06 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewt3019858; Tue, 28 Feb 2017 07:41:04 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:32 +0000 Message-Id: <20170228124056.5074-4-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 28 Feb 2017 12:41:06 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 03/27] migration/vmstate: split up vmstate_base_addr X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Halil Pasic Currently vmstate_base_addr does several things: it pinpoints the field within the struct, possibly allocates memory and possibly does the first pointer dereference. Obviously allocation is needed only for load. Let us split up the functionality in vmstate_base_addr and move the address manipulations (that is everything but the allocation logic) to load and save so it becomes more obvious what is actually going on. Like this all the address calculations (and the handling of the flags controlling these) is in one place and the sequence is more obvious. The newly introduced function vmstate_handle_alloc also fixes the allocation for the unused VMS_VBUFFER|VMS_MULTIPLY|VMS_ALLOC scenario and is substantially simpler than the original vmstate_base_addr. In load and save some asserts are added so it's easier to debug situations where we would end up with a null pointer dereference. Signed-off-by: Halil Pasic Reviewed-by: Dr. David Alan Gilbert Message-Id: <20170222160119.52771-3-pasic@linux.vnet.ibm.com> Signed-off-by: Dr. David Alan Gilbert --- migration/vmstate.c | 39 +++++++++++++++++---------------------- 1 file changed, 17 insertions(+), 22 deletions(-) diff --git a/migration/vmstate.c b/migration/vmstate.c index 36efa80..836a7a4 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -52,29 +52,15 @@ static int vmstate_size(void *opaque, VMStateField *fie= ld) return size; } =20 -static void *vmstate_base_addr(void *opaque, VMStateField *field, bool all= oc) +static void vmstate_handle_alloc(void *ptr, VMStateField *field, void *opa= que) { - void *base_addr =3D opaque + field->offset; - - if (field->flags & VMS_POINTER) { - if (alloc && (field->flags & VMS_ALLOC)) { - gsize size =3D 0; - if (field->flags & VMS_VBUFFER) { - size =3D vmstate_size(opaque, field); - } else { - int n_elems =3D vmstate_n_elems(opaque, field); - if (n_elems) { - size =3D n_elems * field->size; - } - } - if (size) { - *(void **)base_addr =3D g_malloc(size); - } + if (field->flags & VMS_POINTER && field->flags & VMS_ALLOC) { + gsize size =3D vmstate_size(opaque, field); + size *=3D vmstate_n_elems(opaque, field); + if (size) { + *(void **)ptr =3D g_malloc(size); } - base_addr =3D *(void **)base_addr; } - - return base_addr; } =20 int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd, @@ -116,10 +102,15 @@ int vmstate_load_state(QEMUFile *f, const VMStateDesc= ription *vmsd, field->field_exists(opaque, version_id)) || (!field->field_exists && field->version_id <=3D version_id)) { - void *first_elem =3D vmstate_base_addr(opaque, field, true); + void *first_elem =3D opaque + field->offset; int i, n_elems =3D vmstate_n_elems(opaque, field); int size =3D vmstate_size(opaque, field); =20 + vmstate_handle_alloc(first_elem, field, opaque); + if (field->flags & VMS_POINTER) { + first_elem =3D *(void **)first_elem; + assert(first_elem || !n_elems); + } for (i =3D 0; i < n_elems; i++) { void *curr_elem =3D first_elem + size * i; =20 @@ -321,13 +312,17 @@ void vmstate_save_state(QEMUFile *f, const VMStateDes= cription *vmsd, while (field->name) { if (!field->field_exists || field->field_exists(opaque, vmsd->version_id)) { - void *first_elem =3D vmstate_base_addr(opaque, field, false); + void *first_elem =3D opaque + field->offset; int i, n_elems =3D vmstate_n_elems(opaque, field); int size =3D vmstate_size(opaque, field); int64_t old_offset, written_bytes; QJSON *vmdesc_loop =3D vmdesc; =20 trace_vmstate_save_state_loop(vmsd->name, field->name, n_elems= ); + if (field->flags & VMS_POINTER) { + first_elem =3D *(void **)first_elem; + assert(first_elem || !n_elems); + } for (i =3D 0; i < n_elems; i++) { void *curr_elem =3D first_elem + size * i; =20 --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286322106367.72579395713876; Tue, 28 Feb 2017 04:52:02 -0800 (PST) Received: from localhost ([::1]:60760 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihG4-0000np-Tm for importer@patchew.org; Tue, 28 Feb 2017 07:52:00 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39924) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5Z-0007zV-Jl for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5Y-0000DK-G1 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:09 -0500 Received: from mx1.redhat.com ([209.132.183.28]:54790) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5Y-0000Cr-7z for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:08 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5A86B81239; Tue, 28 Feb 2017 12:41:08 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewt4019858; Tue, 28 Feb 2017 07:41:06 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:33 +0000 Message-Id: <20170228124056.5074-5-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 28 Feb 2017 12:41:08 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 04/27] migration/vmstate: fix array of ptr with nullptrs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Halil Pasic Make VMS_ARRAY_OF_POINTER cope with null pointers. Previously the reward for trying to migrate an array with some null pointers in it was an illegal memory access, that is a swift and painless death of the process. Let's make vmstate cope with this scenario. The general approach is, when we encounter a null pointer (element), instead of following the pointer to save/load the data behind it, we save/load a placeholder. This way we can detect if we expected a null pointer at the load side but not null data was saved instead. Signed-off-by: Halil Pasic Reviewed-by: Guenther Hutzl Reviewed-by: Dr. David Alan Gilbert Message-Id: <20170222160119.52771-4-pasic@linux.vnet.ibm.com> Signed-off-by: Dr. David Alan Gilbert --- include/migration/vmstate.h | 4 ++++ migration/vmstate.c | 40 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 63e7b02..f2dbf84 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -253,6 +253,10 @@ extern const VMStateInfo vmstate_info_uint16; extern const VMStateInfo vmstate_info_uint32; extern const VMStateInfo vmstate_info_uint64; =20 +/** Put this in the stream when migrating a null pointer.*/ +#define VMS_NULLPTR_MARKER (0x30U) /* '0' */ +extern const VMStateInfo vmstate_info_nullptr; + extern const VMStateInfo vmstate_info_float64; extern const VMStateInfo vmstate_info_cpudouble; =20 diff --git a/migration/vmstate.c b/migration/vmstate.c index 836a7a4..78b3cd4 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -117,7 +117,11 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescr= iption *vmsd, if (field->flags & VMS_ARRAY_OF_POINTER) { curr_elem =3D *(void **)curr_elem; } - if (field->flags & VMS_STRUCT) { + if (!curr_elem) { + /* if null pointer check placeholder and do not follow= */ + assert(field->flags & VMS_ARRAY_OF_POINTER); + ret =3D vmstate_info_nullptr.get(f, curr_elem, size, N= ULL); + } else if (field->flags & VMS_STRUCT) { ret =3D vmstate_load_state(f, field->vmsd, curr_elem, field->vmsd->version_id); } else { @@ -332,7 +336,11 @@ void vmstate_save_state(QEMUFile *f, const VMStateDesc= ription *vmsd, assert(curr_elem); curr_elem =3D *(void **)curr_elem; } - if (field->flags & VMS_STRUCT) { + if (!curr_elem) { + /* if null pointer write placeholder and do not follow= */ + assert(field->flags & VMS_ARRAY_OF_POINTER); + vmstate_info_nullptr.put(f, curr_elem, size, NULL, NUL= L); + } else if (field->flags & VMS_STRUCT) { vmstate_save_state(f, field->vmsd, curr_elem, vmdesc_l= oop); } else { field->info->put(f, curr_elem, size, field, vmdesc_loo= p); @@ -747,6 +755,34 @@ const VMStateInfo vmstate_info_uint64 =3D { .put =3D put_uint64, }; =20 +static int get_nullptr(QEMUFile *f, void *pv, size_t size, VMStateField *f= ield) + +{ + if (qemu_get_byte(f) =3D=3D VMS_NULLPTR_MARKER) { + return 0; + } + error_report("vmstate: get_nullptr expected VMS_NULLPTR_MARKER"); + return -EINVAL; +} + +static int put_nullptr(QEMUFile *f, void *pv, size_t size, + VMStateField *field, QJSON *vmdesc) + +{ + if (pv =3D=3D NULL) { + qemu_put_byte(f, VMS_NULLPTR_MARKER); + return 0; + } + error_report("vmstate: put_nullptr must be called with pv =3D=3D NULL"= ); + return -EINVAL; +} + +const VMStateInfo vmstate_info_nullptr =3D { + .name =3D "uint64", + .get =3D get_nullptr, + .put =3D put_nullptr, +}; + /* 64 bit unsigned int. See that the received value is the same than the o= ne in the field */ =20 --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 148828659417643.0129715731224; Tue, 28 Feb 2017 04:56:34 -0800 (PST) Received: from localhost ([::1]:60784 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihKS-00058C-Va for importer@patchew.org; Tue, 28 Feb 2017 07:56:33 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39960) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5d-00082Z-48 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5a-0000EW-GE for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:13 -0500 Received: from mx1.redhat.com ([209.132.183.28]:47662) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5a-0000EG-7h for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:10 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 55F467E9D1; Tue, 28 Feb 2017 12:41:10 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewt5019858; Tue, 28 Feb 2017 07:41:08 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:34 +0000 Message-Id: <20170228124056.5074-6-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 28 Feb 2017 12:41:10 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 05/27] tests/test-vmstate.c: test array of ptr with null X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Halil Pasic Add test for VMSTATE_ARRAY_OF_POINTER_TO_STRUCT with an array containing some null pointer. Signed-off-by: Halil Pasic Reviewed-by: Dr. David Alan Gilbert Message-Id: <20170222160119.52771-5-pasic@linux.vnet.ibm.com> Signed-off-by: Dr. David Alan Gilbert Fixed type case in assert to uintptr_t rather than uint64_t --- tests/test-vmstate.c | 74 ++++++++++++++++++++++++++++++++++++++++--------= ---- 1 file changed, 57 insertions(+), 17 deletions(-) diff --git a/tests/test-vmstate.c b/tests/test-vmstate.c index 39f338a..2e20756 100644 --- a/tests/test-vmstate.c +++ b/tests/test-vmstate.c @@ -476,6 +476,8 @@ const VMStateDescription vmsd_tst =3D { } }; =20 +/* test array migration */ + #define AR_SIZE 4 =20 typedef struct { @@ -492,20 +494,22 @@ const VMStateDescription vmsd_arps =3D { VMSTATE_END_OF_LIST() } }; + +static uint8_t wire_arr_ptr_no0[] =3D { + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x01, + 0x00, 0x00, 0x00, 0x02, + 0x00, 0x00, 0x00, 0x03, + QEMU_VM_EOF +}; + static void test_arr_ptr_str_no0_save(void) { TestStructTriv ar[AR_SIZE] =3D {{.i =3D 0}, {.i =3D 1}, {.i =3D 2}, {.= i =3D 3} }; TestArrayOfPtrToStuct sample =3D {.ar =3D {&ar[0], &ar[1], &ar[2], &ar= [3]} }; - uint8_t wire_sample[] =3D { - 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x01, - 0x00, 0x00, 0x00, 0x02, - 0x00, 0x00, 0x00, 0x03, - QEMU_VM_EOF - }; =20 save_vmstate(&vmsd_arps, &sample); - compare_vmstate(wire_sample, sizeof(wire_sample)); + compare_vmstate(wire_arr_ptr_no0, sizeof(wire_arr_ptr_no0)); } =20 static void test_arr_ptr_str_no0_load(void) @@ -514,21 +518,54 @@ static void test_arr_ptr_str_no0_load(void) TestStructTriv ar[AR_SIZE] =3D {}; TestArrayOfPtrToStuct obj =3D {.ar =3D {&ar[0], &ar[1], &ar[2], &ar[3]= } }; int idx; - uint8_t wire_sample[] =3D { - 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x01, - 0x00, 0x00, 0x00, 0x02, - 0x00, 0x00, 0x00, 0x03, - QEMU_VM_EOF - }; =20 - save_buffer(wire_sample, sizeof(wire_sample)); + save_buffer(wire_arr_ptr_no0, sizeof(wire_arr_ptr_no0)); + SUCCESS(load_vmstate_one(&vmsd_arps, &obj, 1, + wire_arr_ptr_no0, sizeof(wire_arr_ptr_no0))); + for (idx =3D 0; idx < AR_SIZE; ++idx) { + /* compare the target array ar with the ground truth array ar_gt */ + g_assert_cmpint(ar_gt[idx].i, =3D=3D, ar[idx].i); + } +} + +static uint8_t wire_arr_ptr_0[] =3D { + 0x00, 0x00, 0x00, 0x00, + VMS_NULLPTR_MARKER, + 0x00, 0x00, 0x00, 0x02, + 0x00, 0x00, 0x00, 0x03, + QEMU_VM_EOF +}; + +static void test_arr_ptr_str_0_save(void) +{ + TestStructTriv ar[AR_SIZE] =3D {{.i =3D 0}, {.i =3D 1}, {.i =3D 2}, {.= i =3D 3} }; + TestArrayOfPtrToStuct sample =3D {.ar =3D {&ar[0], NULL, &ar[2], &ar[3= ]} }; + + save_vmstate(&vmsd_arps, &sample); + compare_vmstate(wire_arr_ptr_0, sizeof(wire_arr_ptr_0)); +} + +static void test_arr_ptr_str_0_load(void) +{ + TestStructTriv ar_gt[AR_SIZE] =3D {{.i =3D 0}, {.i =3D 0}, {.i =3D 2},= {.i =3D 3} }; + TestStructTriv ar[AR_SIZE] =3D {}; + TestArrayOfPtrToStuct obj =3D {.ar =3D {&ar[0], NULL, &ar[2], &ar[3]} = }; + int idx; + + save_buffer(wire_arr_ptr_0, sizeof(wire_arr_ptr_0)); SUCCESS(load_vmstate_one(&vmsd_arps, &obj, 1, - wire_sample, sizeof(wire_sample))); + wire_arr_ptr_0, sizeof(wire_arr_ptr_0))); for (idx =3D 0; idx < AR_SIZE; ++idx) { /* compare the target array ar with the ground truth array ar_gt */ g_assert_cmpint(ar_gt[idx].i, =3D=3D, ar[idx].i); } + for (idx =3D 0; idx < AR_SIZE; ++idx) { + if (idx =3D=3D 1) { + g_assert_cmpint((uintptr_t)(obj.ar[idx]), =3D=3D, 0); + } else { + g_assert_cmpint((uintptr_t)(obj.ar[idx]), !=3D, 0); + } + } } =20 /* test QTAILQ migration */ @@ -781,6 +818,9 @@ int main(int argc, char **argv) test_arr_ptr_str_no0_save); g_test_add_func("/vmstate/array/ptr/str/no0/load", test_arr_ptr_str_no0_load); + g_test_add_func("/vmstate/array/ptr/str/0/save", test_arr_ptr_str_0_sa= ve); + g_test_add_func("/vmstate/array/ptr/str/0/load", + test_arr_ptr_str_0_load); g_test_add_func("/vmstate/qtailq/save/saveq", test_save_q); g_test_add_func("/vmstate/qtailq/load/loadq", test_load_q); g_test_add_func("/vmstate/tmp_struct", test_tmp_struct); --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286845819175.25242036992586; Tue, 28 Feb 2017 05:00:45 -0800 (PST) Received: from localhost ([::1]:60805 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihOV-0000M7-IL for importer@patchew.org; Tue, 28 Feb 2017 08:00:43 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39963) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5d-000831-Eg for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5c-0000Ey-G5 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:13 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51490) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5c-0000Eh-7p for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:12 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 55E203B709; Tue, 28 Feb 2017 12:41:12 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewt6019858; Tue, 28 Feb 2017 07:41:10 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:35 +0000 Message-Id: <20170228124056.5074-7-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 28 Feb 2017 12:41:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 06/27] tests/test-vmstate.c: test array of ptr to primitive X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Halil Pasic Let's have a test for ptr arrays to some primitive type with some not-null and null ptrs intermixed. Signed-off-by: Halil Pasic Reviewed-by: Dr. David Alan Gilbert Message-Id: <20170222160119.52771-6-pasic@linux.vnet.ibm.com> Signed-off-by: Dr. David Alan Gilbert --- tests/test-vmstate.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/tests/test-vmstate.c b/tests/test-vmstate.c index 2e20756..f694a89 100644 --- a/tests/test-vmstate.c +++ b/tests/test-vmstate.c @@ -568,6 +568,50 @@ static void test_arr_ptr_str_0_load(void) } } =20 +typedef struct TestArrayOfPtrToInt { + int32_t *ar[AR_SIZE]; +} TestArrayOfPtrToInt; + +const VMStateDescription vmsd_arpp =3D { + .name =3D "test/arps", + .version_id =3D 1, + .minimum_version_id =3D 1, + .fields =3D (VMStateField[]) { + VMSTATE_ARRAY_OF_POINTER(ar, TestArrayOfPtrToInt, + AR_SIZE, 0, vmstate_info_int32, int32_t*), + VMSTATE_END_OF_LIST() + } +}; + +static void test_arr_ptr_prim_0_save(void) +{ + int32_t ar[AR_SIZE] =3D {0 , 1, 2, 3}; + TestArrayOfPtrToInt sample =3D {.ar =3D {&ar[0], NULL, &ar[2], &ar[3]= } }; + + save_vmstate(&vmsd_arpp, &sample); + compare_vmstate(wire_arr_ptr_0, sizeof(wire_arr_ptr_0)); +} + +static void test_arr_ptr_prim_0_load(void) +{ + int32_t ar_gt[AR_SIZE] =3D {0, 1, 2, 3}; + int32_t ar[AR_SIZE] =3D {3 , 42, 1, 0}; + TestArrayOfPtrToInt obj =3D {.ar =3D {&ar[0], NULL, &ar[2], &ar[3]} }; + int idx; + + save_buffer(wire_arr_ptr_0, sizeof(wire_arr_ptr_0)); + SUCCESS(load_vmstate_one(&vmsd_arpp, &obj, 1, + wire_arr_ptr_0, sizeof(wire_arr_ptr_0))); + for (idx =3D 0; idx < AR_SIZE; ++idx) { + /* compare the target array ar with the ground truth array ar_gt */ + if (idx =3D=3D 1) { + g_assert_cmpint(42, =3D=3D, ar[idx]); + } else { + g_assert_cmpint(ar_gt[idx], =3D=3D, ar[idx]); + } + } +} + /* test QTAILQ migration */ typedef struct TestQtailqElement TestQtailqElement; =20 @@ -821,6 +865,10 @@ int main(int argc, char **argv) g_test_add_func("/vmstate/array/ptr/str/0/save", test_arr_ptr_str_0_sa= ve); g_test_add_func("/vmstate/array/ptr/str/0/load", test_arr_ptr_str_0_load); + g_test_add_func("/vmstate/array/ptr/prim/0/save", + test_arr_ptr_prim_0_save); + g_test_add_func("/vmstate/array/ptr/prim/0/load", + test_arr_ptr_prim_0_load); g_test_add_func("/vmstate/qtailq/save/saveq", test_save_q); g_test_add_func("/vmstate/qtailq/load/loadq", test_load_q); g_test_add_func("/vmstate/tmp_struct", test_tmp_struct); --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286045150269.15296060929086; Tue, 28 Feb 2017 04:47:25 -0800 (PST) Received: from localhost ([::1]:60737 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihBa-00056G-2N for importer@patchew.org; Tue, 28 Feb 2017 07:47:22 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39991) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5f-00085i-Oc for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5e-0000Fi-En for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:15 -0500 Received: from mx1.redhat.com ([209.132.183.28]:34308) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5e-0000FQ-8z for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:14 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 53D1661D29; Tue, 28 Feb 2017 12:41:14 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewt7019858; Tue, 28 Feb 2017 07:41:12 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:36 +0000 Message-Id: <20170228124056.5074-8-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 28 Feb 2017 12:41:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 07/27] vmstate-static-checker: update white list with spapr_pci X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Laurent Vivier To fix migration between 2.7 and 2.8, some fields have been renamed and managed with the help of a PHB property (pre_2_8_migration): 5c4537b spapr: Fix 2.7<->2.8 migration of PCI host bridge So we need to add them to the white list: dma_liobn[0], mem_win_addr, mem_win_size, io_win_addr, io_win_size become mig_liobn, mig_mem_win_addr, mig_mem_win_size, mig_io_win_addr, mig_io_win_size CC: David Gibson CC: Dr. David Alan Gilbert CC: Thomas Huth CC: Greg Kurz CC: Alexey Kardashevskiy Signed-off-by: Laurent Vivier Message-Id: <20170214133331.28997-1-lvivier@redhat.com> Reviewed-by: Thomas Huth Reviewed-by: David Gibson Signed-off-by: Dr. David Alan Gilbert --- scripts/vmstate-static-checker.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/scripts/vmstate-static-checker.py b/scripts/vmstate-static-che= cker.py index 14a27e7..bcef7ee 100755 --- a/scripts/vmstate-static-checker.py +++ b/scripts/vmstate-static-checker.py @@ -85,6 +85,11 @@ def check_fields_match(name, s_field, d_field): 'xio3130-express-upstream-port': ['br.dev', 'parent_obj.parent_obj= ', 'br.dev.exp.aer_log', 'parent_obj.parent_obj.exp.aer_l= og'], + 'spapr_pci': ['dma_liobn[0]', 'mig_liobn', + 'mem_win_addr', 'mig_mem_win_addr', + 'mem_win_size', 'mig_mem_win_size', + 'io_win_addr', 'mig_io_win_addr', + 'io_win_size', 'mig_io_win_size'], } =20 if not name in changed_names: --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 148828579747365.10709291572834; Tue, 28 Feb 2017 04:43:17 -0800 (PST) Received: from localhost ([::1]:60714 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih7c-00017S-5I for importer@patchew.org; Tue, 28 Feb 2017 07:43:16 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40003) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5h-00087f-MX for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5g-0000GF-EH for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:17 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57672) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5g-0000G6-5H for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:16 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4872E4E4CB; Tue, 28 Feb 2017 12:41:16 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewt8019858; Tue, 28 Feb 2017 07:41:14 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:37 +0000 Message-Id: <20170228124056.5074-9-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 28 Feb 2017 12:41:16 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 08/27] migrate: Introduce a 'dc->vmsd' check to avoid segfault for --only-migratable X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Ashijeet Acharya Commit a3a3d8c7 introduced a segfault bug while checking for 'dc->vmsd->unmigratable' which caused QEMU to crash when trying to add devices which do no set their 'dc->vmsd' yet while initialization. Place a 'dc->vmsd' check prior to it so that we do not segfault for such devices. NOTE: This doesn't compromise the functioning of --only-migratable option as all the unmigratable devices do set their 'dc->vmsd'. Introduce a new function check_migratable() and move the only_migratable check inside it, also use stubs to avoid user-mode qemu build failures. Signed-off-by: Ashijeet Acharya Message-Id: <1487009088-23891-1-git-send-email-ashijeetacharya@gmail.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- hw/core/qdev.c | 7 +++++++ hw/usb/bus.c | 19 ------------------- include/migration/migration.h | 3 +++ migration/migration.c | 15 +++++++++++++++ qdev-monitor.c | 9 --------- stubs/vmstate.c | 6 ++++++ 6 files changed, 31 insertions(+), 28 deletions(-) diff --git a/hw/core/qdev.c b/hw/core/qdev.c index 06ba02e..dd1ca1c 100644 --- a/hw/core/qdev.c +++ b/hw/core/qdev.c @@ -37,6 +37,7 @@ #include "hw/boards.h" #include "hw/sysbus.h" #include "qapi-event.h" +#include "migration/migration.h" =20 int qdev_hotplug =3D 0; static bool qdev_hot_added =3D false; @@ -889,6 +890,7 @@ static void device_set_realized(Object *obj, bool value= , Error **errp) Error *local_err =3D NULL; bool unattached_parent =3D false; static int unattached_count; + int ret; =20 if (dev->hotplugged && !dc->hotpluggable) { error_setg(errp, QERR_DEVICE_NO_HOTPLUG, object_get_typename(obj)); @@ -896,6 +898,11 @@ static void device_set_realized(Object *obj, bool valu= e, Error **errp) } =20 if (value && !dev->realized) { + ret =3D check_migratable(obj, &local_err); + if (ret < 0) { + goto fail; + } + if (!obj->parent) { gchar *name =3D g_strdup_printf("device[%d]", unattached_count= ++); =20 diff --git a/hw/usb/bus.c b/hw/usb/bus.c index efe4b8e..24f1608 100644 --- a/hw/usb/bus.c +++ b/hw/usb/bus.c @@ -8,7 +8,6 @@ #include "monitor/monitor.h" #include "trace.h" #include "qemu/cutils.h" -#include "migration/migration.h" =20 static void usb_bus_dev_print(Monitor *mon, DeviceState *qdev, int indent); =20 @@ -688,8 +687,6 @@ USBDevice *usbdevice_create(const char *cmdline) const char *params; int len; USBDevice *dev; - ObjectClass *klass; - DeviceClass *dc; =20 params =3D strchr(cmdline,':'); if (params) { @@ -724,22 +721,6 @@ USBDevice *usbdevice_create(const char *cmdline) return NULL; } =20 - klass =3D object_class_by_name(f->name); - if (klass =3D=3D NULL) { - error_report("Device '%s' not found", f->name); - return NULL; - } - - dc =3D DEVICE_CLASS(klass); - - if (only_migratable) { - if (dc->vmsd->unmigratable) { - error_report("Device %s is not migratable, but --only-migratab= le " - "was specified", f->name); - return NULL; - } - } - if (f->usbdevice_init) { dev =3D f->usbdevice_init(bus, params); } else { diff --git a/include/migration/migration.h b/include/migration/migration.h index 1735d66..34a383e 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -22,6 +22,7 @@ #include "qapi-types.h" #include "exec/cpu-common.h" #include "qemu/coroutine_int.h" +#include "qom/object.h" =20 #define QEMU_VM_FILE_MAGIC 0x5145564d #define QEMU_VM_FILE_VERSION_COMPAT 0x00000002 @@ -313,6 +314,8 @@ int migrate_add_blocker(Error *reason, Error **errp); */ void migrate_del_blocker(Error *reason); =20 +int check_migratable(Object *obj, Error **err); + bool migrate_release_ram(void); bool migrate_postcopy_ram(void); bool migrate_zero_blocks(void); diff --git a/migration/migration.c b/migration/migration.c index 4fab538..481c540 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1150,6 +1150,21 @@ void migrate_del_blocker(Error *reason) migration_blockers =3D g_slist_remove(migration_blockers, reason); } =20 +int check_migratable(Object *obj, Error **err) +{ + DeviceClass *dc =3D DEVICE_GET_CLASS(obj); + if (only_migratable && dc->vmsd) { + if (dc->vmsd->unmigratable) { + error_setg(err, "Device %s is not migratable, but " + "--only-migratable was specified", + object_get_typename(obj)); + return -1; + } + } + + return 0; +} + void qmp_migrate_incoming(const char *uri, Error **errp) { Error *local_err =3D NULL; diff --git a/qdev-monitor.c b/qdev-monitor.c index 549f45f..5f2fcdf 100644 --- a/qdev-monitor.c +++ b/qdev-monitor.c @@ -29,7 +29,6 @@ #include "qemu/error-report.h" #include "qemu/help_option.h" #include "sysemu/block-backend.h" -#include "migration/migration.h" =20 /* * Aliases were a bad idea from the start. Let's keep them @@ -579,14 +578,6 @@ DeviceState *qdev_device_add(QemuOpts *opts, Error **e= rrp) return NULL; } =20 - if (only_migratable) { - if (dc->vmsd->unmigratable) { - error_setg(errp, "Device %s is not migratable, but " - "--only-migratable was specified", driver); - return NULL; - } - } - /* find bus */ path =3D qemu_opt_get(opts, "bus"); if (path !=3D NULL) { diff --git a/stubs/vmstate.c b/stubs/vmstate.c index bbe158f..6d52f29 100644 --- a/stubs/vmstate.c +++ b/stubs/vmstate.c @@ -1,6 +1,7 @@ #include "qemu/osdep.h" #include "qemu-common.h" #include "migration/vmstate.h" +#include "migration/migration.h" =20 const VMStateDescription vmstate_dummy =3D {}; =20 @@ -19,3 +20,8 @@ void vmstate_unregister(DeviceState *dev, void *opaque) { } + +int check_migratable(Object *obj, Error **err) +{ + return 0; +} --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 148828632005013.61959541904821; Tue, 28 Feb 2017 04:52:00 -0800 (PST) Received: from localhost ([::1]:60759 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihG2-0000mN-Px for importer@patchew.org; Tue, 28 Feb 2017 07:51:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40015) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5j-000891-7n for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5i-0000Hk-Ay for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:19 -0500 Received: from mx1.redhat.com ([209.132.183.28]:49568) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5i-0000HU-5X for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:18 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3B36CC05681A; Tue, 28 Feb 2017 12:41:18 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewt9019858; Tue, 28 Feb 2017 07:41:16 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:38 +0000 Message-Id: <20170228124056.5074-10-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 28 Feb 2017 12:41:18 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 09/27] migration: fix id leak regression X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Marc-Andr=C3=A9 Lureau This leak was introduced in commit 581f08bac22bdd5e081ae07f68071a0fc3c5c2c7. (it stands out quickly with ASAN once the rest of the leaks are also removed from make check with this series) Cc: Dr. David Alan Gilbert Cc: Juan Quintela Signed-off-by: Marc-Andr=C3=A9 Lureau Message-Id: <20170221141451.28305-31-marcandre.lureau@redhat.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 1 + 1 file changed, 1 insertion(+) diff --git a/migration/savevm.c b/migration/savevm.c index 5ecd264..87c7a00 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -688,6 +688,7 @@ int vmstate_register_with_alias_id(DeviceState *dev, in= t instance_id, =20 return -1; } + g_free(id); =20 se->compat =3D g_new0(CompatEntry, 1); pstrcpy(se->compat->idstr, sizeof(se->compat->idstr), vmsd->na= me); --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286061587160.8881821241248; Tue, 28 Feb 2017 04:47:41 -0800 (PST) Received: from localhost ([::1]:60738 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihBs-0005SI-9K for importer@patchew.org; Tue, 28 Feb 2017 07:47:40 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40035) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5m-0008Bd-FC for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5k-0000IB-GK for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:22 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59376) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5k-0000Hz-7n for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:20 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5677EC04B328; Tue, 28 Feb 2017 12:41:20 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtA019858; Tue, 28 Feb 2017 07:41:18 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:39 +0000 Message-Id: <20170228124056.5074-11-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 28 Feb 2017 12:41:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 10/27] migration: Update docs to discourage version bumps X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Version bumps break backwards migration; update the docs to explain to people that's bad and how to avoid it. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Markus Armbruster Message-Id: <20170210110359.8210-1-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- docs/migration.txt | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 58 insertions(+) diff --git a/docs/migration.txt b/docs/migration.txt index 6503c17..c8b0304 100644 --- a/docs/migration.txt +++ b/docs/migration.txt @@ -161,6 +161,11 @@ include/hw/hw.h. =20 =3D=3D=3D More about versions =3D=3D=3D =20 +Version numbers are intended for major incompatible changes to the +migration of a device, and using them breaks backwards-migration +compatibility; in general most changes can be made by adding Subsections +(see below) or _TEST macros (see below) which won't break compatibility. + You can see that there are several version fields: =20 - version_id: the maximum version_id supported by VMState for that device. @@ -175,6 +180,9 @@ version_id. And the function load_state_old() (if pres= ent) is able to load state from minimum_version_id_old to minimum_version_id. This function is deprecated and will be removed when no more users are left. =20 +Saving state will always create a section with the 'version_id' value +and thus can't be loaded by any older QEMU. + =3D=3D=3D Massaging functions =3D=3D=3D =20 Sometimes, it is not enough to be able to save the state directly @@ -292,6 +300,56 @@ save/send this state when we are in the middle of a pi= o operation not enabled, the values on that fields are garbage and don't need to be sent. =20 +Using a condition function that checks a 'property' to determine whether +to send a subsection allows backwards migration compatibility when +new subsections are added. + +For example; + a) Add a new property using DEFINE_PROP_BOOL - e.g. support-foo and + default it to true. + b) Add an entry to the HW_COMPAT_ for the previous version + that sets the property to false. + c) Add a static bool support_foo function that tests the property. + d) Add a subsection with a .needed set to the support_foo function + e) (potentially) Add a pre_load that sets up a default value for 'foo' + to be used if the subsection isn't loaded. + +Now that subsection will not be generated when using an older +machine type and the migration stream will be accepted by older +QEMU versions. pre-load functions can be used to initialise state +on the newer version so that they default to suitable values +when loading streams created by older QEMU versions that do not +generate the subsection. + +In some cases subsections are added for data that had been accidentally +omitted by earlier versions; if the missing data causes the migration +process to succeed but the guest to behave badly then it may be better +to send the subsection and cause the migration to explicitly fail +with the unknown subsection error. If the bad behaviour only happens +with certain data values, making the subsection conditional on +the data value (rather than the machine type) allows migrations to succeed +in most cases. In general the preference is to tie the subsection to +the machine type, and allow reliable migrations, unless the behaviour +from omission of the subsection is really bad. + +=3D Not sending existing elements =3D + +Sometimes members of the VMState are no longer needed; + removing them will break migration compatibility + making them version dependent and bumping the version will break backwar= ds + migration compatibility. + +The best way is to: + a) Add a new property/compatibility/function in the same way for subsect= ions + above. + b) replace the VMSTATE macro with the _TEST version of the macro, e.g.: + VMSTATE_UINT32(foo, barstruct) + becomes + VMSTATE_UINT32_TEST(foo, barstruct, pre_version_baz) + + Sometime in the future when we no longer care about the ancient +versions these can be killed off. + =3D Return path =3D =20 In most migration scenarios there is only a single data path that runs --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286593119140.49119861258475; Tue, 28 Feb 2017 04:56:33 -0800 (PST) Received: from localhost ([::1]:60783 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihKR-00056B-K7 for importer@patchew.org; Tue, 28 Feb 2017 07:56:31 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40045) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5n-0008CN-9q for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5m-0000J2-BS for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:23 -0500 Received: from mx1.redhat.com ([209.132.183.28]:54862) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5m-0000Ir-6d for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:22 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4E2FB85363; Tue, 28 Feb 2017 12:41:22 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtB019858; Tue, 28 Feb 2017 07:41:20 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:40 +0000 Message-Id: <20170228124056.5074-12-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 28 Feb 2017 12:41:22 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 11/27] migration: fix use-after-free of to_dst_file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy hmp_savevm calls qemu_savevm_state(f), which sets to_dst_file=3Df in global migration state. Then hmp_savevm closes f (g_free called). Next access to to_dst_file in migration state (for example, qmp_migrate_set_speed) will use it after it was freed. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Dr. David Alan Gilbert Message-Id: <20170225193155.447462-5-vsementsov@virtuozzo.com> Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index 87c7a00..26d2c44 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1277,6 +1277,11 @@ done: status =3D MIGRATION_STATUS_COMPLETED; } migrate_set_state(&ms->state, MIGRATION_STATUS_SETUP, status); + + /* f is outer parameter, it should not stay in global migration state = after + * this function finished */ + ms->to_dst_file =3D NULL; + return ret; } =20 --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488287134552873.4586561952515; Tue, 28 Feb 2017 05:05:34 -0800 (PST) Received: from localhost ([::1]:60824 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihT9-00045a-25 for importer@patchew.org; Tue, 28 Feb 2017 08:05:31 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40070) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5p-0008ES-Lg for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5o-0000Jd-H3 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:25 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51540) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5o-0000JP-7k for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:24 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 58BA13B709; Tue, 28 Feb 2017 12:41:24 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtC019858; Tue, 28 Feb 2017 07:41:22 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:41 +0000 Message-Id: <20170228124056.5074-13-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 28 Feb 2017 12:41:24 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 12/27] postcopy: Transmit ram size summary word X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Replace the host page-size in the 'advise' command by a pagesize summary bitmap; if the VM is just using normal RAM then this will be exactly the same as before, however if they're using huge pages they'll be different, and thus: a) Migration from/to old qemu's that don't understand huge pages will fail early. b) Migrations with different size RAMBlocks will also fail early. This catches it very early; earlier than the detailed per-block check in the next patch. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Reviewed-by: Juan Quintela Message-Id: <20170224182844.32452-2-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 1 + migration/ram.c | 17 +++++++++++++++++ migration/savevm.c | 32 +++++++++++++++++++++----------- 3 files changed, 39 insertions(+), 11 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index 34a383e..6272adf 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -378,6 +378,7 @@ void global_state_store_running(void); void flush_page_queue(MigrationState *ms); int ram_save_queue_pages(MigrationState *ms, const char *rbname, ram_addr_t start, ram_addr_t len); +uint64_t ram_pagesize_summary(void); =20 PostcopyState postcopy_state_get(void); /* Set the state and return the old state */ diff --git a/migration/ram.c b/migration/ram.c index f289fcd..1f6397a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -600,6 +600,23 @@ static void migration_bitmap_sync_init(void) iterations_prev =3D 0; } =20 +/* Returns a summary bitmap of the page sizes of all RAMBlocks; + * for VMs with just normal pages this is equivalent to the + * host page size. If it's got some huge pages then it's the OR + * of all the different page sizes. + */ +uint64_t ram_pagesize_summary(void) +{ + RAMBlock *block; + uint64_t summary =3D 0; + + QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + summary |=3D block->page_size; + } + + return summary; +} + static void migration_bitmap_sync(void) { RAMBlock *block; diff --git a/migration/savevm.c b/migration/savevm.c index 26d2c44..3b19a4a 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -870,7 +870,7 @@ int qemu_savevm_send_packaged(QEMUFile *f, const uint8_= t *buf, size_t len) void qemu_savevm_send_postcopy_advise(QEMUFile *f) { uint64_t tmp[2]; - tmp[0] =3D cpu_to_be64(getpagesize()); + tmp[0] =3D cpu_to_be64(ram_pagesize_summary()); tmp[1] =3D cpu_to_be64(1ul << qemu_target_page_bits()); =20 trace_qemu_savevm_send_postcopy_advise(); @@ -1352,7 +1352,7 @@ static int qemu_loadvm_state_main(QEMUFile *f, Migrat= ionIncomingState *mis); static int loadvm_postcopy_handle_advise(MigrationIncomingState *mis) { PostcopyState ps =3D postcopy_state_set(POSTCOPY_INCOMING_ADVISE); - uint64_t remote_hps, remote_tps; + uint64_t remote_pagesize_summary, local_pagesize_summary, remote_tps; =20 trace_loadvm_postcopy_handle_advise(); if (ps !=3D POSTCOPY_INCOMING_NONE) { @@ -1365,17 +1365,27 @@ static int loadvm_postcopy_handle_advise(MigrationI= ncomingState *mis) return -1; } =20 - remote_hps =3D qemu_get_be64(mis->from_src_file); - if (remote_hps !=3D getpagesize()) { + remote_pagesize_summary =3D qemu_get_be64(mis->from_src_file); + local_pagesize_summary =3D ram_pagesize_summary(); + + if (remote_pagesize_summary !=3D local_pagesize_summary) { /* - * Some combinations of mismatch are probably possible but it gets - * a bit more complicated. In particular we need to place whole - * host pages on the dest at once, and we need to ensure that we - * handle dirtying to make sure we never end up sending part of - * a hostpage on it's own. + * This detects two potential causes of mismatch: + * a) A mismatch in host page sizes + * Some combinations of mismatch are probably possible but it= gets + * a bit more complicated. In particular we need to place wh= ole + * host pages on the dest at once, and we need to ensure that= we + * handle dirtying to make sure we never end up sending part = of + * a hostpage on it's own. + * b) The use of different huge page sizes on source/destination + * a more fine grain test is performed during RAM block migra= tion + * but this test here causes a nice early clear failure, and + * also fails when passed to an older qemu that doesn't + * do huge pages. */ - error_report("Postcopy needs matching host page sizes (s=3D%d d=3D= %d)", - (int)remote_hps, getpagesize()); + error_report("Postcopy needs matching RAM page sizes (s=3D%" PRIx64 + " d=3D%" PRIx= 64 ")", + remote_pagesize_summary, local_pagesize_summary); return -1; } =20 --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286845114257.8770943059658; Tue, 28 Feb 2017 05:00:45 -0800 (PST) Received: from localhost ([::1]:60804 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihOU-0000JQ-K1 for importer@patchew.org; Tue, 28 Feb 2017 08:00:42 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40092) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5r-0008FS-No for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5q-0000K3-CH for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:27 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57726) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5q-0000Jv-6N for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:26 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4D5214E4E6; Tue, 28 Feb 2017 12:41:26 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtD019858; Tue, 28 Feb 2017 07:41:24 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:42 +0000 Message-Id: <20170228124056.5074-14-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 28 Feb 2017 12:41:26 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 13/27] postcopy: Transmit and compare individual page sizes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" When using postcopy with hugepages, we require the source and destination page sizes for any RAMBlock to match; note that different RAMBlocks in the same VM can have different page sizes. Transmit them as part of the RAM information header and fail if there's a difference. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Reviewed-by: Juan Quintela Message-Id: <20170224182844.32452-3-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 1f6397a..fbd987a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2027,6 +2027,9 @@ static int ram_save_setup(QEMUFile *f, void *opaque) qemu_put_byte(f, strlen(block->idstr)); qemu_put_buffer(f, (uint8_t *)block->idstr, strlen(block->idstr)); qemu_put_be64(f, block->used_length); + if (migrate_postcopy_ram() && block->page_size !=3D qemu_host_page= _size) { + qemu_put_be64(f, block->page_size); + } } =20 rcu_read_unlock(); @@ -2528,6 +2531,8 @@ static int ram_load(QEMUFile *f, void *opaque, int ve= rsion_id) * be atomic */ bool postcopy_running =3D postcopy_state_get() >=3D POSTCOPY_INCOMING_= LISTENING; + /* ADVISE is earlier, it shows the source has the postcopy capability = on */ + bool postcopy_advised =3D postcopy_state_get() >=3D POSTCOPY_INCOMING_= ADVISE; =20 seq_iter++; =20 @@ -2592,6 +2597,18 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) error_report_err(local_err); } } + /* For postcopy we need to check hugepage sizes match = */ + if (postcopy_advised && + block->page_size !=3D qemu_host_page_size) { + uint64_t remote_page_size =3D qemu_get_be64(f); + if (remote_page_size !=3D block->page_size) { + error_report("Mismatched RAM page size %s " + "(local) %zd !=3D %" PRId64, + id, block->page_size, + remote_page_size); + ret =3D -EINVAL; + } + } ram_control_load_hook(f, RAM_CONTROL_BLOCK_REG, block->idstr); } else { --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286339497620.0444498811239; Tue, 28 Feb 2017 04:52:19 -0800 (PST) Received: from localhost ([::1]:60761 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihGM-000107-AX for importer@patchew.org; Tue, 28 Feb 2017 07:52:18 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40115) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5v-0008It-Gi for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5s-0000KN-E6 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:31 -0500 Received: from mx1.redhat.com ([209.132.183.28]:42020) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5s-0000KE-7f for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:28 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4E3853A768D; Tue, 28 Feb 2017 12:41:28 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtE019858; Tue, 28 Feb 2017 07:41:26 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:43 +0000 Message-Id: <20170228124056.5074-15-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 28 Feb 2017 12:41:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 14/27] postcopy: Chunk discards for hugepages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" At the start of the postcopy phase, partially sent huge pages must be discarded. The code for dealing with host page sizes larger than the target page size can be reused for this case. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-4-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index fbd987a..7c5cdf9 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1672,12 +1672,17 @@ static void postcopy_chunk_hostpages_pass(Migration= State *ms, bool unsent_pass, { unsigned long *bitmap; unsigned long *unsentmap; - unsigned int host_ratio =3D qemu_host_page_size / TARGET_PAGE_SIZE; + unsigned int host_ratio =3D block->page_size / TARGET_PAGE_SIZE; unsigned long first =3D block->offset >> TARGET_PAGE_BITS; unsigned long len =3D block->used_length >> TARGET_PAGE_BITS; unsigned long last =3D first + (len - 1); unsigned long run_start; =20 + if (block->page_size =3D=3D TARGET_PAGE_SIZE) { + /* Easy case - TPS=3D=3DHPS for a non-huge page RAMBlock */ + return; + } + bitmap =3D atomic_rcu_read(&migration_bitmap_rcu)->bmap; unsentmap =3D atomic_rcu_read(&migration_bitmap_rcu)->unsentmap; =20 @@ -1781,7 +1786,8 @@ static void postcopy_chunk_hostpages_pass(MigrationSt= ate *ms, bool unsent_pass, * Utility for the outgoing postcopy code. * * Discard any partially sent host-page size chunks, mark any partially - * dirty host-page size chunks as all dirty. + * dirty host-page size chunks as all dirty. In this case the host-page + * is the host-page for the particular RAMBlock, i.e. it might be a huge p= age * * Returns: 0 on success */ @@ -1789,11 +1795,6 @@ static int postcopy_chunk_hostpages(MigrationState *= ms) { struct RAMBlock *block; =20 - if (qemu_host_page_size =3D=3D TARGET_PAGE_SIZE) { - /* Easy case - TPS=3D=3DHPS - nothing to be done */ - return 0; - } - /* Easiest way to make sure we don't resume in the middle of a host-pa= ge */ last_seen_block =3D NULL; last_sent_block =3D NULL; @@ -1849,7 +1850,7 @@ int ram_postcopy_send_discard_bitmap(MigrationState *= ms) return -EINVAL; } =20 - /* Deal with TPS !=3D HPS */ + /* Deal with TPS !=3D HPS and huge pages */ ret =3D postcopy_chunk_hostpages(ms); if (ret) { rcu_read_unlock(); --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286234253977.1045431637925; Tue, 28 Feb 2017 04:50:34 -0800 (PST) Received: from localhost ([::1]:60749 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihEe-00080d-R2 for importer@patchew.org; Tue, 28 Feb 2017 07:50:32 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40119) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5v-0008JF-UI for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5u-0000Ki-HZ for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:31 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59420) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5u-0000Kd-9O for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:30 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 52E50C04B943; Tue, 28 Feb 2017 12:41:30 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtF019858; Tue, 28 Feb 2017 07:41:28 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:44 +0000 Message-Id: <20170228124056.5074-16-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 28 Feb 2017 12:41:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 15/27] exec: ram_block_discard_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Create ram_block_discard_range in exec.c to replace postcopy_ram_discard_range and most of ram_discard_range. Those two routines are a bit of a weird combination, and ram_discard_range is about to get more complex for hugepages. It's OS dependent code (so shouldn't be in migration/ram.c) but it needs quite a bit of the innards of RAMBlock so doesn't belong in the os*.c. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-5-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- exec.c | 54 ++++++++++++++++++++++++++++++++++++= ++++ include/exec/cpu-common.h | 1 + include/migration/postcopy-ram.h | 7 ------ migration/postcopy-ram.c | 30 +--------------------- migration/ram.c | 24 +++--------------- migration/trace-events | 2 +- 6 files changed, 60 insertions(+), 58 deletions(-) diff --git a/exec.c b/exec.c index 3adf2b1..8ac556a 100644 --- a/exec.c +++ b/exec.c @@ -45,6 +45,7 @@ #include "exec/address-spaces.h" #include "sysemu/xen-mapcache.h" #include "trace-root.h" + #endif #include "exec/cpu-all.h" #include "qemu/rcu_queue.h" @@ -3294,4 +3295,57 @@ int qemu_ram_foreach_block(RAMBlockIterFunc func, vo= id *opaque) rcu_read_unlock(); return ret; } + +/* + * Unmap pages of memory from start to start+length such that + * they a) read as 0, b) Trigger whatever fault mechanism + * the OS provides for postcopy. + * The pages must be unmapped by the end of the function. + * Returns: 0 on success, none-0 on failure + * + */ +int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length) +{ + int ret =3D -1; + + uint8_t *host_startaddr =3D rb->host + start; + + if ((uintptr_t)host_startaddr & (rb->page_size - 1)) { + error_report("ram_block_discard_range: Unaligned start address: %p= ", + host_startaddr); + goto err; + } + + if ((start + length) <=3D rb->used_length) { + uint8_t *host_endaddr =3D host_startaddr + length; + if ((uintptr_t)host_endaddr & (rb->page_size - 1)) { + error_report("ram_block_discard_range: Unaligned end address: = %p", + host_endaddr); + goto err; + } + + errno =3D ENOTSUP; /* If we are missing MADVISE etc */ + +#if defined(CONFIG_MADVISE) + /* Note: We need the madvise MADV_DONTNEED behaviour of definitely + * freeing the page. + */ + ret =3D madvise(host_startaddr, length, MADV_DONTNEED); +#endif + if (ret) { + ret =3D -errno; + error_report("ram_block_discard_range: Failed to discard range= " + "%s:%" PRIx64 " +%zx (%d)", + rb->idstr, start, length, ret); + } + } else { + error_report("ram_block_discard_range: Overrun block '%s' (%" PRIu= 64 + "/%zx/" RAM_ADDR_FMT")", + rb->idstr, start, length, rb->used_length); + } + +err: + return ret; +} + #endif diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index bd15853..1350c2e 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -105,6 +105,7 @@ typedef int (RAMBlockIterFunc)(const char *block_name, = void *host_addr, ram_addr_t offset, ram_addr_t length, void *opaque); =20 int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque); +int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length); =20 #endif =20 diff --git a/include/migration/postcopy-ram.h b/include/migration/postcopy-= ram.h index b6a7491f..43bbbca 100644 --- a/include/migration/postcopy-ram.h +++ b/include/migration/postcopy-ram.h @@ -35,13 +35,6 @@ int postcopy_ram_incoming_init(MigrationIncomingState *m= is, size_t ram_pages); int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis); =20 /* - * Discard the contents of 'length' bytes from 'start' - * We can assume that if we've been called postcopy_ram_hosttest returned = true - */ -int postcopy_ram_discard_range(MigrationIncomingState *mis, uint8_t *start, - size_t length); - -/* * Userfault requires us to mark RAM as NOHUGEPAGE prior to discard * however leaving it until after precopy means that most of the precopy * data is still THPd diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index a40dddb..1e3d22f 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -200,27 +200,6 @@ out: return ret; } =20 -/** - * postcopy_ram_discard_range: Discard a range of memory. - * We can assume that if we've been called postcopy_ram_hosttest returned = true. - * - * @mis: Current incoming migration state. - * @start, @length: range of memory to discard. - * - * returns: 0 on success. - */ -int postcopy_ram_discard_range(MigrationIncomingState *mis, uint8_t *start, - size_t length) -{ - trace_postcopy_ram_discard_range(start, length); - if (madvise(start, length, MADV_DONTNEED)) { - error_report("%s MADV_DONTNEED: %s", __func__, strerror(errno)); - return -1; - } - - return 0; -} - /* * Setup an area of RAM so that it *can* be used for postcopy later; this * must be done right at the start prior to pre-copy. @@ -239,7 +218,7 @@ static int init_range(const char *block_name, void *hos= t_addr, * - we're going to get the copy from the source anyway. * (Precopy will just overwrite this data, so doesn't need the discard) */ - if (postcopy_ram_discard_range(mis, host_addr, length)) { + if (ram_discard_range(mis, block_name, 0, length)) { return -1; } =20 @@ -658,13 +637,6 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSta= te *mis) return -1; } =20 -int postcopy_ram_discard_range(MigrationIncomingState *mis, uint8_t *start, - size_t length) -{ - assert(0); - return -1; -} - int postcopy_ram_prepare_discard(MigrationIncomingState *mis) { assert(0); diff --git a/migration/ram.c b/migration/ram.c index 7c5cdf9..44fe960 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1890,6 +1890,8 @@ int ram_discard_range(MigrationIncomingState *mis, { int ret =3D -1; =20 + trace_ram_discard_range(block_name, start, length); + rcu_read_lock(); RAMBlock *rb =3D qemu_ram_block_by_name(block_name); =20 @@ -1899,27 +1901,7 @@ int ram_discard_range(MigrationIncomingState *mis, goto err; } =20 - uint8_t *host_startaddr =3D rb->host + start; - - if ((uintptr_t)host_startaddr & (qemu_host_page_size - 1)) { - error_report("ram_discard_range: Unaligned start address: %p", - host_startaddr); - goto err; - } - - if ((start + length) <=3D rb->used_length) { - uint8_t *host_endaddr =3D host_startaddr + length; - if ((uintptr_t)host_endaddr & (qemu_host_page_size - 1)) { - error_report("ram_discard_range: Unaligned end address: %p", - host_endaddr); - goto err; - } - ret =3D postcopy_ram_discard_range(mis, host_startaddr, length); - } else { - error_report("ram_discard_range: Overrun block '%s' (%" PRIu64 - "/%zx/" RAM_ADDR_FMT")", - block_name, start, length, rb->used_length); - } + ret =3D ram_block_discard_range(rb, start, length); =20 err: rcu_read_unlock(); diff --git a/migration/trace-events b/migration/trace-events index fa660e3..7372ce2 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -68,6 +68,7 @@ get_queued_page_not_dirty(const char *block_name, uint64_= t tmp_offset, uint64_t migration_bitmap_sync_start(void) "" migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 migration_throttle(void) "" +ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: sta= rt: %" PRIx64 " %zx" ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x" ram_postcopy_send_discard_bitmap(void) "" ram_save_queue_pages(const char *rbname, size_t start, size_t len) "%s: st= art: %zx len: %zx" @@ -176,7 +177,6 @@ rdma_start_outgoing_migration_after_rdma_source_init(vo= id) "" # migration/postcopy-ram.c postcopy_discard_send_finish(const char *ramblock, int nwords, int ncmds) = "%s mask words sent=3D%d in %d commands" postcopy_discard_send_range(const char *ramblock, unsigned long start, uns= igned long length) "%s:%lx/%lx" -postcopy_ram_discard_range(void *start, size_t length) "%p,+%zx" postcopy_cleanup_range(const char *ramblock, void *host_addr, size_t offse= t, size_t length) "%s: %p offset=3D%zx length=3D%zx" postcopy_init_range(const char *ramblock, void *host_addr, size_t offset, = size_t length) "%s: %p offset=3D%zx length=3D%zx" postcopy_nhp_range(const char *ramblock, void *host_addr, size_t offset, s= ize_t length) "%s: %p offset=3D%zx length=3D%zx" --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286606473879.6080495453206; Tue, 28 Feb 2017 04:56:46 -0800 (PST) Received: from localhost ([::1]:60785 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihKf-0005ME-3v for importer@patchew.org; Tue, 28 Feb 2017 07:56:45 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40135) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5x-0008Kr-CJ for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5w-0000L3-CI for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:33 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51582) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5w-0000Ko-5s for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:32 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3C0873B71F; Tue, 28 Feb 2017 12:41:32 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtG019858; Tue, 28 Feb 2017 07:41:30 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:45 +0000 Message-Id: <20170228124056.5074-17-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 28 Feb 2017 12:41:32 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 16/27] postcopy: enhance ram_block_discard_range for hugepages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Unfortunately madvise DONTNEED doesn't work on hugepagetlb so use fallocate(FALLOC_FL_PUNCH_HOLE) qemu_fd_getpagesize only sets the page based off a file if the file is from hugetlbfs. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-6-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- exec.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/exec.c b/exec.c index 8ac556a..24cdf64 100644 --- a/exec.c +++ b/exec.c @@ -46,6 +46,11 @@ #include "sysemu/xen-mapcache.h" #include "trace-root.h" =20 +#ifdef CONFIG_FALLOCATE_PUNCH_HOLE +#include +#include +#endif + #endif #include "exec/cpu-all.h" #include "qemu/rcu_queue.h" @@ -3326,12 +3331,23 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t = start, size_t length) =20 errno =3D ENOTSUP; /* If we are missing MADVISE etc */ =20 + if (rb->page_size =3D=3D qemu_host_page_size) { #if defined(CONFIG_MADVISE) - /* Note: We need the madvise MADV_DONTNEED behaviour of definitely - * freeing the page. - */ - ret =3D madvise(host_startaddr, length, MADV_DONTNEED); + /* Note: We need the madvise MADV_DONTNEED behaviour of defini= tely + * freeing the page. + */ + ret =3D madvise(host_startaddr, length, MADV_DONTNEED); #endif + } else { + /* Huge page case - unfortunately it can't do DONTNEED, but + * it can do the equivalent by FALLOC_FL_PUNCH_HOLE in the + * huge page file. + */ +#ifdef CONFIG_FALLOCATE_PUNCH_HOLE + ret =3D fallocate(rb->fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEE= P_SIZE, + start, length); +#endif + } if (ret) { ret =3D -errno; error_report("ram_block_discard_range: Failed to discard range= " --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286861482652.7950300389297; Tue, 28 Feb 2017 05:01:01 -0800 (PST) Received: from localhost ([::1]:60806 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihOk-0000X3-65 for importer@patchew.org; Tue, 28 Feb 2017 08:00:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40147) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih5z-0008O9-98 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih5y-0000LJ-9U for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:35 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43040) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih5y-0000LB-3X for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:34 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3023380461; Tue, 28 Feb 2017 12:41:34 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtH019858; Tue, 28 Feb 2017 07:41:32 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:46 +0000 Message-Id: <20170228124056.5074-18-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 28 Feb 2017 12:41:34 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 17/27] postcopy: Record largest page size X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Record the largest page size in use; we'll need it soon for allocating temporary buffers. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-7-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- exec.c | 13 +++++++++++++ include/exec/cpu-common.h | 1 + include/migration/migration.h | 1 + migration/migration.c | 1 + 4 files changed, 16 insertions(+) diff --git a/exec.c b/exec.c index 24cdf64..785d20f 100644 --- a/exec.c +++ b/exec.c @@ -1524,6 +1524,19 @@ size_t qemu_ram_pagesize(RAMBlock *rb) return rb->page_size; } =20 +/* Returns the largest size of page in use */ +size_t qemu_ram_pagesize_largest(void) +{ + RAMBlock *block; + size_t largest =3D 0; + + QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + largest =3D MAX(largest, qemu_ram_pagesize(block)); + } + + return largest; +} + static int memory_try_enable_merging(void *addr, size_t len) { if (!machine_mem_merge(current_machine)) { diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index 1350c2e..8c305aa 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -64,6 +64,7 @@ void qemu_ram_set_idstr(RAMBlock *block, const char *name= , DeviceState *dev); void qemu_ram_unset_idstr(RAMBlock *block); const char *qemu_ram_get_idstr(RAMBlock *rb); size_t qemu_ram_pagesize(RAMBlock *block); +size_t qemu_ram_pagesize_largest(void); =20 void cpu_physical_memory_rw(hwaddr addr, uint8_t *buf, int len, int is_write); diff --git a/include/migration/migration.h b/include/migration/migration.h index 6272adf..08b98e9 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -93,6 +93,7 @@ struct MigrationIncomingState { */ QemuEvent main_thread_load_event; =20 + size_t largest_page_size; bool have_fault_thread; QemuThread fault_thread; QemuSemaphore fault_thread_sem; diff --git a/migration/migration.c b/migration/migration.c index 481c540..3dab684 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -387,6 +387,7 @@ static void process_incoming_migration_co(void *opaque) int ret; =20 mis->from_src_file =3D f; + mis->largest_page_size =3D qemu_ram_pagesize_largest(); postcopy_state_set(POSTCOPY_INCOMING_NONE); migrate_set_state(&mis->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_ACTIVE); --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286573082592.5639736994325; Tue, 28 Feb 2017 04:56:13 -0800 (PST) Received: from localhost ([::1]:60782 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihK4-0004fo-Nh for importer@patchew.org; Tue, 28 Feb 2017 07:56:08 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40158) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih61-0008ST-VE for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih60-0000NN-Dj for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:38 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43058) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih60-0000Lm-48 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:36 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3660180463; Tue, 28 Feb 2017 12:41:36 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtI019858; Tue, 28 Feb 2017 07:41:34 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:47 +0000 Message-Id: <20170228124056.5074-19-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 28 Feb 2017 12:41:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 18/27] postcopy: Plumb pagesize down into place helpers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Now we deal with normal size pages and huge pages we need to tell the place handlers the size we're dealing with and make sure the temporary page is large enough. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-8-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- include/migration/postcopy-ram.h | 6 +++-- migration/postcopy-ram.c | 47 ++++++++++++++++++++++++------------= ---- migration/ram.c | 15 +++++++------ 3 files changed, 40 insertions(+), 28 deletions(-) diff --git a/include/migration/postcopy-ram.h b/include/migration/postcopy-= ram.h index 43bbbca..8e036b9 100644 --- a/include/migration/postcopy-ram.h +++ b/include/migration/postcopy-ram.h @@ -74,13 +74,15 @@ void postcopy_discard_send_finish(MigrationState *ms, * to use other postcopy_ routines to allocate. * returns 0 on success */ -int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m); +int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m, + size_t pagesize); =20 /* * Place a zero page at (host) atomically * returns 0 on success */ -int postcopy_place_page_zero(MigrationIncomingState *mis, void *host); +int postcopy_place_page_zero(MigrationIncomingState *mis, void *host, + size_t pagesize); =20 /* * Allocate a page of memory that can be mapped at a later point in time diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 1e3d22f..a8b7fed 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -321,7 +321,7 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingStat= e *mis) migrate_send_rp_shut(mis, qemu_file_get_error(mis->from_src_file) !=3D= 0); =20 if (mis->postcopy_tmp_page) { - munmap(mis->postcopy_tmp_page, getpagesize()); + munmap(mis->postcopy_tmp_page, mis->largest_page_size); mis->postcopy_tmp_page =3D NULL; } trace_postcopy_ram_incoming_cleanup_exit(); @@ -543,13 +543,14 @@ int postcopy_ram_enable_notify(MigrationIncomingState= *mis) * Place a host page (from) at (host) atomically * returns 0 on success */ -int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m) +int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m, + size_t pagesize) { struct uffdio_copy copy_struct; =20 copy_struct.dst =3D (uint64_t)(uintptr_t)host; copy_struct.src =3D (uint64_t)(uintptr_t)from; - copy_struct.len =3D getpagesize(); + copy_struct.len =3D pagesize; copy_struct.mode =3D 0; =20 /* copy also acks to the kernel waking the stalled thread up @@ -559,8 +560,8 @@ int postcopy_place_page(MigrationIncomingState *mis, vo= id *host, void *from) */ if (ioctl(mis->userfault_fd, UFFDIO_COPY, ©_struct)) { int e =3D errno; - error_report("%s: %s copy host: %p from: %p", - __func__, strerror(e), host, from); + error_report("%s: %s copy host: %p from: %p (size: %zd)", + __func__, strerror(e), host, from, pagesize); =20 return -e; } @@ -573,23 +574,29 @@ int postcopy_place_page(MigrationIncomingState *mis, = void *host, void *from) * Place a zero page at (host) atomically * returns 0 on success */ -int postcopy_place_page_zero(MigrationIncomingState *mis, void *host) +int postcopy_place_page_zero(MigrationIncomingState *mis, void *host, + size_t pagesize) { - struct uffdio_zeropage zero_struct; + trace_postcopy_place_page_zero(host); =20 - zero_struct.range.start =3D (uint64_t)(uintptr_t)host; - zero_struct.range.len =3D getpagesize(); - zero_struct.mode =3D 0; + if (pagesize =3D=3D getpagesize()) { + struct uffdio_zeropage zero_struct; + zero_struct.range.start =3D (uint64_t)(uintptr_t)host; + zero_struct.range.len =3D getpagesize(); + zero_struct.mode =3D 0; =20 - if (ioctl(mis->userfault_fd, UFFDIO_ZEROPAGE, &zero_struct)) { - int e =3D errno; - error_report("%s: %s zero host: %p", - __func__, strerror(e), host); + if (ioctl(mis->userfault_fd, UFFDIO_ZEROPAGE, &zero_struct)) { + int e =3D errno; + error_report("%s: %s zero host: %p", + __func__, strerror(e), host); =20 - return -e; + return -e; + } + } else { + /* TODO: The kernel can't use UFFDIO_ZEROPAGE for hugepages */ + assert(0); } =20 - trace_postcopy_place_page_zero(host); return 0; } =20 @@ -604,7 +611,7 @@ int postcopy_place_page_zero(MigrationIncomingState *mi= s, void *host) void *postcopy_get_tmp_page(MigrationIncomingState *mis) { if (!mis->postcopy_tmp_page) { - mis->postcopy_tmp_page =3D mmap(NULL, getpagesize(), + mis->postcopy_tmp_page =3D mmap(NULL, mis->largest_page_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (mis->postcopy_tmp_page =3D=3D MAP_FAILED) { @@ -649,13 +656,15 @@ int postcopy_ram_enable_notify(MigrationIncomingState= *mis) return -1; } =20 -int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m) +int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m, + size_t pagesize) { assert(0); return -1; } =20 -int postcopy_place_page_zero(MigrationIncomingState *mis, void *host) +int postcopy_place_page_zero(MigrationIncomingState *mis, void *host, + size_t pagesize) { assert(0); return -1; diff --git a/migration/ram.c b/migration/ram.c index 44fe960..ff86664 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2402,6 +2402,7 @@ static int ram_load_postcopy(QEMUFile *f) void *host =3D NULL; void *page_buffer =3D NULL; void *place_source =3D NULL; + RAMBlock *block =3D NULL; uint8_t ch; =20 addr =3D qemu_get_be64(f); @@ -2411,7 +2412,7 @@ static int ram_load_postcopy(QEMUFile *f) trace_ram_load_postcopy_loop((uint64_t)addr, flags); place_needed =3D false; if (flags & (RAM_SAVE_FLAG_COMPRESS | RAM_SAVE_FLAG_PAGE)) { - RAMBlock *block =3D ram_block_from_stream(f, flags); + block =3D ram_block_from_stream(f, flags); =20 host =3D host_from_ram_block_offset(block, addr); if (!host) { @@ -2486,14 +2487,14 @@ static int ram_load_postcopy(QEMUFile *f) =20 if (place_needed) { /* This gets called at the last target page in the host page */ + void *place_dest =3D host + TARGET_PAGE_SIZE - block->page_siz= e; + if (all_zero) { - ret =3D postcopy_place_page_zero(mis, - host + TARGET_PAGE_SIZE - - qemu_host_page_size); + ret =3D postcopy_place_page_zero(mis, place_dest, + block->page_size); } else { - ret =3D postcopy_place_page(mis, host + TARGET_PAGE_SIZE - - qemu_host_page_size, - place_source); + ret =3D postcopy_place_page(mis, place_dest, + place_source, block->page_size); } } if (!ret) { --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488287148484956.932490683662; Tue, 28 Feb 2017 05:05:48 -0800 (PST) Received: from localhost ([::1]:60828 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihTN-0004Ig-2C for importer@patchew.org; Tue, 28 Feb 2017 08:05:45 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40177) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih65-0008WK-6n for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih62-0000Px-92 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:41 -0500 Received: from mx1.redhat.com ([209.132.183.28]:34454) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih62-0000PE-0a for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:38 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 17E4461E4C; Tue, 28 Feb 2017 12:41:38 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtJ019858; Tue, 28 Feb 2017 07:41:36 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:48 +0000 Message-Id: <20170228124056.5074-20-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 28 Feb 2017 12:41:38 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 19/27] postcopy: Use temporary for placing zero huge pages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" The kernel can't do UFFDIO_ZEROPAGE for huge pages, so we have to allocate a temporary (always zero) page and use UFFDIO_COPYPAGE on it. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-9-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 1 + migration/postcopy-ram.c | 23 +++++++++++++++++++++-- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index 08b98e9..5720c88 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -109,6 +109,7 @@ struct MigrationIncomingState { QEMUFile *to_src_file; QemuMutex rp_mutex; /* We send replies from multiple threads */ void *postcopy_tmp_page; + void *postcopy_tmp_zero_page; =20 QEMUBH *bh; =20 diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index a8b7fed..4c736d2 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -324,6 +324,10 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSta= te *mis) munmap(mis->postcopy_tmp_page, mis->largest_page_size); mis->postcopy_tmp_page =3D NULL; } + if (mis->postcopy_tmp_zero_page) { + munmap(mis->postcopy_tmp_zero_page, mis->largest_page_size); + mis->postcopy_tmp_zero_page =3D NULL; + } trace_postcopy_ram_incoming_cleanup_exit(); return 0; } @@ -593,8 +597,23 @@ int postcopy_place_page_zero(MigrationIncomingState *m= is, void *host, return -e; } } else { - /* TODO: The kernel can't use UFFDIO_ZEROPAGE for hugepages */ - assert(0); + /* The kernel can't use UFFDIO_ZEROPAGE for hugepages */ + if (!mis->postcopy_tmp_zero_page) { + mis->postcopy_tmp_zero_page =3D mmap(NULL, mis->largest_page_s= ize, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, + -1, 0); + if (mis->postcopy_tmp_zero_page =3D=3D MAP_FAILED) { + int e =3D errno; + mis->postcopy_tmp_zero_page =3D NULL; + error_report("%s: %s mapping large zero page", + __func__, strerror(e)); + return -e; + } + memset(mis->postcopy_tmp_zero_page, '\0', mis->largest_page_si= ze); + } + return postcopy_place_page(mis, host, mis->postcopy_tmp_zero_page, + pagesize); } =20 return 0; --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488287131605782.9959487182575; Tue, 28 Feb 2017 05:05:31 -0800 (PST) Received: from localhost ([::1]:60823 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihT6-00043J-7m for importer@patchew.org; Tue, 28 Feb 2017 08:05:28 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40178) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih65-0008WL-6o for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih64-0000So-9k for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:41 -0500 Received: from mx1.redhat.com ([209.132.183.28]:54972) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih64-0000RV-1L for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:40 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1EB4C81239; Tue, 28 Feb 2017 12:41:40 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtK019858; Tue, 28 Feb 2017 07:41:38 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:49 +0000 Message-Id: <20170228124056.5074-21-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 28 Feb 2017 12:41:40 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 20/27] postcopy: Load huge pages in one go X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" The existing postcopy RAM load loop already ensures that it glues together whole host-pages from the target page size chunks sent over the wire. Modify the definition of host page that it uses to be the RAM block page size and thus be huge pages where appropriate. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-10-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ff86664..9f28da2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2390,7 +2390,7 @@ static int ram_load_postcopy(QEMUFile *f) { int flags =3D 0, ret =3D 0; bool place_needed =3D false; - bool matching_page_sizes =3D qemu_host_page_size =3D=3D TARGET_PAGE_SI= ZE; + bool matching_page_sizes =3D false; MigrationIncomingState *mis =3D migration_incoming_get_current(); /* Temporary page that is later 'placed' */ void *postcopy_host_page =3D postcopy_get_tmp_page(mis); @@ -2420,8 +2420,11 @@ static int ram_load_postcopy(QEMUFile *f) ret =3D -EINVAL; break; } + matching_page_sizes =3D block->page_size =3D=3D TARGET_PAGE_SI= ZE; /* - * Postcopy requires that we place whole host pages atomically. + * Postcopy requires that we place whole host pages atomically; + * these may be huge pages for RAMBlocks that are backed by + * hugetlbfs. * To make it atomic, the data is read into a temporary page * that's moved into place later. * The migration protocol uses, possibly smaller, target-pages @@ -2429,9 +2432,9 @@ static int ram_load_postcopy(QEMUFile *f) * of a host page in order. */ page_buffer =3D postcopy_host_page + - ((uintptr_t)host & ~qemu_host_page_mask); + ((uintptr_t)host & (block->page_size - 1)); /* If all TP are zero then we can optimise the place */ - if (!((uintptr_t)host & ~qemu_host_page_mask)) { + if (!((uintptr_t)host & (block->page_size - 1))) { all_zero =3D true; } else { /* not the 1st TP within the HP */ @@ -2449,7 +2452,7 @@ static int ram_load_postcopy(QEMUFile *f) * page */ place_needed =3D (((uintptr_t)host + TARGET_PAGE_SIZE) & - ~qemu_host_page_mask) =3D=3D 0; + (block->page_size - 1)) =3D=3D 0; place_source =3D postcopy_host_page; } last_host =3D host; --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488286822456136.2025296784543; Tue, 28 Feb 2017 05:00:22 -0800 (PST) Received: from localhost ([::1]:60798 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihO9-0008NU-5m for importer@patchew.org; Tue, 28 Feb 2017 08:00:21 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40211) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih67-00006z-6f for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih66-0000UZ-7w for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:43 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43084) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih66-0000T7-1R for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:42 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 19BBE8553C; Tue, 28 Feb 2017 12:41:42 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtL019858; Tue, 28 Feb 2017 07:41:40 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:50 +0000 Message-Id: <20170228124056.5074-22-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 28 Feb 2017 12:41:42 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 21/27] postcopy: Mask fault addresses to huge page boundary X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Currently the fault address received by userfault is rounded to the host page boundary and a host page is requested from the source. Use the current RAMBlock page size instead of the general host page size so that for RAMBlocks backed by huge pages we request the whole huge page. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-11-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 4c736d2..03cbd6e 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -403,7 +403,6 @@ static void *postcopy_ram_fault_thread(void *opaque) MigrationIncomingState *mis =3D opaque; struct uffd_msg msg; int ret; - size_t hostpagesize =3D getpagesize(); RAMBlock *rb =3D NULL; RAMBlock *last_rb =3D NULL; /* last RAMBlock we sent part of */ =20 @@ -470,7 +469,7 @@ static void *postcopy_ram_fault_thread(void *opaque) break; } =20 - rb_offset &=3D ~(hostpagesize - 1); + rb_offset &=3D ~(qemu_ram_pagesize(rb) - 1); trace_postcopy_ram_fault_thread_request(msg.arg.pagefault.address, qemu_ram_get_idstr(rb), rb_offset); @@ -482,11 +481,11 @@ static void *postcopy_ram_fault_thread(void *opaque) if (rb !=3D last_rb) { last_rb =3D rb; migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), - rb_offset, hostpagesize); + rb_offset, qemu_ram_pagesize(rb)); } else { /* Save some space */ migrate_send_rp_req_pages(mis, NULL, - rb_offset, hostpagesize); + rb_offset, qemu_ram_pagesize(rb)); } } trace_postcopy_ram_fault_thread_exit(); --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488287413386904.6737614628579; Tue, 28 Feb 2017 05:10:13 -0800 (PST) Received: from localhost ([::1]:60850 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihXf-0008EJ-Us for importer@patchew.org; Tue, 28 Feb 2017 08:10:12 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40228) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih69-0000Ac-3X for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih68-0000Xo-7D for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:45 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43100) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih68-0000WQ-0W for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:44 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1578380461; Tue, 28 Feb 2017 12:41:44 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtM019858; Tue, 28 Feb 2017 07:41:42 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:51 +0000 Message-Id: <20170228124056.5074-23-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 28 Feb 2017 12:41:44 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 22/27] postcopy: Send whole huge pages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" The RAM save code uses ram_save_host_page to send whole host pages at a time; change this to use the host page size associated with the RAM Block which may be a huge page. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-12-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 9f28da2..719425b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1302,6 +1302,8 @@ static int ram_save_target_page(MigrationState *ms, Q= EMUFile *f, * offset to point into the middle of a host page * in which case the remainder of the hostpage is sent. * Only dirty target pages are sent. + * Note that the host page size may be a huge page for= this + * block. * * Returns: Number of pages written. * @@ -1320,6 +1322,8 @@ static int ram_save_host_page(MigrationState *ms, QEM= UFile *f, ram_addr_t dirty_ram_abs) { int tmppages, pages =3D 0; + size_t pagesize =3D qemu_ram_pagesize(pss->block); + do { tmppages =3D ram_save_target_page(ms, f, pss, last_stage, bytes_transferred, dirty_ram_abs); @@ -1330,7 +1334,7 @@ static int ram_save_host_page(MigrationState *ms, QEM= UFile *f, pages +=3D tmppages; pss->offset +=3D TARGET_PAGE_SIZE; dirty_ram_abs +=3D TARGET_PAGE_SIZE; - } while (pss->offset & (qemu_host_page_size - 1)); + } while (pss->offset & (pagesize - 1)); =20 /* The offset we leave with is the last one we looked at */ pss->offset -=3D TARGET_PAGE_SIZE; --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488287399642511.2904274854666; Tue, 28 Feb 2017 05:09:59 -0800 (PST) Received: from localhost ([::1]:60848 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihXO-0007yX-BR for importer@patchew.org; Tue, 28 Feb 2017 08:09:54 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40244) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih6A-0000E3-Ty for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih6A-0000Zf-3O for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:46 -0500 Received: from mx1.redhat.com ([209.132.183.28]:42152) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih69-0000ZV-Ta for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:46 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 095E7128D; Tue, 28 Feb 2017 12:41:46 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtN019858; Tue, 28 Feb 2017 07:41:44 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:52 +0000 Message-Id: <20170228124056.5074-24-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 28 Feb 2017 12:41:46 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 23/27] postcopy: Allow hugepages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Allow huge pages in postcopy. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-13-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 25 +------------------------ 1 file changed, 1 insertion(+), 24 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 03cbd6e..6b30b43 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -85,24 +85,6 @@ static bool ufd_version_check(int ufd) } =20 /* - * Check for things that postcopy won't support; returns 0 if the block - * is fine. - */ -static int check_range(const char *block_name, void *host_addr, - ram_addr_t offset, ram_addr_t length, void *opaque) -{ - RAMBlock *rb =3D qemu_ram_block_by_name(block_name); - - if (qemu_ram_pagesize(rb) > getpagesize()) { - error_report("Postcopy doesn't support large page sizes yet (%s)", - block_name); - return -E2BIG; - } - - return 0; -} - -/* * Note: This has the side effect of munlock'ing all of RAM, that's * normally fine since if the postcopy succeeds it gets turned back on at = the * end. @@ -122,12 +104,6 @@ bool postcopy_ram_supported_by_host(void) goto out; } =20 - /* Check for anything about the RAMBlocks we don't support */ - if (qemu_ram_foreach_block(check_range, NULL)) { - /* check_range will have printed its own error */ - goto out; - } - ufd =3D syscall(__NR_userfaultfd, O_CLOEXEC); if (ufd =3D=3D -1) { error_report("%s: userfaultfd not available: %s", __func__, @@ -139,6 +115,7 @@ bool postcopy_ram_supported_by_host(void) if (!ufd_version_check(ufd)) { goto out; } + /* TODO: Only allow huge pages if the kernel supports it */ =20 /* * userfault and mlock don't go together; we'll put it back later if --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488287655545103.06690016326468; Tue, 28 Feb 2017 05:14:15 -0800 (PST) Received: from localhost ([::1]:60873 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihbY-0004gx-CE for importer@patchew.org; Tue, 28 Feb 2017 08:14:12 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40261) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih6E-0000HL-3w for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih6C-0000ak-8h for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:50 -0500 Received: from mx1.redhat.com ([209.132.183.28]:42874) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih6C-0000ab-06 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:48 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1EC2080F8E; Tue, 28 Feb 2017 12:41:48 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtO019858; Tue, 28 Feb 2017 07:41:46 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:53 +0000 Message-Id: <20170228124056.5074-25-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 28 Feb 2017 12:41:48 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 24/27] postcopy: Update userfaultfd.h header X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Just the userfaultfd.h update from Paolo's header update run; * Drop this patch after Paolo's update goes in * Signed-off-by: Dr. David Alan Gilbert Message-Id: <20170224182844.32452-14-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- linux-headers/linux/userfaultfd.h | 67 +++++++++++++++++++++++++++++++++--= ---- 1 file changed, 58 insertions(+), 9 deletions(-) diff --git a/linux-headers/linux/userfaultfd.h b/linux-headers/linux/userfa= ultfd.h index 19e8453..2ed5dc3 100644 --- a/linux-headers/linux/userfaultfd.h +++ b/linux-headers/linux/userfaultfd.h @@ -11,13 +11,18 @@ =20 #include =20 -#define UFFD_API ((__u64)0xAA) /* - * After implementing the respective features it will become: - * #define UFFD_API_FEATURES (UFFD_FEATURE_PAGEFAULT_FLAG_WP | \ - * UFFD_FEATURE_EVENT_FORK) + * If the UFFDIO_API is upgraded someday, the UFFDIO_UNREGISTER and + * UFFDIO_WAKE ioctls should be defined as _IOW and not as _IOR. In + * userfaultfd.h we assumed the kernel was reading (instead _IOC_READ + * means the userland is reading). */ -#define UFFD_API_FEATURES (0) +#define UFFD_API ((__u64)0xAA) +#define UFFD_API_FEATURES (UFFD_FEATURE_EVENT_FORK | \ + UFFD_FEATURE_EVENT_REMAP | \ + UFFD_FEATURE_EVENT_MADVDONTNEED | \ + UFFD_FEATURE_MISSING_HUGETLBFS | \ + UFFD_FEATURE_MISSING_SHMEM) #define UFFD_API_IOCTLS \ ((__u64)1 << _UFFDIO_REGISTER | \ (__u64)1 << _UFFDIO_UNREGISTER | \ @@ -26,6 +31,9 @@ ((__u64)1 << _UFFDIO_WAKE | \ (__u64)1 << _UFFDIO_COPY | \ (__u64)1 << _UFFDIO_ZEROPAGE) +#define UFFD_API_RANGE_IOCTLS_BASIC \ + ((__u64)1 << _UFFDIO_WAKE | \ + (__u64)1 << _UFFDIO_COPY) =20 /* * Valid ioctl command number range with this API is from 0x00 to @@ -72,6 +80,21 @@ struct uffd_msg { } pagefault; =20 struct { + __u32 ufd; + } fork; + + struct { + __u64 from; + __u64 to; + __u64 len; + } remap; + + struct { + __u64 start; + __u64 end; + } madv_dn; + + struct { /* unused reserved fields */ __u64 reserved1; __u64 reserved2; @@ -84,9 +107,9 @@ struct uffd_msg { * Start at 0x12 and not at 0 to be more strict against bugs. */ #define UFFD_EVENT_PAGEFAULT 0x12 -#if 0 /* not available yet */ #define UFFD_EVENT_FORK 0x13 -#endif +#define UFFD_EVENT_REMAP 0x14 +#define UFFD_EVENT_MADVDONTNEED 0x15 =20 /* flags for UFFD_EVENT_PAGEFAULT */ #define UFFD_PAGEFAULT_FLAG_WRITE (1<<0) /* If this was a write fault */ @@ -104,11 +127,37 @@ struct uffdio_api { * Note: UFFD_EVENT_PAGEFAULT and UFFD_PAGEFAULT_FLAG_WRITE * are to be considered implicitly always enabled in all kernels as * long as the uffdio_api.api requested matches UFFD_API. + * + * UFFD_FEATURE_MISSING_HUGETLBFS means an UFFDIO_REGISTER + * with UFFDIO_REGISTER_MODE_MISSING mode will succeed on + * hugetlbfs virtual memory ranges. Adding or not adding + * UFFD_FEATURE_MISSING_HUGETLBFS to uffdio_api.features has + * no real functional effect after UFFDIO_API returns, but + * it's only useful for an initial feature set probe at + * UFFDIO_API time. There are two ways to use it: + * + * 1) by adding UFFD_FEATURE_MISSING_HUGETLBFS to the + * uffdio_api.features before calling UFFDIO_API, an error + * will be returned by UFFDIO_API on a kernel without + * hugetlbfs missing support + * + * 2) the UFFD_FEATURE_MISSING_HUGETLBFS can not be added in + * uffdio_api.features and instead it will be set by the + * kernel in the uffdio_api.features if the kernel supports + * it, so userland can later check if the feature flag is + * present in uffdio_api.features after UFFDIO_API + * succeeded. + * + * UFFD_FEATURE_MISSING_SHMEM works the same as + * UFFD_FEATURE_MISSING_HUGETLBFS, but it applies to shmem + * (i.e. tmpfs and other shmem based APIs). */ -#if 0 /* not available yet */ #define UFFD_FEATURE_PAGEFAULT_FLAG_WP (1<<0) #define UFFD_FEATURE_EVENT_FORK (1<<1) -#endif +#define UFFD_FEATURE_EVENT_REMAP (1<<2) +#define UFFD_FEATURE_EVENT_MADVDONTNEED (1<<3) +#define UFFD_FEATURE_MISSING_HUGETLBFS (1<<4) +#define UFFD_FEATURE_MISSING_SHMEM (1<<5) __u64 features; =20 __u64 ioctls; --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488287919885501.3157630380158; Tue, 28 Feb 2017 05:18:39 -0800 (PST) Received: from localhost ([::1]:60905 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihfp-0000Jn-JS for importer@patchew.org; Tue, 28 Feb 2017 08:18:37 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40280) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih6G-0000JU-5g for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:53 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih6F-0000bK-Ev for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:52 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43148) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih6F-0000b7-8d for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:51 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 587A880475; Tue, 28 Feb 2017 12:41:51 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtP019858; Tue, 28 Feb 2017 07:41:49 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:54 +0000 Message-Id: <20170228124056.5074-26-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 28 Feb 2017 12:41:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 25/27] postcopy: Check for userfault+hugepage feature X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" We need extra Linux kernel support (~4.11) to support userfaults on hugetlbfs; check for them. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-15-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 6b30b43..102fb61 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -81,6 +81,17 @@ static bool ufd_version_check(int ufd) return false; } =20 + if (getpagesize() !=3D ram_pagesize_summary()) { + bool have_hp =3D false; + /* We've got a huge page */ +#ifdef UFFD_FEATURE_MISSING_HUGETLBFS + have_hp =3D api_struct.features & UFFD_FEATURE_MISSING_HUGETLBFS; +#endif + if (!have_hp) { + error_report("Userfault on this host does not support huge pag= es"); + return false; + } + } return true; } =20 @@ -115,7 +126,6 @@ bool postcopy_ram_supported_by_host(void) if (!ufd_version_check(ufd)) { goto out; } - /* TODO: Only allow huge pages if the kernel supports it */ =20 /* * userfault and mlock don't go together; we'll put it back later if --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488287662797357.79272283164335; Tue, 28 Feb 2017 05:14:22 -0800 (PST) Received: from localhost ([::1]:60874 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihbg-00051g-MK for importer@patchew.org; Tue, 28 Feb 2017 08:14:20 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40307) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih6I-0000LC-4c for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih6H-0000bp-DF for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:54 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57906) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih6H-0000bY-79 for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:53 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4F8D74E4CB; Tue, 28 Feb 2017 12:41:53 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtQ019858; Tue, 28 Feb 2017 07:41:51 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:55 +0000 Message-Id: <20170228124056.5074-27-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 28 Feb 2017 12:41:53 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 26/27] postcopy: Add doc about hugepages and postcopy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-16-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- docs/migration.txt | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/migration.txt b/docs/migration.txt index c8b0304..1b940a8 100644 --- a/docs/migration.txt +++ b/docs/migration.txt @@ -540,3 +540,16 @@ request for a page that has already been sent is ignor= ed. Duplicate requests such as this can happen as a page is sent at about the same time the destination accesses it. =20 +=3D=3D=3D Postcopy with hugepages =3D=3D=3D + +Postcopy now works with hugetlbfs backed memory: + a) The linux kernel on the destination must support userfault on hugepag= es. + b) The huge-page configuration on the source and destination VMs must be + identical; i.e. RAMBlocks on both sides must use the same page size. + c) Note that -mem-path /dev/hugepages will fall back to allocating norm= al + RAM if it doesn't have enough hugepages, triggering (b) to fail. + Using -mem-prealloc enforces the allocation using hugepages. + d) Care should be taken with the size of hugepage used; postcopy with 2MB + hugepages works well, however 1GB hugepages are likely to be problema= tic + since it takes ~1 second to transfer a 1GB hugepage across a 10Gbps l= ink, + and until the full page is transferred the destination thread is bloc= ked. --=20 2.9.3 From nobody Thu May 2 03:21:13 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1488287104979944.694558985889; Tue, 28 Feb 2017 05:05:04 -0800 (PST) Received: from localhost ([::1]:60822 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cihSg-0003kE-Do for importer@patchew.org; Tue, 28 Feb 2017 08:05:02 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40332) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cih6J-0000Mq-TX for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cih6J-0000dI-8g for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:55 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59598) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cih6J-0000cT-3t for qemu-devel@nongnu.org; Tue, 28 Feb 2017 07:41:55 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 37624C04B92D; Tue, 28 Feb 2017 12:41:55 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-149.ams2.redhat.com [10.36.117.149]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v1SCewtR019858; Tue, 28 Feb 2017 07:41:53 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Tue, 28 Feb 2017 12:40:56 +0000 Message-Id: <20170228124056.5074-28-dgilbert@redhat.com> In-Reply-To: <20170228124056.5074-1-dgilbert@redhat.com> References: <20170228124056.5074-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 28 Feb 2017 12:41:55 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 27/27] postcopy: Add extra check for COPY function X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, pasic@linux.vnet.ibm.com, vsementsov@virtuozzo.com, quintela@redhat.com, ashijeetacharya@gmail.com, marcandre.lureau@redhat.com, danielhb@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" As an extra sanity check, make sure the region we're registering can perform UFFDIO_COPY; the COPY will fail later but this gives a cleaner failure. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Message-Id: <20170224182844.32452-17-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 102fb61..effbeb6 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -378,6 +378,10 @@ static int ram_block_enable_notify(const char *block_n= ame, void *host_addr, error_report("%s userfault register: %s", __func__, strerror(errno= )); return -1; } + if (!(reg_struct.ioctls & ((__u64)1 << _UFFDIO_COPY))) { + error_report("%s userfault: Region doesn't support COPY", __func__= ); + return -1; + } =20 return 0; } --=20 2.9.3