From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486402686965162.68269462371848; Mon, 6 Feb 2017 09:38:06 -0800 (PST) Received: from localhost ([::1]:49864 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canEr-0007PD-KO for importer@patchew.org; Mon, 06 Feb 2017 12:38:05 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45300) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canA8-0003sU-Lf for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canA6-0006O5-RF for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:12 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51078) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canA6-0006Nt-IE for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:10 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B4A7CC0016A4 for ; Mon, 6 Feb 2017 17:33:10 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8R6012611; Mon, 6 Feb 2017 12:33:09 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:32:51 +0000 Message-Id: <20170206173306.20603-2-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 06 Feb 2017 17:33:10 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 01/16] postcopy: Transmit ram size summary word X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Replace the host page-size in the 'advise' command by a pagesize summary bitmap; if the VM is just using normal RAM then this will be exactly the same as before, however if they're using huge pages they'll be different, and thus: a) Migration from/to old qemu's that don't understand huge pages will fail early. b) Migrations with different size RAMBlocks will also fail early. This catches it very early; earlier than the detailed per-block check in the next patch. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- include/migration/migration.h | 1 + migration/ram.c | 17 +++++++++++++++++ migration/savevm.c | 32 +++++++++++++++++++++----------- 3 files changed, 39 insertions(+), 11 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index af9135f..96c9d6e 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -366,6 +366,7 @@ void global_state_store_running(void); void flush_page_queue(MigrationState *ms); int ram_save_queue_pages(MigrationState *ms, const char *rbname, ram_addr_t start, ram_addr_t len); +uint64_t ram_pagesize_summary(void); =20 PostcopyState postcopy_state_get(void); /* Set the state and return the old state */ diff --git a/migration/ram.c b/migration/ram.c index ef8fadf..b405e4a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -600,6 +600,23 @@ static void migration_bitmap_sync_init(void) iterations_prev =3D 0; } =20 +/* Returns a summary bitmap of the page sizes of all RAMBlocks; + * for VMs with just normal pages this is equivalent to the + * host page size. If it's got some huge pages then it's the OR + * of all the different page sizes. + */ +uint64_t ram_pagesize_summary(void) +{ + RAMBlock *block; + uint64_t summary =3D 0; + + QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + summary |=3D block->page_size; + } + + return summary; +} + static void migration_bitmap_sync(void) { RAMBlock *block; diff --git a/migration/savevm.c b/migration/savevm.c index de86db0..e83d01a 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -869,7 +869,7 @@ int qemu_savevm_send_packaged(QEMUFile *f, const uint8_= t *buf, size_t len) void qemu_savevm_send_postcopy_advise(QEMUFile *f) { uint64_t tmp[2]; - tmp[0] =3D cpu_to_be64(getpagesize()); + tmp[0] =3D cpu_to_be64(ram_pagesize_summary()); tmp[1] =3D cpu_to_be64(1ul << qemu_target_page_bits()); =20 trace_qemu_savevm_send_postcopy_advise(); @@ -1346,7 +1346,7 @@ static int qemu_loadvm_state_main(QEMUFile *f, Migrat= ionIncomingState *mis); static int loadvm_postcopy_handle_advise(MigrationIncomingState *mis) { PostcopyState ps =3D postcopy_state_set(POSTCOPY_INCOMING_ADVISE); - uint64_t remote_hps, remote_tps; + uint64_t remote_pagesize_summary, local_pagesize_summary, remote_tps; =20 trace_loadvm_postcopy_handle_advise(); if (ps !=3D POSTCOPY_INCOMING_NONE) { @@ -1359,17 +1359,27 @@ static int loadvm_postcopy_handle_advise(MigrationI= ncomingState *mis) return -1; } =20 - remote_hps =3D qemu_get_be64(mis->from_src_file); - if (remote_hps !=3D getpagesize()) { + remote_pagesize_summary =3D qemu_get_be64(mis->from_src_file); + local_pagesize_summary =3D ram_pagesize_summary(); + + if (remote_pagesize_summary !=3D local_pagesize_summary) { /* - * Some combinations of mismatch are probably possible but it gets - * a bit more complicated. In particular we need to place whole - * host pages on the dest at once, and we need to ensure that we - * handle dirtying to make sure we never end up sending part of - * a hostpage on it's own. + * This detects two potential causes of mismatch: + * a) A mismatch in host page sizes + * Some combinations of mismatch are probably possible but it= gets + * a bit more complicated. In particular we need to place wh= ole + * host pages on the dest at once, and we need to ensure that= we + * handle dirtying to make sure we never end up sending part = of + * a hostpage on it's own. + * b) The use of different huge page sizes on source/destination + * a more fine grain test is performed during RAM block migra= tion + * but this test here causes a nice early clear failure, and + * also fails when passed to an older qemu that doesn't + * do huge pages. */ - error_report("Postcopy needs matching host page sizes (s=3D%d d=3D= %d)", - (int)remote_hps, getpagesize()); + error_report("Postcopy needs matching RAM page sizes (s=3D%" PRIx64 + " d=3D%" PRIx= 64 ")", + remote_pagesize_summary, local_pagesize_summary); return -1; } =20 --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486402588249509.97191585198755; Mon, 6 Feb 2017 09:36:28 -0800 (PST) Received: from localhost ([::1]:49858 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canDF-00060m-Rq for importer@patchew.org; Mon, 06 Feb 2017 12:36:25 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45302) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canA8-0003sV-MA for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canA7-0006OJ-St for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:12 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51854) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canA7-0006OB-MR for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:11 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D98C681222 for ; Mon, 6 Feb 2017 17:33:11 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8R7012611; Mon, 6 Feb 2017 12:33:10 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:32:52 +0000 Message-Id: <20170206173306.20603-3-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 06 Feb 2017 17:33:11 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 02/16] postcopy: Transmit and compare individual page sizes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" When using postcopy with hugepages, we require the source and destination page sizes for any RAMBlock to match; note that different RAMBlocks in the same VM can have different page sizes. Transmit them as part of the RAM information header and fail if there's a difference. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- migration/ram.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index b405e4a..5726563 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1979,6 +1979,9 @@ static int ram_save_setup(QEMUFile *f, void *opaque) qemu_put_byte(f, strlen(block->idstr)); qemu_put_buffer(f, (uint8_t *)block->idstr, strlen(block->idstr)); qemu_put_be64(f, block->used_length); + if (migrate_postcopy_ram() && block->page_size !=3D qemu_host_page= _size) { + qemu_put_be64(f, block->page_size); + } } =20 rcu_read_unlock(); @@ -2480,6 +2483,8 @@ static int ram_load(QEMUFile *f, void *opaque, int ve= rsion_id) * be atomic */ bool postcopy_running =3D postcopy_state_get() >=3D POSTCOPY_INCOMING_= LISTENING; + /* ADVISE is earlier, it shows the source has the postcopy capability = on */ + bool postcopy_advised =3D postcopy_state_get() >=3D POSTCOPY_INCOMING_= ADVISE; =20 seq_iter++; =20 @@ -2544,6 +2549,18 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) error_report_err(local_err); } } + /* For postcopy we need to check hugepage sizes match = */ + if (postcopy_advised && + block->page_size !=3D qemu_host_page_size) { + uint64_t remote_page_size =3D qemu_get_be64(f); + if (remote_page_size !=3D block->page_size) { + error_report("Mismatched RAM page size %s " + "(local) %zd !=3D %" PRId64, + id, block->page_size, + remote_page_size); + ret =3D -EINVAL; + } + } ram_control_load_hook(f, RAM_CONTROL_BLOCK_REG, block->idstr); } else { --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486402528020319.3916415042163; Mon, 6 Feb 2017 09:35:28 -0800 (PST) Received: from localhost ([::1]:49847 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canCI-0005BW-Fn for importer@patchew.org; Mon, 06 Feb 2017 12:35:26 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45334) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canA9-0003t6-UB for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canA8-0006Oz-W2 for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:13 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51858) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canA8-0006OQ-RT for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:12 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0F1AB8123A for ; Mon, 6 Feb 2017 17:33:13 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8R8012611; Mon, 6 Feb 2017 12:33:12 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:32:53 +0000 Message-Id: <20170206173306.20603-4-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Mon, 06 Feb 2017 17:33:13 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 03/16] postcopy: Chunk discards for hugepages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" At the start of the postcopy phase, partially sent huge pages must be discarded. The code for dealing with host page sizes larger than the target page size can be reused for this case. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- migration/ram.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 5726563..d33bd21 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1627,12 +1627,17 @@ static void postcopy_chunk_hostpages_pass(Migration= State *ms, bool unsent_pass, { unsigned long *bitmap; unsigned long *unsentmap; - unsigned int host_ratio =3D qemu_host_page_size / TARGET_PAGE_SIZE; + unsigned int host_ratio =3D block->page_size / TARGET_PAGE_SIZE; unsigned long first =3D block->offset >> TARGET_PAGE_BITS; unsigned long len =3D block->used_length >> TARGET_PAGE_BITS; unsigned long last =3D first + (len - 1); unsigned long run_start; =20 + if (block->page_size =3D=3D TARGET_PAGE_SIZE) { + /* Easy case - TPS=3D=3DHPS for a non-huge page RAMBlock */ + return; + } + bitmap =3D atomic_rcu_read(&migration_bitmap_rcu)->bmap; unsentmap =3D atomic_rcu_read(&migration_bitmap_rcu)->unsentmap; =20 @@ -1736,7 +1741,8 @@ static void postcopy_chunk_hostpages_pass(MigrationSt= ate *ms, bool unsent_pass, * Utility for the outgoing postcopy code. * * Discard any partially sent host-page size chunks, mark any partially - * dirty host-page size chunks as all dirty. + * dirty host-page size chunks as all dirty. In this case the host-page + * is the host-page for the particular RAMBlock, i.e. it might be a huge p= age * * Returns: 0 on success */ @@ -1744,11 +1750,6 @@ static int postcopy_chunk_hostpages(MigrationState *= ms) { struct RAMBlock *block; =20 - if (qemu_host_page_size =3D=3D TARGET_PAGE_SIZE) { - /* Easy case - TPS=3D=3DHPS - nothing to be done */ - return 0; - } - /* Easiest way to make sure we don't resume in the middle of a host-pa= ge */ last_seen_block =3D NULL; last_sent_block =3D NULL; @@ -1804,7 +1805,7 @@ int ram_postcopy_send_discard_bitmap(MigrationState *= ms) return -EINVAL; } =20 - /* Deal with TPS !=3D HPS */ + /* Deal with TPS !=3D HPS and huge pages */ ret =3D postcopy_chunk_hostpages(ms); if (ret) { rcu_read_unlock(); --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486402947058105.10074194997912; Mon, 6 Feb 2017 09:42:27 -0800 (PST) Received: from localhost ([::1]:49887 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canJ3-0002wO-GV for importer@patchew.org; Mon, 06 Feb 2017 12:42:25 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45383) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAE-0003vD-Li for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAA-0006PL-AB for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:18 -0500 Received: from mx1.redhat.com ([209.132.183.28]:38552) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAA-0006P6-1k for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:14 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3D3AA369C3 for ; Mon, 6 Feb 2017 17:33:14 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8R9012611; Mon, 6 Feb 2017 12:33:13 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:32:54 +0000 Message-Id: <20170206173306.20603-5-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Feb 2017 17:33:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 04/16] exec: ram_block_discard_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Create ram_block_discard_range in exec.c to replace postcopy_ram_discard_range and most of ram_discard_range. Those two routines are a bit of a weird combination, and ram_discard_range is about to get more complex for hugepages. It's OS dependent code (so shouldn't be in migration/ram.c) but it needs quite a bit of the innards of RAMBlock so doesn't belong in the os*.c. Signed-off-by: Dr. David Alan Gilbert Tested-by: Laurent Vivier --- exec.c | 59 +++++++++++++++++++++++++++++++++++++++++++= ++++ include/exec/cpu-common.h | 1 + 2 files changed, 60 insertions(+) diff --git a/exec.c b/exec.c index 8b9ed73..e040cdf 100644 --- a/exec.c +++ b/exec.c @@ -45,6 +45,12 @@ #include "exec/address-spaces.h" #include "sysemu/xen-mapcache.h" #include "trace-root.h" + +#ifdef CONFIG_FALLOCATE_PUNCH_HOLE +#include +#include +#endif + #endif #include "exec/cpu-all.h" #include "qemu/rcu_queue.h" @@ -3286,4 +3292,57 @@ int qemu_ram_foreach_block(RAMBlockIterFunc func, vo= id *opaque) rcu_read_unlock(); return ret; } + +/* + * Unmap pages of memory from start to start+length such that + * they a) read as 0, b) Trigger whatever fault mechanism + * the OS provides for postcopy. + * The pages must be unmapped by the end of the function. + * Returns: 0 on success, none-0 on failure + * + */ +int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length) +{ + int ret =3D -1; + + rcu_read_lock(); + uint8_t *host_startaddr =3D rb->host + start; + + if ((uintptr_t)host_startaddr & (rb->page_size - 1)) { + error_report("ram_block_discard_range: Unaligned start address: %p= ", + host_startaddr); + goto err; + } + + if ((start + length) <=3D rb->used_length) { + uint8_t *host_endaddr =3D host_startaddr + length; + if ((uintptr_t)host_endaddr & (rb->page_size - 1)) { + error_report("ram_block_discard_range: Unaligned end address: = %p", + host_endaddr); + goto err; + } + + errno =3D ENOTSUP; /* If we are missing MADVISE etc */ + +#if defined(CONFIG_MADVISE) + ret =3D qemu_madvise(host_startaddr, length, QEMU_MADV_DONTNEED); +#endif + if (ret) { + ret =3D -errno; + error_report("ram_block_discard_range: Failed to discard range= " + "%s:%" PRIx64 " +%zx (%d)", + rb->idstr, start, length, ret); + } + } else { + error_report("ram_block_discard_range: Overrun block '%s' (%" PRIu= 64 + "/%zx/" RAM_ADDR_FMT")", + rb->idstr, start, length, rb->used_length); + } + +err: + rcu_read_unlock(); + + return ret; +} + #endif diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index bd15853..1350c2e 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -105,6 +105,7 @@ typedef int (RAMBlockIterFunc)(const char *block_name, = void *host_addr, ram_addr_t offset, ram_addr_t length, void *opaque); =20 int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque); +int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length); =20 #endif =20 --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486402677276947.1590128959862; Mon, 6 Feb 2017 09:37:57 -0800 (PST) Received: from localhost ([::1]:49863 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canEi-0007H0-0v for importer@patchew.org; Mon, 06 Feb 2017 12:37:56 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45355) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAC-0003t7-DG for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAB-0006Pf-DR for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:16 -0500 Received: from mx1.redhat.com ([209.132.183.28]:56464) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAB-0006PQ-7O for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:15 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 63F2B63144 for ; Mon, 6 Feb 2017 17:33:15 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RA012611; Mon, 6 Feb 2017 12:33:14 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:32:55 +0000 Message-Id: <20170206173306.20603-6-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 06 Feb 2017 17:33:15 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 05/16] postcopy: enhance ram_block_discard_range for hugepages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Unfortunately madvise DONTNEED doesn't work on hugepagetlb so use fallocate(FALLOC_FL_PUNCH_HOLE) qemu_fd_getpagesize only sets the page based off a file if the file is from hugetlbfs. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- exec.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/exec.c b/exec.c index e040cdf..c25f6b3 100644 --- a/exec.c +++ b/exec.c @@ -3324,9 +3324,20 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t s= tart, size_t length) =20 errno =3D ENOTSUP; /* If we are missing MADVISE etc */ =20 + if (rb->page_size =3D=3D qemu_host_page_size) { #if defined(CONFIG_MADVISE) - ret =3D qemu_madvise(host_startaddr, length, QEMU_MADV_DONTNEED); + ret =3D qemu_madvise(host_startaddr, length, QEMU_MADV_DONTNEE= D); #endif + } else { + /* Huge page case - unfortunately it can't do DONTNEED, but + * it can do the equivalent by FALLOC_FL_PUNCH_HOLE in the + * huge page file. + */ +#ifdef CONFIG_FALLOCATE_PUNCH_HOLE + ret =3D fallocate(rb->fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEE= P_SIZE, + start, length); +#endif + } if (ret) { ret =3D -errno; error_report("ram_block_discard_range: Failed to discard range= " --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486402850245343.9944671400358; Mon, 6 Feb 2017 09:40:50 -0800 (PST) Received: from localhost ([::1]:49881 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canHU-0001PV-FY for importer@patchew.org; Mon, 06 Feb 2017 12:40:48 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45367) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAD-0003uU-Ow for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAC-0006Pw-JJ for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:17 -0500 Received: from mx1.redhat.com ([209.132.183.28]:38566) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAC-0006Pj-At for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:16 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 85FFE3D955 for ; Mon, 6 Feb 2017 17:33:16 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RB012611; Mon, 6 Feb 2017 12:33:15 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:32:56 +0000 Message-Id: <20170206173306.20603-7-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 06 Feb 2017 17:33:16 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 06/16] Fold postcopy_ram_discard_range into ram_discard_range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Using the previously created ram_block_discard_range, kill off postcopy_ram_discard_range. ram_discard_range is just a wrapper that does the name lookup. Signed-off-by: Dr. David Alan Gilbert Tested-by: Laurent Vivier --- include/migration/postcopy-ram.h | 7 ------- migration/postcopy-ram.c | 30 +----------------------------- migration/ram.c | 24 +++--------------------- migration/trace-events | 2 +- 4 files changed, 5 insertions(+), 58 deletions(-) diff --git a/include/migration/postcopy-ram.h b/include/migration/postcopy-= ram.h index b6a7491f..43bbbca 100644 --- a/include/migration/postcopy-ram.h +++ b/include/migration/postcopy-ram.h @@ -35,13 +35,6 @@ int postcopy_ram_incoming_init(MigrationIncomingState *m= is, size_t ram_pages); int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis); =20 /* - * Discard the contents of 'length' bytes from 'start' - * We can assume that if we've been called postcopy_ram_hosttest returned = true - */ -int postcopy_ram_discard_range(MigrationIncomingState *mis, uint8_t *start, - size_t length); - -/* * Userfault requires us to mark RAM as NOHUGEPAGE prior to discard * however leaving it until after precopy means that most of the precopy * data is still THPd diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index a40dddb..1e3d22f 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -200,27 +200,6 @@ out: return ret; } =20 -/** - * postcopy_ram_discard_range: Discard a range of memory. - * We can assume that if we've been called postcopy_ram_hosttest returned = true. - * - * @mis: Current incoming migration state. - * @start, @length: range of memory to discard. - * - * returns: 0 on success. - */ -int postcopy_ram_discard_range(MigrationIncomingState *mis, uint8_t *start, - size_t length) -{ - trace_postcopy_ram_discard_range(start, length); - if (madvise(start, length, MADV_DONTNEED)) { - error_report("%s MADV_DONTNEED: %s", __func__, strerror(errno)); - return -1; - } - - return 0; -} - /* * Setup an area of RAM so that it *can* be used for postcopy later; this * must be done right at the start prior to pre-copy. @@ -239,7 +218,7 @@ static int init_range(const char *block_name, void *hos= t_addr, * - we're going to get the copy from the source anyway. * (Precopy will just overwrite this data, so doesn't need the discard) */ - if (postcopy_ram_discard_range(mis, host_addr, length)) { + if (ram_discard_range(mis, block_name, 0, length)) { return -1; } =20 @@ -658,13 +637,6 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSta= te *mis) return -1; } =20 -int postcopy_ram_discard_range(MigrationIncomingState *mis, uint8_t *start, - size_t length) -{ - assert(0); - return -1; -} - int postcopy_ram_prepare_discard(MigrationIncomingState *mis) { assert(0); diff --git a/migration/ram.c b/migration/ram.c index d33bd21..136996a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1845,6 +1845,8 @@ int ram_discard_range(MigrationIncomingState *mis, { int ret =3D -1; =20 + trace_ram_discard_range(block_name, start, length); + rcu_read_lock(); RAMBlock *rb =3D qemu_ram_block_by_name(block_name); =20 @@ -1854,27 +1856,7 @@ int ram_discard_range(MigrationIncomingState *mis, goto err; } =20 - uint8_t *host_startaddr =3D rb->host + start; - - if ((uintptr_t)host_startaddr & (qemu_host_page_size - 1)) { - error_report("ram_discard_range: Unaligned start address: %p", - host_startaddr); - goto err; - } - - if ((start + length) <=3D rb->used_length) { - uint8_t *host_endaddr =3D host_startaddr + length; - if ((uintptr_t)host_endaddr & (qemu_host_page_size - 1)) { - error_report("ram_discard_range: Unaligned end address: %p", - host_endaddr); - goto err; - } - ret =3D postcopy_ram_discard_range(mis, host_startaddr, length); - } else { - error_report("ram_discard_range: Overrun block '%s' (%" PRIu64 - "/%zx/" RAM_ADDR_FMT")", - block_name, start, length, rb->used_length); - } + ret =3D ram_block_discard_range(rb, start, length); =20 err: rcu_read_unlock(); diff --git a/migration/trace-events b/migration/trace-events index fa660e3..7372ce2 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -68,6 +68,7 @@ get_queued_page_not_dirty(const char *block_name, uint64_= t tmp_offset, uint64_t migration_bitmap_sync_start(void) "" migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 migration_throttle(void) "" +ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: sta= rt: %" PRIx64 " %zx" ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x" ram_postcopy_send_discard_bitmap(void) "" ram_save_queue_pages(const char *rbname, size_t start, size_t len) "%s: st= art: %zx len: %zx" @@ -176,7 +177,6 @@ rdma_start_outgoing_migration_after_rdma_source_init(vo= id) "" # migration/postcopy-ram.c postcopy_discard_send_finish(const char *ramblock, int nwords, int ncmds) = "%s mask words sent=3D%d in %d commands" postcopy_discard_send_range(const char *ramblock, unsigned long start, uns= igned long length) "%s:%lx/%lx" -postcopy_ram_discard_range(void *start, size_t length) "%p,+%zx" postcopy_cleanup_range(const char *ramblock, void *host_addr, size_t offse= t, size_t length) "%s: %p offset=3D%zx length=3D%zx" postcopy_init_range(const char *ramblock, void *host_addr, size_t offset, = size_t length) "%s: %p offset=3D%zx length=3D%zx" postcopy_nhp_range(const char *ramblock, void *host_addr, size_t offset, s= ize_t length) "%s: %p offset=3D%zx length=3D%zx" --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486403323837256.77547752752537; Mon, 6 Feb 2017 09:48:43 -0800 (PST) Received: from localhost ([::1]:49919 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canP8-0008HI-K2 for importer@patchew.org; Mon, 06 Feb 2017 12:48:42 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45399) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAH-0003xo-2E for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAD-0006QA-O2 for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:21 -0500 Received: from mx1.redhat.com ([209.132.183.28]:11528) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAD-0006Q2-Hs for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:17 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B1F4980F8F for ; Mon, 6 Feb 2017 17:33:17 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RC012611; Mon, 6 Feb 2017 12:33:16 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:32:57 +0000 Message-Id: <20170206173306.20603-8-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Mon, 06 Feb 2017 17:33:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 07/16] postcopy: Record largest page size X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Record the largest page size in use; we'll need it soon for allocating temporary buffers. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- exec.c | 13 +++++++++++++ include/exec/cpu-common.h | 1 + include/migration/migration.h | 1 + migration/migration.c | 1 + 4 files changed, 16 insertions(+) diff --git a/exec.c b/exec.c index c25f6b3..59f3b6b 100644 --- a/exec.c +++ b/exec.c @@ -1524,6 +1524,19 @@ size_t qemu_ram_pagesize(RAMBlock *rb) return rb->page_size; } =20 +/* Returns the largest size of page in use */ +size_t qemu_ram_pagesize_largest(void) +{ + RAMBlock *block; + size_t largest =3D 0; + + QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + largest =3D MAX(largest, qemu_ram_pagesize(block)); + } + + return largest; +} + static int memory_try_enable_merging(void *addr, size_t len) { if (!machine_mem_merge(current_machine)) { diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index 1350c2e..8c305aa 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -64,6 +64,7 @@ void qemu_ram_set_idstr(RAMBlock *block, const char *name= , DeviceState *dev); void qemu_ram_unset_idstr(RAMBlock *block); const char *qemu_ram_get_idstr(RAMBlock *rb); size_t qemu_ram_pagesize(RAMBlock *block); +size_t qemu_ram_pagesize_largest(void); =20 void cpu_physical_memory_rw(hwaddr addr, uint8_t *buf, int len, int is_write); diff --git a/include/migration/migration.h b/include/migration/migration.h index 96c9d6e..c9c1d5f 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -92,6 +92,7 @@ struct MigrationIncomingState { */ QemuEvent main_thread_load_event; =20 + size_t largest_page_size; bool have_fault_thread; QemuThread fault_thread; QemuSemaphore fault_thread_sem; diff --git a/migration/migration.c b/migration/migration.c index 283677c..e0fdafc 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -387,6 +387,7 @@ static void process_incoming_migration_co(void *opaque) int ret; =20 mis =3D migration_incoming_state_new(f); + mis->largest_page_size =3D qemu_ram_pagesize_largest(); postcopy_state_set(POSTCOPY_INCOMING_NONE); migrate_set_state(&mis->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_ACTIVE); --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486402802223671.7001143002041; Mon, 6 Feb 2017 09:40:02 -0800 (PST) Received: from localhost ([::1]:49872 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canGi-0000ex-P5 for importer@patchew.org; Mon, 06 Feb 2017 12:40:00 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45411) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAH-0003y8-RB for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAE-0006Qh-Ty for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:21 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43644) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAE-0006QM-MA for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:18 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DADFF4DB14 for ; Mon, 6 Feb 2017 17:33:18 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RD012611; Mon, 6 Feb 2017 12:33:17 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:32:58 +0000 Message-Id: <20170206173306.20603-9-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 06 Feb 2017 17:33:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 08/16] postcopy: Plumb pagesize down into place helpers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Now we deal with normal size pages and huge pages we need to tell the place handlers the size we're dealing with and make sure the temporary page is large enough. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- include/migration/postcopy-ram.h | 6 +++-- migration/postcopy-ram.c | 47 ++++++++++++++++++++++++------------= ---- migration/ram.c | 15 +++++++------ 3 files changed, 40 insertions(+), 28 deletions(-) diff --git a/include/migration/postcopy-ram.h b/include/migration/postcopy-= ram.h index 43bbbca..8e036b9 100644 --- a/include/migration/postcopy-ram.h +++ b/include/migration/postcopy-ram.h @@ -74,13 +74,15 @@ void postcopy_discard_send_finish(MigrationState *ms, * to use other postcopy_ routines to allocate. * returns 0 on success */ -int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m); +int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m, + size_t pagesize); =20 /* * Place a zero page at (host) atomically * returns 0 on success */ -int postcopy_place_page_zero(MigrationIncomingState *mis, void *host); +int postcopy_place_page_zero(MigrationIncomingState *mis, void *host, + size_t pagesize); =20 /* * Allocate a page of memory that can be mapped at a later point in time diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 1e3d22f..a8b7fed 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -321,7 +321,7 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingStat= e *mis) migrate_send_rp_shut(mis, qemu_file_get_error(mis->from_src_file) !=3D= 0); =20 if (mis->postcopy_tmp_page) { - munmap(mis->postcopy_tmp_page, getpagesize()); + munmap(mis->postcopy_tmp_page, mis->largest_page_size); mis->postcopy_tmp_page =3D NULL; } trace_postcopy_ram_incoming_cleanup_exit(); @@ -543,13 +543,14 @@ int postcopy_ram_enable_notify(MigrationIncomingState= *mis) * Place a host page (from) at (host) atomically * returns 0 on success */ -int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m) +int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m, + size_t pagesize) { struct uffdio_copy copy_struct; =20 copy_struct.dst =3D (uint64_t)(uintptr_t)host; copy_struct.src =3D (uint64_t)(uintptr_t)from; - copy_struct.len =3D getpagesize(); + copy_struct.len =3D pagesize; copy_struct.mode =3D 0; =20 /* copy also acks to the kernel waking the stalled thread up @@ -559,8 +560,8 @@ int postcopy_place_page(MigrationIncomingState *mis, vo= id *host, void *from) */ if (ioctl(mis->userfault_fd, UFFDIO_COPY, ©_struct)) { int e =3D errno; - error_report("%s: %s copy host: %p from: %p", - __func__, strerror(e), host, from); + error_report("%s: %s copy host: %p from: %p (size: %zd)", + __func__, strerror(e), host, from, pagesize); =20 return -e; } @@ -573,23 +574,29 @@ int postcopy_place_page(MigrationIncomingState *mis, = void *host, void *from) * Place a zero page at (host) atomically * returns 0 on success */ -int postcopy_place_page_zero(MigrationIncomingState *mis, void *host) +int postcopy_place_page_zero(MigrationIncomingState *mis, void *host, + size_t pagesize) { - struct uffdio_zeropage zero_struct; + trace_postcopy_place_page_zero(host); =20 - zero_struct.range.start =3D (uint64_t)(uintptr_t)host; - zero_struct.range.len =3D getpagesize(); - zero_struct.mode =3D 0; + if (pagesize =3D=3D getpagesize()) { + struct uffdio_zeropage zero_struct; + zero_struct.range.start =3D (uint64_t)(uintptr_t)host; + zero_struct.range.len =3D getpagesize(); + zero_struct.mode =3D 0; =20 - if (ioctl(mis->userfault_fd, UFFDIO_ZEROPAGE, &zero_struct)) { - int e =3D errno; - error_report("%s: %s zero host: %p", - __func__, strerror(e), host); + if (ioctl(mis->userfault_fd, UFFDIO_ZEROPAGE, &zero_struct)) { + int e =3D errno; + error_report("%s: %s zero host: %p", + __func__, strerror(e), host); =20 - return -e; + return -e; + } + } else { + /* TODO: The kernel can't use UFFDIO_ZEROPAGE for hugepages */ + assert(0); } =20 - trace_postcopy_place_page_zero(host); return 0; } =20 @@ -604,7 +611,7 @@ int postcopy_place_page_zero(MigrationIncomingState *mi= s, void *host) void *postcopy_get_tmp_page(MigrationIncomingState *mis) { if (!mis->postcopy_tmp_page) { - mis->postcopy_tmp_page =3D mmap(NULL, getpagesize(), + mis->postcopy_tmp_page =3D mmap(NULL, mis->largest_page_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (mis->postcopy_tmp_page =3D=3D MAP_FAILED) { @@ -649,13 +656,15 @@ int postcopy_ram_enable_notify(MigrationIncomingState= *mis) return -1; } =20 -int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m) +int postcopy_place_page(MigrationIncomingState *mis, void *host, void *fro= m, + size_t pagesize) { assert(0); return -1; } =20 -int postcopy_place_page_zero(MigrationIncomingState *mis, void *host) +int postcopy_place_page_zero(MigrationIncomingState *mis, void *host, + size_t pagesize) { assert(0); return -1; diff --git a/migration/ram.c b/migration/ram.c index 136996a..ff448ef 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2354,6 +2354,7 @@ static int ram_load_postcopy(QEMUFile *f) void *host =3D NULL; void *page_buffer =3D NULL; void *place_source =3D NULL; + RAMBlock *block =3D NULL; uint8_t ch; =20 addr =3D qemu_get_be64(f); @@ -2363,7 +2364,7 @@ static int ram_load_postcopy(QEMUFile *f) trace_ram_load_postcopy_loop((uint64_t)addr, flags); place_needed =3D false; if (flags & (RAM_SAVE_FLAG_COMPRESS | RAM_SAVE_FLAG_PAGE)) { - RAMBlock *block =3D ram_block_from_stream(f, flags); + block =3D ram_block_from_stream(f, flags); =20 host =3D host_from_ram_block_offset(block, addr); if (!host) { @@ -2438,14 +2439,14 @@ static int ram_load_postcopy(QEMUFile *f) =20 if (place_needed) { /* This gets called at the last target page in the host page */ + void *place_dest =3D host + TARGET_PAGE_SIZE - block->page_siz= e; + if (all_zero) { - ret =3D postcopy_place_page_zero(mis, - host + TARGET_PAGE_SIZE - - qemu_host_page_size); + ret =3D postcopy_place_page_zero(mis, place_dest, + block->page_size); } else { - ret =3D postcopy_place_page(mis, host + TARGET_PAGE_SIZE - - qemu_host_page_size, - place_source); + ret =3D postcopy_place_page(mis, place_dest, + place_source, block->page_size); } } if (!ret) { --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486403438283666.0807083116572; Mon, 6 Feb 2017 09:50:38 -0800 (PST) Received: from localhost ([::1]:49927 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canQz-0001E2-1Y for importer@patchew.org; Mon, 06 Feb 2017 12:50:37 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45410) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAH-0003y7-Qy for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAG-0006Qv-03 for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:21 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37760) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAF-0006Qn-RP for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:19 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0C4647E9D7 for ; Mon, 6 Feb 2017 17:33:20 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RE012611; Mon, 6 Feb 2017 12:33:19 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:32:59 +0000 Message-Id: <20170206173306.20603-10-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 06 Feb 2017 17:33:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 09/16] postcopy: Use temporary for placing zero huge pages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" The kernel can't do UFFDIO_ZEROPAGE for huge pages, so we have to allocate a temporary (always zero) page and use UFFDIO_COPYPAGE on it. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- include/migration/migration.h | 1 + migration/postcopy-ram.c | 23 +++++++++++++++++++++-- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index c9c1d5f..bd399fc 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -108,6 +108,7 @@ struct MigrationIncomingState { QEMUFile *to_src_file; QemuMutex rp_mutex; /* We send replies from multiple threads */ void *postcopy_tmp_page; + void *postcopy_tmp_zero_page; =20 QEMUBH *bh; =20 diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index a8b7fed..4c736d2 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -324,6 +324,10 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSta= te *mis) munmap(mis->postcopy_tmp_page, mis->largest_page_size); mis->postcopy_tmp_page =3D NULL; } + if (mis->postcopy_tmp_zero_page) { + munmap(mis->postcopy_tmp_zero_page, mis->largest_page_size); + mis->postcopy_tmp_zero_page =3D NULL; + } trace_postcopy_ram_incoming_cleanup_exit(); return 0; } @@ -593,8 +597,23 @@ int postcopy_place_page_zero(MigrationIncomingState *m= is, void *host, return -e; } } else { - /* TODO: The kernel can't use UFFDIO_ZEROPAGE for hugepages */ - assert(0); + /* The kernel can't use UFFDIO_ZEROPAGE for hugepages */ + if (!mis->postcopy_tmp_zero_page) { + mis->postcopy_tmp_zero_page =3D mmap(NULL, mis->largest_page_s= ize, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, + -1, 0); + if (mis->postcopy_tmp_zero_page =3D=3D MAP_FAILED) { + int e =3D errno; + mis->postcopy_tmp_zero_page =3D NULL; + error_report("%s: %s mapping large zero page", + __func__, strerror(e)); + return -e; + } + memset(mis->postcopy_tmp_zero_page, '\0', mis->largest_page_si= ze); + } + return postcopy_place_page(mis, host, mis->postcopy_tmp_zero_page, + pagesize); } =20 return 0; --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486403208761766.5449253397718; Mon, 6 Feb 2017 09:46:48 -0800 (PST) Received: from localhost ([::1]:49909 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canNF-0006bs-G0 for importer@patchew.org; Mon, 06 Feb 2017 12:46:45 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45417) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAI-0003yA-4v for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAH-0006RD-6z for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:22 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51136) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAH-0006R0-1m for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:21 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 37656C04FF82 for ; Mon, 6 Feb 2017 17:33:21 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RF012611; Mon, 6 Feb 2017 12:33:20 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:33:00 +0000 Message-Id: <20170206173306.20603-11-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 06 Feb 2017 17:33:21 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 10/16] postcopy: Load huge pages in one go X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" The existing postcopy RAM load loop already ensures that it glues together whole host-pages from the target page size chunks sent over the wire. Modify the definition of host page that it uses to be the RAM block page size and thus be huge pages where appropriate. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- migration/ram.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ff448ef..88d9444 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2342,7 +2342,7 @@ static int ram_load_postcopy(QEMUFile *f) { int flags =3D 0, ret =3D 0; bool place_needed =3D false; - bool matching_page_sizes =3D qemu_host_page_size =3D=3D TARGET_PAGE_SI= ZE; + bool matching_page_sizes =3D false; MigrationIncomingState *mis =3D migration_incoming_get_current(); /* Temporary page that is later 'placed' */ void *postcopy_host_page =3D postcopy_get_tmp_page(mis); @@ -2372,8 +2372,11 @@ static int ram_load_postcopy(QEMUFile *f) ret =3D -EINVAL; break; } + matching_page_sizes =3D block->page_size =3D=3D TARGET_PAGE_SI= ZE; /* - * Postcopy requires that we place whole host pages atomically. + * Postcopy requires that we place whole host pages atomically; + * these may be huge pages for RAMBlocks that are backed by + * hugetlbfs. * To make it atomic, the data is read into a temporary page * that's moved into place later. * The migration protocol uses, possibly smaller, target-pages @@ -2381,9 +2384,9 @@ static int ram_load_postcopy(QEMUFile *f) * of a host page in order. */ page_buffer =3D postcopy_host_page + - ((uintptr_t)host & ~qemu_host_page_mask); + ((uintptr_t)host & (block->page_size - 1)); /* If all TP are zero then we can optimise the place */ - if (!((uintptr_t)host & ~qemu_host_page_mask)) { + if (!((uintptr_t)host & (block->page_size - 1))) { all_zero =3D true; } else { /* not the 1st TP within the HP */ @@ -2401,7 +2404,7 @@ static int ram_load_postcopy(QEMUFile *f) * page */ place_needed =3D (((uintptr_t)host + TARGET_PAGE_SIZE) & - ~qemu_host_page_mask) =3D=3D 0; + (block->page_size - 1)) =3D=3D 0; place_source =3D postcopy_host_page; } last_host =3D host; --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486403111803883.550368640551; Mon, 6 Feb 2017 09:45:11 -0800 (PST) Received: from localhost ([::1]:49899 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canLi-0005Gz-H8 for importer@patchew.org; Mon, 06 Feb 2017 12:45:10 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45438) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAJ-0003yZ-C2 for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAI-0006Rj-AO for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:23 -0500 Received: from mx1.redhat.com ([209.132.183.28]:3902) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAI-0006RI-5I for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:22 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5D6114E35B for ; Mon, 6 Feb 2017 17:33:22 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RG012611; Mon, 6 Feb 2017 12:33:21 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:33:01 +0000 Message-Id: <20170206173306.20603-12-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 06 Feb 2017 17:33:22 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 11/16] postcopy: Mask fault addresses to huge page boundary X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Currently the fault address received by userfault is rounded to the host page boundary and a host page is requested from the source. Use the current RAMBlock page size instead of the general host page size so that for RAMBlocks backed by huge pages we request the whole huge page. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- include/exec/memory.h | 1 - migration/postcopy-ram.c | 7 +++---- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index 987f925..c428891 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1614,7 +1614,6 @@ MemTxResult address_space_read_continue(AddressSpace = *as, hwaddr addr, MemTxResult address_space_read_full(AddressSpace *as, hwaddr addr, MemTxAttrs attrs, uint8_t *buf, int le= n); void *qemu_map_ram_ptr(RAMBlock *ram_block, ram_addr_t addr); - static inline bool memory_access_is_direct(MemoryRegion *mr, bool is_write) { if (is_write) { diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 4c736d2..03cbd6e 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -403,7 +403,6 @@ static void *postcopy_ram_fault_thread(void *opaque) MigrationIncomingState *mis =3D opaque; struct uffd_msg msg; int ret; - size_t hostpagesize =3D getpagesize(); RAMBlock *rb =3D NULL; RAMBlock *last_rb =3D NULL; /* last RAMBlock we sent part of */ =20 @@ -470,7 +469,7 @@ static void *postcopy_ram_fault_thread(void *opaque) break; } =20 - rb_offset &=3D ~(hostpagesize - 1); + rb_offset &=3D ~(qemu_ram_pagesize(rb) - 1); trace_postcopy_ram_fault_thread_request(msg.arg.pagefault.address, qemu_ram_get_idstr(rb), rb_offset); @@ -482,11 +481,11 @@ static void *postcopy_ram_fault_thread(void *opaque) if (rb !=3D last_rb) { last_rb =3D rb; migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), - rb_offset, hostpagesize); + rb_offset, qemu_ram_pagesize(rb)); } else { /* Save some space */ migrate_send_rp_req_pages(mis, NULL, - rb_offset, hostpagesize); + rb_offset, qemu_ram_pagesize(rb)); } } trace_postcopy_ram_fault_thread_exit(); --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486403361015636.9236726595785; Mon, 6 Feb 2017 09:49:21 -0800 (PST) Received: from localhost ([::1]:49921 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canPj-0000Hs-M2 for importer@patchew.org; Mon, 06 Feb 2017 12:49:19 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45484) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAN-00045U-DJ for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAJ-0006S3-He for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:27 -0500 Received: from mx1.redhat.com ([209.132.183.28]:46204) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAJ-0006Rs-Cq for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:23 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8B2FA80475 for ; Mon, 6 Feb 2017 17:33:23 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RH012611; Mon, 6 Feb 2017 12:33:22 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:33:02 +0000 Message-Id: <20170206173306.20603-13-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 06 Feb 2017 17:33:23 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 12/16] postcopy: Send whole huge pages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" The RAM save code uses ram_save_host_page to send whole host pages at a time; change this to use the host page size associated with the RAM Block which may be a huge page. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- migration/ram.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 88d9444..2350f71 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1281,6 +1281,8 @@ static int ram_save_target_page(MigrationState *ms, Q= EMUFile *f, * offset to point into the middle of a host page * in which case the remainder of the hostpage is sent. * Only dirty target pages are sent. + * Note that the host page size may be a huge page for= this + * block. * * Returns: Number of pages written. * @@ -1299,6 +1301,8 @@ static int ram_save_host_page(MigrationState *ms, QEM= UFile *f, ram_addr_t dirty_ram_abs) { int tmppages, pages =3D 0; + size_t pagesize =3D qemu_ram_pagesize(pss->block); + do { tmppages =3D ram_save_target_page(ms, f, pss, last_stage, bytes_transferred, dirty_ram_abs); @@ -1309,7 +1313,7 @@ static int ram_save_host_page(MigrationState *ms, QEM= UFile *f, pages +=3D tmppages; pss->offset +=3D TARGET_PAGE_SIZE; dirty_ram_abs +=3D TARGET_PAGE_SIZE; - } while (pss->offset & (qemu_host_page_size - 1)); + } while (pss->offset & (pagesize - 1)); =20 /* The offset we leave with is the last one we looked at */ pss->offset -=3D TARGET_PAGE_SIZE; --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486403665211306.18507729291923; Mon, 6 Feb 2017 09:54:25 -0800 (PST) Received: from localhost ([::1]:49950 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canUd-0004DC-US for importer@patchew.org; Mon, 06 Feb 2017 12:54:24 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45458) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAL-00042S-Jo for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAK-0006SN-L9 for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:25 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43312) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAK-0006SG-Fv for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:24 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AC3AAC054903 for ; Mon, 6 Feb 2017 17:33:24 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RI012611; Mon, 6 Feb 2017 12:33:23 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:33:03 +0000 Message-Id: <20170206173306.20603-14-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 06 Feb 2017 17:33:24 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 13/16] postcopy: Allow hugepages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Allow huge pages in postcopy. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- migration/postcopy-ram.c | 25 +------------------------ 1 file changed, 1 insertion(+), 24 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 03cbd6e..6b30b43 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -85,24 +85,6 @@ static bool ufd_version_check(int ufd) } =20 /* - * Check for things that postcopy won't support; returns 0 if the block - * is fine. - */ -static int check_range(const char *block_name, void *host_addr, - ram_addr_t offset, ram_addr_t length, void *opaque) -{ - RAMBlock *rb =3D qemu_ram_block_by_name(block_name); - - if (qemu_ram_pagesize(rb) > getpagesize()) { - error_report("Postcopy doesn't support large page sizes yet (%s)", - block_name); - return -E2BIG; - } - - return 0; -} - -/* * Note: This has the side effect of munlock'ing all of RAM, that's * normally fine since if the postcopy succeeds it gets turned back on at = the * end. @@ -122,12 +104,6 @@ bool postcopy_ram_supported_by_host(void) goto out; } =20 - /* Check for anything about the RAMBlocks we don't support */ - if (qemu_ram_foreach_block(check_range, NULL)) { - /* check_range will have printed its own error */ - goto out; - } - ufd =3D syscall(__NR_userfaultfd, O_CLOEXEC); if (ufd =3D=3D -1) { error_report("%s: userfaultfd not available: %s", __func__, @@ -139,6 +115,7 @@ bool postcopy_ram_supported_by_host(void) if (!ufd_version_check(ufd)) { goto out; } + /* TODO: Only allow huge pages if the kernel supports it */ =20 /* * userfault and mlock don't go together; we'll put it back later if --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 148640360462650.60562228215326; Mon, 6 Feb 2017 09:53:24 -0800 (PST) Received: from localhost ([::1]:49945 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canTb-0003Re-8w for importer@patchew.org; Mon, 06 Feb 2017 12:53:19 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45481) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAN-00045N-Aq for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAL-0006Sm-Ve for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:27 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37786) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAL-0006SW-ND for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:25 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D586C8EB54 for ; Mon, 6 Feb 2017 17:33:25 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RJ012611; Mon, 6 Feb 2017 12:33:24 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:33:04 +0000 Message-Id: <20170206173306.20603-15-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 06 Feb 2017 17:33:25 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 14/16] postcopy: Update userfaultfd.h header X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" We use a new userfaultfd define, so update the header. (Not needed if someone just runs the update script once it's gone into the main kernel). Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- linux-headers/linux/userfaultfd.h | 81 ++++++++++++++++++++++++++++++++++-= ---- 1 file changed, 71 insertions(+), 10 deletions(-) diff --git a/linux-headers/linux/userfaultfd.h b/linux-headers/linux/userfa= ultfd.h index 19e8453..a7c1a62 100644 --- a/linux-headers/linux/userfaultfd.h +++ b/linux-headers/linux/userfaultfd.h @@ -11,13 +11,19 @@ =20 #include =20 -#define UFFD_API ((__u64)0xAA) /* - * After implementing the respective features it will become: - * #define UFFD_API_FEATURES (UFFD_FEATURE_PAGEFAULT_FLAG_WP | \ - * UFFD_FEATURE_EVENT_FORK) + * If the UFFDIO_API is upgraded someday, the UFFDIO_UNREGISTER and + * UFFDIO_WAKE ioctls should be defined as _IOW and not as _IOR. In + * userfaultfd.h we assumed the kernel was reading (instead _IOC_READ + * means the userland is reading). */ -#define UFFD_API_FEATURES (0) +#define UFFD_API ((__u64)0xAA) +#define UFFD_API_FEATURES (UFFD_FEATURE_PAGEFAULT_FLAG_WP | \ + UFFD_FEATURE_EVENT_FORK | \ + UFFD_FEATURE_EVENT_REMAP | \ + UFFD_FEATURE_EVENT_MADVDONTNEED | \ + UFFD_FEATURE_MISSING_HUGETLBFS | \ + UFFD_FEATURE_MISSING_SHMEM) #define UFFD_API_IOCTLS \ ((__u64)1 << _UFFDIO_REGISTER | \ (__u64)1 << _UFFDIO_UNREGISTER | \ @@ -25,7 +31,11 @@ #define UFFD_API_RANGE_IOCTLS \ ((__u64)1 << _UFFDIO_WAKE | \ (__u64)1 << _UFFDIO_COPY | \ - (__u64)1 << _UFFDIO_ZEROPAGE) + (__u64)1 << _UFFDIO_ZEROPAGE | \ + (__u64)1 << _UFFDIO_WRITEPROTECT) +#define UFFD_API_RANGE_IOCTLS_BASIC \ + ((__u64)1 << _UFFDIO_WAKE | \ + (__u64)1 << _UFFDIO_COPY) =20 /* * Valid ioctl command number range with this API is from 0x00 to @@ -40,6 +50,7 @@ #define _UFFDIO_WAKE (0x02) #define _UFFDIO_COPY (0x03) #define _UFFDIO_ZEROPAGE (0x04) +#define _UFFDIO_WRITEPROTECT (0x05) #define _UFFDIO_API (0x3F) =20 /* userfaultfd ioctl ids */ @@ -56,6 +67,8 @@ struct uffdio_copy) #define UFFDIO_ZEROPAGE _IOWR(UFFDIO, _UFFDIO_ZEROPAGE, \ struct uffdio_zeropage) +#define UFFDIO_WRITEPROTECT _IOWR(UFFDIO, _UFFDIO_WRITEPROTECT, \ + struct uffdio_writeprotect) =20 /* read() structure */ struct uffd_msg { @@ -72,6 +85,21 @@ struct uffd_msg { } pagefault; =20 struct { + __u32 ufd; + } fork; + + struct { + __u64 from; + __u64 to; + __u64 len; + } remap; + + struct { + __u64 start; + __u64 end; + } madv_dn; + + struct { /* unused reserved fields */ __u64 reserved1; __u64 reserved2; @@ -84,9 +112,9 @@ struct uffd_msg { * Start at 0x12 and not at 0 to be more strict against bugs. */ #define UFFD_EVENT_PAGEFAULT 0x12 -#if 0 /* not available yet */ #define UFFD_EVENT_FORK 0x13 -#endif +#define UFFD_EVENT_REMAP 0x14 +#define UFFD_EVENT_MADVDONTNEED 0x15 =20 /* flags for UFFD_EVENT_PAGEFAULT */ #define UFFD_PAGEFAULT_FLAG_WRITE (1<<0) /* If this was a write fault */ @@ -104,11 +132,37 @@ struct uffdio_api { * Note: UFFD_EVENT_PAGEFAULT and UFFD_PAGEFAULT_FLAG_WRITE * are to be considered implicitly always enabled in all kernels as * long as the uffdio_api.api requested matches UFFD_API. + * + * UFFD_FEATURE_MISSING_HUGETLBFS means an UFFDIO_REGISTER + * with UFFDIO_REGISTER_MODE_MISSING mode will succeed on + * hugetlbfs virtual memory ranges. Adding or not adding + * UFFD_FEATURE_MISSING_HUGETLBFS to uffdio_api.features has + * no real functional effect after UFFDIO_API returns, but + * it's only useful for an initial feature set probe at + * UFFDIO_API time. There are two ways to use it: + * + * 1) by adding UFFD_FEATURE_MISSING_HUGETLBFS to the + * uffdio_api.features before calling UFFDIO_API, an error + * will be returned by UFFDIO_API on a kernel without + * hugetlbfs missing support + * + * 2) the UFFD_FEATURE_MISSING_HUGETLBFS can not be added in + * uffdio_api.features and instead it will be set by the + * kernel in the uffdio_api.features if the kernel supports + * it, so userland can later check if the feature flag is + * present in uffdio_api.features after UFFDIO_API + * succeeded. + * + * UFFD_FEATURE_MISSING_SHMEM works the same as + * UFFD_FEATURE_MISSING_HUGETLBFS, but it applies to shmem + * (i.e. tmpfs and other shmem based APIs). */ -#if 0 /* not available yet */ #define UFFD_FEATURE_PAGEFAULT_FLAG_WP (1<<0) #define UFFD_FEATURE_EVENT_FORK (1<<1) -#endif +#define UFFD_FEATURE_EVENT_REMAP (1<<2) +#define UFFD_FEATURE_EVENT_MADVDONTNEED (1<<3) +#define UFFD_FEATURE_MISSING_HUGETLBFS (1<<4) +#define UFFD_FEATURE_MISSING_SHMEM (1<<5) __u64 features; =20 __u64 ioctls; @@ -164,4 +218,11 @@ struct uffdio_zeropage { __s64 zeropage; }; =20 +struct uffdio_writeprotect { + struct uffdio_range range; + /* !WP means undo writeprotect. DONTWAKE is valid only with !WP */ +#define UFFDIO_WRITEPROTECT_MODE_WP ((__u64)1<<0) +#define UFFDIO_WRITEPROTECT_MODE_DONTWAKE ((__u64)1<<1) + __u64 mode; +}; #endif /* _LINUX_USERFAULTFD_H */ --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486403869414803.6251586136298; Mon, 6 Feb 2017 09:57:49 -0800 (PST) Received: from localhost ([::1]:49974 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canXw-0006w1-40 for importer@patchew.org; Mon, 06 Feb 2017 12:57:48 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45526) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAQ-0004AD-5E for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAN-0006TG-1M for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:30 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43688) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAM-0006Sq-Rk for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:26 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0A3053DE3F for ; Mon, 6 Feb 2017 17:33:27 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RK012611; Mon, 6 Feb 2017 12:33:26 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:33:05 +0000 Message-Id: <20170206173306.20603-16-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 06 Feb 2017 17:33:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 15/16] postcopy: Check for userfault+hugepage feature X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" We need extra Linux kernel support (~4.11) to support userfaults on hugetlbfs; check for them. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- migration/postcopy-ram.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 6b30b43..102fb61 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -81,6 +81,17 @@ static bool ufd_version_check(int ufd) return false; } =20 + if (getpagesize() !=3D ram_pagesize_summary()) { + bool have_hp =3D false; + /* We've got a huge page */ +#ifdef UFFD_FEATURE_MISSING_HUGETLBFS + have_hp =3D api_struct.features & UFFD_FEATURE_MISSING_HUGETLBFS; +#endif + if (!have_hp) { + error_report("Userfault on this host does not support huge pag= es"); + return false; + } + } return true; } =20 @@ -115,7 +126,6 @@ bool postcopy_ram_supported_by_host(void) if (!ufd_version_check(ufd)) { goto out; } - /* TODO: Only allow huge pages if the kernel supports it */ =20 /* * userfault and mlock don't go together; we'll put it back later if --=20 2.9.3 From nobody Sun Apr 28 11:26:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1486403819487781.1559036560565; Mon, 6 Feb 2017 09:56:59 -0800 (PST) Received: from localhost ([::1]:49972 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canX7-0006Pz-23 for importer@patchew.org; Mon, 06 Feb 2017 12:56:57 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45528) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1canAQ-0004AF-5Q for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1canAO-0006Uj-5I for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:30 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37798) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1canAO-0006TT-0B for qemu-devel@nongnu.org; Mon, 06 Feb 2017 12:33:28 -0500 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 308927E9EF for ; Mon, 6 Feb 2017 17:33:28 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-81.ams2.redhat.com [10.36.117.81]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v16HX8RL012611; Mon, 6 Feb 2017 12:33:27 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com Date: Mon, 6 Feb 2017 17:33:06 +0000 Message-Id: <20170206173306.20603-17-dgilbert@redhat.com> In-Reply-To: <20170206173306.20603-1-dgilbert@redhat.com> References: <20170206173306.20603-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 06 Feb 2017 17:33:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 16/16] postcopy: Add doc about hugepages and postcopy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aarcange@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Laurent Vivier Tested-by: Laurent Vivier --- docs/migration.txt | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/migration.txt b/docs/migration.txt index 6503c17..b462ead 100644 --- a/docs/migration.txt +++ b/docs/migration.txt @@ -482,3 +482,16 @@ request for a page that has already been sent is ignor= ed. Duplicate requests such as this can happen as a page is sent at about the same time the destination accesses it. =20 +=3D=3D=3D Postcopy with hugepages =3D=3D=3D + +Postcopy now works with hugetlbfs backed memory: + a) The linux kernel on the destination must support userfault on hugepag= es. + b) The huge-page configuration on the source and destination VMs must be + identical; i.e. RAMBlocks on both sides must use the same page size. + c) Note that -mem-path /dev/hugepages will fall back to allocating norm= al + RAM if it doesn't have enough hugepages, triggering (b) to fail. + Using -mem-prealloc enforces the allocation using hugepages. + d) Care should be taken with the size of hugepage used; postcopy with 2MB + hugepages works well, however 1GB hugepages are likely to be problema= tic + since it takes ~1 second to transfer a 1GB hugepage across a 10Gbps l= ink, + and until the full page is transferred the destination thread is bloc= ked. --=20 2.9.3