From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676579916572.6538865228588; Wed, 28 Jun 2017 12:02:59 -0700 (PDT) Received: from localhost ([::1]:35248 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEr-0005F7-HY for importer@patchew.org; Wed, 28 Jun 2017 15:02:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40140) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDB-0004B9-JQ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQID6-0007rA-PF for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42958) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQID6-0007pF-Fq for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:08 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4535D3DBE9; Wed, 28 Jun 2017 19:01:07 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 053AE17D24; Wed, 28 Jun 2017 19:00:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 4535D3DBE9 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 4535D3DBE9 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:19 +0100 Message-Id: <20170628190047.26159-2-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 28 Jun 2017 19:01:07 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 01/29] RAMBlock/migration: Add migration flags X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Add a migration flags field to each RAMBlock so that the migration code can hold a set of private flags on the RAMBlock. Add accessors. Signed-off-by: Dr. David Alan Gilbert --- exec.c | 10 ++++++++++ include/exec/cpu-common.h | 2 ++ include/exec/ram_addr.h | 2 ++ 3 files changed, 14 insertions(+) diff --git a/exec.c b/exec.c index 42ad1eaedd..69fc5c9b07 100644 --- a/exec.c +++ b/exec.c @@ -1741,6 +1741,16 @@ size_t qemu_ram_pagesize_largest(void) return largest; } =20 +uint32_t qemu_ram_get_migration_flags(const RAMBlock *rb) +{ + return rb->migration_flags; +} + +void qemu_ram_set_migration_flags(RAMBlock *rb, uint32_t flags) +{ + rb->migration_flags =3D flags; +} + static int memory_try_enable_merging(void *addr, size_t len) { if (!machine_mem_merge(current_machine)) { diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index 4d45a72ea9..4af179b543 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -72,6 +72,8 @@ const char *qemu_ram_get_idstr(RAMBlock *rb); bool qemu_ram_is_shared(RAMBlock *rb); size_t qemu_ram_pagesize(RAMBlock *block); size_t qemu_ram_pagesize_largest(void); +void qemu_ram_set_migration_flags(RAMBlock *rb, uint32_t flags); +uint32_t qemu_ram_get_migration_flags(const RAMBlock *rb); =20 void cpu_physical_memory_rw(hwaddr addr, uint8_t *buf, int len, int is_write); diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h index af5bf26080..0cb6c5cb73 100644 --- a/include/exec/ram_addr.h +++ b/include/exec/ram_addr.h @@ -32,6 +32,8 @@ struct RAMBlock { ram_addr_t max_length; void (*resized)(const char*, uint64_t length, void *host); uint32_t flags; + /* These flags are owned by migration, initialised to 0 */ + uint32_t migration_flags; /* Protected by iothread lock. */ char idstr[256]; /* RCU-enabled, writes protected by the ramlist lock */ --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676599087501.0713282347431; Wed, 28 Jun 2017 12:03:19 -0700 (PDT) Received: from localhost ([::1]:35251 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIFB-0005XE-Ll for importer@patchew.org; Wed, 28 Jun 2017 15:03:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40181) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDM-0004Iq-QO for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDH-0007yE-Uj for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45556) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDH-0007y0-LX for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:19 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6F2E230AF51; Wed, 28 Jun 2017 19:01:18 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2167C17D5D; Wed, 28 Jun 2017 19:01:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6F2E230AF51 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 6F2E230AF51 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:20 +0100 Message-Id: <20170628190047.26159-3-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 28 Jun 2017 19:01:18 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 02/29] migrate: Update ram_block_discard_range for shared X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" The choice of call to discard a block is getting more complicated for other cases. We use fallocate PUNCH_HOLE in any file cases; it works for both hugepage and for tmpfs. We use the DONTNEED for non-hugepage cases either where they're anonymous or where they're private. Care should be taken when trying other backing files. Signed-off-by: Dr. David Alan Gilbert --- exec.c | 28 ++++++++++++++++------------ trace-events | 3 +++ 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/exec.c b/exec.c index 69fc5c9b07..4e61226a16 100644 --- a/exec.c +++ b/exec.c @@ -3557,6 +3557,7 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t st= art, size_t length) } =20 if ((start + length) <=3D rb->used_length) { + bool need_madvise, need_fallocate; uint8_t *host_endaddr =3D host_startaddr + length; if ((uintptr_t)host_endaddr & (rb->page_size - 1)) { error_report("ram_block_discard_range: Unaligned end address: = %p", @@ -3566,23 +3567,26 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t = start, size_t length) =20 errno =3D ENOTSUP; /* If we are missing MADVISE etc */ =20 - if (rb->page_size =3D=3D qemu_host_page_size) { -#if defined(CONFIG_MADVISE) - /* Note: We need the madvise MADV_DONTNEED behaviour of defini= tely - * freeing the page. - */ - ret =3D madvise(host_startaddr, length, MADV_DONTNEED); -#endif - } else { - /* Huge page case - unfortunately it can't do DONTNEED, but - * it can do the equivalent by FALLOC_FL_PUNCH_HOLE in the - * huge page file. - */ + /* The logic here is messy; + * madvise DONTNEED fails for hugepages + * fallocate works on hugepages and shmem + */ + need_madvise =3D (rb->page_size =3D=3D qemu_host_page_size) && + (rb->fd =3D=3D -1 || !(rb->flags & RAM_SHARED)); + need_fallocate =3D rb->fd !=3D -1; + if (ret =3D=3D -1 && need_fallocate) { #ifdef CONFIG_FALLOCATE_PUNCH_HOLE ret =3D fallocate(rb->fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEE= P_SIZE, start, length); #endif } + if (need_madvise && (!need_fallocate || (ret =3D=3D 0))) { +#if defined(CONFIG_MADVISE) + ret =3D madvise(host_startaddr, length, MADV_DONTNEED); +#endif + } + trace_ram_block_discard_range(rb->idstr, host_startaddr, + need_madvise, need_fallocate, ret); if (ret) { ret =3D -errno; error_report("ram_block_discard_range: Failed to discard range= " diff --git a/trace-events b/trace-events index bae63fdb1d..2c6e8d2160 100644 --- a/trace-events +++ b/trace-events @@ -55,6 +55,9 @@ dma_complete(void *dbs, int ret, void *cb) "dbs=3D%p ret= =3D%d cb=3D%p" dma_blk_cb(void *dbs, int ret) "dbs=3D%p ret=3D%d" dma_map_wait(void *dbs) "dbs=3D%p" =20 +# exec.c +ram_block_discard_range(const char *rbname, void *hva, bool need_madvise, = bool need_fallocate, int ret) "%s@%p: madvise: %d fallocate: %d ret: %d" + # memory.c memory_region_ops_read(int cpu_index, void *mr, uint64_t addr, uint64_t va= lue, unsigned size) "cpu %d mr %p addr %#"PRIx64" value %#"PRIx64" size %u" memory_region_ops_write(int cpu_index, void *mr, uint64_t addr, uint64_t v= alue, unsigned size) "cpu %d mr %p addr %#"PRIx64" value %#"PRIx64" size %u" --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676753417227.50580226008287; Wed, 28 Jun 2017 12:05:53 -0700 (PDT) Received: from localhost ([::1]:35262 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIHe-00086x-Td for importer@patchew.org; Wed, 28 Jun 2017 15:05:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40180) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDM-0004Ip-Q7 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDK-0007yt-BD for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43336) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDK-0007yg-4u for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:22 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AE9483B703; Wed, 28 Jun 2017 19:01:20 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id AFED64F9A6; Wed, 28 Jun 2017 19:01:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com AE9483B703 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com AE9483B703 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:21 +0100 Message-Id: <20170628190047.26159-4-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 28 Jun 2017 19:01:21 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 03/29] qemu_ram_block_host_offset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Utility to give the offset of a host pointer within a RAMBlock (assuming we already know it's in that RAMBlock) Signed-off-by: Dr. David Alan Gilbert --- exec.c | 6 ++++++ include/exec/cpu-common.h | 1 + 2 files changed, 7 insertions(+) diff --git a/exec.c b/exec.c index 4e61226a16..a1499b9bee 100644 --- a/exec.c +++ b/exec.c @@ -2218,6 +2218,12 @@ static void *qemu_ram_ptr_length(RAMBlock *ram_block= , ram_addr_t addr, return ramblock_ptr(block, addr); } =20 +/* Return the offset of a hostpointer within a ramblock */ +ram_addr_t qemu_ram_block_host_offset(RAMBlock *rb, void *host) +{ + return (uint8_t *)host - (uint8_t *)rb->host; +} + /* * Translates a host ptr back to a RAMBlock, a ram_addr and an offset * in that RAMBlock. diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index 4af179b543..fa1ec22d66 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -66,6 +66,7 @@ ram_addr_t qemu_ram_addr_from_host(void *ptr); RAMBlock *qemu_ram_block_by_name(const char *name); RAMBlock *qemu_ram_block_from_host(void *ptr, bool round_offset, ram_addr_t *offset); +ram_addr_t qemu_ram_block_host_offset(RAMBlock *rb, void *host); void qemu_ram_set_idstr(RAMBlock *block, const char *name, DeviceState *de= v); void qemu_ram_unset_idstr(RAMBlock *block); const char *qemu_ram_get_idstr(RAMBlock *rb); --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676601294293.5097751206205; Wed, 28 Jun 2017 12:03:21 -0700 (PDT) Received: from localhost ([::1]:35250 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIFC-0005Wr-NY for importer@patchew.org; Wed, 28 Jun 2017 15:03:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40179) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDM-0004Io-Q5 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDL-0007zg-WC for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56429) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDL-0007zQ-Qu for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:23 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D64AF4E884; Wed, 28 Jun 2017 19:01:22 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id D4E804F9B5; Wed, 28 Jun 2017 19:01:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D64AF4E884 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D64AF4E884 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:22 +0100 Message-Id: <20170628190047.26159-5-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 28 Jun 2017 19:01:23 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 04/29] migration/ram: ramblock_recv_bitmap_test_byte_offset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Utility for testing the map when you already know the offset in the RAMBlock. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 5 +++++ migration/ram.h | 1 + 2 files changed, 6 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 3daa69b2c3..6dcb9e8b43 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -165,6 +165,11 @@ int ramblock_recv_bitmap_test(void *host_addr, RAMBloc= k *rb) rb->receivedmap); } =20 +bool ramblock_recv_bitmap_test_byte_offset(uint64_t byte_offset, RAMBlock = *rb) +{ + return test_bit(byte_offset >> TARGET_PAGE_BITS, rb->receivedmap); +} + void ramblock_recv_bitmap_set(void *host_addr, RAMBlock *rb) { set_bit_atomic(ramblock_recv_bitmap_offset(host_addr, rb), rb->receive= dmap); diff --git a/migration/ram.h b/migration/ram.h index 98d68df03d..0fcf85d904 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -55,6 +55,7 @@ void ram_handle_compressed(void *host, uint8_t ch, uint64= _t size); =20 void ramblock_recv_map_init(void); int ramblock_recv_bitmap_test(void *host_addr, RAMBlock *rb); +bool ramblock_recv_bitmap_test_byte_offset(uint64_t byte_offset, RAMBlock = *rb); void ramblock_recv_bitmap_set(void *host_addr, RAMBlock *rb); void ramblock_recv_bitmap_clear(void *host_addr, RAMBlock *rb); =20 --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676905038462.0127164568594; Wed, 28 Jun 2017 12:08:25 -0700 (PDT) Received: from localhost ([::1]:35272 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIK5-0001yf-Ie for importer@patchew.org; Wed, 28 Jun 2017 15:08:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40238) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDR-0004N3-6c for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDQ-00081i-DV for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34241) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDQ-00081T-7B for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:28 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D8010C0567B3; Wed, 28 Jun 2017 19:01:26 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2A2CA17D24; Wed, 28 Jun 2017 19:01:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D8010C0567B3 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D8010C0567B3 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:23 +0100 Message-Id: <20170628190047.26159-6-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 28 Jun 2017 19:01:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 05/29] postcopy: use UFFDIO_ZEROPAGE only when available X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Use the recently added migration flag to hold whether each RAMBlock has the UFFDIO_ZEROPAGE capability, use it when it's available. This allows the use of postcopy on tmpfs as well as hugepage backed files. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.h | 4 ++++ migration/postcopy-ram.c | 12 +++++++++--- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index d9a268a3af..d109635d08 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -20,6 +20,10 @@ #include "exec/cpu-common.h" #include "qemu/coroutine_int.h" =20 +/* Migration flags to be set using qemu_ram_set_migration_flags */ +/* Postcopy can atomically zero pages in this RAMBlock */ +#define QEMU_MIGFLAG_POSTCOPY_ZERO 0x00000001 + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index be2a8f8e02..96338a8070 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -408,6 +408,12 @@ static int ram_block_enable_notify(const char *block_n= ame, void *host_addr, error_report("%s userfault: Region doesn't support COPY", __func__= ); return -1; } + if (reg_struct.ioctls & ((__u64)1 << _UFFDIO_ZEROPAGE)) { + RAMBlock *rb =3D qemu_ram_block_by_name(block_name); + qemu_ram_set_migration_flags(rb, qemu_ram_get_migration_flags(rb) | + QEMU_MIGFLAG_POSTCOPY_ZERO); + } + =20 return 0; } @@ -620,11 +626,11 @@ int postcopy_place_page(MigrationIncomingState *mis, = void *host, void *from, int postcopy_place_page_zero(MigrationIncomingState *mis, void *host, RAMBlock *rb) { + size_t pagesize =3D qemu_ram_pagesize(rb); trace_postcopy_place_page_zero(host); =20 - if (qemu_ram_pagesize(rb) =3D=3D getpagesize()) { - if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, NULL, getpagesize= (), - rb)) { + if (qemu_ram_get_migration_flags(rb) & QEMU_MIGFLAG_POSTCOPY_ZERO) { + if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, NULL, pagesize, r= b)) { int e =3D errno; error_report("%s: %s zero host: %p", __func__, strerror(e), host); --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676712888603.0015417864048; Wed, 28 Jun 2017 12:05:12 -0700 (PDT) Received: from localhost ([::1]:35256 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIH1-0007Hs-Ii for importer@patchew.org; Wed, 28 Jun 2017 15:05:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40298) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDY-0004SJ-38 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDV-00083p-O9 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34866) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDV-00083a-FI for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:33 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5820F74859; Wed, 28 Jun 2017 19:01:32 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF7474F9A6; Wed, 28 Jun 2017 19:01:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5820F74859 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 5820F74859 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:24 +0100 Message-Id: <20170628190047.26159-7-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 28 Jun 2017 19:01:32 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 06/29] postcopy: Add notifier chain X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Add a notifier chain for postcopy with a 'reason' flag and an opportunity for a notifier member to return an error. Call it when enabling postcopy. This will initially used to enable devices to declare they're unable to postcopy and later to notify of devices of stages within postcopy. Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 41 +++++++++++++++++++++++++++++++++++++++++ migration/postcopy-ram.h | 26 ++++++++++++++++++++++++++ vl.c | 4 +++- 3 files changed, 70 insertions(+), 1 deletion(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 96338a8070..64f5a8b003 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -23,6 +23,8 @@ #include "savevm.h" #include "postcopy-ram.h" #include "ram.h" +#include "qapi/error.h" +#include "qemu/notify.h" #include "sysemu/sysemu.h" #include "sysemu/balloon.h" #include "qemu/error-report.h" @@ -45,6 +47,38 @@ struct PostcopyDiscardState { unsigned int nsentcmds; }; =20 +/* A notifier chain for postcopy + * The notifier should return 0 if it's OK, or a + * -errno on error. + * The notifier should expect an Error ** as it's data + */ +static NotifierWithReturnList postcopy_notifier_list; + +void postcopy_infrastructure_init(void) +{ + notifier_with_return_list_init(&postcopy_notifier_list); +} + +void postcopy_add_notifier(NotifierWithReturn *nn) +{ + notifier_with_return_list_add(&postcopy_notifier_list, nn); +} + +void postcopy_remove_notifier(NotifierWithReturn *n) +{ + notifier_with_return_remove(n); +} + +int postcopy_notify(enum PostcopyNotifyReason reason, Error **errp) +{ + struct PostcopyNotifyData pnd; + pnd.reason =3D reason; + pnd.errp =3D errp; + + return notifier_with_return_list_notify(&postcopy_notifier_list, + &pnd); +} + /* Postcopy needs to detect accesses to pages that haven't yet been copied * across, and efficiently map new pages in, the techniques for doing this * are target OS specific. @@ -133,6 +167,7 @@ bool postcopy_ram_supported_by_host(void) struct uffdio_register reg_struct; struct uffdio_range range_struct; uint64_t feature_mask; + Error *local_err =3D NULL; =20 if (qemu_target_page_size() > pagesize) { error_report("Target page size bigger than host page size"); @@ -146,6 +181,12 @@ bool postcopy_ram_supported_by_host(void) goto out; } =20 + /* Give devices a chance to object */ + if (postcopy_notify(POSTCOPY_NOTIFY_PROBE, &local_err)) { + error_report_err(local_err); + goto out; + } + /* Version and features check */ if (!ufd_version_check(ufd)) { goto out; diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 78a3591322..d688411674 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -114,4 +114,30 @@ PostcopyState postcopy_state_get(void); /* Set the state and return the old state */ PostcopyState postcopy_state_set(PostcopyState new_state); =20 +/* + * To be called once at the start before any device initialisation + */ +void postcopy_infrastructure_init(void); + +/* Add a notifier to a list to be called when checking whether the devices + * can support postcopy. + * It's data is a *PostcopyNotifyData + * It should return 0 if OK, or a negative value on failure. + * On failure it must set the data->errp to an error. + * + */ +enum PostcopyNotifyReason { + POSTCOPY_NOTIFY_PROBE =3D 0, +}; + +struct PostcopyNotifyData { + enum PostcopyNotifyReason reason; + Error **errp; +}; + +void postcopy_add_notifier(NotifierWithReturn *nn); +void postcopy_remove_notifier(NotifierWithReturn *n); +/* Call the notifier list set by postcopy_add_start_notifier */ +int postcopy_notify(enum PostcopyNotifyReason reason, Error **errp); + #endif diff --git a/vl.c b/vl.c index a2bd69f4e0..b6c660a703 100644 --- a/vl.c +++ b/vl.c @@ -93,8 +93,9 @@ int main(int argc, char **argv) #include "sysemu/dma.h" #include "hw/audio/soundhw.h" #include "audio/audio.h" -#include "sysemu/cpus.h" #include "migration/colo.h" +#include "migration/postcopy-ram.h" +#include "sysemu/cpus.h" #include "sysemu/kvm.h" #include "sysemu/hax.h" #include "qapi/qobject-input-visitor.h" @@ -3060,6 +3061,7 @@ int main(int argc, char **argv, char **envp) module_call_init(MODULE_INIT_OPTS); =20 runstate_init(); + postcopy_infrastructure_init(); =20 if (qcrypto_init(&err) < 0) { error_reportf_err(err, "cannot initialize crypto: "); --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149867673911617.78684458861551; Wed, 28 Jun 2017 12:05:39 -0700 (PDT) Received: from localhost ([::1]:35260 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIHP-0007ms-Hp for importer@patchew.org; Wed, 28 Jun 2017 15:05:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40350) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDc-0004Vk-BG for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDb-00086K-Fc for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45058) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDb-00085x-69 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:39 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E17E94E4F3; Wed, 28 Jun 2017 19:01:37 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9331A17D24; Wed, 28 Jun 2017 19:01:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com E17E94E4F3 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com E17E94E4F3 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:25 +0100 Message-Id: <20170628190047.26159-8-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 28 Jun 2017 19:01:38 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 07/29] postcopy: Add vhost-user flag for postcopy and check it X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Add a vhost feature flag for postcopy support, and use the postcopy notifier to check it before allowing postcopy. Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.h | 1 + hw/virtio/vhost-user.c | 39 +++++++++++++++++++++++++++++++= +++- 2 files changed, 39 insertions(+), 1 deletion(-) diff --git a/contrib/libvhost-user/libvhost-user.h b/contrib/libvhost-user/= libvhost-user.h index 53ef222c0b..82a6bf4549 100644 --- a/contrib/libvhost-user/libvhost-user.h +++ b/contrib/libvhost-user/libvhost-user.h @@ -34,6 +34,7 @@ enum VhostUserProtocolFeature { VHOST_USER_PROTOCOL_F_MQ =3D 0, VHOST_USER_PROTOCOL_F_LOG_SHMFD =3D 1, VHOST_USER_PROTOCOL_F_RARP =3D 2, + VHOST_USER_PROTOCOL_F_POSTCOPY =3D 6, =20 VHOST_USER_PROTOCOL_F_MAX }; diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 958ee09bcb..78eaf9b022 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -17,6 +17,8 @@ #include "sysemu/kvm.h" #include "qemu/error-report.h" #include "qemu/sockets.h" +#include "migration/migration.h" +#include "migration/postcopy-ram.h" =20 #include #include @@ -33,7 +35,7 @@ enum VhostUserProtocolFeature { VHOST_USER_PROTOCOL_F_REPLY_ACK =3D 3, VHOST_USER_PROTOCOL_F_NET_MTU =3D 4, VHOST_USER_PROTOCOL_F_SLAVE_REQ =3D 5, - + VHOST_USER_PROTOCOL_F_POSTCOPY =3D 6, VHOST_USER_PROTOCOL_F_MAX }; =20 @@ -121,8 +123,10 @@ static VhostUserMsg m __attribute__ ((unused)); #define VHOST_USER_VERSION (0x1) =20 struct vhost_user { + struct vhost_dev *dev; CharBackend *chr; int slave_fd; + NotifierWithReturn postcopy_notifier; }; =20 static bool ioeventfd_enabled(void) @@ -701,6 +705,33 @@ out: return ret; } =20 +static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier, + void *opaque) +{ + struct PostcopyNotifyData *pnd =3D opaque; + struct vhost_user *u =3D container_of(notifier, struct vhost_user, + postcopy_notifier); + struct vhost_dev *dev =3D u->dev; + + switch (pnd->reason) { + case POSTCOPY_NOTIFY_PROBE: + if (!virtio_has_feature(dev->protocol_features, + VHOST_USER_PROTOCOL_F_POSTCOPY)) { + /* TODO: Get the device name into this error somehow */ + error_setg(pnd->errp, + "vhost-user backend not capable of postcopy"); + return -ENOENT; + } + break; + + default: + /* We ignore notifications we don't know */ + break; + } + + return 0; +} + static int vhost_user_init(struct vhost_dev *dev, void *opaque) { uint64_t features, protocol_features; @@ -712,6 +743,7 @@ static int vhost_user_init(struct vhost_dev *dev, void = *opaque) u =3D g_new0(struct vhost_user, 1); u->chr =3D opaque; u->slave_fd =3D -1; + u->dev =3D dev; dev->opaque =3D u; =20 err =3D vhost_user_get_features(dev, &features); @@ -763,11 +795,15 @@ static int vhost_user_init(struct vhost_dev *dev, voi= d *opaque) "VHOST_USER_PROTOCOL_F_LOG_SHMFD feature."); } =20 + u->postcopy_notifier.notify =3D vhost_user_postcopy_notifier; + err =3D vhost_setup_slave_channel(dev); if (err < 0) { return err; } =20 + postcopy_add_notifier(&u->postcopy_notifier); + return 0; } =20 @@ -778,6 +814,7 @@ static int vhost_user_cleanup(struct vhost_dev *dev) assert(dev->vhost_ops->backend_type =3D=3D VHOST_BACKEND_TYPE_USER); =20 u =3D dev->opaque; + postcopy_remove_notifier(&u->postcopy_notifier); if (u->slave_fd >=3D 0) { close(u->slave_fd); u->slave_fd =3D -1; --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676755280285.18377405666615; Wed, 28 Jun 2017 12:05:55 -0700 (PDT) Received: from localhost ([::1]:35263 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIHg-0008Cm-Vt for importer@patchew.org; Wed, 28 Jun 2017 15:05:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40430) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDj-0004d2-DZ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDe-000880-Hl for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44052) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDe-00087L-9T for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:42 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0F8B93D970; Wed, 28 Jun 2017 19:01:41 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2F2334F9A6; Wed, 28 Jun 2017 19:01:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 0F8B93D970 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 0F8B93D970 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:26 +0100 Message-Id: <20170628190047.26159-9-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 28 Jun 2017 19:01:41 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 08/29] vhost-user: Add 'VHOST_USER_POSTCOPY_ADVISE' message X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Wire up a notifier to send a VHOST_USER_POSTCOPY_ADVISE message on an incoming advise. Later patches will fill in the behaviour/contents of the message. Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.c | 14 ++++++++++ contrib/libvhost-user/libvhost-user.h | 1 + hw/virtio/vhost-user.c | 48 +++++++++++++++++++++++++++++++= ++++ migration/postcopy-ram.h | 1 + migration/savevm.c | 6 +++++ 5 files changed, 70 insertions(+) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index 9efb9dac0e..ca7af5453a 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -63,6 +63,7 @@ vu_request_to_string(int req) REQ(VHOST_USER_SET_VRING_ENABLE), REQ(VHOST_USER_SEND_RARP), REQ(VHOST_USER_INPUT_GET_CONFIG), + REQ(VHOST_USER_POSTCOPY_ADVISE), REQ(VHOST_USER_MAX), }; #undef REQ @@ -744,6 +745,17 @@ vu_set_vring_enable_exec(VuDev *dev, VhostUserMsg *vms= g) } =20 static bool +vu_set_postcopy_advise(VuDev *dev, VhostUserMsg *vmsg) +{ + /* TODO: Open ufd, pass it back in the request + /* TODO: Add addresses=20 + */ + vmsg->payload.u64 =3D 0xcafe; + vmsg->size =3D sizeof(vmsg->payload.u64); + return true; /* =3D send a reply */ +} + +static bool vu_process_message(VuDev *dev, VhostUserMsg *vmsg) { int do_reply =3D 0; @@ -806,6 +818,8 @@ vu_process_message(VuDev *dev, VhostUserMsg *vmsg) return vu_get_queue_num_exec(dev, vmsg); case VHOST_USER_SET_VRING_ENABLE: return vu_set_vring_enable_exec(dev, vmsg); + case VHOST_USER_POSTCOPY_ADVISE: + return vu_set_postcopy_advise(dev, vmsg); default: vmsg_close_fds(vmsg); vu_panic(dev, "Unhandled request: %d", vmsg->request); diff --git a/contrib/libvhost-user/libvhost-user.h b/contrib/libvhost-user/= libvhost-user.h index 82a6bf4549..8bb35582ea 100644 --- a/contrib/libvhost-user/libvhost-user.h +++ b/contrib/libvhost-user/libvhost-user.h @@ -63,6 +63,7 @@ typedef enum VhostUserRequest { VHOST_USER_SET_VRING_ENABLE =3D 18, VHOST_USER_SEND_RARP =3D 19, VHOST_USER_INPUT_GET_CONFIG =3D 20, + VHOST_USER_POSTCOPY_ADVISE =3D 23, VHOST_USER_MAX } VhostUserRequest; =20 diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 78eaf9b022..ee9a1ac8a3 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -65,6 +65,7 @@ typedef enum VhostUserRequest { VHOST_USER_NET_SET_MTU =3D 20, VHOST_USER_SET_SLAVE_REQ_FD =3D 21, VHOST_USER_IOTLB_MSG =3D 22, + VHOST_USER_POSTCOPY_ADVISE =3D 23, VHOST_USER_MAX } VhostUserRequest; =20 @@ -705,6 +706,50 @@ out: return ret; } =20 +/* + * Called at the start of an inbound postcopy on reception of the + * 'advise' command. + */ +static int vhost_user_postcopy_advise(struct vhost_dev *dev, Error **errp) +{ + struct vhost_user *u =3D dev->opaque; + CharBackend *chr =3D u->chr; + int ufd; + VhostUserMsg msg =3D { + .request =3D VHOST_USER_POSTCOPY_ADVISE, + .flags =3D VHOST_USER_VERSION, + }; + + if (vhost_user_write(dev, &msg, NULL, 0) < 0) { + error_setg(errp, "Failed to send postcopy_advise to vhost"); + return -1; + } + + if (vhost_user_read(dev, &msg) < 0) { + error_setg(errp, "Failed to get postcopy_advise reply from vhost"); + return -1; + } + + if (msg.request !=3D VHOST_USER_POSTCOPY_ADVISE) { + error_setg(errp, "Unexpected msg type. Expected %d received %d", + VHOST_USER_POSTCOPY_ADVISE, msg.request); + return -1; + } + + if (msg.size !=3D sizeof(msg.payload.u64)) { + error_setg(errp, "Received bad msg size."); + return -1; + } + ufd =3D qemu_chr_fe_get_msgfd(chr); + if (ufd < 0) { + error_setg(errp, "%s: Failed to get ufd", __func__); + return -1; + } + + /* TODO: register ufd with userfault thread */ + return 0; +} + static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier, void *opaque) { @@ -724,6 +769,9 @@ static int vhost_user_postcopy_notifier(NotifierWithRet= urn *notifier, } break; =20 + case POSTCOPY_NOTIFY_INBOUND_ADVISE: + return vhost_user_postcopy_advise(dev, pnd->errp); + default: /* We ignore notifications we don't know */ break; diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index d688411674..70d4b09659 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -128,6 +128,7 @@ void postcopy_infrastructure_init(void); */ enum PostcopyNotifyReason { POSTCOPY_NOTIFY_PROBE =3D 0, + POSTCOPY_NOTIFY_INBOUND_ADVISE, }; =20 struct PostcopyNotifyData { diff --git a/migration/savevm.c b/migration/savevm.c index a12637ef76..4977baf9b7 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1365,6 +1365,7 @@ static int loadvm_postcopy_handle_advise(MigrationInc= omingState *mis) { PostcopyState ps =3D postcopy_state_set(POSTCOPY_INCOMING_ADVISE); uint64_t remote_pagesize_summary, local_pagesize_summary, remote_tps; + Error *local_err =3D NULL; =20 trace_loadvm_postcopy_handle_advise(); if (ps !=3D POSTCOPY_INCOMING_NONE) { @@ -1412,6 +1413,11 @@ static int loadvm_postcopy_handle_advise(MigrationIn= comingState *mis) return -1; } =20 + if (postcopy_notify(POSTCOPY_NOTIFY_INBOUND_ADVISE, &local_err)) { + error_report_err(local_err); + return -1; + } + if (ram_postcopy_incoming_init(mis)) { return -1; } --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677055758938.6820825081693; Wed, 28 Jun 2017 12:10:55 -0700 (PDT) Received: from localhost ([::1]:35286 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIMW-0004mP-6i for importer@patchew.org; Wed, 28 Jun 2017 15:10:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40429) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDj-0004d0-DU for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDf-00088O-G6 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52382) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDf-000886-8M for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:43 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 343118553E; Wed, 28 Jun 2017 19:01:42 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4E82617D32; Wed, 28 Jun 2017 19:01:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 343118553E Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 343118553E From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:27 +0100 Message-Id: <20170628190047.26159-10-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 28 Jun 2017 19:01:42 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 09/29] vhub: Support sending fds back to qemu X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Allow replies with fds (for postcopy) Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.c | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index ca7af5453a..e3a32755cf 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -214,6 +214,30 @@ vu_message_write(VuDev *dev, int conn_fd, VhostUserMsg= *vmsg) { int rc; uint8_t *p =3D (uint8_t *)vmsg; + char control[CMSG_SPACE(VHOST_MEMORY_MAX_NREGIONS * sizeof(int))] =3D = { }; + struct iovec iov =3D { + .iov_base =3D (char *)vmsg, + .iov_len =3D VHOST_USER_HDR_SIZE, + }; + struct msghdr msg =3D { + .msg_iov =3D &iov, + .msg_iovlen =3D 1, + .msg_control =3D control, + }; + struct cmsghdr *cmsg; + + memset(control, 0, sizeof(control)); + if (vmsg->fds) { + size_t fdsize =3D vmsg->fd_num * sizeof(int); + msg.msg_controllen =3D CMSG_SPACE(fdsize); + cmsg =3D CMSG_FIRSTHDR(&msg); + cmsg->cmsg_len =3D CMSG_LEN(fdsize); + cmsg->cmsg_level =3D SOL_SOCKET; + cmsg->cmsg_type =3D SCM_RIGHTS; + memcpy(CMSG_DATA(cmsg), vmsg->fds, fdsize); + } else { + msg.msg_controllen =3D 0; + } =20 /* Set the version in the flags when sending the reply */ vmsg->flags &=3D ~VHOST_USER_VERSION_MASK; @@ -221,7 +245,7 @@ vu_message_write(VuDev *dev, int conn_fd, VhostUserMsg = *vmsg) vmsg->flags |=3D VHOST_USER_REPLY_MASK; =20 do { - rc =3D write(conn_fd, p, VHOST_USER_HDR_SIZE); + rc =3D sendmsg(conn_fd, &msg, 0); } while (rc < 0 && (errno =3D=3D EINTR || errno =3D=3D EAGAIN)); =20 do { @@ -314,6 +338,7 @@ vu_get_features_exec(VuDev *dev, VhostUserMsg *vmsg) } =20 vmsg->size =3D sizeof(vmsg->payload.u64); + vmsg->fd_num =3D 0; =20 DPRINT("Sending back to guest u64: 0x%016"PRIx64"\n", vmsg->payload.u6= 4); =20 @@ -455,6 +480,7 @@ vu_set_log_base_exec(VuDev *dev, VhostUserMsg *vmsg) dev->log_size =3D log_mmap_size; =20 vmsg->size =3D sizeof(vmsg->payload.u64); + vmsg->fd_num =3D 0; =20 return true; } @@ -699,6 +725,7 @@ vu_get_protocol_features_exec(VuDev *dev, VhostUserMsg = *vmsg) =20 vmsg->payload.u64 =3D features; vmsg->size =3D sizeof(vmsg->payload.u64); + vmsg->fd_num =3D 0; =20 return true; } --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676866513942.2917269436958; Wed, 28 Jun 2017 12:07:46 -0700 (PDT) Received: from localhost ([::1]:35270 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIJQ-0001Mc-Uu for importer@patchew.org; Wed, 28 Jun 2017 15:07:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40428) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDj-0004cz-DH for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDi-0008Al-3n for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43112) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDh-0008AF-Tq for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:46 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C51BAC04B31E; Wed, 28 Jun 2017 19:01:44 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6EEE817D32; Wed, 28 Jun 2017 19:01:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C51BAC04B31E Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C51BAC04B31E From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:28 +0100 Message-Id: <20170628190047.26159-11-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 28 Jun 2017 19:01:44 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 10/29] vhub: Open userfaultfd X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Open a userfaultfd (on a postcopy_advise) and send it back in the reply to the qemu for it to monitor. Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.c | 24 +++++++++++++++++++++--- contrib/libvhost-user/libvhost-user.h | 3 +++ 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index e3a32755cf..62e97f6b84 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -15,6 +15,7 @@ =20 #include #include +#include #include =20 #include "qemu/atomic.h" @@ -774,11 +775,28 @@ vu_set_vring_enable_exec(VuDev *dev, VhostUserMsg *vm= sg) static bool vu_set_postcopy_advise(VuDev *dev, VhostUserMsg *vmsg) { - /* TODO: Open ufd, pass it back in the request - /* TODO: Add addresses=20 - */ + struct uffdio_api api_struct; + + dev->postcopy_ufd =3D syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK= ); + /* TODO: Add addresses */ vmsg->payload.u64 =3D 0xcafe; vmsg->size =3D sizeof(vmsg->payload.u64); + + if (dev->postcopy_ufd =3D=3D -1) { + vu_panic(dev, "Userfaultfd not available: %s", strerror(errno)); + return false; + } + api_struct.api =3D UFFD_API; + api_struct.features =3D 0; + if (ioctl(dev->postcopy_ufd, UFFDIO_API, &api_struct)) { + vu_panic(dev, "Failed UFFDIO_API: %s", strerror(errno)); + close(dev->postcopy_ufd); + return false; + } + /* TODO: Stash feature flags somewhere */ + /* Return a ufd to the QEMU */ + vmsg->fd_num =3D 1; + vmsg->fds[0] =3D dev->postcopy_ufd; return true; /* =3D send a reply */ } =20 diff --git a/contrib/libvhost-user/libvhost-user.h b/contrib/libvhost-user/= libvhost-user.h index 8bb35582ea..3e65a962da 100644 --- a/contrib/libvhost-user/libvhost-user.h +++ b/contrib/libvhost-user/libvhost-user.h @@ -231,6 +231,9 @@ struct VuDev { * re-initialize */ vu_panic_cb panic; const VuDevIface *iface; + + /* Postcopy data */ + int postcopy_ufd; }; =20 typedef struct VuVirtqElement { --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677019889530.1036724233898; Wed, 28 Jun 2017 12:10:19 -0700 (PDT) Received: from localhost ([::1]:35279 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQILw-0004CG-6l for importer@patchew.org; Wed, 28 Jun 2017 15:10:16 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40499) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDp-0004iV-HX for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDn-0008DZ-Vx for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:53 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41078) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDn-0008D1-NC for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:51 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 93A8EC05689C; Wed, 28 Jun 2017 19:01:50 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id D58F04F9A6; Wed, 28 Jun 2017 19:01:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 93A8EC05689C Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 93A8EC05689C From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:29 +0100 Message-Id: <20170628190047.26159-12-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 28 Jun 2017 19:01:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 11/29] postcopy: Allow registering of fd handler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Allow other userfaultfd's to be registered into the fault thread so that handlers for shared memory can get responses. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 3 + migration/migration.h | 2 + migration/postcopy-ram.c | 212 +++++++++++++++++++++++++++++++++++--------= ---- migration/postcopy-ram.h | 21 +++++ migration/trace-events | 2 + 5 files changed, 186 insertions(+), 54 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 89df832625..a88b007a8b 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -134,6 +134,8 @@ MigrationIncomingState *migration_incoming_get_current(= void) if (!once) { mis_current.state =3D MIGRATION_STATUS_NONE; memset(&mis_current, 0, sizeof(MigrationIncomingState)); + mis_current.postcopy_remote_fds =3D g_array_new(FALSE, TRUE, + sizeof(struct PostCopyF= D)); qemu_mutex_init(&mis_current.rp_mutex); qemu_event_init(&mis_current.main_thread_load_event, false); once =3D true; @@ -157,6 +159,7 @@ void migration_incoming_state_destroy(void) qemu_fclose(mis->from_src_file); mis->from_src_file =3D NULL; } + g_array_free(mis->postcopy_remote_fds, TRUE); =20 qemu_event_destroy(&mis->main_thread_load_event); } diff --git a/migration/migration.h b/migration/migration.h index d109635d08..1210b217a1 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -51,6 +51,8 @@ struct MigrationIncomingState { QemuMutex rp_mutex; /* We send replies from multiple threads */ void *postcopy_tmp_page; void *postcopy_tmp_zero_page; + /* PostCopyFD's for external userfaultfds & handlers of shared memory = */ + GArray *postcopy_remote_fds; =20 QEMUBH *bh; =20 diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 64f5a8b003..d43a73c108 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -467,29 +467,43 @@ static void *postcopy_ram_fault_thread(void *opaque) MigrationIncomingState *mis =3D opaque; struct uffd_msg msg; int ret; + size_t index; RAMBlock *rb =3D NULL; RAMBlock *last_rb =3D NULL; /* last RAMBlock we sent part of */ =20 trace_postcopy_ram_fault_thread_entry(); qemu_sem_post(&mis->fault_thread_sem); =20 + struct pollfd *pfd; + size_t pfd_len =3D 2 + mis->postcopy_remote_fds->len; + + pfd =3D g_new0(struct pollfd, pfd_len); + + pfd[0].fd =3D mis->userfault_fd; + pfd[0].events =3D POLLIN; + pfd[1].fd =3D mis->userfault_quit_fd; + pfd[1].events =3D POLLIN; /* Waiting for eventfd to go positive */ + trace_postcopy_ram_fault_thread_fds_core(pfd[0].fd, pfd[1].fd); + for (index =3D 0; index < mis->postcopy_remote_fds->len; index++) { + struct PostCopyFD *pcfd =3D &g_array_index(mis->postcopy_remote_fd= s, + struct PostCopyFD, index); + pfd[2 + index].fd =3D pcfd->fd; + pfd[2 + index].events =3D POLLIN; + trace_postcopy_ram_fault_thread_fds_extra(2 + index, pcfd->idstr, + pcfd->fd); + } + while (true) { ram_addr_t rb_offset; - struct pollfd pfd[2]; + int poll_result; =20 /* * We're mainly waiting for the kernel to give us a faulting HVA, * however we can be told to quit via userfault_quit_fd which is * an eventfd */ - pfd[0].fd =3D mis->userfault_fd; - pfd[0].events =3D POLLIN; - pfd[0].revents =3D 0; - pfd[1].fd =3D mis->userfault_quit_fd; - pfd[1].events =3D POLLIN; /* Waiting for eventfd to go positive */ - pfd[1].revents =3D 0; - - if (poll(pfd, 2, -1 /* Wait forever */) =3D=3D -1) { + poll_result =3D poll(pfd, pfd_len, -1 /* Wait forever */); + if (poll_result =3D=3D -1) { error_report("%s: userfault poll: %s", __func__, strerror(errn= o)); break; } @@ -499,57 +513,118 @@ static void *postcopy_ram_fault_thread(void *opaque) break; } =20 - ret =3D read(mis->userfault_fd, &msg, sizeof(msg)); - if (ret !=3D sizeof(msg)) { - if (errno =3D=3D EAGAIN) { - /* - * if a wake up happens on the other thread just after - * the poll, there is nothing to read. - */ - continue; + if (pfd[0].revents) { + poll_result--; + ret =3D read(mis->userfault_fd, &msg, sizeof(msg)); + if (ret !=3D sizeof(msg)) { + if (errno =3D=3D EAGAIN) { + /* + * if a wake up happens on the other thread just after + * the poll, there is nothing to read. + */ + continue; + } + if (ret < 0) { + error_report("%s: Failed to read full userfault " + "message: %s", + __func__, strerror(errno)); + break; + } else { + error_report("%s: Read %d bytes from userfaultfd " + "expected %zd", + __func__, ret, sizeof(msg)); + break; /* Lost alignment, don't know what we'd read ne= xt */ + } + } + if (msg.event !=3D UFFD_EVENT_PAGEFAULT) { + error_report("%s: Read unexpected event %ud from userfault= fd", + __func__, msg.event); + continue; /* It's not a page fault, shouldn't happen */ } - if (ret < 0) { - error_report("%s: Failed to read full userfault message: %= s", - __func__, strerror(errno)); + + rb =3D qemu_ram_block_from_host( + (void *)(uintptr_t)msg.arg.pagefault.address, + true, &rb_offset); + if (!rb) { + error_report("postcopy_ram_fault_thread: Fault outside gue= st: %" + PRIx64, (uint64_t)msg.arg.pagefault.address); break; - } else { - error_report("%s: Read %d bytes from userfaultfd expected = %zd", - __func__, ret, sizeof(msg)); - break; /* Lost alignment, don't know what we'd read next */ } - } - if (msg.event !=3D UFFD_EVENT_PAGEFAULT) { - error_report("%s: Read unexpected event %ud from userfaultfd", - __func__, msg.event); - continue; /* It's not a page fault, shouldn't happen */ - } =20 - rb =3D qemu_ram_block_from_host( - (void *)(uintptr_t)msg.arg.pagefault.address, - true, &rb_offset); - if (!rb) { - error_report("postcopy_ram_fault_thread: Fault outside guest: = %" - PRIx64, (uint64_t)msg.arg.pagefault.address); - break; - } + rb_offset &=3D ~(qemu_ram_pagesize(rb) - 1); + trace_postcopy_ram_fault_thread_request(msg.arg.pagefault.addr= ess, + qemu_ram_get_idstr(rb), + rb_offset); =20 - rb_offset &=3D ~(qemu_ram_pagesize(rb) - 1); - trace_postcopy_ram_fault_thread_request(msg.arg.pagefault.address, - qemu_ram_get_idstr(rb), - rb_offset); + /* + * Send the request to the source - we want to request one + * of our host page sizes (which is >=3D TPS) + */ + if (rb !=3D last_rb) { + last_rb =3D rb; + migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), + rb_offset, qemu_ram_pagesize(rb)); + } else { + /* Save some space */ + migrate_send_rp_req_pages(mis, NULL, + rb_offset, qemu_ram_pagesize(rb)); + } + } =20 - /* - * Send the request to the source - we want to request one - * of our host page sizes (which is >=3D TPS) - */ - if (rb !=3D last_rb) { - last_rb =3D rb; - migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), - rb_offset, qemu_ram_pagesize(rb)); - } else { - /* Save some space */ - migrate_send_rp_req_pages(mis, NULL, - rb_offset, qemu_ram_pagesize(rb)); + /* Now handle any requests from external processes on shared memor= y */ + /* TODO: May need to handle devices deregistering during postcopy = */ + for (index =3D 2; index < pfd_len && poll_result; index++) { + if (pfd[index].revents) { + struct PostCopyFD *pcfd =3D + &g_array_index(mis->postcopy_remote_fds, + struct PostCopyFD, index - 2); + + poll_result--; + if (pfd[index].revents & POLLERR) { + error_report("%s: POLLERR on poll %zd fd=3D%d", + __func__, index, pcfd->fd); + pfd[index].events =3D 0; + continue; + } + + ret =3D read(pcfd->fd, &msg, sizeof(msg)); + if (ret !=3D sizeof(msg)) { + if (errno =3D=3D EAGAIN) { + /* + * if a wake up happens on the other thread just a= fter + * the poll, there is nothing to read. + */ + continue; + } + if (ret < 0) { + error_report("%s: Failed to read full userfault " + "message: %s (shared) revents=3D%d", + __func__, strerror(errno), + pfd[index].revents); + /*TODO: Could just disable this sharer */ + break; + } else { + error_report("%s: Read %d bytes from userfaultfd " + "expected %zd (shared)", + __func__, ret, sizeof(msg)); + /*TODO: Could just disable this sharer */ + break; /*Lost alignment,don't know what we'd read = next*/ + } + } + if (msg.event !=3D UFFD_EVENT_PAGEFAULT) { + error_report("%s: Read unexpected event %ud " + "from userfaultfd (shared)", + __func__, msg.event); + continue; /* It's not a page fault, shouldn't happen */ + } + /* Call the device handler registered with us */ + ret =3D pcfd->handler(pcfd, &msg); + if (ret) { + error_report("%s: Failed to resolve shared fault on %z= d/%s", + __func__, index, pcfd->idstr); + /* TODO: Fail? Disable this sharer? */ + } + } } } trace_postcopy_ram_fault_thread_exit(); @@ -879,3 +954,32 @@ PostcopyState postcopy_state_set(PostcopyState new_sta= te) { return atomic_xchg(&incoming_postcopy_state, new_state); } + +/* Register a handler for external shared memory postcopy + * called on the destination. + */ +void postcopy_register_shared_ufd(struct PostCopyFD *pcfd) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + mis->postcopy_remote_fds =3D g_array_append_val(mis->postcopy_remote_f= ds, + *pcfd); +} + +/* Unregister a handler for external shared memory postcopy + */ +void postcopy_unregister_shared_ufd(struct PostCopyFD *pcfd) +{ + guint i; + MigrationIncomingState *mis =3D migration_incoming_get_current(); + GArray *pcrfds =3D mis->postcopy_remote_fds; + + for (i =3D 0; i < pcrfds->len; i++) { + struct PostCopyFD *cur =3D &g_array_index(pcrfds, struct PostCopyF= D, i); + if (cur->fd =3D=3D pcfd->fd) { + mis->postcopy_remote_fds =3D g_array_remove_index(pcrfds, i); + return; + } + } +} + diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 70d4b09659..ba8a8ffec5 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -141,4 +141,25 @@ void postcopy_remove_notifier(NotifierWithReturn *n); /* Call the notifier list set by postcopy_add_start_notifier */ int postcopy_notify(enum PostcopyNotifyReason reason, Error **errp); =20 +struct PostCopyFD; + +/* ufd is a pointer to the struct uffd_msg *TODO: more Portable! */ +typedef int (*pcfdhandler)(struct PostCopyFD *pcfd, void *ufd); + +struct PostCopyFD { + int fd; + /* Data to pass to handler */ + void *data; + /* Handler to be called whenever we get a poll event */ + pcfdhandler handler; + /* A string to use in error messages */ + char *idstr; +}; + +/* Register a userfaultfd owned by an external process for + * shared memory. + */ +void postcopy_register_shared_ufd(struct PostCopyFD *pcfd); +void postcopy_unregister_shared_ufd(struct PostCopyFD *pcfd); + #endif diff --git a/migration/trace-events b/migration/trace-events index cb2c4b5b40..522cfa170d 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -189,6 +189,8 @@ postcopy_place_page_zero(void *host_addr) "host=3D%p" postcopy_ram_enable_notify(void) "" postcopy_ram_fault_thread_entry(void) "" postcopy_ram_fault_thread_exit(void) "" +postcopy_ram_fault_thread_fds_core(int baseufd, int quitfd) "ufd: %d quitf= d: %d" +postcopy_ram_fault_thread_fds_extra(size_t index, const char *name, int fd= ) "%zd/%s: %d" postcopy_ram_fault_thread_quit(void) "" postcopy_ram_fault_thread_request(uint64_t hostaddr, const char *ramblock,= size_t offset) "Request for HVA=3D%" PRIx64 " rb=3D%s offset=3D%zx" postcopy_ram_incoming_cleanup_closeuf(void) "" --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677152463729.4895889105293; Wed, 28 Jun 2017 12:12:32 -0700 (PDT) Received: from localhost ([::1]:35293 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIO6-0006CW-38 for importer@patchew.org; Wed, 28 Jun 2017 15:12:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40547) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDu-0004n9-KZ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDt-0008Fi-IX for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:58 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45698) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDt-0008FZ-9f for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:57 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 20EC24E4F3; Wed, 28 Jun 2017 19:01:56 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id DAD844F6B4; Wed, 28 Jun 2017 19:01:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 20EC24E4F3 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 20EC24E4F3 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:30 +0100 Message-Id: <20170628190047.26159-13-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 28 Jun 2017 19:01:56 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 12/29] vhost+postcopy: Register shared ufd with postcopy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Register the UFD that comes in as the response to the 'advise' method with the postcopy code. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/vhost-user.c | 23 ++++++++++++++++++++++- migration/postcopy-ram.h | 2 +- 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index ee9a1ac8a3..b98fbe4834 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -24,6 +24,7 @@ #include #include #include +#include =20 #define VHOST_MEMORY_MAX_NREGIONS 8 #define VHOST_USER_F_PROTOCOL_FEATURES 30 @@ -128,6 +129,7 @@ struct vhost_user { CharBackend *chr; int slave_fd; NotifierWithReturn postcopy_notifier; + struct PostCopyFD postcopy_fd; }; =20 static bool ioeventfd_enabled(void) @@ -707,6 +709,19 @@ out: } =20 /* + * Called back from the postcopy fault thread when a fault is received on = our + * ufd. + * TODO: This is Linux specific + */ +static int vhost_user_postcopy_fault_handler(struct PostCopyFD *pcfd, + void *ufd) +{ + struct uffd_msg *msg =3D ufd; + + return 0; +} + +/* * Called at the start of an inbound postcopy on reception of the * 'advise' command. */ @@ -745,8 +760,14 @@ static int vhost_user_postcopy_advise(struct vhost_dev= *dev, Error **errp) error_setg(errp, "%s: Failed to get ufd", __func__); return -1; } + fcntl(ufd, F_SETFL, O_NONBLOCK); =20 - /* TODO: register ufd with userfault thread */ + /* register ufd with userfault thread */ + u->postcopy_fd.fd =3D ufd; + u->postcopy_fd.data =3D dev; + u->postcopy_fd.handler =3D vhost_user_postcopy_fault_handler; + u->postcopy_fd.idstr =3D "vhost-user"; /* Need to find unique name */ + postcopy_register_shared_ufd(&u->postcopy_fd); return 0; } =20 diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index ba8a8ffec5..28c216cc7a 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -153,7 +153,7 @@ struct PostCopyFD { /* Handler to be called whenever we get a poll event */ pcfdhandler handler; /* A string to use in error messages */ - char *idstr; + const char *idstr; }; =20 /* Register a userfaultfd owned by an external process for --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676906438327.1281042294273; Wed, 28 Jun 2017 12:08:26 -0700 (PDT) Received: from localhost ([::1]:35273 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIK8-00020y-SA for importer@patchew.org; Wed, 28 Jun 2017 15:08:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40564) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDw-0004pQ-R7 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDv-0008GU-Oe for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:00 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47658) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDv-0008GF-H8 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:01:59 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 620D97CE0A; Wed, 28 Jun 2017 19:01:58 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 61B8B5DC1D; Wed, 28 Jun 2017 19:01:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 620D97CE0A Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 620D97CE0A From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:31 +0100 Message-Id: <20170628190047.26159-14-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 28 Jun 2017 19:01:58 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 13/29] vhost+postcopy: Transmit 'listen' to client X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Notify the vhost-user client on reception of the 'postcopy-listen' event from the source. Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.c | 16 ++++++++++++++++ contrib/libvhost-user/libvhost-user.h | 2 ++ hw/virtio/trace-events | 3 +++ hw/virtio/vhost-user.c | 23 +++++++++++++++++++++++ migration/postcopy-ram.h | 1 + migration/savevm.c | 7 +++++++ 6 files changed, 52 insertions(+) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index 62e97f6b84..6de339fb7a 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -15,7 +15,9 @@ =20 #include #include +#include #include +#include #include =20 #include "qemu/atomic.h" @@ -65,6 +67,7 @@ vu_request_to_string(int req) REQ(VHOST_USER_SEND_RARP), REQ(VHOST_USER_INPUT_GET_CONFIG), REQ(VHOST_USER_POSTCOPY_ADVISE), + REQ(VHOST_USER_POSTCOPY_LISTEN), REQ(VHOST_USER_MAX), }; #undef REQ @@ -801,6 +804,17 @@ vu_set_postcopy_advise(VuDev *dev, VhostUserMsg *vmsg) } =20 static bool +vu_set_postcopy_listen(VuDev *dev, VhostUserMsg *vmsg) +{ + if (dev->nregions) { + vu_panic(dev, "Regions already registered at postcopy-listen"); + return false; + } + dev->postcopy_listening =3D true; + + return false; +} +static bool vu_process_message(VuDev *dev, VhostUserMsg *vmsg) { int do_reply =3D 0; @@ -865,6 +879,8 @@ vu_process_message(VuDev *dev, VhostUserMsg *vmsg) return vu_set_vring_enable_exec(dev, vmsg); case VHOST_USER_POSTCOPY_ADVISE: return vu_set_postcopy_advise(dev, vmsg); + case VHOST_USER_POSTCOPY_LISTEN: + return vu_set_postcopy_listen(dev, vmsg); default: vmsg_close_fds(vmsg); vu_panic(dev, "Unhandled request: %d", vmsg->request); diff --git a/contrib/libvhost-user/libvhost-user.h b/contrib/libvhost-user/= libvhost-user.h index 3e65a962da..86e1934ddb 100644 --- a/contrib/libvhost-user/libvhost-user.h +++ b/contrib/libvhost-user/libvhost-user.h @@ -64,6 +64,7 @@ typedef enum VhostUserRequest { VHOST_USER_SEND_RARP =3D 19, VHOST_USER_INPUT_GET_CONFIG =3D 20, VHOST_USER_POSTCOPY_ADVISE =3D 23, + VHOST_USER_POSTCOPY_LISTEN =3D 24, VHOST_USER_MAX } VhostUserRequest; =20 @@ -234,6 +235,7 @@ struct VuDev { =20 /* Postcopy data */ int postcopy_ufd; + bool postcopy_listening; }; =20 typedef struct VuVirtqElement { diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index e24d8fa997..1076dbbb1d 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -1,5 +1,8 @@ # See docs/tracing.txt for syntax documentation. =20 +# hw/virtio/vhost-user.c +vhost_user_postcopy_listen(void) "" + # hw/virtio/virtio.c virtqueue_alloc_element(void *elem, size_t sz, unsigned in_num, unsigned o= ut_num) "elem %p size %zd in_num %u out_num %u" virtqueue_fill(void *vq, const void *elem, unsigned int len, unsigned int = idx) "vq %p elem %p len %u idx %u" diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index b98fbe4834..1f70f5760f 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -67,6 +67,7 @@ typedef enum VhostUserRequest { VHOST_USER_SET_SLAVE_REQ_FD =3D 21, VHOST_USER_IOTLB_MSG =3D 22, VHOST_USER_POSTCOPY_ADVISE =3D 23, + VHOST_USER_POSTCOPY_LISTEN =3D 24, VHOST_USER_MAX } VhostUserRequest; =20 @@ -771,6 +772,25 @@ static int vhost_user_postcopy_advise(struct vhost_dev= *dev, Error **errp) return 0; } =20 +/* + * Called at the switch to postcopy on reception of the 'listen' command. + */ +static int vhost_user_postcopy_listen(struct vhost_dev *dev, Error **errp) +{ + VhostUserMsg msg =3D { + .request =3D VHOST_USER_POSTCOPY_LISTEN, + .flags =3D VHOST_USER_VERSION, + }; + + trace_vhost_user_postcopy_listen(); + if (vhost_user_write(dev, &msg, NULL, 0) < 0) { + error_setg(errp, "Failed to send postcopy_listen to vhost"); + return -1; + } + + return 0; +} + static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier, void *opaque) { @@ -793,6 +813,9 @@ static int vhost_user_postcopy_notifier(NotifierWithRet= urn *notifier, case POSTCOPY_NOTIFY_INBOUND_ADVISE: return vhost_user_postcopy_advise(dev, pnd->errp); =20 + case POSTCOPY_NOTIFY_INBOUND_LISTEN: + return vhost_user_postcopy_listen(dev, pnd->errp); + default: /* We ignore notifications we don't know */ break; diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 28c216cc7a..873c147b68 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -129,6 +129,7 @@ void postcopy_infrastructure_init(void); enum PostcopyNotifyReason { POSTCOPY_NOTIFY_PROBE =3D 0, POSTCOPY_NOTIFY_INBOUND_ADVISE, + POSTCOPY_NOTIFY_INBOUND_LISTEN, }; =20 struct PostcopyNotifyData { diff --git a/migration/savevm.c b/migration/savevm.c index 4977baf9b7..4b6294b7d4 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1579,6 +1579,8 @@ static int loadvm_postcopy_handle_listen(MigrationInc= omingState *mis) { PostcopyState ps =3D postcopy_state_set(POSTCOPY_INCOMING_LISTENING); trace_loadvm_postcopy_handle_listen(); + Error *local_err =3D NULL; + if (ps !=3D POSTCOPY_INCOMING_ADVISE && ps !=3D POSTCOPY_INCOMING_DISC= ARD) { error_report("CMD_POSTCOPY_LISTEN in wrong postcopy state (%d)", p= s); return -1; @@ -1600,6 +1602,11 @@ static int loadvm_postcopy_handle_listen(MigrationIn= comingState *mis) return -1; } =20 + if (postcopy_notify(POSTCOPY_NOTIFY_INBOUND_LISTEN, &local_err)) { + error_report_err(local_err); + return -1; + } + if (mis->have_listen_thread) { error_report("CMD_POSTCOPY_RAM_LISTEN already has a listen thread"= ); return -1; --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677273418710.0338726610079; Wed, 28 Jun 2017 12:14:33 -0700 (PDT) Received: from localhost ([::1]:35301 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIQ3-0007we-Ox for importer@patchew.org; Wed, 28 Jun 2017 15:14:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40580) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIDz-0004rn-97 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIDy-0008HX-8W for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:03 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35674) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIDx-0008HJ-WE for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:02 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 09A5A74859; Wed, 28 Jun 2017 19:02:01 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE9365DD71; Wed, 28 Jun 2017 19:01:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 09A5A74859 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 09A5A74859 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:32 +0100 Message-Id: <20170628190047.26159-15-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 28 Jun 2017 19:02:01 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 14/29] vhost+postcopy: Register new regions with the ufd X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" When new regions are sent to the client using SET_MEM_TABLE, register them with the userfaultfd. Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.c | 33 +++++++++++++++++++++++++++++++= ++ 1 file changed, 33 insertions(+) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index 6de339fb7a..be7470e3a9 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -450,6 +450,39 @@ vu_set_mem_table_exec(VuDev *dev, VhostUserMsg *vmsg) dev_region->mmap_addr); } =20 + if (dev->postcopy_listening) { + /* We should already have an open ufd need to mark each memory + * range as ufd. + * Note: Do we need any madvises? Well it's not been accessed + * yet, still probably need no THP to be safe, discard to be s= afe? + */ + struct uffdio_register reg_struct; + /* Note: We might need to go back to using mmap_addr and + * len + mmap_offset for * huge pages, but then we do hope not= to + * see accesses in that area below the offset + */ + reg_struct.range.start =3D (uintptr_t)(dev_region->mmap_addr + + dev_region->mmap_offset); + reg_struct.range.len =3D dev_region->size; + reg_struct.mode =3D UFFDIO_REGISTER_MODE_MISSING; + + if (ioctl(dev->postcopy_ufd, UFFDIO_REGISTER, ®_struct)) { + vu_panic(dev, "%s: Failed to userfault region %d: (ufd=3D%= d)%s\n", + __func__, i, strerror(errno), dev->postcopy_ufd); + continue; + } + if (!(reg_struct.ioctls & ((__u64)1 << _UFFDIO_COPY))) { + vu_panic(dev, "%s Region (%d) doesn't support COPY", + __func__, i); + continue; + } + DPRINT("%s: region %d: Registered userfault for %llx + %llx\n", + __func__, i, reg_struct.range.start, reg_struct.range.= len); + /* TODO: Stash 'zero' support flags somewhere */ + /* TODO: Get address back to QEMU */ + + } + close(vmsg->fds[i]); } =20 --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677394353349.858261943824; Wed, 28 Jun 2017 12:16:34 -0700 (PDT) Received: from localhost ([::1]:35313 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIS0-0001N1-QW for importer@patchew.org; Wed, 28 Jun 2017 15:16:32 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40633) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIE5-0004xU-Gv for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIE0-0008I5-KI for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:09 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53172) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIE0-0008Hs-6n for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:04 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 272208046F; Wed, 28 Jun 2017 19:02:03 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 22AD317D14; Wed, 28 Jun 2017 19:02:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 272208046F Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 272208046F From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:33 +0100 Message-Id: <20170628190047.26159-16-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 28 Jun 2017 19:02:03 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 15/29] vhost+postcopy: Send address back to qemu X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" We need a better way, but at the moment we need the address of the mappings sent back to qemu so it can interpret the messages on the userfaultfd it reads. Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.c | 15 +++++++++- hw/virtio/trace-events | 1 + hw/virtio/vhost-user.c | 54 +++++++++++++++++++++++++++++++= ++++ 3 files changed, 69 insertions(+), 1 deletion(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index be7470e3a9..0658b6e847 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -479,13 +479,26 @@ vu_set_mem_table_exec(VuDev *dev, VhostUserMsg *vmsg) DPRINT("%s: region %d: Registered userfault for %llx + %llx\n", __func__, i, reg_struct.range.start, reg_struct.range.= len); /* TODO: Stash 'zero' support flags somewhere */ - /* TODO: Get address back to QEMU */ =20 + /* TODO: We need to find a way for the qemu not to see the vir= tual + * addresses of the clients, so as to keep better separation. + */ + /* Return the address to QEMU so that it can translate the ufd + * fault addresses back. + */ + msg_region->userspace_addr =3D (uintptr_t)(mmap_addr + + dev_region->mmap_offs= et); } =20 close(vmsg->fds[i]); } =20 + if (dev->postcopy_listening) { + /* Need to return the addresses - send the updated message back */ + vmsg->fd_num =3D 0; + return true; + } + return false; } =20 diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 1076dbbb1d..f7be340a45 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -2,6 +2,7 @@ =20 # hw/virtio/vhost-user.c vhost_user_postcopy_listen(void) "" +vhost_user_set_mem_table_postcopy(uint64_t client_addr, uint64_t qhva, int= reply_i, int region_i) "client:%"PRIx64" for hva: %"PRIx64" reply %d regio= n %d" =20 # hw/virtio/virtio.c virtqueue_alloc_element(void *elem, size_t sz, unsigned in_num, unsigned o= ut_num) "elem %p size %zd in_num %u out_num %u" diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 1f70f5760f..6be3e7ff2d 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -19,6 +19,7 @@ #include "qemu/sockets.h" #include "migration/migration.h" #include "migration/postcopy-ram.h" +#include "trace.h" =20 #include #include @@ -131,6 +132,7 @@ struct vhost_user { int slave_fd; NotifierWithReturn postcopy_notifier; struct PostCopyFD postcopy_fd; + uint64_t postcopy_client_bases[VHOST_MEMORY_MAX_NREGIONS]; }; =20 static bool ioeventfd_enabled(void) @@ -298,6 +300,7 @@ static int vhost_user_set_log_base(struct vhost_dev *de= v, uint64_t base, static int vhost_user_set_mem_table(struct vhost_dev *dev, struct vhost_memory *mem) { + struct vhost_user *u =3D dev->opaque; int fds[VHOST_MEMORY_MAX_NREGIONS]; int i, fd; size_t fd_num =3D 0; @@ -348,6 +351,57 @@ static int vhost_user_set_mem_table(struct vhost_dev *= dev, return -1; } =20 + if (u->postcopy_fd.handler) { + VhostUserMsg msg_reply; + int region_i, reply_i; + if (vhost_user_read(dev, &msg_reply) < 0) { + return -1; + } + + if (msg_reply.request !=3D VHOST_USER_SET_MEM_TABLE) { + error_report("%s: Received unexpected msg type." + "Expected %d received %d", __func__, + VHOST_USER_SET_MEM_TABLE, msg_reply.request); + return -1; + } + /* We're using the same structure, just reusing one of the + * fields, so it should be the same size. + */ + if (msg_reply.size !=3D msg.size) { + error_report("%s: Unexpected size for postcopy reply " + "%d vs %d", __func__, msg_reply.size, msg.size); + return -1; + } + + memset(u->postcopy_client_bases, 0, + sizeof(uint64_t) * VHOST_MEMORY_MAX_NREGIONS); + + /* They're in the same order as the regions that were sent + * but some of the regions were skipped (above) if they + * didn't have fd's + */ + for (reply_i =3D 0, region_i =3D 0; + region_i < dev->mem->nregions; + region_i++) { + if (reply_i < fd_num && + msg_reply.payload.memory.regions[region_i].guest_phys_addr= =3D=3D + dev->mem->regions[region_i].guest_phys_addr) { + u->postcopy_client_bases[region_i] =3D + msg_reply.payload.memory.regions[reply_i].userspace_ad= dr; + trace_vhost_user_set_mem_table_postcopy( + msg_reply.payload.memory.regions[reply_i].userspace_ad= dr, + msg.payload.memory.regions[reply_i].userspace_addr, + reply_i, region_i); + reply_i++; + } + } + if (reply_i !=3D fd_num) { + error_report("%s: postcopy reply not fully consumed " + "%d vs %zd", + __func__, reply_i, fd_num); + return -1; + } + } if (reply_supported) { return process_message_reply(dev, &msg); } --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677057892168.3535136815558; Wed, 28 Jun 2017 12:10:57 -0700 (PDT) Received: from localhost ([::1]:35287 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIMX-0004nL-Gr for importer@patchew.org; Wed, 28 Jun 2017 15:10:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40706) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEJ-000556-1X for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEE-0008Mi-55 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45404) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIED-0008MO-Ty for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:18 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ECB4740F11; Wed, 28 Jun 2017 19:02:16 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5311D17D24; Wed, 28 Jun 2017 19:02:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com ECB4740F11 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com ECB4740F11 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:34 +0100 Message-Id: <20170628190047.26159-17-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 28 Jun 2017 19:02:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 16/29] vhost+postcopy: Stash RAMBlock and offset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Stash the RAMBlock and offset for later use looking up addresses. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/trace-events | 1 + hw/virtio/vhost-user.c | 11 +++++++++++ 2 files changed, 12 insertions(+) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index f7be340a45..1fd194363a 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -3,6 +3,7 @@ # hw/virtio/vhost-user.c vhost_user_postcopy_listen(void) "" vhost_user_set_mem_table_postcopy(uint64_t client_addr, uint64_t qhva, int= reply_i, int region_i) "client:%"PRIx64" for hva: %"PRIx64" reply %d regio= n %d" +vhost_user_set_mem_table_withfd(int index, const char *name, uint64_t memo= ry_size, uint64_t guest_phys_addr, uint64_t userspace_addr, uint64_t offset= ) "%d:%s: size:%"PRIx64" GPA:%"PRIx64" QVA/userspace:%"PRIx64" RB offset:%"= PRIx64 =20 # hw/virtio/virtio.c virtqueue_alloc_element(void *elem, size_t sz, unsigned in_num, unsigned o= ut_num) "elem %p size %zd in_num %u out_num %u" diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 6be3e7ff2d..3185af7a45 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -133,6 +133,11 @@ struct vhost_user { NotifierWithReturn postcopy_notifier; struct PostCopyFD postcopy_fd; uint64_t postcopy_client_bases[VHOST_MEMORY_MAX_NREGIONS]; + RAMBlock *region_rb[VHOST_MEMORY_MAX_NREGIONS]; + /* The offset from the start of the RAMBlock to the start of the + * vhost region. + */ + ram_addr_t region_rb_offset[VHOST_MEMORY_MAX_NREGIONS]; }; =20 static bool ioeventfd_enabled(void) @@ -324,8 +329,14 @@ static int vhost_user_set_mem_table(struct vhost_dev *= dev, assert((uintptr_t)reg->userspace_addr =3D=3D reg->userspace_addr); mr =3D memory_region_from_host((void *)(uintptr_t)reg->userspace_a= ddr, &offset); + u->region_rb_offset[i] =3D offset; + u->region_rb[i] =3D mr->ram_block; fd =3D memory_region_get_fd(mr); if (fd > 0) { + trace_vhost_user_set_mem_table_withfd(fd_num, mr->name, + reg->memory_size, + reg->guest_phys_addr, + reg->userspace_addr, off= set); msg.payload.memory.regions[fd_num].userspace_addr =3D reg->use= rspace_addr; msg.payload.memory.regions[fd_num].memory_size =3D reg->memor= y_size; msg.payload.memory.regions[fd_num].guest_phys_addr =3D reg->gu= est_phys_addr; --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677183636139.31988963006904; Wed, 28 Jun 2017 12:13:03 -0700 (PDT) Received: from localhost ([::1]:35296 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIOY-0006Yp-Io for importer@patchew.org; Wed, 28 Jun 2017 15:12:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40705) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEJ-000555-0S for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEG-0008NC-IZ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEG-0008N2-9V for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:20 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3D45C81F03; Wed, 28 Jun 2017 19:02:19 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 45A885DC1D; Wed, 28 Jun 2017 19:02:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 3D45C81F03 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 3D45C81F03 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:35 +0100 Message-Id: <20170628190047.26159-18-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 28 Jun 2017 19:02:19 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 17/29] vhost+postcopy: Send requests to source for shared pages X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Send requests back to the source for shared page requests. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.h | 2 ++ migration/postcopy-ram.c | 31 ++++++++++++++++++++++++++++--- migration/postcopy-ram.h | 3 +++ migration/trace-events | 2 ++ 4 files changed, 35 insertions(+), 3 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 1210b217a1..5fa0bb7b53 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -49,6 +49,8 @@ struct MigrationIncomingState { int userfault_quit_fd; QEMUFile *to_src_file; QemuMutex rp_mutex; /* We send replies from multiple threads */ + /* RAMBlock of last request sent to source */ + RAMBlock *last_rb; void *postcopy_tmp_page; void *postcopy_tmp_zero_page; /* PostCopyFD's for external userfaultfds & handlers of shared memory = */ diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index d43a73c108..cbf10236f0 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -460,6 +460,31 @@ static int ram_block_enable_notify(const char *block_n= ame, void *host_addr, } =20 /* + * Callback from shared fault handlers to ask for a page, + * the page must be specified by a RAMBlock and an offset in that rb + */ +int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb, + uint64_t client_addr, uint64_t rb_offset) +{ + size_t pagesize =3D qemu_ram_pagesize(rb); + uint64_t aligned_rbo =3D rb_offset & ~(pagesize - 1); + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + trace_postcopy_request_shared_page(pcfd->idstr, qemu_ram_get_idstr(rb), + rb_offset); + /* TODO: Check bitmap to see if we already have the page */ + if (rb !=3D mis->last_rb) { + mis->last_rb =3D rb; + migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), + aligned_rbo, pagesize); + } else { + /* Save some space */ + migrate_send_rp_req_pages(mis, NULL, aligned_rbo, pagesize); + } + return 0; +} + +/* * Handle faults detected by the USERFAULT markings */ static void *postcopy_ram_fault_thread(void *opaque) @@ -469,9 +494,9 @@ static void *postcopy_ram_fault_thread(void *opaque) int ret; size_t index; RAMBlock *rb =3D NULL; - RAMBlock *last_rb =3D NULL; /* last RAMBlock we sent part of */ =20 trace_postcopy_ram_fault_thread_entry(); + mis->last_rb =3D NULL; /* last RAMBlock we sent part of */ qemu_sem_post(&mis->fault_thread_sem); =20 struct pollfd *pfd; @@ -560,8 +585,8 @@ static void *postcopy_ram_fault_thread(void *opaque) * Send the request to the source - we want to request one * of our host page sizes (which is >=3D TPS) */ - if (rb !=3D last_rb) { - last_rb =3D rb; + if (rb !=3D mis->last_rb) { + mis->last_rb =3D rb; migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), rb_offset, qemu_ram_pagesize(rb)); } else { diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 873c147b68..69e88b0174 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -162,5 +162,8 @@ struct PostCopyFD { */ void postcopy_register_shared_ufd(struct PostCopyFD *pcfd); void postcopy_unregister_shared_ufd(struct PostCopyFD *pcfd); +/* Callback from shared fault handlers to ask for a page */ +int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb, + uint64_t client_addr, uint64_t offset); =20 #endif diff --git a/migration/trace-events b/migration/trace-events index 522cfa170d..7ac8f9cf41 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -197,6 +197,8 @@ postcopy_ram_incoming_cleanup_closeuf(void) "" postcopy_ram_incoming_cleanup_entry(void) "" postcopy_ram_incoming_cleanup_exit(void) "" postcopy_ram_incoming_cleanup_join(void) "" +postcopy_request_shared_page(const char *sharer, const char *rb, uint64_t = rb_offset) "for %s in %s offset %"PRIx64 + save_xbzrle_page_skipping(void) "" save_xbzrle_page_overflow(void) "" ram_save_iterate_big_wait(uint64_t milliconds, int iterations) "big wait: = %" PRIu64 " milliseconds, %d iterations" --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677302522870.096259070349; Wed, 28 Jun 2017 12:15:02 -0700 (PDT) Received: from localhost ([::1]:35304 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIQX-0008JO-64 for importer@patchew.org; Wed, 28 Jun 2017 15:15:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40722) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEK-000568-31 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEJ-0008O7-50 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45598) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEI-0008Nk-Sq for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:23 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C8C2761288; Wed, 28 Jun 2017 19:02:21 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 84ACA4F9A6; Wed, 28 Jun 2017 19:02:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com C8C2761288 Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com C8C2761288 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:36 +0100 Message-Id: <20170628190047.26159-19-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 28 Jun 2017 19:02:21 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 18/29] vhost+postcopy: Resolve client address X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Resolve fault addresses read off the clients UFD into RAMBlock and offset, and call back to the postcopy code to ask for the page. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/trace-events | 3 +++ hw/virtio/vhost-user.c | 28 +++++++++++++++++++++++++++- 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 1fd194363a..3cec81bb1e 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -1,6 +1,9 @@ # See docs/tracing.txt for syntax documentation. =20 # hw/virtio/vhost-user.c +vhost_user_postcopy_fault_handler(const char *name, uint64_t fault_address= , int nregions) "%s: @%"PRIx64" nregions:%d" +vhost_user_postcopy_fault_handler_loop(int i, uint64_t client_base, uint64= _t size) "%d: client%"PRIx64" +%"PRIx64 +vhost_user_postcopy_fault_handler_found(int i, uint64_t region_offset, uin= t64_t rb_offset) "%d: region_offset: %"PRIx64" rb_offset:%"PRIx64 vhost_user_postcopy_listen(void) "" vhost_user_set_mem_table_postcopy(uint64_t client_addr, uint64_t qhva, int= reply_i, int region_i) "client:%"PRIx64" for hva: %"PRIx64" reply %d regio= n %d" vhost_user_set_mem_table_withfd(int index, const char *name, uint64_t memo= ry_size, uint64_t guest_phys_addr, uint64_t userspace_addr, uint64_t offset= ) "%d:%s: size:%"PRIx64" GPA:%"PRIx64" QVA/userspace:%"PRIx64" RB offset:%"= PRIx64 diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 3185af7a45..92620830e4 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -782,9 +782,35 @@ out: static int vhost_user_postcopy_fault_handler(struct PostCopyFD *pcfd, void *ufd) { + struct vhost_dev *dev =3D pcfd->data; + struct vhost_user *u =3D dev->opaque; struct uffd_msg *msg =3D ufd; + uint64_t faultaddr =3D msg->arg.pagefault.address; + RAMBlock *rb =3D NULL; + uint64_t rb_offset; + int i; =20 - return 0; + trace_vhost_user_postcopy_fault_handler(pcfd->idstr, faultaddr, + dev->mem->nregions); + for (i =3D 0; i < dev->mem->nregions; i++) { + trace_vhost_user_postcopy_fault_handler_loop(i, + u->postcopy_client_bases[i], dev->mem->regions[i].memory_s= ize); + if (faultaddr >=3D u->postcopy_client_bases[i]) { + /* Ofset of the fault address in the vhost region */ + uint64_t region_offset =3D faultaddr - u->postcopy_client_base= s[i]; + if (region_offset <=3D dev->mem->regions[i].memory_size) { + rb_offset =3D region_offset + u->region_rb_offset[i]; + trace_vhost_user_postcopy_fault_handler_found(i, + region_offset, rb_offset); + rb =3D u->region_rb[i]; + return postcopy_request_shared_page(pcfd, rb, faultaddr, + rb_offset); + } + } + } + error_report("%s: Failed to find region for fault %" PRIx64, + __func__, faultaddr); + return -1; } =20 /* --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677498383243.97275368827775; Wed, 28 Jun 2017 12:18:18 -0700 (PDT) Received: from localhost ([::1]:35326 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQITf-0002je-J4 for importer@patchew.org; Wed, 28 Jun 2017 15:18:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40785) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEU-0005Et-Hi for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEP-0008SK-La for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:34 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54024) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEP-0008S8-DD for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:29 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6BBEB80082; Wed, 28 Jun 2017 19:02:28 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1B12F5DD71; Wed, 28 Jun 2017 19:02:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6BBEB80082 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 6BBEB80082 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:37 +0100 Message-Id: <20170628190047.26159-20-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 28 Jun 2017 19:02:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 19/29] postcopy: wake shared X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Send a 'wake' request on a userfaultfd for a shared process. The address in the clients address space is specified together with the RAMBlock it was resolved to. Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 26 ++++++++++++++++++++++++++ migration/postcopy-ram.h | 6 ++++++ migration/trace-events | 1 + 3 files changed, 33 insertions(+) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index cbf10236f0..072b355991 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -459,6 +459,25 @@ static int ram_block_enable_notify(const char *block_n= ame, void *host_addr, return 0; } =20 +int postcopy_wake_shared(struct PostCopyFD *pcfd, + uint64_t client_addr, + RAMBlock *rb) +{ + size_t pagesize =3D qemu_ram_pagesize(rb); + struct uffdio_range range; + int ret; + trace_postcopy_wake_shared(client_addr, qemu_ram_get_idstr(rb)); + range.start =3D client_addr & ~(pagesize - 1); + range.len =3D pagesize; + ret =3D ioctl(pcfd->fd, UFFDIO_WAKE, &range); + if (ret) { + error_report("%s: Failed to wake: %zx in %s (%s)", + __func__, client_addr, qemu_ram_get_idstr(rb), + strerror(errno)); + } + return ret; +} + /* * Callback from shared fault handlers to ask for a page, * the page must be specified by a RAMBlock and an offset in that rb @@ -877,6 +896,13 @@ void *postcopy_get_tmp_page(MigrationIncomingState *mi= s) return NULL; } =20 +int postcopy_wake_shared(struct PostCopyFD *pcfd, + uint64_t client_addr, + RAMBlock *rb) +{ + assert(0); + return -1; +} #endif =20 /* -----------------------------------------------------------------------= -- */ diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 69e88b0174..d2b2f5f4aa 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -162,6 +162,12 @@ struct PostCopyFD { */ void postcopy_register_shared_ufd(struct PostCopyFD *pcfd); void postcopy_unregister_shared_ufd(struct PostCopyFD *pcfd); +/* Notify a client ufd that a page is available + * Note: The 'client_address' is in the address space of the client + * program not QEMU + */ +int postcopy_wake_shared(struct PostCopyFD *pcfd, uint64_t client_addr, + RAMBlock *rb); /* Callback from shared fault handlers to ask for a page */ int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb, uint64_t client_addr, uint64_t offset); diff --git a/migration/trace-events b/migration/trace-events index 7ac8f9cf41..85a35be518 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -198,6 +198,7 @@ postcopy_ram_incoming_cleanup_entry(void) "" postcopy_ram_incoming_cleanup_exit(void) "" postcopy_ram_incoming_cleanup_join(void) "" postcopy_request_shared_page(const char *sharer, const char *rb, uint64_t = rb_offset) "for %s in %s offset %"PRIx64 +postcopy_wake_shared(uint64_t client_addr, const char *rb) "at %"PRIx64" i= n %s" =20 save_xbzrle_page_skipping(void) "" save_xbzrle_page_overflow(void) "" --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677229333568.9974819516034; Wed, 28 Jun 2017 12:13:49 -0700 (PDT) Received: from localhost ([::1]:35298 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIPL-0007Dh-T9 for importer@patchew.org; Wed, 28 Jun 2017 15:13:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40817) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEX-0005HZ-OK for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEW-0008V0-PV for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47226) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEW-0008Un-GY for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:36 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 67FCB80F94; Wed, 28 Jun 2017 19:02:35 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id B50075DC1D; Wed, 28 Jun 2017 19:02:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 67FCB80F94 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 67FCB80F94 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:38 +0100 Message-Id: <20170628190047.26159-21-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Wed, 28 Jun 2017 19:02:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 20/29] postcopy: postcopy_notify_shared_wake X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Add a hook to allow a client userfaultfd to be 'woken' when a page arrives, and a walker that calls that hook for relevant clients given a RAMBlock and offset. Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 16 ++++++++++++++++ migration/postcopy-ram.h | 10 ++++++++++ 2 files changed, 26 insertions(+) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 072b355991..e6b8160f09 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -753,6 +753,22 @@ static int qemu_ufd_copy_ioctl(int userfault_fd, void = *host_addr, return ret; } =20 +int postcopy_notify_shared_wake(RAMBlock *rb, uint64_t offset) +{ + int i; + MigrationIncomingState *mis =3D migration_incoming_get_current(); + GArray *pcrfds =3D mis->postcopy_remote_fds; + + for (i =3D 0; i < pcrfds->len; i++) { + struct PostCopyFD *cur =3D &g_array_index(pcrfds, struct PostCopyF= D, i); + int ret =3D cur->waker(cur, rb, offset); + if (ret) { + return ret; + } + } + return 0; +} + /* * Place a host page (from) at (host) atomically * returns 0 on success diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index d2b2f5f4aa..ecf731c689 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -146,6 +146,10 @@ struct PostCopyFD; =20 /* ufd is a pointer to the struct uffd_msg *TODO: more Portable! */ typedef int (*pcfdhandler)(struct PostCopyFD *pcfd, void *ufd); +/* Notification to wake, either on place or on reception of + * a fault on something that's already arrived (race) + */ +typedef int (*pcfdwake)(struct PostCopyFD *pcfd, RAMBlock *rb, uint64_t of= fset); =20 struct PostCopyFD { int fd; @@ -153,6 +157,8 @@ struct PostCopyFD { void *data; /* Handler to be called whenever we get a poll event */ pcfdhandler handler; + /* Notification to wake shared client */ + pcfdwake waker; /* A string to use in error messages */ const char *idstr; }; @@ -162,6 +168,10 @@ struct PostCopyFD { */ void postcopy_register_shared_ufd(struct PostCopyFD *pcfd); void postcopy_unregister_shared_ufd(struct PostCopyFD *pcfd); +/* Call each of the shared 'waker's registerd telling them of + * availability of a block. + */ +int postcopy_notify_shared_wake(RAMBlock *rb, uint64_t offset); /* Notify a client ufd that a page is available * Note: The 'client_address' is in the address space of the client * program not QEMU --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677352336991.1556582989796; Wed, 28 Jun 2017 12:15:52 -0700 (PDT) Received: from localhost ([::1]:35311 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIRL-0000ed-1Q for importer@patchew.org; Wed, 28 Jun 2017 15:15:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40863) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEe-0005Nv-Qo for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEZ-00004x-VL for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52520) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEZ-0008WA-NQ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:39 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 48ECA61BBB; Wed, 28 Jun 2017 19:02:38 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id A750517D14; Wed, 28 Jun 2017 19:02:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 48ECA61BBB Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 48ECA61BBB From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:39 +0100 Message-Id: <20170628190047.26159-22-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 28 Jun 2017 19:02:38 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 21/29] vhost+postcopy: Add vhost waker X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Register a waker function in vhost-user code to be notified when pages arrive or requests to previously mapped pages get requested. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/trace-events | 3 +++ hw/virtio/vhost-user.c | 26 ++++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 3cec81bb1e..e320a90941 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -5,6 +5,9 @@ vhost_user_postcopy_fault_handler(const char *name, uint64_= t fault_address, int vhost_user_postcopy_fault_handler_loop(int i, uint64_t client_base, uint64= _t size) "%d: client%"PRIx64" +%"PRIx64 vhost_user_postcopy_fault_handler_found(int i, uint64_t region_offset, uin= t64_t rb_offset) "%d: region_offset: %"PRIx64" rb_offset:%"PRIx64 vhost_user_postcopy_listen(void) "" +vhost_user_postcopy_waker(const char *rb, uint64_t rb_offset) "%s + %"PRIx= 64 +vhost_user_postcopy_waker_found(uint64_t client_addr) "%"PRIx64 +vhost_user_postcopy_waker_nomatch(const char *rb, uint64_t rb_offset) "%s = + %"PRIx64 vhost_user_set_mem_table_postcopy(uint64_t client_addr, uint64_t qhva, int= reply_i, int region_i) "client:%"PRIx64" for hva: %"PRIx64" reply %d regio= n %d" vhost_user_set_mem_table_withfd(int index, const char *name, uint64_t memo= ry_size, uint64_t guest_phys_addr, uint64_t userspace_addr, uint64_t offset= ) "%d:%s: size:%"PRIx64" GPA:%"PRIx64" QVA/userspace:%"PRIx64" RB offset:%"= PRIx64 =20 diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 92620830e4..0f2e05f817 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -813,6 +813,31 @@ static int vhost_user_postcopy_fault_handler(struct Po= stCopyFD *pcfd, return -1; } =20 +static int vhost_user_postcopy_waker(struct PostCopyFD *pcfd, RAMBlock *rb, + uint64_t offset) +{ + struct vhost_dev *dev =3D pcfd->data; + struct vhost_user *u =3D dev->opaque; + int i; + + trace_vhost_user_postcopy_waker(qemu_ram_get_idstr(rb), offset); + /* Translate the offset into an address in the clients address space */ + for (i =3D 0; i < dev->mem->nregions; i++) { + if (u->region_rb[i] =3D=3D rb && + offset >=3D u->region_rb_offset[i] && + offset < (u->region_rb_offset[i] + + dev->mem->regions[i].memory_size)) { + uint64_t client_addr =3D (offset - u->region_rb_offset[i]) + + u->postcopy_client_bases[i]; + trace_vhost_user_postcopy_waker_found(client_addr); + return postcopy_wake_shared(pcfd, client_addr, rb); + } + } + + trace_vhost_user_postcopy_waker_nomatch(qemu_ram_get_idstr(rb), offset= ); + return 0; +} + /* * Called at the start of an inbound postcopy on reception of the * 'advise' command. @@ -858,6 +883,7 @@ static int vhost_user_postcopy_advise(struct vhost_dev = *dev, Error **errp) u->postcopy_fd.fd =3D ufd; u->postcopy_fd.data =3D dev; u->postcopy_fd.handler =3D vhost_user_postcopy_fault_handler; + u->postcopy_fd.waker =3D vhost_user_postcopy_waker; u->postcopy_fd.idstr =3D "vhost-user"; /* Need to find unique name */ postcopy_register_shared_ufd(&u->postcopy_fd); return 0; --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677418134269.74091145643786; Wed, 28 Jun 2017 12:16:58 -0700 (PDT) Received: from localhost ([::1]:35321 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQISO-0001mH-PU for importer@patchew.org; Wed, 28 Jun 2017 15:16:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40862) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEe-0005Nu-QM for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEb-00005R-J2 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37272) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEb-000059-AZ for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:41 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3CD627D0E8; Wed, 28 Jun 2017 19:02:40 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3908D17D24; Wed, 28 Jun 2017 19:02:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 3CD627D0E8 Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 3CD627D0E8 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:40 +0100 Message-Id: <20170628190047.26159-23-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 28 Jun 2017 19:02:40 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 22/29] vhost+postcopy: Call wakeups X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Cause the vhost-user client to be woken up whenever: a) We place a page in postcopy mode b) We get a fault and the page has already been received Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 14 ++++++++++---- migration/trace-events | 1 + 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index e6b8160f09..b97fc9398b 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -491,7 +491,11 @@ int postcopy_request_shared_page(struct PostCopyFD *pc= fd, RAMBlock *rb, =20 trace_postcopy_request_shared_page(pcfd->idstr, qemu_ram_get_idstr(rb), rb_offset); - /* TODO: Check bitmap to see if we already have the page */ + if (ramblock_recv_bitmap_test_byte_offset(aligned_rbo, rb)) { + trace_postcopy_request_shared_page_present(pcfd->idstr, + qemu_ram_get_idstr(rb), rb_offset); + return postcopy_wake_shared(pcfd, client_addr, rb); + } if (rb !=3D mis->last_rb) { mis->last_rb =3D rb; migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), @@ -792,7 +796,8 @@ int postcopy_place_page(MigrationIncomingState *mis, vo= id *host, void *from, } =20 trace_postcopy_place_page(host); - return 0; + return postcopy_notify_shared_wake(rb, + qemu_ram_block_host_offset(rb, host= )); } =20 /* @@ -813,6 +818,9 @@ int postcopy_place_page_zero(MigrationIncomingState *mi= s, void *host, =20 return -e; } + return postcopy_notify_shared_wake(rb, + qemu_ram_block_host_offset(rb, + host= )); } else { /* The kernel can't use UFFDIO_ZEROPAGE for hugepages */ if (!mis->postcopy_tmp_zero_page) { @@ -832,8 +840,6 @@ int postcopy_place_page_zero(MigrationIncomingState *mi= s, void *host, return postcopy_place_page(mis, host, mis->postcopy_tmp_zero_page, rb); } - - return 0; } =20 /* diff --git a/migration/trace-events b/migration/trace-events index 85a35be518..b2f7d85704 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -198,6 +198,7 @@ postcopy_ram_incoming_cleanup_entry(void) "" postcopy_ram_incoming_cleanup_exit(void) "" postcopy_ram_incoming_cleanup_join(void) "" postcopy_request_shared_page(const char *sharer, const char *rb, uint64_t = rb_offset) "for %s in %s offset %"PRIx64 +postcopy_request_shared_page_present(const char *sharer, const char *rb, u= int64_t rb_offset) "%s already %s offset %"PRIx64 postcopy_wake_shared(uint64_t client_addr, const char *rb) "at %"PRIx64" i= n %s" =20 save_xbzrle_page_skipping(void) "" --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677608547903.9016379947091; Wed, 28 Jun 2017 12:20:08 -0700 (PDT) Received: from localhost ([::1]:35331 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIVS-0004g0-4W for importer@patchew.org; Wed, 28 Jun 2017 15:20:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40896) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEi-0005Rl-K3 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEh-00008W-EN for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:48 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48090) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEh-00008E-63 for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:47 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1ED3C4E4D0; Wed, 28 Jun 2017 19:02:46 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8015F5DC1D; Wed, 28 Jun 2017 19:02:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1ED3C4E4D0 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 1ED3C4E4D0 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:41 +0100 Message-Id: <20170628190047.26159-24-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 28 Jun 2017 19:02:46 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 23/29] vub+postcopy: madvises X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Clear the area and turn off THP. Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index 0658b6e847..ceddeac74f 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -451,11 +451,39 @@ vu_set_mem_table_exec(VuDev *dev, VhostUserMsg *vmsg) } =20 if (dev->postcopy_listening) { + int ret; /* We should already have an open ufd need to mark each memory * range as ufd. - * Note: Do we need any madvises? Well it's not been accessed - * yet, still probably need no THP to be safe, discard to be s= afe? */ + + /* Discard any mapping we have here; note I can't use MADV_REM= OVE + * or fallocate to make the hole since I don't want to lose + * data that's already arrived in the shared process. + * TODO: How to do hugepage + */ + ret =3D madvise((void *)dev_region->mmap_addr, + dev_region->size + dev_region->mmap_offset, + MADV_DONTNEED); + if (ret) { + fprintf(stderr, + "%s: Failed to madvise(DONTNEED) region %d: %s\n", + __func__, i, strerror(errno)); + } + /* Turn off transparent hugepages so we dont get lose wakeups + * in neighbouring pages. + * TODO: Turn this backon later. + */ + ret =3D madvise((void *)dev_region->mmap_addr, + dev_region->size + dev_region->mmap_offset, + MADV_NOHUGEPAGE); + if (ret) { + /* Note: This can happen legally on kernels that are confi= gured + * without madvise'able hugepages + */ + fprintf(stderr, + "%s: Failed to madvise(NOHUGEPAGE) region %d: %s\n= ", + __func__, i, strerror(errno)); + } struct uffdio_register reg_struct; /* Note: We might need to go back to using mmap_addr and * len + mmap_offset for * huge pages, but then we do hope not= to --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677704165371.85861174875913; Wed, 28 Jun 2017 12:21:44 -0700 (PDT) Received: from localhost ([::1]:35341 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIWz-0005od-Mc for importer@patchew.org; Wed, 28 Jun 2017 15:21:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40944) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEp-0005Xh-0n for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEl-0000EV-Rz for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45222) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEl-0000ED-Jr for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:51 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 96B24C04B302; Wed, 28 Jun 2017 19:02:50 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 688AE17D14; Wed, 28 Jun 2017 19:02:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 96B24C04B302 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 96B24C04B302 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:42 +0100 Message-Id: <20170628190047.26159-25-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 28 Jun 2017 19:02:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 24/29] vhost+postcopy: Lock around set_mem_table X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" **HACK - better solution needed ** We have the situation where: qemu bridge send set_mem_table map memory a) mark area with UFD send reply with map addresses b) start using c) receive reply As soon as (a) happens qemu might start seeing faults from memory accesses (but doesn't until b); but it can't process those faults until (c) when it's received the mmap addresses. Make the fault handler spin until it gets the reply in (c). At the very least this needs some proper locks, but preferably we need to split the message. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/vhost-user.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 0f2e05f817..74e4313782 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -138,6 +138,7 @@ struct vhost_user { * vhost region. */ ram_addr_t region_rb_offset[VHOST_MEMORY_MAX_NREGIONS]; + uint64_t in_set_mem_table; /*Hack! 1 while waiting for set_m= em_table reply */ }; =20 static bool ioeventfd_enabled(void) @@ -321,6 +322,7 @@ static int vhost_user_set_mem_table(struct vhost_dev *d= ev, msg.flags |=3D VHOST_USER_NEED_REPLY_MASK; } =20 + atomic_set(&u->in_set_mem_table, true); for (i =3D 0; i < dev->mem->nregions; ++i) { struct vhost_memory_region *reg =3D dev->mem->regions + i; ram_addr_t offset; @@ -351,14 +353,15 @@ static int vhost_user_set_mem_table(struct vhost_dev = *dev, if (!fd_num) { error_report("Failed initializing vhost-user memory map, " "consider using -object memory-backend-file share=3Do= n"); + atomic_set(&u->in_set_mem_table, false); return -1; } =20 msg.size =3D sizeof(msg.payload.memory.nregions); msg.size +=3D sizeof(msg.payload.memory.padding); msg.size +=3D fd_num * sizeof(VhostUserMemoryRegion); - if (vhost_user_write(dev, &msg, fds, fd_num) < 0) { + atomic_set(&u->in_set_mem_table, false); return -1; } =20 @@ -373,6 +376,7 @@ static int vhost_user_set_mem_table(struct vhost_dev *d= ev, error_report("%s: Received unexpected msg type." "Expected %d received %d", __func__, VHOST_USER_SET_MEM_TABLE, msg_reply.request); + atomic_set(&u->in_set_mem_table, false); return -1; } /* We're using the same structure, just reusing one of the @@ -381,6 +385,7 @@ static int vhost_user_set_mem_table(struct vhost_dev *d= ev, if (msg_reply.size !=3D msg.size) { error_report("%s: Unexpected size for postcopy reply " "%d vs %d", __func__, msg_reply.size, msg.size); + atomic_set(&u->in_set_mem_table, false); return -1; } =20 @@ -410,9 +415,11 @@ static int vhost_user_set_mem_table(struct vhost_dev *= dev, error_report("%s: postcopy reply not fully consumed " "%d vs %zd", __func__, reply_i, fd_num); + atomic_set(&u->in_set_mem_table, false); return -1; } } + atomic_set(&u->in_set_mem_table, false); if (reply_supported) { return process_message_reply(dev, &msg); } @@ -821,6 +828,11 @@ static int vhost_user_postcopy_waker(struct PostCopyFD= *pcfd, RAMBlock *rb, int i; =20 trace_vhost_user_postcopy_waker(qemu_ram_get_idstr(rb), offset); + while (atomic_mb_read(&u->in_set_mem_table)) { + fprintf(stderr, "%s: Spin waiting for memtable\n", __func__); + usleep(1000*100); + } + /* Translate the offset into an address in the clients address space */ for (i =3D 0; i < dev->mem->nregions; i++) { if (u->region_rb[i] =3D=3D rb && --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677526277799.0466678625729; Wed, 28 Jun 2017 12:18:46 -0700 (PDT) Received: from localhost ([::1]:35327 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIU7-0002zn-TI for importer@patchew.org; Wed, 28 Jun 2017 15:18:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40957) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEq-0005Yv-7e for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEp-0000Fg-ID for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43170) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEp-0000FK-Cz for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:55 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 67B5EC0587E4; Wed, 28 Jun 2017 19:02:54 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id B050717D24; Wed, 28 Jun 2017 19:02:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 67B5EC0587E4 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 67B5EC0587E4 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:43 +0100 Message-Id: <20170628190047.26159-26-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 28 Jun 2017 19:02:54 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 25/29] vhu: enable = false on get_vring_base X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" When we receive a GET_VRING_BASE message set enable =3D false to stop any new received packets modifying the ring. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Maxime Coquelin --- contrib/libvhost-user/libvhost-user.c | 1 + 1 file changed, 1 insertion(+) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index ceddeac74f..d37052b7b0 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -652,6 +652,7 @@ vu_get_vring_base_exec(VuDev *dev, VhostUserMsg *vmsg) vmsg->size =3D sizeof(vmsg->payload.state); =20 dev->vq[index].started =3D false; + dev->vq[index].enable =3D false; if (dev->iface->queue_set_started) { dev->iface->queue_set_started(dev, index, false); } --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498676889501103.89833997484698; Wed, 28 Jun 2017 12:08:09 -0700 (PDT) Received: from localhost ([::1]:35271 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIJr-0001kt-05 for importer@patchew.org; Wed, 28 Jun 2017 15:08:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40976) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEt-0005c4-7P for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEs-0000Gc-Cf for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58026) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEs-0000GE-4D for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:02:58 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 192887F3E3; Wed, 28 Jun 2017 19:02:57 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id AEBE317D24; Wed, 28 Jun 2017 19:02:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 192887F3E3 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 192887F3E3 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:44 +0100 Message-Id: <20170628190047.26159-27-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 28 Jun 2017 19:02:57 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 26/29] vhost: Add VHOST_USER_POSTCOPY_END message X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" This message is sent just before the end of postcopy to get the client to stop using userfault since we wont respond to any more requests. It should close userfaultfd so that any other pages get mapped to the backing file automatically by the kernel, since at this point we know we've received everything. Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.c | 23 +++++++++++++++++++++++ contrib/libvhost-user/libvhost-user.h | 1 + hw/virtio/vhost-user.c | 1 + 3 files changed, 25 insertions(+) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index d37052b7b0..c1716d1a62 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -68,6 +68,7 @@ vu_request_to_string(int req) REQ(VHOST_USER_INPUT_GET_CONFIG), REQ(VHOST_USER_POSTCOPY_ADVISE), REQ(VHOST_USER_POSTCOPY_LISTEN), + REQ(VHOST_USER_POSTCOPY_END), REQ(VHOST_USER_MAX), }; #undef REQ @@ -889,6 +890,26 @@ vu_set_postcopy_listen(VuDev *dev, VhostUserMsg *vmsg) =20 return false; } + +static bool +vu_set_postcopy_end(VuDev *dev, VhostUserMsg *vmsg) +{ + fprintf(stderr, "%s: Entry\n", __func__); + dev->postcopy_listening =3D false; + if (dev->postcopy_ufd > 0) { + close(dev->postcopy_ufd); + dev->postcopy_ufd =3D -1; + fprintf(stderr, "%s: Done close\n", __func__); + } + + vmsg->fd_num =3D 0; + vmsg->payload.u64 =3D 0; + vmsg->size =3D sizeof(vmsg->payload.u64); + vmsg->flags =3D VHOST_USER_VERSION | VHOST_USER_REPLY_MASK; + fprintf(stderr, "%s: exit\n", __func__); + return true; +} + static bool vu_process_message(VuDev *dev, VhostUserMsg *vmsg) { @@ -956,6 +977,8 @@ vu_process_message(VuDev *dev, VhostUserMsg *vmsg) return vu_set_postcopy_advise(dev, vmsg); case VHOST_USER_POSTCOPY_LISTEN: return vu_set_postcopy_listen(dev, vmsg); + case VHOST_USER_POSTCOPY_END: + return vu_set_postcopy_end(dev, vmsg); default: vmsg_close_fds(vmsg); vu_panic(dev, "Unhandled request: %d", vmsg->request); diff --git a/contrib/libvhost-user/libvhost-user.h b/contrib/libvhost-user/= libvhost-user.h index 86e1934ddb..1665c729f0 100644 --- a/contrib/libvhost-user/libvhost-user.h +++ b/contrib/libvhost-user/libvhost-user.h @@ -65,6 +65,7 @@ typedef enum VhostUserRequest { VHOST_USER_INPUT_GET_CONFIG =3D 20, VHOST_USER_POSTCOPY_ADVISE =3D 23, VHOST_USER_POSTCOPY_LISTEN =3D 24, + VHOST_USER_POSTCOPY_END =3D 25, VHOST_USER_MAX } VhostUserRequest; =20 diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 74e4313782..b29a141703 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -69,6 +69,7 @@ typedef enum VhostUserRequest { VHOST_USER_IOTLB_MSG =3D 22, VHOST_USER_POSTCOPY_ADVISE =3D 23, VHOST_USER_POSTCOPY_LISTEN =3D 24, + VHOST_USER_POSTCOPY_END =3D 25, VHOST_USER_MAX } VhostUserRequest; =20 --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677630711659.0060775108519; Wed, 28 Jun 2017 12:20:30 -0700 (PDT) Received: from localhost ([::1]:35336 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIVo-00050D-AH for importer@patchew.org; Wed, 28 Jun 2017 15:20:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41002) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIEv-0005ee-QR for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIEu-0000Ha-PL for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:01 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58132) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIEu-0000HH-HN for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:00 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8696C7F3E3; Wed, 28 Jun 2017 19:02:59 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6410817D24; Wed, 28 Jun 2017 19:02:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8696C7F3E3 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 8696C7F3E3 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:45 +0100 Message-Id: <20170628190047.26159-28-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 28 Jun 2017 19:02:59 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 27/29] vhost+postcopy: Wire up POSTCOPY_END notify X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Wire up a call to VHOST_USER_POSTCOPY_END message to the vhost clients right before we ask the listener thread to shutdown. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/trace-events | 2 ++ hw/virtio/vhost-user.c | 30 ++++++++++++++++++++++++++++++ migration/postcopy-ram.c | 5 +++++ migration/postcopy-ram.h | 1 + 4 files changed, 38 insertions(+) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index e320a90941..f18b67a1f2 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -1,6 +1,8 @@ # See docs/tracing.txt for syntax documentation. =20 # hw/virtio/vhost-user.c +vhost_user_postcopy_end_entry(void) "" +vhost_user_postcopy_end_exit(void) "" vhost_user_postcopy_fault_handler(const char *name, uint64_t fault_address= , int nregions) "%s: @%"PRIx64" nregions:%d" vhost_user_postcopy_fault_handler_loop(int i, uint64_t client_base, uint64= _t size) "%d: client%"PRIx64" +%"PRIx64 vhost_user_postcopy_fault_handler_found(int i, uint64_t region_offset, uin= t64_t rb_offset) "%d: region_offset: %"PRIx64" rb_offset:%"PRIx64 diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index b29a141703..4388ce7c69 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -921,6 +921,33 @@ static int vhost_user_postcopy_listen(struct vhost_dev= *dev, Error **errp) return 0; } =20 +/* + * Called at the end of postcopy + */ +static int vhost_user_postcopy_end(struct vhost_dev *dev, Error **errp) +{ + VhostUserMsg msg =3D { + .request =3D VHOST_USER_POSTCOPY_END, + .flags =3D VHOST_USER_VERSION | VHOST_USER_NEED_REPLY_MASK, + }; + int ret; + + trace_vhost_user_postcopy_end_entry(); + if (vhost_user_write(dev, &msg, NULL, 0) < 0) { + error_setg(errp, "Failed to send postcopy_end to vhost"); + return -1; + } + + ret =3D process_message_reply(dev, &msg); + if (ret) { + error_setg(errp, "Failed to receive reply to postcopy_end"); + return ret; + } + trace_vhost_user_postcopy_end_exit(); + + return 0; +} + static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier, void *opaque) { @@ -946,6 +973,9 @@ static int vhost_user_postcopy_notifier(NotifierWithRet= urn *notifier, case POSTCOPY_NOTIFY_INBOUND_LISTEN: return vhost_user_postcopy_listen(dev, pnd->errp); =20 + case POSTCOPY_NOTIFY_INBOUND_END: + return vhost_user_postcopy_end(dev, pnd->errp); + default: /* We ignore notifications we don't know */ break; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index b97fc9398b..fdd53cdf1e 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -337,7 +337,12 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSta= te *mis) =20 if (mis->have_fault_thread) { uint64_t tmp64; + Error *local_err =3D NULL; =20 + if (postcopy_notify(POSTCOPY_NOTIFY_INBOUND_END, &local_err)) { + error_report_err(local_err); + return -1; + } if (qemu_ram_foreach_block(cleanup_range, mis)) { return -1; } diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index ecf731c689..d0dc838001 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -130,6 +130,7 @@ enum PostcopyNotifyReason { POSTCOPY_NOTIFY_PROBE =3D 0, POSTCOPY_NOTIFY_INBOUND_ADVISE, POSTCOPY_NOTIFY_INBOUND_LISTEN, + POSTCOPY_NOTIFY_INBOUND_END, }; =20 struct PostcopyNotifyData { --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 14986777863261008.988942920982; Wed, 28 Jun 2017 12:23:06 -0700 (PDT) Received: from localhost ([::1]:35353 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIYL-0006nr-4H for importer@patchew.org; Wed, 28 Jun 2017 15:23:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41035) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIF1-0005iI-Ev for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIF0-0000JP-HN for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55478) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIF0-0000JC-BO for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:06 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 51B2A8047E; Wed, 28 Jun 2017 19:03:05 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id C0C1A5DC1D; Wed, 28 Jun 2017 19:02:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 51B2A8047E Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 51B2A8047E From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:46 +0100 Message-Id: <20170628190047.26159-29-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 28 Jun 2017 19:03:05 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 28/29] postcopy: Allow shared memory X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Now that we have the mechanisms in here, allow shared memory in a postcopy. Signed-off-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index fdd53cdf1e..736d272965 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -138,12 +138,6 @@ static int test_ramblock_postcopiable(const char *bloc= k_name, void *host_addr, RAMBlock *rb =3D qemu_ram_block_by_name(block_name); size_t pagesize =3D qemu_ram_pagesize(rb); =20 - if (qemu_ram_is_shared(rb)) { - error_report("Postcopy on shared RAM (%s) is not yet supported", - block_name); - return 1; - } - if (length % pagesize) { error_report("Postcopy requires RAM blocks to be a page size multi= ple," " block %s is 0x" RAM_ADDR_FMT " bytes with a " --=20 2.13.0 From nobody Sun Apr 28 15:05:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1498677727504485.1262406435666; Wed, 28 Jun 2017 12:22:07 -0700 (PDT) Received: from localhost ([::1]:35342 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIXM-00066H-VT for importer@patchew.org; Wed, 28 Jun 2017 15:22:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41096) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dQIFC-0005rQ-If for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dQIF7-0000Lh-MI for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55742) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dQIF7-0000LO-Fp for qemu-devel@nongnu.org; Wed, 28 Jun 2017 15:03:13 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5D27880082; Wed, 28 Jun 2017 19:03:12 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-117-210.ams2.redhat.com [10.36.117.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id 91AF35DD71; Wed, 28 Jun 2017 19:03:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5D27880082 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dgilbert@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 5D27880082 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, maxime.coquelin@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com Date: Wed, 28 Jun 2017 20:00:47 +0100 Message-Id: <20170628190047.26159-30-dgilbert@redhat.com> In-Reply-To: <20170628190047.26159-1-dgilbert@redhat.com> References: <20170628190047.26159-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 28 Jun 2017 19:03:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 29/29] vhost-user: Claim support for postcopy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Tell QEMU we understand the protocol features needed for postcopy. Signed-off-by: Dr. David Alan Gilbert --- contrib/libvhost-user/libvhost-user.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/= libvhost-user.c index c1716d1a62..1c46aecfb3 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -797,7 +797,8 @@ vu_set_vring_err_exec(VuDev *dev, VhostUserMsg *vmsg) static bool vu_get_protocol_features_exec(VuDev *dev, VhostUserMsg *vmsg) { - uint64_t features =3D 1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD; + uint64_t features =3D 1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD | + 1ULL << VHOST_USER_PROTOCOL_F_POSTCOPY; =20 if (dev->iface->get_protocol_features) { features |=3D dev->iface->get_protocol_features(dev); --=20 2.13.0