From nobody Wed Nov 5 15:55:52 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1497544842848689.2058182390458; Thu, 15 Jun 2017 09:40:42 -0700 (PDT) Received: from localhost ([::1]:55068 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dLXp3-00035C-J6 for importer@patchew.org; Thu, 15 Jun 2017 12:40:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47855) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dLXl9-00087D-S6 for qemu-devel@nongnu.org; Thu, 15 Jun 2017 12:36:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dLXl5-0001ZN-09 for qemu-devel@nongnu.org; Thu, 15 Jun 2017 12:36:39 -0400 Received: from mailout2.w1.samsung.com ([210.118.77.12]:31501) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dLXl4-0001Xf-Nj for qemu-devel@nongnu.org; Thu, 15 Jun 2017 12:36:34 -0400 Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0ORL00GVDKSVDC40@mailout2.w1.samsung.com> for qemu-devel@nongnu.org; Thu, 15 Jun 2017 17:36:31 +0100 (BST) Received: from eusmges2.samsung.com (unknown [203.254.199.241]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170615163631eucas1p1332d7368ed12eb790b9d037940498f88~IWN8o3fXm1514815148eucas1p1a; Thu, 15 Jun 2017 16:36:31 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2.samsung.com (EUCPMTA) with SMTP id 7B.E2.04459.F87B2495; Thu, 15 Jun 2017 17:36:31 +0100 (BST) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170615163630eucas1p24f8f92a8acc7e408849696645f9557f2~IWN76ZEYz1661516615eucas1p2k; Thu, 15 Jun 2017 16:36:30 +0000 (GMT) Received: from eusync2.samsung.com ( [203.254.199.212]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 3B.C2.20206.E87B2495; Thu, 15 Jun 2017 17:36:30 +0100 (BST) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync2.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0ORL009RKKSLTR30@eusync2.samsung.com>; Thu, 15 Jun 2017 17:36:30 +0100 (BST) X-AuditID: cbfec7f1-f796e6d00000116b-60-5942b78fcace From: Alexey Perevalov To: qemu-devel@nongnu.org Date: Thu, 15 Jun 2017 19:36:15 +0300 Message-id: <1497544575-18628-4-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 1.9.1 In-reply-to: <1497544575-18628-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrDIsWRmVeSWpSXmKPExsWy7djP87r9250iDdbdELGYe/c8i0Xvtnvs Flfaf7JbbNn/jd3ieO8OFos7W/qYHNg8nlzbzOTxft9VNo++LasYA5ijuGxSUnMyy1KL9O0S uDIaLvxnK7iuU3Fszj32Bsbpyl2MnBwSAiYSHz7OZISwxSQu3FvPBmILCSxllFgzSaOLkQvI /swocXTyKTaYhl3XFzFDJJYxSqxed48Nwulmkji9to+pi5GDg03AQGLfPVuQBhEBSYnfXafB GpgFWhkl2vseMYMkhAXsJRq+zGIFsVkEVCXa//8Es3kF3CWmvf7LDrFNTuLksclgcU4BD4mH C/rAlkkI3GeT2Ne8nB1kmYSArMSmA8wQ9S4SGw6cZ4WwhSVeHd8CNUdG4vLkbhaI3nZGie6d nawQzgRGiTPTYbbZS5y6eZUJxGYW4JOYtG06M8QCXomONiGIEg+JpqctTBC2o8T2B8eZIL6f zSjRt3U66wRGmQWMDKsYRVJLi3PTU4uN9IoTc4tL89L1kvNzNzEC4/T0v+MfdzC+P2F1iFGA g1GJh1ehwSlSiDWxrLgy9xCjBAezkgjv1DlAId6UxMqq1KL8+KLSnNTiQ4zSHCxK4rxcp65F CAmkJ5akZqemFqQWwWSZODilGhiz7u5cvz7x/bto6f6i+Sc40vsnbbM+G9N61kpdUIHzgrRO EofYMsmjxeo7572+Yh6j1SLgFPkornPFDdMjTY6LVr8JfHpX+lzltvuWMil1l3OdsuaqfDt+ +d4V1UrDlWzt3O5ea2ZaGP6YKHh8ikKv91dza6tVfN6SG1tW5jfM/sFic/fAHRMlluKMREMt 5qLiRAC8SV7pzwIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrFLMWRmVeSWpSXmKPExsVy+t/xK7p9250iDfZdVrKYe/c8i0Xvtnvs Flfaf7JbbNn/jd3ieO8OFos7W/qYHNg8nlzbzOTxft9VNo++LasYA5ij3GwyUhNTUosUUvOS 81My89JtlUJD3HQtlBTyEnNTbZUidH1DgpQUyhJzSoE8IwM04OAc4B6spG+X4JbRcOE/W8F1 nYpjc+6xNzBOV+5i5OSQEDCR2HV9ETOELSZx4d56ti5GLg4hgSWMEjNXLWYHSQgJ9DJJzF6r 0MXIwcEmYCCx754tSFhEQFLid9dpZpB6ZoFWRoknjbPA6oUF7CUavsxiBbFZBFQl2v//BLN5 Bdwlpr3+yw6xTE7i5LHJYHFOAQ+Jhwv62CB2uUu8n7mQbQIj7wJGhlWMIqmlxbnpucVGesWJ ucWleel6yfm5mxiB4brt2M8tOxi73gUfYhTgYFTi4a1ocooUYk0sK67MPcQowcGsJMI7dQ5Q iDclsbIqtSg/vqg0J7X4EKMp0FETmaVEk/OBsZRXEm9oYmhuaWhkbGFhbmSkJM479cOVcCGB 9MSS1OzU1ILUIpg+Jg5OqQZG6dBd0w6JtAds+DXzsvO75z1vDNc+qZjqtfKbUf3h3WdctscV Zm3b1F8nYRq7e5ZImfF8m5C2BtupUi+Ev8hbWqatYrcqf7d9C0/l/4MBWyT+vX+yQ8gq+6dg 3swtl+4demqj3Fy4zn3qwyVf+t/MaHllEPns8saDpjf2e88s2SShaZMl9jWuT4mlOCPRUIu5 qDgRANzr/QltAgAA X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20170615163630eucas1p24f8f92a8acc7e408849696645f9557f2 X-Msg-Generator: CA X-Sender-IP: 182.198.249.180 X-Local-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG+yCvOyEseyghOyekBtTZW5pb3IgRW5naW5lZXI=?= X-Global-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG1NhbXN1bmcgRWxlY3Ryb25pY3MbU2VuaW9yIEVuZ2luZWVy?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20170615163630eucas1p24f8f92a8acc7e408849696645f9557f2 X-RootMTR: 20170615163630eucas1p24f8f92a8acc7e408849696645f9557f2 References: <1497544575-18628-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 210.118.77.12 Subject: [Qemu-devel] [PATCH v2 3/3] migration: add bitmap for received page X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: i.maximets@samsung.com, dgilbert@redhat.com, Alexey Perevalov , peterx@redhat.com, quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch adds ability to track down already received pages, it's necessary for calculation vCPU block time in postcopy migration feature, maybe for restore after postcopy migration failure. Also it's necessary to solve shared memory issue in postcopy livemigration. Information about received pages will be transferred to the software virtual bridge (e.g. OVS-VSWITCHD), to avoid fallocate (unmap) for already received pages. fallocate syscall is required for remmaped shared memory, due to remmaping itself blocks ioctl(UFFDIO_COPY, ioctl in this case will end with EEXIT error (struct page is exists after remmap). Bitmap is placed into RAMBlock as another postcopy/precopy related bitmaps. Signed-off-by: Alexey Perevalov Reviewed-by: Dr. David Alan Gilbert --- include/exec/ram_addr.h | 2 ++ migration/postcopy-ram.c | 11 ++++++++--- migration/ram.c | 33 +++++++++++++++++++++++++++++++++ migration/ram.h | 5 +++++ migration/savevm.c | 1 + 5 files changed, 49 insertions(+), 3 deletions(-) diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h index 140efa8..c2c1dfe 100644 --- a/include/exec/ram_addr.h +++ b/include/exec/ram_addr.h @@ -47,6 +47,8 @@ struct RAMBlock { * of the postcopy phase */ unsigned long *unsentmap; + /* bitmap of already received pages in postcopy */ + unsigned long *receivedmap; }; =20 static inline bool offset_in_ramblock(RAMBlock *b, ram_addr_t offset) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 38a429a..5ac3ed2 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -562,8 +562,13 @@ int postcopy_ram_enable_notify(MigrationIncomingState = *mis) } =20 static int qemu_ufd_copy_ioctl(int userfault_fd, void *host_addr, - void *from_addr, uint64_t pagesize) + void *from_addr, uint64_t pagesize, RAMBlock *rb) { + /* received page isn't feature of blocktime calculation, + * it's more general entity, so keep it here, + * but gup betwean two following operation could be high, + * and in this case blocktime for such small interval will be lost */ + set_receivedmap_by_addr(host_addr, rb); if (from_addr) { struct uffdio_copy copy_struct; copy_struct.dst =3D (uint64_t)(uintptr_t)host_addr; @@ -595,7 +600,7 @@ int postcopy_place_page(MigrationIncomingState *mis, vo= id *host, void *from, * which would be slightly cheaper, but we'd have to be careful * of the order of updating our page state. */ - if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, from, pagesize)) { + if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, from, pagesize, rb)) { int e =3D errno; error_report("%s: %s copy host: %p from: %p (size: %zd)", __func__, strerror(e), host, from, pagesize); @@ -619,7 +624,7 @@ int postcopy_place_page_zero(MigrationIncomingState *mi= s, void *host, =20 pagesize =3D qemu_ram_pagesize(rb); if (pagesize =3D=3D getpagesize()) { - if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, 0, getpagesize())= ) { + if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, 0, getpagesize(),= rb)) { int e =3D errno; error_report("%s: %s zero host: %p", __func__, strerror(e), host); diff --git a/migration/ram.c b/migration/ram.c index f50479d..f5ea3c2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -151,6 +151,34 @@ out: return ret; } =20 +void init_receivedmap(void) +{ + RAMBlock *rb; + + RAMBLOCK_FOREACH(rb) { + unsigned long pages; + pages =3D rb->max_length >> TARGET_PAGE_BITS; + rb->receivedmap =3D bitmap_new(pages); + } +} + +static unsigned long int get_received_bit_offset(void *host_addr, RAMBlock= *rb) +{ + uint64_t host_addr_offset =3D (uint64_t)(uintptr_t)(host_addr + - (void *)rb->host); + return host_addr_offset >> TARGET_PAGE_BITS; +} + +int test_receivedmap_by_addr(void *host_addr, RAMBlock *rb) +{ + return test_bit(get_received_bit_offset(host_addr, rb), rb->receivedma= p); +} + +void set_receivedmap_by_addr(void *host_addr, RAMBlock *rb) +{ + set_bit_atomic(get_received_bit_offset(host_addr, rb), rb->receivedmap= ); +} + /* * An outstanding page request, on the source, having been received * and queued @@ -2324,8 +2352,13 @@ static int ram_load_setup(QEMUFile *f, void *opaque) =20 static int ram_load_cleanup(void *opaque) { + RAMBlock *rb; xbzrle_load_cleanup(); compress_threads_load_cleanup(); + + RAMBLOCK_FOREACH(rb) { + g_free(rb->receivedmap); + } return 0; } =20 diff --git a/migration/ram.h b/migration/ram.h index c081fde..7048ff9 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -52,4 +52,9 @@ int ram_discard_range(const char *block_name, uint64_t st= art, size_t length); int ram_postcopy_incoming_init(MigrationIncomingState *mis); =20 void ram_handle_compressed(void *host, uint8_t ch, uint64_t size); + +void init_receivedmap(void); +int test_receivedmap_by_addr(void *host_addr, RAMBlock *rb); +void set_receivedmap_by_addr(void *host_addr, RAMBlock *rb); + #endif diff --git a/migration/savevm.c b/migration/savevm.c index 31158da..668d3bb 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1372,6 +1372,7 @@ static int loadvm_postcopy_handle_advise(MigrationInc= omingState *mis) return -1; } =20 + init_receivedmap(); remote_pagesize_summary =3D qemu_get_be64(mis->from_src_file); local_pagesize_summary =3D ram_pagesize_summary(); =20 --=20 1.9.1