From nobody Sat May 4 08:46:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 148985007034280.51318758786181; Sat, 18 Mar 2017 08:14:30 -0700 (PDT) Received: from localhost ([::1]:53644 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cpG3o-0002ka-Bt for importer@patchew.org; Sat, 18 Mar 2017 11:14:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60997) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cpG32-0002jm-OM for qemu-devel@nongnu.org; Sat, 18 Mar 2017 11:13:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cpG2z-0000MU-KC for qemu-devel@nongnu.org; Sat, 18 Mar 2017 11:13:40 -0400 Received: from mailout3.w1.samsung.com ([210.118.77.13]:54589) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cpG2z-0000Kh-Dc for qemu-devel@nongnu.org; Sat, 18 Mar 2017 11:13:37 -0400 Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout3.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0ON0002OZNMKE660@mailout3.w1.samsung.com> for qemu-devel@nongnu.org; Sat, 18 Mar 2017 15:13:32 +0000 (GMT) Received: from eusmges2.samsung.com (unknown [203.254.199.241]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170318151331eucas1p22895e14258f1765479f64eb00a9a67b6~tArEYdQQk3118131181eucas1p2A; Sat, 18 Mar 2017 15:13:31 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges2.samsung.com (EUCPMTA) with SMTP id AA.19.30614.B9E4DC85; Sat, 18 Mar 2017 15:13:31 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170318151330eucas1p102b9378cc23a399528fa7f2440846650~tArDt3H_k1886718867eucas1p14; Sat, 18 Mar 2017 15:13:30 +0000 (GMT) Received: from eusync4.samsung.com ( [203.254.199.214]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 0E.4C.10233.8AE4DC85; Sat, 18 Mar 2017 15:13:44 +0000 (GMT) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync4.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0ON0000UPNMF5X60@eusync4.samsung.com>; Sat, 18 Mar 2017 15:13:30 +0000 (GMT) X-AuditID: cbfec7f1-f793f6d000007796-c6-58cd4e9b67be From: Alexey Perevalov To: "Dr. David Alan Gilbert" , qemu-devel@nongnu.org Date: Sat, 18 Mar 2017 18:13:22 +0300 Message-id: <1489850003-5652-2-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 1.9.1 In-reply-to: <1489850003-5652-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrIIsWRmVeSWpSXmKPExsWy7djPc7qz/c5GGOxtYLeYe/c8i0Xvtnvs Flfaf7JbHO/dweLA4vHk2mYmj/f7rrJ59G1ZxRjAHMVlk5Kak1mWWqRvl8CVMevkTtaCt8wV e+c9Ympg7GPuYuTkkBAwkXjWeAHKFpO4cG89WxcjF4eQwFJGif2dK5khnM+MEhMaZsN1rFq4 HSqxjFHi4IklUC3dTBJzO9czdTFycLAJGEjsu2cL0iAi4CnReeAeK4jNLGAssWbGKzBbWMBU YlL/TXYQm0VAVeLRwQtMIDavgJvE7a3LWSCWyUmcPDYZrJ5TwF3i7vvZLCC7JAQus0m8mdPO BrJLQkBWYtMBqONcJNo3/YTqFZZ4dXwLO4QtI3F5cjdUbzujRPfOTlYIZwKjxJnpf6Gq7CVO 3bzKBHEpn8SkbdOZIRbwSnS0CUGUeEj87/8BFXaUOHS7CuL3WYwS5863ME5glFnAyLCKUSS1 tDg3PbXYSK84Mbe4NC9dLzk/dxMjMCJP/zv+cQfj+xNWhxgFOBiVeHgNnp+OEGJNLCuuzD3E KMHBrCTC+8DybIQQb0piZVVqUX58UWlOavEhRmkOFiVx3j0LroQLCaQnlqRmp6YWpBbBZJk4 OKUaGOdo8Xf8+tX+4sXD2Y82Pyt46e0Tuv9t8Wzp5e49bYHbvu42y2XSvRnzWfnK9zU14d0r 3DZXhWaUfgtTLDigHuxWtsMuOmD+W5btCTJnQ8wftH7e/W1H2vFppSFXXOSfm35fb6zJOtNw y5ob0iZlzp+zJ6isMgudruNrUPDlKH9f79rasutf7iixFGckGmoxFxUnAgD1rERQxAIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrALMWRmVeSWpSXmKPExsVy+t/xa7or/M5GGByap2Qx9+55FovebffY La60/2S3ON67g8WBxePJtc1MHu/3XWXz6NuyijGAOcrNJiM1MSW1SCE1Lzk/JTMv3VYpNMRN 10JJIS8xN9VWKULXNyRISaEsMacUyDMyQAMOzgHuwUr6dgluGbNO7mQteMtcsXfeI6YGxj7m LkZODgkBE4lVC7dD2WISF+6tZ+ti5OIQEljCKLH1VT8rhNPLJLH48nfGLkYODjYBA4l992xB GkQEPCU6D9xjBbGZBYwl1sx4BWYLC5hKTOq/yQ5iswioSjw6eIEJxOYVcJO4vXU5C8QyOYmT xyaD1XMKuEvcfT8bLC4EVHP47FTGCYy8CxgZVjGKpJYW56bnFhvpFSfmFpfmpesl5+duYgSG 5rZjP7fsYOx6F3yIUYCDUYmH98aT0xFCrIllxZW5hxglOJiVRHgfWJ6NEOJNSaysSi3Kjy8q zUktPsRoCnTURGYp0eR8YNzklcQbmhiaWxoaGVtYmBsZKYnzTv1wJVxIID2xJDU7NbUgtQim j4mDU6qB0Uf1brSUp64C77rKiH/LP9teY12ndE24tIXz3JTDmTMu27bFnn53iDvu8ZZq79vn f7XXZE+cdvvc5i+HDzp9OCYZf227iFes5topUhekK3+7c/Jq2IstzTydLnR3uvbn5Wf7XLe9 luiIPHvyQlHrBIsDm0+aXp7IpfHjSuHj5s2J+v7HjL+erVBiKc5INNRiLipOBABecvvWYwIA AA== X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20170318151330eucas1p102b9378cc23a399528fa7f2440846650 X-Msg-Generator: CA X-Sender-IP: 182.198.249.180 X-Local-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG+yCvOyEseyghOyekBtTZW5pb3IgRW5naW5lZXI=?= X-Global-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG1NhbXN1bmcgRWxlY3Ryb25pY3MbU2VuaW9yIEVuZ2luZWVy?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20170318151330eucas1p102b9378cc23a399528fa7f2440846650 X-RootMTR: 20170318151330eucas1p102b9378cc23a399528fa7f2440846650 References: <1489850003-5652-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 210.118.77.13 Subject: [Qemu-devel] [PATCH 1/2] userfault: add pid into uffd_msg X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: i.maximets@samsung.com, a.perevalov@samsung.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Alexey Perevalov --- linux-headers/linux/userfaultfd.h | 1 + 1 file changed, 1 insertion(+) diff --git a/linux-headers/linux/userfaultfd.h b/linux-headers/linux/userfa= ultfd.h index 2ed5dc3..7b299a2 100644 --- a/linux-headers/linux/userfaultfd.h +++ b/linux-headers/linux/userfaultfd.h @@ -77,6 +77,7 @@ struct uffd_msg { struct { __u64 flags; __u64 address; + pid_t pid; } pagefault; =20 struct { --=20 1.8.3.1 From nobody Sat May 4 08:46:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1489850160731100.42555704920767; Sat, 18 Mar 2017 08:16:00 -0700 (PDT) Received: from localhost ([::1]:53655 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cpG5H-0003pS-Bx for importer@patchew.org; Sat, 18 Mar 2017 11:15:59 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60993) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cpG32-0002jk-Kq for qemu-devel@nongnu.org; Sat, 18 Mar 2017 11:13:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cpG2z-0000Mg-Pb for qemu-devel@nongnu.org; Sat, 18 Mar 2017 11:13:40 -0400 Received: from mailout4.w1.samsung.com ([210.118.77.14]:57687) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cpG2z-0000Jv-Gh for qemu-devel@nongnu.org; Sat, 18 Mar 2017 11:13:37 -0400 Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout4.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0ON000EPONML7160@mailout4.w1.samsung.com> for qemu-devel@nongnu.org; Sat, 18 Mar 2017 15:13:33 +0000 (GMT) Received: from eusmges3.samsung.com (unknown [203.254.199.242]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170318151333eucas1p11e410ae044b38a08558e3032f344091f~tArGBpIkv1897918979eucas1p1b; Sat, 18 Mar 2017 15:13:33 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3.samsung.com (EUCPMTA) with SMTP id B0.C7.09557.C9E4DC85; Sat, 18 Mar 2017 15:13:32 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170318151332eucas1p2e375c85a501455f59d6f132da36ae844~tArFLKbDH1799217992eucas1p20; Sat, 18 Mar 2017 15:13:32 +0000 (GMT) Received: from eusync4.samsung.com ( [203.254.199.214]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 78.E9.06687.40F4DC85; Sat, 18 Mar 2017 15:15:16 +0000 (GMT) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync4.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0ON0000UPNMF5X60@eusync4.samsung.com>; Sat, 18 Mar 2017 15:13:32 +0000 (GMT) X-AuditID: cbfec7f2-f790f6d000002555-fe-58cd4e9c2e00 From: Alexey Perevalov To: "Dr. David Alan Gilbert" , qemu-devel@nongnu.org Date: Sat, 18 Mar 2017 18:13:23 +0300 Message-id: <1489850003-5652-3-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 1.9.1 In-reply-to: <1489850003-5652-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrIIsWRmVeSWpSXmKPExsWy7djP87pz/M5GGFxcbWox9+55FovebffY La60/2S3ON67g8WBxePJtc1MHu/3XWXz6NuyijGAOYrLJiU1J7MstUjfLoEro2nRR8aCE/2M FYvnfmNtYDyQ28XIySEhYCJx6u8kdghbTOLCvfVsXYxcHEICSxklfh5fzgjhfGaU6D97lg2m Y0bzARaIxDJGif39E5khnG4miftPfwFlODjYBAwk9t2zBWkQEfCU6DxwjxXEZhYwllgz4xWY LSxgJ/H4xA2w1SwCqhKrdq1jArF5Bdwkeo91M0Isk5M4eWwyWD2ngLvE3fezWSDil9kkXm6r AFklISArsekAM4TpInF7izNEhbDEq+NboB6TkejsOMgEcqWEQDujRPfOTlYIZwKjxJnpf6Gq 7CVO3bzKBHEnn8SkbdOhhvJKdLQJQZR4SHS8nAxV7ijxdskmVojXZzFKbJz5hGkCo8wCRoZV jCKppcW56anFxnrFibnFpXnpesn5uZsYgRF5+t/xTzsYv56wOsQowMGoxMN748npCCHWxLLi ytxDjBIczEoivA8sz0YI8aYkVlalFuXHF5XmpBYfYpTmYFES592z4Eq4kEB6YklqdmpqQWoR TJaJg1OqgdGVyy5n3fbWw6emmZo7reayKbXp2tM8R+/sWuFfNbHur7YcdVpS+Enk7WOhJS3W gZr2v3/PeOlu33Tu5h99xpkrM9f9Uzj75IlH7rzuyrBPPDltS55xKk5TObHXKXWFTPzUL05L 9LJ09A3rSxam3la07nG28spatHiaVMMU4YdRtb/kvebF/lFiKc5INNRiLipOBAD6bq8jxAIA AA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrALMWRmVeSWpSXmKPExsVy+t/xa7os/mcjDDbuVLOYe/c8i0Xvtnvs Flfaf7JbHO/dweLA4vHk2mYmj/f7rrJ59G1ZxRjAHOVmk5GamJJapJCal5yfkpmXbqsUGuKm a6GkkJeYm2qrFKHrGxKkpFCWmFMK5BkZoAEH5wD3YCV9uwS3jKZFHxkLTvQzViye+421gfFA bhcjJ4eEgInEjOYDLBC2mMSFe+vZuhi5OIQEljBK7Hx3lQ0kISTQyyTReju+i5GDg03AQGLf PVuQsIiAp0TngXusIDazgLHEmhmvwGxhATuJxydusIPYLAKqEqt2rWMCsXkF3CR6j3UzQuyS kzh5bDJYPaeAu8Td97NZIFa5SRw+O5VxAiPvAkaGVYwiqaXFuem5xYZ6xYm5xaV56XrJ+bmb GIGhue3Yz807GC9tDD7EKMDBqMTDe+PJ6Qgh1sSy4srcQ4wSHMxKIrwPLM9GCPGmJFZWpRbl xxeV5qQWH2I0BTpqIrOUaHI+MG7ySuINTQzNLQ2NjC0szI2MlMR5Sz5cCRcSSE8sSc1OTS1I LYLpY+LglGpgrHHICYgQ3Hx5lyvn/pddJp8K2Tc/7N33b8P158tUb1c1sZxdHHCjqu2j3wK+ u+1zNZ2/3pxwVmhH3hXGmsef5TLlP69tfs4XcvFKbMK/v8kCE9sD9c69Zpm3oClxX63zpAcx 7OIuXPpbf2huW67LlFSa7Sen9/SAmlzUKwsZv8f391h6Xn4trcRSnJFoqMVcVJwIAKD8ZPBj AgAA X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20170318151332eucas1p2e375c85a501455f59d6f132da36ae844 X-Msg-Generator: CA X-Sender-IP: 182.198.249.179 X-Local-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG+yCvOyEseyghOyekBtTZW5pb3IgRW5naW5lZXI=?= X-Global-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG1NhbXN1bmcgRWxlY3Ryb25pY3MbU2VuaW9yIEVuZ2luZWVy?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20170318151332eucas1p2e375c85a501455f59d6f132da36ae844 X-RootMTR: 20170318151332eucas1p2e375c85a501455f59d6f132da36ae844 References: <1489850003-5652-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 210.118.77.14 Subject: [Qemu-devel] [PATCH 2/2] migration: calculate downtime on dst side X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: i.maximets@samsung.com, a.perevalov@samsung.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch provides downtime calculation per vCPU, as a summary and as a overlapped value for all vCPUs. This approach just keeps tree with page fault addr as a key, and t1-t2 interval of pagefault time and page copy time, with affected vCPU bit mask. For more implementation details please see comment to get_postcopy_total_downtime function. Signed-off-by: Alexey Perevalov --- include/migration/migration.h | 11 ++ migration/migration.c | 238 ++++++++++++++++++++++++++++++++++++++= +++- migration/postcopy-ram.c | 61 ++++++++++- migration/savevm.c | 2 + migration/trace-events | 10 +- 5 files changed, 318 insertions(+), 4 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index 5720c88..8f9af77 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -123,10 +123,21 @@ struct MigrationIncomingState { =20 /* See savevm.c */ LoadStateEntry_Head loadvm_handlers; + + /* + * Tree for keeping postcopy downtime, + * necessary to calculate correct downtime, during multiple + * vm suspends, it keeps host page address as a key and + * DowntimeDuration as a data + */ + GTree *postcopy_downtime; }; =20 MigrationIncomingState *migration_incoming_get_current(void); void migration_incoming_state_destroy(void); +void mark_postcopy_downtime_begin(uint64_t addr, int cpu); +void mark_postcopy_downtime_end(uint64_t addr); +int64_t get_postcopy_total_downtime(void); =20 /* * An outstanding page request, on the source, having been received diff --git a/migration/migration.c b/migration/migration.c index 54060f7..57d71e1 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -77,6 +77,12 @@ static NotifierList migration_state_notifiers =3D =20 static bool deferred_incoming; =20 +typedef struct { + int64_t begin; + int64_t end; + uint64_t cpus; +} DowntimeDuration; + /* * Current state of incoming postcopy; note this is not part of * MigrationIncomingState since it's state is used during cleanup @@ -117,6 +123,21 @@ MigrationState *migrate_get_current(void) return ¤t_migration; } =20 +static gint addr_compare(gconstpointer a, gconstpointer b, + gpointer user_data G_GNUC_UNUSED) +{ + if (a =3D=3D b) + return 0; + else if (a > b) + return 1; + return -1; +} + +static void destroy_downtime_duration(gpointer data) +{ + free(data); +} + MigrationIncomingState *migration_incoming_get_current(void) { static bool once; @@ -128,6 +149,9 @@ MigrationIncomingState *migration_incoming_get_current(= void) QLIST_INIT(&mis_current.loadvm_handlers); qemu_mutex_init(&mis_current.rp_mutex); qemu_event_init(&mis_current.main_thread_load_event, false); + mis_current.postcopy_downtime =3D g_tree_new_full(addr_compare, + NULL, NULL, + destroy_downtime_duration); once =3D true; } return &mis_current; @@ -138,10 +162,13 @@ void migration_incoming_state_destroy(void) struct MigrationIncomingState *mis =3D migration_incoming_get_current(= ); =20 qemu_event_destroy(&mis->main_thread_load_event); + if (mis->postcopy_downtime) { + g_tree_destroy(mis->postcopy_downtime); + mis->postcopy_downtime =3D NULL; + } loadvm_free_handlers(mis); } =20 - typedef struct { bool optional; uint32_t size; @@ -1119,7 +1146,6 @@ MigrationState *migrate_init(const MigrationParams *p= arams) s->last_req_rb =3D NULL; error_free(s->error); s->error =3D NULL; - migrate_set_state(&s->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_S= ETUP); =20 QSIMPLEQ_INIT(&s->src_page_requests); @@ -2109,3 +2135,211 @@ PostcopyState postcopy_state_set(PostcopyState new_= state) return atomic_xchg(&incoming_postcopy_state, new_state); } =20 +void mark_postcopy_downtime_begin(uint64_t addr, int cpu) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + DowntimeDuration *dd; + if (!mis || !mis->postcopy_downtime) { + error_report("Migration incoming state should exists mis %p", mis); + return; + } + + dd =3D g_tree_lookup(mis->postcopy_downtime, (gpointer)addr); /* !!! c= ast */ + if (!dd) { + dd =3D (DowntimeDuration *)g_malloc0(sizeof(DowntimeDuration)); + g_tree_insert(mis->postcopy_downtime, (gpointer)addr, (gpointer)dd= ); + } + + if (cpu < 0) + /* assume in this situation all vCPUs are sleeping */ + dd->cpus =3D ~0u; + else + set_bit(cpu, &dd->cpus); + + /* + * overwrite previously set dd->begin, if that page already was + * faulted on another cpu + */ + dd->begin =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + trace_mark_postcopy_downtime_begin(addr, dd, dd->begin, cpu); +} + +void mark_postcopy_downtime_end(uint64_t addr) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + DowntimeDuration *dd; + if (!mis || !mis->postcopy_downtime) { + error_report("Migration incoming state should exists mis %p", mis); + return; + } + + dd =3D g_tree_lookup(mis->postcopy_downtime, (gpointer)addr); + if (!dd) { + /* error_report("Could not populate downtime duration completion t= ime \n\ + There is no downtime duration for 0x%"PRIx64, addr= ); */ + return; + } + + dd->end =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + trace_mark_postcopy_downtime_end(addr, dd, dd->end); +} + +typedef struct { + int64_t tp; /* point in time */ + bool is_end; + int64_t cpus; /* cpus bit mask */ +} OverlapDowntime; + +static gboolean populate_downtime_points(gpointer key, gpointer value, + gpointer data) +{ + DowntimeDuration *dd =3D (DowntimeDuration *)value; + GPtrArray *interval =3D (GPtrArray *)data; + OverlapDowntime *od_begin =3D g_malloc0(sizeof(OverlapDowntime)); + OverlapDowntime *od_end =3D g_malloc0(sizeof(OverlapDowntime)); + + od_begin->tp =3D dd->begin; + od_begin->is_end =3D false; + od_begin->cpus =3D dd->cpus; + g_ptr_array_add(interval, od_begin); + + od_end->tp =3D dd->end; + od_end->is_end =3D true; + od_end->cpus =3D dd->cpus; + g_ptr_array_add(interval, od_end); + + if (dd->end && dd->begin) + trace_sumup_downtime_duration(dd->end - dd->begin, (uint64_t)key, = dd->cpus); + return FALSE; +} + +static gboolean calculate_per_cpu(gpointer key, gpointer value, + gpointer data) +{ + int *downtime_cpu =3D (int *)data; + DowntimeDuration *dd =3D (DowntimeDuration *)value; + int cpu_iter; + for (cpu_iter =3D 0; cpu_iter < smp_cpus; cpu_iter++) { + if (test_bit(cpu_iter, &dd->cpus) && dd->end && dd->begin) + downtime_cpu[cpu_iter] +=3D dd->end - dd->begin; + } + return FALSE; +} + +static gint compare_downtime(gconstpointer a, gconstpointer b) +{ + DowntimeDuration *dda =3D (DowntimeDuration *)a; + DowntimeDuration *ddb =3D (DowntimeDuration *)b; + return dda->begin - ddb->begin; +} + +static uint64_t get_sufficient_smp_cpus(void) +{ + int i; + static uint64_t sufficient_cpus; + for (i =3D 0; i < smp_cpus; i++) + { + set_bit(i, &sufficient_cpus); + } + return sufficient_cpus; +} + +/* + * This function calculates downtime per cpu and trace it + * + * Also it calculates total downtime as an interval's overlap, + * for many vCPU. + * + * The approach is following: + * Initially intervals are represented in tree where key is + * pagefault address, and values: + * begin - page fault time + * end - page load time + * cpus - bit mask shows affected cpus + * + * To calculate overlap on all cpus, intervals converted into + * array of points in time (downtime_points), the size of + * array is 2 * number of nodes in tree of intervals (2 array + * elements per one in element of interval). + * Each element is marked as end (E) or as start (S) of interval. + * The overlap downtime will be calculated for SE, only in case + * there is sequence S(0..N)E(M) for every vCPU. + * + * As example we have 3 CPU + * + * S1 E1 S1 E1 + * -----***********------------xxx***************------------------------>= CPU1 + * + * S2 E2 + * ------------****************xxx--------------------------------------->= CPU2 + * + * S3 E3 + * ------------------------****xxx********------------------------------->= CPU3 + * + * We have sequence S1,S2,E1,S3,S1,E2,E3,E1 + * S2,E1 - doesn't match condition due to sequence S1,S2,E1 doesn't includ= e CPU3, + * S3,S1,E2 - sequenece includes all CPUs, in this case overlap will be S1= ,E2 + * + */ +int64_t get_postcopy_total_downtime(void) +{ + MigrationIncomingState *mis =3D migration_incoming_get_current(); + int64_t total_downtime =3D 0; /* for total overlapped downtime */ + const int intervals =3D g_tree_nnodes(mis->postcopy_downtime); + const int points =3D 2 * intervals; + uint64_t sufficient_smp_cpus =3D get_sufficient_smp_cpus(); + int point_iter, start_point_iter; + GPtrArray *downtime_points =3D g_ptr_array_sized_new(points); + /* for summary downtime per cpu */ + int *downtime_cpu =3D g_malloc0(smp_cpus * sizeof(int)); + if (!mis || !mis->postcopy_downtime) { + error_report("Migration incoming state should exists, mis %p", mis= ); + return -1; + } + + /* make downtime points S/E from interval */ + g_tree_foreach(mis->postcopy_downtime, populate_downtime_points, + downtime_points); + g_tree_foreach(mis->postcopy_downtime, calculate_per_cpu, downtime_cpu= ); + + /* just for RFC patch */ + for (point_iter =3D 0; point_iter < smp_cpus; point_iter++) + { + trace_downtime_per_cpu(point_iter, downtime_cpu[point_iter]); + } + + g_ptr_array_sort(downtime_points, compare_downtime); + + for (point_iter =3D 1; point_iter < points; point_iter++) { + OverlapDowntime *od =3D g_ptr_array_index(downtime_points, point_i= ter); + uint64_t cur_cpus =3D od->cpus; + int smp_cpus_i =3D smp_cpus; + OverlapDowntime *prev_od =3D g_ptr_array_index(downtime_points, + point_iter - 1); + /* we need sequence SE */ + if (!od->is_end || prev_od->is_end) + continue; + + for (start_point_iter =3D point_iter - 1; + start_point_iter >=3D 0 && smp_cpus_i; + start_point_iter--, smp_cpus_i--) { + OverlapDowntime *t_od =3D g_ptr_array_index(downtime_points, + start_point_iter); + /* should be S */ + if (t_od->is_end) + break; + + cur_cpus |=3D t_od->cpus; + if (sufficient_smp_cpus & cur_cpus) { + total_downtime +=3D od->tp - prev_od->tp; + /* situation when one S point represents all vCPU is possi= ble */ + break; + } + } + } + trace_get_postcopy_total_downtime(g_tree_nnodes(mis->postcopy_downtime= ), + total_downtime); + + g_ptr_array_free(downtime_points, TRUE); + return total_downtime; +} diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index dc80dbb..3bd9db0 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -23,6 +23,7 @@ #include "migration/postcopy-ram.h" #include "sysemu/sysemu.h" #include "sysemu/balloon.h" +#include #include "qemu/error-report.h" #include "trace.h" =20 @@ -404,6 +405,60 @@ static int ram_block_enable_notify(const char *block_n= ame, void *host_addr, return 0; } =20 +#define PROC_LEN 1024 +static void trace_for_thread(const char *msg, pid_t thread_id) +{ + const char *status_file_frmt =3D "/proc/%d/status"; + char status_file_path[MAXPATHLEN]; + char proc_name[PROC_LEN]; + char proc_status[PROC_LEN]; + char *line =3D NULL; + FILE *f; + ssize_t read; + size_t len; + + sprintf(status_file_path, status_file_frmt, thread_id); + f =3D fopen(status_file_path, "r"); + if (!f) { + error_report("can't open %s", status_file_path); + return; + } + + memset(proc_name, 0, sizeof(proc_name)); + memset(proc_status, 0, sizeof(proc_status)); + + while ((read =3D getline(&line, &len, f)) !=3D -1) { + if (strstr(line, "Name")) + strncpy(proc_name, line, sizeof(proc_name)); + if (strstr(line, "State")) + strncpy(proc_status, line, sizeof(proc_status)); + } + + free(line); + trace_vcpu_thread_status(msg, thread_id, proc_name, proc_status); +} + +static int defined_mem_fault_cpu_index(pid_t pid) +{ + CPUState *cpu_iter; + + CPU_FOREACH(cpu_iter) { + if (cpu_iter->thread_id =3D=3D pid) + return cpu_iter->cpu_index; + } + trace_for_thread("can't find cpu_index for thread id", pid); + return -1; +} + +static void trace_cpu_state(void) +{ + CPUState *cpu_iter; + CPU_FOREACH(cpu_iter) { + trace_for_thread("vCPU", cpu_iter->thread_id); + trace_postcopy_vcpu_running(cpu_iter->cpu_index, cpu_iter->running= ); + } +} + /* * Handle faults detected by the USERFAULT markings */ @@ -445,6 +500,7 @@ static void *postcopy_ram_fault_thread(void *opaque) } =20 ret =3D read(mis->userfault_fd, &msg, sizeof(msg)); + trace_cpu_state(); if (ret !=3D sizeof(msg)) { if (errno =3D=3D EAGAIN) { /* @@ -481,8 +537,10 @@ static void *postcopy_ram_fault_thread(void *opaque) rb_offset &=3D ~(qemu_ram_pagesize(rb) - 1); trace_postcopy_ram_fault_thread_request(msg.arg.pagefault.address, qemu_ram_get_idstr(rb), - rb_offset); + rb_offset, msg.arg.pagefau= lt.pid); =20 + mark_postcopy_downtime_begin(msg.arg.pagefault.address, + defined_mem_fault_cpu_index(msg.arg.pagefault.pid)); /* * Send the request to the source - we want to request one * of our host page sizes (which is >=3D TPS) @@ -577,6 +635,7 @@ int postcopy_place_page(MigrationIncomingState *mis, vo= id *host, void *from, =20 return -e; } + mark_postcopy_downtime_end((uint64_t)host); =20 trace_postcopy_place_page(host); return 0; diff --git a/migration/savevm.c b/migration/savevm.c index 3b19a4a..e12c0a2 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1630,6 +1630,7 @@ static void loadvm_postcopy_handle_run_bh(void *opaqu= e) =20 if (autostart) { /* Hold onto your hats, starting the CPU */ + trace_loadvm_postcopy_vm_start(get_postcopy_total_downtime()); vm_start(); } else { /* leave it paused and let management decide when to start the CPU= */ @@ -1930,6 +1931,7 @@ qemu_loadvm_section_part_end(QEMUFile *f, MigrationIn= comingState *mis) return -EINVAL; } =20 + trace_loadvm_postcopy_vm_start(get_postcopy_total_downtime()); return 0; } =20 diff --git a/migration/trace-events b/migration/trace-events index 7372ce2..8a21684 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -44,6 +44,7 @@ vmstate_subsection_save_loop(const char *name, const char= *sub) "%s/%s" vmstate_subsection_save_top(const char *idstr) "%s" vmstate_load(const char *idstr, const char *vmsd_name) "%s, %s" qemu_announce_self_iter(const char *mac) "%s" +loadvm_postcopy_vm_start(int64_t downtime) "%"PRId64 =20 # migration/vmstate.c vmstate_load_field_error(const char *field, int ret) "field \"%s\" load fa= iled, ret =3D %d" @@ -110,6 +111,11 @@ process_incoming_migration_co_end(int ret, int ps) "re= t=3D%d postcopy-state=3D%d" process_incoming_migration_co_postcopy_end_main(void) "" migration_set_incoming_channel(void *ioc, const char *ioctype) "ioc=3D%p i= octype=3D%s" migration_set_outgoing_channel(void *ioc, const char *ioctype, const char = *hostname) "ioc=3D%p ioctype=3D%s hostname=3D%s" +mark_postcopy_downtime_begin(uint64_t addr, void *dd, int64_t time, int cp= u) "addr 0x%" PRIx64 " dd %p time %" PRId64 " cpu %d" +mark_postcopy_downtime_end(uint64_t addr, void *dd, int64_t time) "addr 0x= %" PRIx64 " dd %p time %" PRId64 +get_postcopy_total_downtime(int num, int64_t total) "faults %d, total down= time %" PRId64 +sumup_downtime_duration(int64_t downtime, uint64_t addr, int cpubit) "down= time %" PRId64 " addr 0x%" PRIx64 "cpus %d" +downtime_per_cpu(int cpu_index, int downtime) "downtime cpu[%d]=3D%d" =20 # migration/rdma.c qemu_rdma_accept_incoming_migration(void) "" @@ -186,7 +192,7 @@ postcopy_ram_enable_notify(void) "" postcopy_ram_fault_thread_entry(void) "" postcopy_ram_fault_thread_exit(void) "" postcopy_ram_fault_thread_quit(void) "" -postcopy_ram_fault_thread_request(uint64_t hostaddr, const char *ramblock,= size_t offset) "Request for HVA=3D%" PRIx64 " rb=3D%s offset=3D%zx" +postcopy_ram_fault_thread_request(uint64_t hostaddr, const char *ramblock,= size_t offset, int pid) "Request for HVA=3D%" PRIx64 " rb=3D%s offset=3D%z= x %d" postcopy_ram_incoming_cleanup_closeuf(void) "" postcopy_ram_incoming_cleanup_entry(void) "" postcopy_ram_incoming_cleanup_exit(void) "" @@ -195,6 +201,8 @@ save_xbzrle_page_skipping(void) "" save_xbzrle_page_overflow(void) "" ram_save_iterate_big_wait(uint64_t milliconds, int iterations) "big wait: = %" PRIu64 " milliseconds, %d iterations" ram_load_complete(int ret, uint64_t seq_iter) "exit_code %d seq iteration = %" PRIu64 +vcpu_thread_status(const char *msg, int tpid, char *name, char *status) "%= s host_tid %d %s %s" +postcopy_vcpu_running(int cpu_index, int is_running) "cpu %d running %d" =20 # migration/exec.c migration_exec_outgoing(const char *cmd) "cmd=3D%s" --=20 1.8.3.1