From nobody Thu May 16 10:53:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1623029302; cv=none; d=zohomail.com; s=zohoarc; b=RGlx9vMguOokydSNkQ6aZxTeDdMwlJgdO2yAHf6VNGpwsrWtVUTRLGTQqBsK9KtSxJ2XNOKwkH8kwRu4IIDVCutHmkOSE+wPRAo8mRlgQiHS9KVRg3PAaj1LQx9xVFQfdzL/MR/u0iXQknW05BNSxGXIe/VweyRSX8xjui7+kt0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1623029302; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=pGOjf4Q1XVP8E3UwGDslDHtE/tMsk2zkHiQ6red5rV0=; b=AEbVTtrKR3C9DgTrJZgXh21UMoNaV/cQQHQNmzAUTo+0FR9QqbSLV7Cwpa8VqwpQFGOitGyIbv0ty0FmZbZWwP04OLIMQHtQcnNhMMa1tSII9FF7sszZ/IuVrR6loPX8VEC6MA2FXTiEHWnsUe28jwoDgmbanGYjcIOVgtXmcNA= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623029302413444.03022827230336; Sun, 6 Jun 2021 18:28:22 -0700 (PDT) Received: from localhost ([::1]:53940 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq43x-0008Fv-9g for importer@patchew.org; Sun, 06 Jun 2021 21:28:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48170) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lq3yn-0008JG-4A for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:23:01 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.219]:40680 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3yk-0002F0-IT for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:23:00 -0400 Received: from clientip-182.138.181.182?logid-0bf3c2c3c08f48e58021f22b8807bec4 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id C43C728008E; Mon, 7 Jun 2021 09:11:40 +0800 (CST) Received: from ([172.18.0.218]) by app0025 with ESMTP id 0bf3c2c3c08f48e58021f22b8807bec4 for qemu-devel@nongnu.org; Mon Jun 7 09:11:41 2021 HMM_SOURCE_IP: 172.18.0.218:52380.477647274 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 0bf3c2c3c08f48e58021f22b8807bec4 X-filter-score: filter<0> X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.218 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel@nongnu.org Subject: [PATCH v3 1/7] migration/dirtyrate: make sample page count configurable Date: Mon, 7 Jun 2021 09:11:34 +0800 Message-Id: <3103453a3b2796f929269c99a6ad81a9a7f1f405.1623027729.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.219; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , Hyman , "Dr. David Alan Gilbert" , Peter Xu , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" From: Hyman Huang(=E9=BB=84=E5=8B=87) introduce optional sample-pages argument in calc-dirty-rate, making sample page count per GB configurable so that more accurate dirtyrate can be calculated. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- migration/dirtyrate.c | 31 +++++++++++++++++++++++++++---- migration/dirtyrate.h | 8 +++++++- qapi/migration.json | 13 ++++++++++--- 3 files changed, 44 insertions(+), 8 deletions(-) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index ccb98147e8..2ee3890721 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -48,6 +48,12 @@ static bool is_sample_period_valid(int64_t sec) return true; } =20 +static bool is_sample_pages_valid(int64_t pages) +{ + return pages >=3D MIN_SAMPLE_PAGE_COUNT && + pages <=3D MAX_SAMPLE_PAGE_COUNT; +} + static int dirtyrate_set_state(int *state, int old_state, int new_state) { assert(new_state < DIRTY_RATE_STATUS__MAX); @@ -72,13 +78,15 @@ static struct DirtyRateInfo *query_dirty_rate_info(void) info->status =3D CalculatingState; info->start_time =3D DirtyStat.start_time; info->calc_time =3D DirtyStat.calc_time; + info->sample_pages =3D DirtyStat.sample_pages; =20 trace_query_dirty_rate_info(DirtyRateStatus_str(CalculatingState)); =20 return info; } =20 -static void init_dirtyrate_stat(int64_t start_time, int64_t calc_time) +static void init_dirtyrate_stat(int64_t start_time, int64_t calc_time, + uint64_t sample_pages) { DirtyStat.total_dirty_samples =3D 0; DirtyStat.total_sample_count =3D 0; @@ -86,6 +94,7 @@ static void init_dirtyrate_stat(int64_t start_time, int64= _t calc_time) DirtyStat.dirty_rate =3D -1; DirtyStat.start_time =3D start_time; DirtyStat.calc_time =3D calc_time; + DirtyStat.sample_pages =3D sample_pages; } =20 static void update_dirtyrate_stat(struct RamblockDirtyInfo *info) @@ -361,6 +370,7 @@ void *get_dirtyrate_thread(void *arg) int ret; int64_t start_time; int64_t calc_time; + uint64_t sample_pages; =20 ret =3D dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_UNSTA= RTED, DIRTY_RATE_STATUS_MEASURING); @@ -371,7 +381,8 @@ void *get_dirtyrate_thread(void *arg) =20 start_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME) / 1000; calc_time =3D config.sample_period_seconds; - init_dirtyrate_stat(start_time, calc_time); + sample_pages =3D config.sample_pages_per_gigabytes; + init_dirtyrate_stat(start_time, calc_time, sample_pages); =20 calculate_dirtyrate(config); =20 @@ -383,7 +394,8 @@ void *get_dirtyrate_thread(void *arg) return NULL; } =20 -void qmp_calc_dirty_rate(int64_t calc_time, Error **errp) +void qmp_calc_dirty_rate(int64_t calc_time, bool has_sample_pages, + int64_t sample_pages, Error **errp) { static struct DirtyRateConfig config; QemuThread thread; @@ -404,6 +416,17 @@ void qmp_calc_dirty_rate(int64_t calc_time, Error **er= rp) return; } =20 + if (has_sample_pages) { + if (!is_sample_pages_valid(sample_pages)) { + error_setg(errp, "sample-pages is out of range[%d, %d].", + MIN_SAMPLE_PAGE_COUNT, + MAX_SAMPLE_PAGE_COUNT); + return; + } + } else { + sample_pages =3D DIRTYRATE_DEFAULT_SAMPLE_PAGES; + } + /* * Init calculation state as unstarted. */ @@ -415,7 +438,7 @@ void qmp_calc_dirty_rate(int64_t calc_time, Error **err= p) } =20 config.sample_period_seconds =3D calc_time; - config.sample_pages_per_gigabytes =3D DIRTYRATE_DEFAULT_SAMPLE_PAGES; + config.sample_pages_per_gigabytes =3D sample_pages; qemu_thread_create(&thread, "get_dirtyrate", get_dirtyrate_thread, (void *)&config, QEMU_THREAD_DETACHED); } diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h index 6ec429534d..e1fd29089e 100644 --- a/migration/dirtyrate.h +++ b/migration/dirtyrate.h @@ -15,7 +15,6 @@ =20 /* * Sample 512 pages per GB as default. - * TODO: Make it configurable. */ #define DIRTYRATE_DEFAULT_SAMPLE_PAGES 512 =20 @@ -35,6 +34,12 @@ #define MIN_FETCH_DIRTYRATE_TIME_SEC 1 #define MAX_FETCH_DIRTYRATE_TIME_SEC 60 =20 +/* + * Take 1/16 pages in 1G as the maxmum sample page count + */ +#define MIN_SAMPLE_PAGE_COUNT 128 +#define MAX_SAMPLE_PAGE_COUNT 16384 + struct DirtyRateConfig { uint64_t sample_pages_per_gigabytes; /* sample pages per GB */ int64_t sample_period_seconds; /* time duration between two sampling */ @@ -63,6 +68,7 @@ struct DirtyRateStat { int64_t dirty_rate; /* dirty rate in MB/s */ int64_t start_time; /* calculation start time in units of second */ int64_t calc_time; /* time duration of two sampling in units of second= */ + uint64_t sample_pages; /* sample pages per GB */ }; =20 void *get_dirtyrate_thread(void *arg); diff --git a/qapi/migration.json b/qapi/migration.json index 7a5bdf9a0d..770ae54c17 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1740,6 +1740,9 @@ # # @calc-time: time in units of second for sample dirty pages # +# @sample-pages: page count per GB for sample dirty pages +# the default value is 512 (since 6.1) +# # Since: 5.2 # ## @@ -1747,7 +1750,8 @@ 'data': {'*dirty-rate': 'int64', 'status': 'DirtyRateStatus', 'start-time': 'int64', - 'calc-time': 'int64'} } + 'calc-time': 'int64', + 'sample-pages': 'uint64'} } =20 ## # @calc-dirty-rate: @@ -1756,13 +1760,16 @@ # # @calc-time: time in units of second for sample dirty pages # +# @sample-pages: page count per GB for sample dirty pages +# the default value is 512 (since 6.1) +# # Since: 5.2 # # Example: -# {"command": "calc-dirty-rate", "data": {"calc-time": 1} } +# {"command": "calc-dirty-rate", "data": {"calc-time": 1, 'sample-pages'= : 512} } # ## -{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64'} } +{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64', '*sample-pa= ges': 'int'} } =20 ## # @query-dirty-rate: --=20 2.18.2 From nobody Thu May 16 10:53:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1623029104; cv=none; d=zohomail.com; s=zohoarc; b=XEQiucHK3QfSUS47yIYG6C/Qn9sskustP3j4kPPSAC1jz3M0ErvG8qDUT8e185ZlmrVAHY4oM7nx5h5C5X4Dqs7Q6U8x0495unIU2mrBcWWGG3ZiyVeJihcDb+Ozn67IpyQiQGBENaCDekqjGCq0SUzTf1yUL9FYOBUOqt2Qm4U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1623029104; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Z9oanLs1QgKfXqSHkNcUa8nzjBrnciyzxHmgt4WqQ5I=; b=JHubPsplqW5eTZaRr7PV2VE5H5bdu+PhVZjyPrDGqC6/JtmK/uCRRxSe5fCCSUD5jC+SwegX3qPJj2YMvrEPvj4xQ/rp4NUhh+kyXt0OQajaUZE9vM2Js1UETeU/S1uow+jp7csXU8ptO6S5Pl0y4IL20GyxEeDFQAiBA79PVjo= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623029104897575.3658264468064; Sun, 6 Jun 2021 18:25:04 -0700 (PDT) Received: from localhost ([::1]:46920 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq40l-0003Zy-QF for importer@patchew.org; Sun, 06 Jun 2021 21:25:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48220) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lq3yo-0008OU-Vm for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:23:02 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.219]:40672 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3yk-0002Ey-I3 for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:23:02 -0400 Received: from clientip-182.138.181.182?logid-c3aef7583b2e4ecea3360270af8e2d74 (unknown [172.18.0.48]) by chinatelecom.cn (HERMES) with SMTP id 76DF4280097; Mon, 7 Jun 2021 09:12:04 +0800 (CST) Received: from ([172.18.0.48]) by app0024 with ESMTP id c3aef7583b2e4ecea3360270af8e2d74 for qemu-devel@nongnu.org; Mon Jun 7 09:12:03 2021 HMM_SOURCE_IP: 172.18.0.48:56360.2112367000 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: c3aef7583b2e4ecea3360270af8e2d74 X-filter-score: filter<0> X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.48 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel@nongnu.org Subject: [PATCH v3 2/7] hmp: Add "calc_dirty_rate" and "info dirty_rate" cmds Date: Mon, 7 Jun 2021 09:11:57 +0800 Message-Id: <4cc0039fc3ad6145136770cf3b0f056c09a2910b.1623027729.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.219; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , Hyman , "Dr. David Alan Gilbert" , Peter Xu , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" From: Peter Xu These two commands are missing when adding the QMP sister commands. Add them, so developers can play with them easier. Signed-off-by: Peter Xu Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- hmp-commands-info.hx | 13 ++++++++++++ hmp-commands.hx | 14 +++++++++++++ include/monitor/hmp.h | 2 ++ migration/dirtyrate.c | 47 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 76 insertions(+) diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx index b2347a6aea..fb59c27200 100644 --- a/hmp-commands-info.hx +++ b/hmp-commands-info.hx @@ -867,3 +867,16 @@ SRST ``info replay`` Display the record/replay information: mode and the current icount. ERST + + { + .name =3D "dirty_rate", + .args_type =3D "", + .params =3D "", + .help =3D "show dirty rate information", + .cmd =3D hmp_info_dirty_rate, + }, + +SRST + ``info dirty_rate`` + Display the vcpu dirty rate information. +ERST diff --git a/hmp-commands.hx b/hmp-commands.hx index 2d21fe5ad4..84dcc3aae6 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -1727,3 +1727,17 @@ ERST .flags =3D "p", }, =20 +SRST +``calc_dirty_rate`` *second* + Start a round of dirty rate measurement with the period specified in *se= cond*. + The result of the dirty rate measurement may be observed with ``info + dirty_rate`` command. +ERST + + { + .name =3D "calc_dirty_rate", + .args_type =3D "second:l,sample_pages_per_GB:l?", + .params =3D "second [sample_pages_per_GB]", + .help =3D "start a round of guest dirty rate measurement", + .cmd =3D hmp_calc_dirty_rate, + }, diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h index 605d57287a..3baa1058e2 100644 --- a/include/monitor/hmp.h +++ b/include/monitor/hmp.h @@ -129,5 +129,7 @@ void hmp_info_replay(Monitor *mon, const QDict *qdict); void hmp_replay_break(Monitor *mon, const QDict *qdict); void hmp_replay_delete_break(Monitor *mon, const QDict *qdict); void hmp_replay_seek(Monitor *mon, const QDict *qdict); +void hmp_info_dirty_rate(Monitor *mon, const QDict *qdict); +void hmp_calc_dirty_rate(Monitor *mon, const QDict *qdict); =20 #endif diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 2ee3890721..320c56ba2c 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -20,6 +20,9 @@ #include "ram.h" #include "trace.h" #include "dirtyrate.h" +#include "monitor/hmp.h" +#include "monitor/monitor.h" +#include "qapi/qmp/qdict.h" =20 static int CalculatingState =3D DIRTY_RATE_STATUS_UNSTARTED; static struct DirtyRateStat DirtyStat; @@ -447,3 +450,47 @@ struct DirtyRateInfo *qmp_query_dirty_rate(Error **err= p) { return query_dirty_rate_info(); } + +void hmp_info_dirty_rate(Monitor *mon, const QDict *qdict) +{ + DirtyRateInfo *info =3D query_dirty_rate_info(); + + monitor_printf(mon, "Status: %s\n", + DirtyRateStatus_str(info->status)); + monitor_printf(mon, "Start Time: %"PRIi64" (ms)\n", + info->start_time); + monitor_printf(mon, "Sample Pages: %"PRIu64" (per GB)\n", + info->sample_pages); + monitor_printf(mon, "Period: %"PRIi64" (sec)\n", + info->calc_time); + monitor_printf(mon, "Dirty rate: "); + if (info->has_dirty_rate) { + monitor_printf(mon, "%"PRIi64" (MB/s)\n", info->dirty_rate); + } else { + monitor_printf(mon, "(not ready)\n"); + } + g_free(info); +} + +void hmp_calc_dirty_rate(Monitor *mon, const QDict *qdict) +{ + int64_t sec =3D qdict_get_try_int(qdict, "second", 0); + int64_t sample_pages =3D qdict_get_try_int(qdict, "sample_pages_per_GB= ", -1); + bool has_sample_pages =3D (sample_pages !=3D -1); + Error *err =3D NULL; + + if (!sec) { + monitor_printf(mon, "Incorrect period length specified!\n"); + return; + } + + qmp_calc_dirty_rate(sec, has_sample_pages, sample_pages, &err); + if (err) { + hmp_handle_error(mon, err); + return; + } + + monitor_printf(mon, "Starting dirty rate measurement with period %"PRI= i64 + " seconds\n", sec); + monitor_printf(mon, "[Please use 'info dirty_rate' to check results]\n= "); +} --=20 2.18.2 From nobody Thu May 16 10:53:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1623028774; cv=none; d=zohomail.com; s=zohoarc; b=P2PfAzK4ejZldnqk7L6kF6j4GzdPyTBjH/mjwzmLq2ft0ONOyiHe+nC5bkZGW3tEDc4e0UBGWvPaQ8OklIAYVTdGUiv1Ydj4IMrgk+wSJQDmbAlDJpDPAuhJFn18lf+yL8IRAfS527Xa4KPgANT85cwIptRR5d0xCjdr/jb55lA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1623028774; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Kpn+7/w1jq/joSEbdWBoa9JKiOLx7bNNf6ckE3CnT2M=; b=fxlOAQvUM9Dt3zE7EhN31ip7/qE8adLcn6OKLlia/XrgrWx/OUBjUhHJq7IT/sZseQuGmXyprcyc0ngIj//HOfIP45xU98cdnTHJ4S7wC5yhcb1aNNYxNw2vx3z69Z5LZpmpjETrLXNt1/rnQlz9FhuyZulVPnBz1jQ46jBnSGc= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623028773399152.08215374267718; Sun, 6 Jun 2021 18:19:33 -0700 (PDT) Received: from localhost ([::1]:33944 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3vP-0003Dx-RY for importer@patchew.org; Sun, 06 Jun 2021 21:19:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40816) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lq3qa-0005Y2-HY for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:14:32 -0400 Received: from [42.123.76.228] (port=47794 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3pZ-0003yQ-W5 for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:14:08 -0400 Received: from clientip-182.138.181.182?logid-563a6af5169d4a62a342ee8f158fb98f (unknown [172.18.0.48]) by chinatelecom.cn (HERMES) with SMTP id CD15D280132; Mon, 7 Jun 2021 09:12:20 +0800 (CST) Received: from ([172.18.0.48]) by app0024 with ESMTP id 563a6af5169d4a62a342ee8f158fb98f for qemu-devel@nongnu.org; Mon Jun 7 09:12:20 2021 HMM_SOURCE_IP: 172.18.0.48:60748.791983158 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 563a6af5169d4a62a342ee8f158fb98f X-filter-score: filter<0> X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.48 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel@nongnu.org Subject: [PATCH v3 3/7] KVM: introduce dirty_pages and kvm_dirty_ring_enabled Date: Mon, 7 Jun 2021 09:12:14 +0800 Message-Id: <5fd7f4b444d14447649982ed44c5ddc5ebde635c.1623027729.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 42.123.76.228 (deferred) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.228; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, RDNS_NONE=0.793, T_SPF_HELO_TEMPERROR=0.01, T_SPF_TEMPERROR=0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , Hyman , "Dr. David Alan Gilbert" , Peter Xu , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" From: Hyman Huang(=E9=BB=84=E5=8B=87) dirty_pages is used to calculate dirtyrate via dirty ring, when enabled, kvm-reaper will increase the dirty pages after gfns being dirtied. kvm_dirty_ring_enabled shows if kvm-reaper is working. dirtyrate thread could use it to check if measurement can base on dirty ring feature. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- accel/kvm/kvm-all.c | 7 +++++++ include/hw/core/cpu.h | 1 + include/sysemu/kvm.h | 1 + 3 files changed, 9 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index c7ec538850..bc012f0bee 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -469,6 +469,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp) cpu->kvm_fd =3D ret; cpu->kvm_state =3D s; cpu->vcpu_dirty =3D true; + cpu->dirty_pages =3D 0; =20 mmap_size =3D kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0); if (mmap_size < 0) { @@ -743,6 +744,7 @@ static uint32_t kvm_dirty_ring_reap_one(KVMState *s, CP= UState *cpu) count++; } cpu->kvm_fetch_index =3D fetch; + cpu->dirty_pages +=3D count; =20 return count; } @@ -2293,6 +2295,11 @@ bool kvm_vcpu_id_is_valid(int vcpu_id) return vcpu_id >=3D 0 && vcpu_id < kvm_max_vcpu_id(s); } =20 +bool kvm_dirty_ring_enabled(void) +{ + return kvm_state->kvm_dirty_ring_size ? true : false; +} + static int kvm_init(MachineState *ms) { MachineClass *mc =3D MACHINE_GET_CLASS(ms); diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index 4e0ea68efc..80fcb1d563 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -374,6 +374,7 @@ struct CPUState { struct kvm_run *kvm_run; struct kvm_dirty_gfn *kvm_dirty_gfns; uint32_t kvm_fetch_index; + uint64_t dirty_pages; =20 /* Used for events with 'vcpu' and *without* the 'disabled' properties= */ DECLARE_BITMAP(trace_dstate_delayed, CPU_TRACE_DSTATE_MAX_EVENTS); diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index a1ab1ee12d..7b22aeb6ae 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -547,4 +547,5 @@ bool kvm_cpu_check_are_resettable(void); =20 bool kvm_arch_cpu_check_are_resettable(void); =20 +bool kvm_dirty_ring_enabled(void); #endif --=20 2.18.2 From nobody Thu May 16 10:53:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1623028978; cv=none; d=zohomail.com; s=zohoarc; b=cvyzH+ODLtWvtzbXggPk/L9DZAGvh8Z6ZtXZ7Un726hsCoDVoYQxJ07wEHgFzVJRb16gzkFvxXcLznUt8LjcWwo2Py3VhsG3bNfFYeJLE2EhymJGJhnCDizU0aLqWZuHK+QHnm6m8JSXq3+Ep0VjWZI2fRlU+a0/9/V7P6RVofI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1623028978; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hogVn23sVHtnAI/0SoiFK7BN7NqG/Vat9Rf3E3qwCH0=; b=W4oMi9CW9t+Vt3HYYV/cCu/a029MlTUoFyBcmnLMcg49TZ5pgLcwJDKfftpGFICGJ2R5gdz7yVxvh4qz1gsWJQygr1UzSb6k0gxkM1KMYzTcpYjly3YCa1dzeUVcEzu90nkA8F5uEdGa+quZlpWDTBb0Om1fCC+HKEqLWRbTF7E= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623028978281460.22479207206095; Sun, 6 Jun 2021 18:22:58 -0700 (PDT) Received: from localhost ([::1]:39188 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3yj-0006ra-4h for importer@patchew.org; Sun, 06 Jun 2021 21:22:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45198) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lq3wa-0004dC-Sg for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:20:44 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.228]:49309 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3wY-0000Wn-6S for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:20:44 -0400 Received: from clientip-182.138.181.182?logid-0124981a5a3c45efa7eb3d3c061ab6a6 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 4E8E1280137; Mon, 7 Jun 2021 09:12:43 +0800 (CST) Received: from ([172.18.0.218]) by app0025 with ESMTP id 0124981a5a3c45efa7eb3d3c061ab6a6 for qemu-devel@nongnu.org; Mon Jun 7 09:12:42 2021 HMM_SOURCE_IP: 172.18.0.218:41712.897474470 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 0124981a5a3c45efa7eb3d3c061ab6a6 X-filter-score: filter<0> X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.218 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel@nongnu.org Subject: [PATCH v3 4/7] migration/dirtyrate: add per-vcpu option for calc-dirty-rate Date: Mon, 7 Jun 2021 09:12:36 +0800 Message-Id: <19ad91782f5798844c42e34683fda833f9d1928a.1623027729.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.228; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , Hyman , "Dr. David Alan Gilbert" , Peter Xu , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" From: Hyman Huang(=E9=BB=84=E5=8B=87) calculate dirtyrate for each vcpu if vcpu is true, add the dirtyrate of each vcpu to the return value also. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- migration/dirtyrate.c | 45 ++++++++++++++++++++++++++++++++++++++----- migration/dirtyrate.h | 1 + qapi/migration.json | 29 ++++++++++++++++++++++++++-- 3 files changed, 68 insertions(+), 7 deletions(-) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 320c56ba2c..5947c688f6 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -397,8 +397,11 @@ void *get_dirtyrate_thread(void *arg) return NULL; } =20 -void qmp_calc_dirty_rate(int64_t calc_time, bool has_sample_pages, - int64_t sample_pages, Error **errp) +void qmp_calc_dirty_rate(int64_t calc_time, + bool per_vcpu, + bool has_sample_pages, + int64_t sample_pages, + Error **errp) { static struct DirtyRateConfig config; QemuThread thread; @@ -419,6 +422,12 @@ void qmp_calc_dirty_rate(int64_t calc_time, bool has_s= ample_pages, return; } =20 + if (has_sample_pages && per_vcpu) { + error_setg(errp, "per-vcpu and sample-pages are mutually exclusive= , " + "only one of then can be specified!\n"); + return; + } + if (has_sample_pages) { if (!is_sample_pages_valid(sample_pages)) { error_setg(errp, "sample-pages is out of range[%d, %d].", @@ -430,6 +439,15 @@ void qmp_calc_dirty_rate(int64_t calc_time, bool has_s= ample_pages, sample_pages =3D DIRTYRATE_DEFAULT_SAMPLE_PAGES; } =20 + /* + * Vcpu method only works when kvm dirty ring is enabled. + */ + if (per_vcpu && !kvm_dirty_ring_enabled()) { + error_setg(errp, "dirty ring is disabled or conflict with migratio= n" + "use sample method or remeasure later."); + return; + } + /* * Init calculation state as unstarted. */ @@ -442,6 +460,7 @@ void qmp_calc_dirty_rate(int64_t calc_time, bool has_sa= mple_pages, =20 config.sample_period_seconds =3D calc_time; config.sample_pages_per_gigabytes =3D sample_pages; + config.per_vcpu =3D per_vcpu; qemu_thread_create(&thread, "get_dirtyrate", get_dirtyrate_thread, (void *)&config, QEMU_THREAD_DETACHED); } @@ -459,13 +478,22 @@ void hmp_info_dirty_rate(Monitor *mon, const QDict *q= dict) DirtyRateStatus_str(info->status)); monitor_printf(mon, "Start Time: %"PRIi64" (ms)\n", info->start_time); - monitor_printf(mon, "Sample Pages: %"PRIu64" (per GB)\n", - info->sample_pages); monitor_printf(mon, "Period: %"PRIi64" (sec)\n", info->calc_time); + if (info->has_sample_pages) { + monitor_printf(mon, "Sample Pages: %"PRIu64" (per GB)\n", + info->sample_pages); + } monitor_printf(mon, "Dirty rate: "); if (info->has_dirty_rate) { monitor_printf(mon, "%"PRIi64" (MB/s)\n", info->dirty_rate); + if (info->per_vcpu && info->has_vcpu_dirty_rate) { + DirtyRateVcpuList *rate, *head =3D info->vcpu_dirty_rate; + for (rate =3D head; rate !=3D NULL; rate =3D rate->next) { + monitor_printf(mon, "vcpu[%"PRIi64"], Dirty rate: %"PRIi64= "\n", + rate->value->id, rate->value->dirty_rate); + } + } } else { monitor_printf(mon, "(not ready)\n"); } @@ -477,6 +505,7 @@ void hmp_calc_dirty_rate(Monitor *mon, const QDict *qdi= ct) int64_t sec =3D qdict_get_try_int(qdict, "second", 0); int64_t sample_pages =3D qdict_get_try_int(qdict, "sample_pages_per_GB= ", -1); bool has_sample_pages =3D (sample_pages !=3D -1); + bool per_vcpu =3D qdict_get_try_bool(qdict, "per_vcpu", false); Error *err =3D NULL; =20 if (!sec) { @@ -484,7 +513,13 @@ void hmp_calc_dirty_rate(Monitor *mon, const QDict *qd= ict) return; } =20 - qmp_calc_dirty_rate(sec, has_sample_pages, sample_pages, &err); + if (has_sample_pages && per_vcpu) { + monitor_printf(mon, "per_vcpu and sample_pages are mutually exclus= ive, " + "only one of then can be specified!\n"); + return; + } + + qmp_calc_dirty_rate(sec, per_vcpu, has_sample_pages, sample_pages, &er= r); if (err) { hmp_handle_error(mon, err); return; diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h index e1fd29089e..ec82716b40 100644 --- a/migration/dirtyrate.h +++ b/migration/dirtyrate.h @@ -43,6 +43,7 @@ struct DirtyRateConfig { uint64_t sample_pages_per_gigabytes; /* sample pages per GB */ int64_t sample_period_seconds; /* time duration between two sampling */ + bool per_vcpu; /* calculate dirtyrate for each vcpu using dirty ring */ }; =20 /* diff --git a/qapi/migration.json b/qapi/migration.json index 770ae54c17..7eef988182 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1708,6 +1708,21 @@ { 'event': 'UNPLUG_PRIMARY', 'data': { 'device-id': 'str' } } =20 +## +# @DirtyRateVcpu: +# +# Dirty rate of vcpu. +# +# @id: vcpu index. +# +# @dirty-rate: dirty rate. +# +# Since: 6.1 +# +## +{ 'struct': 'DirtyRateVcpu', + 'data': { 'id': 'int', 'dirty-rate': 'int64' } } + ## # @DirtyRateStatus: # @@ -1743,6 +1758,10 @@ # @sample-pages: page count per GB for sample dirty pages # the default value is 512 (since 6.1) # +# @per-vcpu: calculate dirtyrate for each vcpu (Since 6.1) +# +# @vcpu-dirty-rate: dirtyrate for each vcpu (Since 6.1) +# # Since: 5.2 # ## @@ -1751,7 +1770,9 @@ 'status': 'DirtyRateStatus', 'start-time': 'int64', 'calc-time': 'int64', - 'sample-pages': 'uint64'} } + '*sample-pages': 'uint64', + 'per-vcpu': 'bool', + '*vcpu-dirty-rate': [ 'DirtyRateVcpu' ] } } =20 ## # @calc-dirty-rate: @@ -1760,6 +1781,10 @@ # # @calc-time: time in units of second for sample dirty pages # +# @per-vcpu: calculate vcpu dirty rate if true, the default value is +# false, note that the per-vcpu and sample-pages are mutually +# exclusive (since 6.1) +# # @sample-pages: page count per GB for sample dirty pages # the default value is 512 (since 6.1) # @@ -1769,7 +1794,7 @@ # {"command": "calc-dirty-rate", "data": {"calc-time": 1, 'sample-pages'= : 512} } # ## -{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64', '*sample-pa= ges': 'int'} } +{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64', 'per-vcpu':= 'bool', '*sample-pages': 'int'} } =20 ## # @query-dirty-rate: --=20 2.18.2 From nobody Thu May 16 10:53:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1623029379; cv=none; d=zohomail.com; s=zohoarc; b=QS/Ka8emUkUpQpcNV7jHOj8+CwzqPweLE1TRahvQGH1owr7M7LKTZFYBXLkZ0jv0Fo+JXB1RYUOLZGfkq40rigw6lFq5a3h/b9YX5PqRc2mRLV0WFaN9PP0vmu6XNeJ/u3UF1MWjLpZ1C3tVZbX0C0wp7rquCb28UGGmzyI/IDg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1623029379; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yK0Z8UkQevB7qwB34wNUe/R318h71H7OeAjXjxs6Nsw=; b=V0MGbOtOw70W3xI1OL/y/Oa8cc4zKhNUObSjQzW6OTxrpuhJLyoB7IRjrffagtEOYZPc++96xPExc7bjUi14LYHDIeWZau3kyhNY9NsWqEH4gUP0KnttlANDH+Jbld+niQti6LV3ExkwjpptsKgmL4+BRcfkFlIfTiFnWPK61ZU= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623029379289202.4951822465648; Sun, 6 Jun 2021 18:29:39 -0700 (PDT) Received: from localhost ([::1]:56300 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq45C-0001Pl-B4 for importer@patchew.org; Sun, 06 Jun 2021 21:29:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49736) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lq408-0003Rb-Np for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:24:24 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.220]:36077 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3pt-0004FN-36 for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:14:28 -0400 Received: from clientip-182.138.181.182?logid-e7583f06e12f448aa431b444b2fc2cc9 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 1B1DA2800CC; Mon, 7 Jun 2021 09:13:00 +0800 (CST) Received: from ([172.18.0.218]) by app0025 with ESMTP id e7583f06e12f448aa431b444b2fc2cc9 for qemu-devel@nongnu.org; Mon Jun 7 09:12:59 2021 HMM_SOURCE_IP: 172.18.0.218:55792.962152414 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: e7583f06e12f448aa431b444b2fc2cc9 X-filter-score: filter<0> X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.218 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel@nongnu.org Subject: [PATCH v3 5/7] migration/dirtyrate: adjust struct DirtyRateStat Date: Mon, 7 Jun 2021 09:12:54 +0800 Message-Id: <9be862eefa97e5e299dd26d4bf403d25ab317ee6.1623027729.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.220; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: 5 X-Spam_score: 0.5 X-Spam_bar: / X-Spam_report: (0.5 / 5.0 requ) BAYES_00=-1.9, FORGED_SPF_HELO=1.997, KHOP_HELO_FCRDNS=0.398, SPF_HELO_PASS=-0.001, T_SPF_TEMPERROR=0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , Hyman , "Dr. David Alan Gilbert" , Peter Xu , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" From: Hyman Huang(=E9=BB=84=E5=8B=87) use union to store stat data of two mutual exclusive methods. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- migration/dirtyrate.c | 40 +++++++++++++++++++++------------------- migration/dirtyrate.h | 18 +++++++++++++++--- 2 files changed, 36 insertions(+), 22 deletions(-) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 5947c688f6..055145c24c 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -88,33 +88,39 @@ static struct DirtyRateInfo *query_dirty_rate_info(void) return info; } =20 -static void init_dirtyrate_stat(int64_t start_time, int64_t calc_time, - uint64_t sample_pages) +static void init_dirtyrate_stat(int64_t start_time, + struct DirtyRateConfig config) { - DirtyStat.total_dirty_samples =3D 0; - DirtyStat.total_sample_count =3D 0; - DirtyStat.total_block_mem_MB =3D 0; DirtyStat.dirty_rate =3D -1; DirtyStat.start_time =3D start_time; - DirtyStat.calc_time =3D calc_time; - DirtyStat.sample_pages =3D sample_pages; + DirtyStat.calc_time =3D config.sample_period_seconds; + DirtyStat.sample_pages =3D config.sample_pages_per_gigabytes; + + if (config.per_vcpu) { + DirtyStat.method.vcpu.nvcpu =3D -1; + DirtyStat.method.vcpu.rates =3D NULL; + } else { + DirtyStat.method.vm.total_dirty_samples =3D 0; + DirtyStat.method.vm.total_sample_count =3D 0; + DirtyStat.method.vm.total_block_mem_MB =3D 0; + } } =20 static void update_dirtyrate_stat(struct RamblockDirtyInfo *info) { - DirtyStat.total_dirty_samples +=3D info->sample_dirty_count; - DirtyStat.total_sample_count +=3D info->sample_pages_count; + DirtyStat.method.vm.total_dirty_samples +=3D info->sample_dirty_count; + DirtyStat.method.vm.total_sample_count +=3D info->sample_pages_count; /* size of total pages in MB */ - DirtyStat.total_block_mem_MB +=3D (info->ramblock_pages * + DirtyStat.method.vm.total_block_mem_MB +=3D (info->ramblock_pages * TARGET_PAGE_SIZE) >> 20; } =20 static void update_dirtyrate(uint64_t msec) { uint64_t dirtyrate; - uint64_t total_dirty_samples =3D DirtyStat.total_dirty_samples; - uint64_t total_sample_count =3D DirtyStat.total_sample_count; - uint64_t total_block_mem_MB =3D DirtyStat.total_block_mem_MB; + uint64_t total_dirty_samples =3D DirtyStat.method.vm.total_dirty_sampl= es; + uint64_t total_sample_count =3D DirtyStat.method.vm.total_sample_count; + uint64_t total_block_mem_MB =3D DirtyStat.method.vm.total_block_mem_MB; =20 dirtyrate =3D total_dirty_samples * total_block_mem_MB * 1000 / (total_sample_count * msec); @@ -327,7 +333,7 @@ static bool compare_page_hash_info(struct RamblockDirty= Info *info, update_dirtyrate_stat(block_dinfo); } =20 - if (DirtyStat.total_sample_count =3D=3D 0) { + if (DirtyStat.method.vm.total_sample_count =3D=3D 0) { return false; } =20 @@ -372,8 +378,6 @@ void *get_dirtyrate_thread(void *arg) struct DirtyRateConfig config =3D *(struct DirtyRateConfig *)arg; int ret; int64_t start_time; - int64_t calc_time; - uint64_t sample_pages; =20 ret =3D dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_UNSTA= RTED, DIRTY_RATE_STATUS_MEASURING); @@ -383,9 +387,7 @@ void *get_dirtyrate_thread(void *arg) } =20 start_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME) / 1000; - calc_time =3D config.sample_period_seconds; - sample_pages =3D config.sample_pages_per_gigabytes; - init_dirtyrate_stat(start_time, calc_time, sample_pages); + init_dirtyrate_stat(start_time, config); =20 calculate_dirtyrate(config); =20 diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h index ec82716b40..af32e33d5d 100644 --- a/migration/dirtyrate.h +++ b/migration/dirtyrate.h @@ -59,17 +59,29 @@ struct RamblockDirtyInfo { uint32_t *hash_result; /* array of hash result for sampled pages */ }; =20 +typedef struct SampleVMStat { + uint64_t total_dirty_samples; /* total dirty sampled page */ + uint64_t total_sample_count; /* total sampled pages */ + uint64_t total_block_mem_MB; /* size of total sampled pages in MB */ +} SampleVMStat; + +typedef struct VcpuStat { + int nvcpu; /* number of vcpu */ + DirtyRateVcpu *rates; /* array of dirty rate for each vcpu */ +} VcpuStat; + /* * Store calculation statistics for each measure. */ struct DirtyRateStat { - uint64_t total_dirty_samples; /* total dirty sampled page */ - uint64_t total_sample_count; /* total sampled pages */ - uint64_t total_block_mem_MB; /* size of total sampled pages in MB */ int64_t dirty_rate; /* dirty rate in MB/s */ int64_t start_time; /* calculation start time in units of second */ int64_t calc_time; /* time duration of two sampling in units of second= */ uint64_t sample_pages; /* sample pages per GB */ + union { + SampleVMStat vm; + VcpuStat vcpu; + } method; }; =20 void *get_dirtyrate_thread(void *arg); --=20 2.18.2 From nobody Thu May 16 10:53:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1623029107; cv=none; d=zohomail.com; s=zohoarc; b=XJ1S4tuNrq9PK+kgB/Mb9lap14Nwm0+yGzITIPumXTaJEVE4BZPUQPcKgExhMc1IuGHibX7FRFfI2Z1XPHHR/OxfMRqGHXYYbGvVCQvNajr/aOyO9CqpE9WWLm4fGk/6jtM4RlmnRF4pekntmgGDDAaXNxarBLzizC6Dmd94Vis= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1623029107; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UcuerpxZwL3UMrkMEQ1Qc6taq+Ao7HryOB6UdtZVEzY=; b=L1j1Dw4vPVtpo8b+JubwGBxvu+vk0wTgl2PI9pdpOMw2ax1tT4gCGjufcoXPQO9Bp/unDIi0R5mSmtEFhvCrD0ZJFcFMpQHWJao+Up+9l/0dt653YisMzhW623qZuI5hkCmC13tzv2ZG83oWqKJSEMyhCgqgSoBSOCPFdVQajsg= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623029107905700.73573420389; Sun, 6 Jun 2021 18:25:07 -0700 (PDT) Received: from localhost ([::1]:47200 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq40o-0003k0-Py for importer@patchew.org; Sun, 06 Jun 2021 21:25:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45044) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lq3wR-0004N4-97 for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:20:38 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.220]:42280 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3wK-0000NZ-U5 for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:20:35 -0400 Received: from clientip-182.138.181.182?logid-39c91e36111c4c909109fa0d232f1b9e (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 63C792800AE; Mon, 7 Jun 2021 09:13:18 +0800 (CST) Received: from ([172.18.0.218]) by app0025 with ESMTP id 39c91e36111c4c909109fa0d232f1b9e for qemu-devel@nongnu.org; Mon Jun 7 09:13:19 2021 HMM_SOURCE_IP: 172.18.0.218:57632.995063495 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 39c91e36111c4c909109fa0d232f1b9e X-filter-score: filter<0> X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.218 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel@nongnu.org Subject: [PATCH v3 6/7] memory: make global_dirty_log a bitmask Date: Mon, 7 Jun 2021 09:13:12 +0800 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.220; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , Hyman , "Dr. David Alan Gilbert" , Peter Xu , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" From: Hyman Huang(=E9=BB=84=E5=8B=87) dirty rate measurement may start or stop dirty logging during calculation. this conflict with migration because stop dirty log make migration leave dirty pages out then that'll be a problem. make global_dirty_log a bitmask can let both migration and dirty rate measurement work fine. introduce GLOBAL_DIRTY_MIGRATION and GLOBAL_DIRTY_DIRTY_RATE to distinguish what current dirty log aims for, migration or dirty rate. all references to global_dirty_log should be untouched because any bit set there should justify that global dirty logging is enabled. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- include/exec/memory.h | 13 ++++++++++--- migration/ram.c | 8 ++++---- softmmu/memory.c | 36 +++++++++++++++++++++++++++--------- 3 files changed, 41 insertions(+), 16 deletions(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index c158fd7084..94c7088299 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -55,7 +55,10 @@ static inline void fuzz_dma_read_cb(size_t addr, } #endif =20 -extern bool global_dirty_log; +#define GLOBAL_DIRTY_MIGRATION (1U<<0) +#define GLOBAL_DIRTY_DIRTY_RATE (1U<<1) + +extern int global_dirty_log; =20 typedef struct MemoryRegionOps MemoryRegionOps; =20 @@ -2099,13 +2102,17 @@ void memory_listener_unregister(MemoryListener *lis= tener); =20 /** * memory_global_dirty_log_start: begin dirty logging for all regions + * + * @flags: purpose of start dirty log, migration or dirty rate */ -void memory_global_dirty_log_start(void); +void memory_global_dirty_log_start(int flags); =20 /** * memory_global_dirty_log_stop: end dirty logging for all regions + * + * @flags: purpose of stop dirty log, migration or dirty rate */ -void memory_global_dirty_log_stop(void); +void memory_global_dirty_log_stop(int flags); =20 void mtree_info(bool flatview, bool dispatch_tree, bool owner, bool disabl= ed); =20 diff --git a/migration/ram.c b/migration/ram.c index 60ea913c54..9ce31af9d1 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2190,7 +2190,7 @@ static void ram_save_cleanup(void *opaque) /* caller have hold iothread lock or is in a bh, so there is * no writing race against the migration bitmap */ - memory_global_dirty_log_stop(); + memory_global_dirty_log_stop(GLOBAL_DIRTY_MIGRATION); } =20 RAMBLOCK_FOREACH_NOT_IGNORED(block) { @@ -2652,7 +2652,7 @@ static void ram_init_bitmaps(RAMState *rs) ram_list_init_bitmaps(); /* We don't use dirty log with background snapshots */ if (!migrate_background_snapshot()) { - memory_global_dirty_log_start(); + memory_global_dirty_log_start(GLOBAL_DIRTY_MIGRATION); migration_bitmap_sync_precopy(rs); } } @@ -3393,7 +3393,7 @@ void colo_incoming_start_dirty_log(void) /* Discard this dirty bitmap record */ bitmap_zero(block->bmap, block->max_length >> TARGET_PAGE_BITS= ); } - memory_global_dirty_log_start(); + memory_global_dirty_log_start(GLOBAL_DIRTY_MIGRATION); } ram_state->migration_dirty_pages =3D 0; qemu_mutex_unlock_ramlist(); @@ -3405,7 +3405,7 @@ void colo_release_ram_cache(void) { RAMBlock *block; =20 - memory_global_dirty_log_stop(); + memory_global_dirty_log_stop(GLOBAL_DIRTY_MIGRATION); RAMBLOCK_FOREACH_NOT_IGNORED(block) { g_free(block->bmap); block->bmap =3D NULL; diff --git a/softmmu/memory.c b/softmmu/memory.c index c19b0be6b1..b93baba82d 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -39,7 +39,7 @@ static unsigned memory_region_transaction_depth; static bool memory_region_update_pending; static bool ioeventfd_update_pending; -bool global_dirty_log; +int global_dirty_log; =20 static QTAILQ_HEAD(, MemoryListener) memory_listeners =3D QTAILQ_HEAD_INITIALIZER(memory_listeners); @@ -2659,14 +2659,20 @@ void memory_global_after_dirty_log_sync(void) =20 static VMChangeStateEntry *vmstate_change; =20 -void memory_global_dirty_log_start(void) +void memory_global_dirty_log_start(int flags) { if (vmstate_change) { qemu_del_vm_change_state_handler(vmstate_change); vmstate_change =3D NULL; } =20 - global_dirty_log =3D true; + if (flags & GLOBAL_DIRTY_MIGRATION) { + global_dirty_log |=3D GLOBAL_DIRTY_MIGRATION; + } + + if (flags & GLOBAL_DIRTY_DIRTY_RATE) { + global_dirty_log |=3D GLOBAL_DIRTY_DIRTY_RATE; + } =20 MEMORY_LISTENER_CALL_GLOBAL(log_global_start, Forward); =20 @@ -2676,9 +2682,15 @@ void memory_global_dirty_log_start(void) memory_region_transaction_commit(); } =20 -static void memory_global_dirty_log_do_stop(void) +static void memory_global_dirty_log_do_stop(int flags) { - global_dirty_log =3D false; + if (flags & GLOBAL_DIRTY_MIGRATION) { + global_dirty_log &=3D ~GLOBAL_DIRTY_MIGRATION; + } + + if (flags & GLOBAL_DIRTY_DIRTY_RATE) { + global_dirty_log &=3D ~GLOBAL_DIRTY_DIRTY_RATE; + } =20 /* Refresh DIRTY_MEMORY_MIGRATION bit. */ memory_region_transaction_begin(); @@ -2691,8 +2703,10 @@ static void memory_global_dirty_log_do_stop(void) static void memory_vm_change_state_handler(void *opaque, bool running, RunState state) { + int *flags =3D opaque; if (running) { - memory_global_dirty_log_do_stop(); + memory_global_dirty_log_do_stop(*flags); + g_free(opaque); =20 if (vmstate_change) { qemu_del_vm_change_state_handler(vmstate_change); @@ -2701,18 +2715,22 @@ static void memory_vm_change_state_handler(void *op= aque, bool running, } } =20 -void memory_global_dirty_log_stop(void) +void memory_global_dirty_log_stop(int flags) { + int *opaque =3D NULL; if (!runstate_is_running()) { if (vmstate_change) { return; } + + opaque =3D g_malloc0(sizeof(opaque)); + *opaque =3D flags; vmstate_change =3D qemu_add_vm_change_state_handler( - memory_vm_change_state_handler, NULL); + memory_vm_change_state_handler, opaque); return; } =20 - memory_global_dirty_log_do_stop(); + memory_global_dirty_log_do_stop(flags); } =20 static void listener_add_address_space(MemoryListener *listener, --=20 2.18.2 From nobody Thu May 16 10:53:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1623028950; cv=none; d=zohomail.com; s=zohoarc; b=N63bH3F2adZmaPu5g9cS8+moBuU4ZWjvzScNAQer8Gn/S2ytTaDi5LAplyr7P+Vj4pSsQ6bTHPFN4dp2ceX5U8ItuDiZ0Dips3BnQZ3oD0LqNYFbalFjTJFoHPev+PWzjuIY45JpE7krlr1VcvXjoO+XGfMFcJlyDdWRx4Iv3pg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1623028950; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qzK7Zsp9JufPuJSugxG9aJ2fo/cHCDLjWccODrt+mmg=; b=hEEZ2Kf48mDnCyyRZ5bS/f0RQUWVXalSVoKJwkIPp6hDj8Mr1lgHWUfersJryR0zULlssVRf6Xb9yswj0SGclqoasDA5AQnJHgYEBbNW2WA9tSNOPdLbU8Nr9hRAmTTnhYWVldKP7trnpx2YQObUQi/UA0GRhQuLTm2Oa57n4cg= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1623028950645620.3457842985592; Sun, 6 Jun 2021 18:22:30 -0700 (PDT) Received: from localhost ([::1]:38594 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3yG-0006TJ-Qq for importer@patchew.org; Sun, 06 Jun 2021 21:22:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41142) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lq3sH-00067x-Rm for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:16:17 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.223]:58868 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lq3sA-0005Ym-DF for qemu-devel@nongnu.org; Sun, 06 Jun 2021 21:16:17 -0400 Received: from clientip-182.138.181.182?logid-0567c98953064115a6ce7ecfc37a6eca (unknown [172.18.0.48]) by chinatelecom.cn (HERMES) with SMTP id 7EE032800D7; Mon, 7 Jun 2021 09:15:26 +0800 (CST) Received: from ([172.18.0.48]) by app0024 with ESMTP id 0567c98953064115a6ce7ecfc37a6eca for qemu-devel@nongnu.org; Mon Jun 7 09:15:27 2021 HMM_SOURCE_IP: 172.18.0.48:44540.80893431 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 0567c98953064115a6ce7ecfc37a6eca X-filter-score: filter<0> X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.48 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel@nongnu.org Subject: [PATCH v3 7/7] migration/dirtyrate: implement dirty-ring dirtyrate calculation Date: Mon, 7 Jun 2021 09:15:20 +0800 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.223; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , Hyman , "Dr. David Alan Gilbert" , Peter Xu , Chuan Zheng , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" From: Hyman Huang(=E9=BB=84=E5=8B=87) use dirty ring feature to implement dirtyrate calculation. to enable it, set vcpu option as true in calc-dirty-rate. add per_vcpu as mandatory option in calc_dirty_rate, to calculate dirty rate for vcpu, and use hmp cmd: (qemu) calc_dirty_rate 1 on Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- hmp-commands.hx | 7 +- migration/dirtyrate.c | 226 ++++++++++++++++++++++++++++++++++++++--- migration/trace-events | 5 + 3 files changed, 220 insertions(+), 18 deletions(-) diff --git a/hmp-commands.hx b/hmp-commands.hx index 84dcc3aae6..cc24ab2ab1 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -1736,8 +1736,9 @@ ERST =20 { .name =3D "calc_dirty_rate", - .args_type =3D "second:l,sample_pages_per_GB:l?", - .params =3D "second [sample_pages_per_GB]", - .help =3D "start a round of guest dirty rate measurement", + .args_type =3D "second:l,per_vcpu:b,sample_pages_per_GB:l?", + .params =3D "second on|off [sample_pages_per_GB]", + .help =3D "start a round of guest dirty rate measurement, " + "calculate for vcpu use on|off", .cmd =3D hmp_calc_dirty_rate, }, diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 055145c24c..e432118f49 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -16,6 +16,9 @@ #include "cpu.h" #include "exec/ramblock.h" #include "qemu/rcu_queue.h" +#include "qemu/main-loop.h" +#include "sysemu/kvm.h" +#include "sysemu/runstate.h" #include "qapi/qapi-commands-migration.h" #include "ram.h" #include "trace.h" @@ -23,9 +26,38 @@ #include "monitor/hmp.h" #include "monitor/monitor.h" #include "qapi/qmp/qdict.h" +#include "exec/memory.h" + +typedef enum { + CALC_NONE =3D 0, + CALC_DIRTY_RING, + CALC_SAMPLE_PAGES, +} CalcMethod; + +typedef struct DirtyPageRecord { + int64_t start_pages; + int64_t end_pages; +} DirtyPageRecord; + +static DirtyPageRecord *dirty_pages; =20 static int CalculatingState =3D DIRTY_RATE_STATUS_UNSTARTED; static struct DirtyRateStat DirtyStat; +static CalcMethod last_method =3D CALC_NONE; +bool register_powerdown_callback =3D false; + +static void dirtyrate_powerdown_req(Notifier *n, void *opaque) +{ + if (last_method =3D=3D CALC_DIRTY_RING) { + g_free(DirtyStat.method.vcpu.rates); + DirtyStat.method.vcpu.rates =3D NULL; + } + trace_dirtyrate_powerdown_callback(); +} + +static Notifier dirtyrate_powerdown_notifier =3D { + .notify =3D dirtyrate_powerdown_req +}; =20 static int64_t set_sample_page_period(int64_t msec, int64_t initial_time) { @@ -72,6 +104,7 @@ static struct DirtyRateInfo *query_dirty_rate_info(void) { int64_t dirty_rate =3D DirtyStat.dirty_rate; struct DirtyRateInfo *info =3D g_malloc0(sizeof(DirtyRateInfo)); + DirtyRateVcpuList *head =3D NULL, **tail =3D &head; =20 if (qatomic_read(&CalculatingState) =3D=3D DIRTY_RATE_STATUS_MEASURED)= { info->has_dirty_rate =3D true; @@ -81,7 +114,22 @@ static struct DirtyRateInfo *query_dirty_rate_info(void) info->status =3D CalculatingState; info->start_time =3D DirtyStat.start_time; info->calc_time =3D DirtyStat.calc_time; - info->sample_pages =3D DirtyStat.sample_pages; + + if (last_method =3D=3D CALC_DIRTY_RING) { + int i =3D 0; + info->per_vcpu =3D true; + info->has_vcpu_dirty_rate =3D true; + for (i =3D 0; i < DirtyStat.method.vcpu.nvcpu; i++) { + DirtyRateVcpu *rate =3D g_malloc0(sizeof(DirtyRateVcpu)); + rate->id =3D DirtyStat.method.vcpu.rates[i].id; + rate->dirty_rate =3D DirtyStat.method.vcpu.rates[i].dirty_rate; + QAPI_LIST_APPEND(tail, rate); + } + info->vcpu_dirty_rate =3D head; + } else { + info->has_sample_pages =3D true; + info->sample_pages =3D DirtyStat.sample_pages; + } =20 trace_query_dirty_rate_info(DirtyRateStatus_str(CalculatingState)); =20 @@ -94,15 +142,37 @@ static void init_dirtyrate_stat(int64_t start_time, DirtyStat.dirty_rate =3D -1; DirtyStat.start_time =3D start_time; DirtyStat.calc_time =3D config.sample_period_seconds; - DirtyStat.sample_pages =3D config.sample_pages_per_gigabytes; - - if (config.per_vcpu) { - DirtyStat.method.vcpu.nvcpu =3D -1; - DirtyStat.method.vcpu.rates =3D NULL; - } else { - DirtyStat.method.vm.total_dirty_samples =3D 0; - DirtyStat.method.vm.total_sample_count =3D 0; - DirtyStat.method.vm.total_block_mem_MB =3D 0; + DirtyStat.sample_pages =3D + config.per_vcpu ? -1 : config.sample_pages_per_gigabytes; + + if (unlikely(!register_powerdown_callback)) { + qemu_register_powerdown_notifier(&dirtyrate_powerdown_notifier); + register_powerdown_callback =3D true; + } + + switch (last_method) { + case CALC_NONE: + case CALC_SAMPLE_PAGES: + if (config.per_vcpu) { + DirtyStat.method.vcpu.nvcpu =3D -1; + DirtyStat.method.vcpu.rates =3D NULL; + } else { + DirtyStat.method.vm.total_dirty_samples =3D 0; + DirtyStat.method.vm.total_sample_count =3D 0; + DirtyStat.method.vm.total_block_mem_MB =3D 0; + } + break; + case CALC_DIRTY_RING: + if (!config.per_vcpu) { + g_free(DirtyStat.method.vcpu.rates); + DirtyStat.method.vcpu.rates =3D NULL; + DirtyStat.method.vm.total_dirty_samples =3D 0; + DirtyStat.method.vm.total_sample_count =3D 0; + DirtyStat.method.vm.total_block_mem_MB =3D 0; + } + break; + default: + break; } } =20 @@ -316,7 +386,7 @@ find_block_matched(RAMBlock *block, int count, } =20 static bool compare_page_hash_info(struct RamblockDirtyInfo *info, - int block_count) + int block_count) { struct RamblockDirtyInfo *block_dinfo =3D NULL; RAMBlock *block =3D NULL; @@ -340,14 +410,125 @@ static bool compare_page_hash_info(struct RamblockDi= rtyInfo *info, return true; } =20 -static void calculate_dirtyrate(struct DirtyRateConfig config) +static void record_dirtypages(CPUState *cpu, bool start) +{ + if (start) { + dirty_pages[cpu->cpu_index].start_pages =3D cpu->dirty_pages; + } else { + dirty_pages[cpu->cpu_index].end_pages =3D cpu->dirty_pages; + } +} + +static void dirtyrate_global_dirty_log_start(void) +{ + /* dirty logging is enabled already */ + if (global_dirty_log) { + return; + } + + qemu_mutex_lock_iothread(); + memory_global_dirty_log_start(GLOBAL_DIRTY_DIRTY_RATE); + qemu_mutex_unlock_iothread(); + trace_dirtyrate_dirty_log_start(); +} + +static void dirtyrate_global_dirty_log_stop(void) +{ + /* migration is in process, do not stop dirty logging, + * just clear the GLOBAL_DIRTY_DIRTY_RATE bit */ + if (global_dirty_log & GLOBAL_DIRTY_MIGRATION) { + global_dirty_log &=3D ~(GLOBAL_DIRTY_DIRTY_RATE); + return; + } + + qemu_mutex_lock_iothread(); + memory_global_dirty_log_stop(GLOBAL_DIRTY_DIRTY_RATE); + qemu_mutex_unlock_iothread(); + trace_dirtyrate_dirty_log_stop(); +} + +static int64_t do_calculate_dirtyrate_vcpu(int idx) +{ + uint64_t memory_size_MB; + int64_t time_s; + uint64_t start_pages =3D dirty_pages[idx].start_pages; + uint64_t end_pages =3D dirty_pages[idx].end_pages; + uint64_t dirty_pages =3D 0; + + /* uint64_t over the INT64_MAX */ + if (unlikely(end_pages < start_pages)) { + dirty_pages =3D INT64_MAX - start_pages + end_pages + 1; + } else { + dirty_pages =3D end_pages - start_pages; + } + + memory_size_MB =3D (dirty_pages * TARGET_PAGE_SIZE) >> 20; + time_s =3D DirtyStat.calc_time; + + trace_dirtyrate_do_calculate_vcpu(idx, dirty_pages, time_s); + + return memory_size_MB / time_s; +} + +static void calculate_dirtyrate_vcpu(struct DirtyRateConfig config) +{ + CPUState *cpu; + int64_t msec =3D 0; + int64_t start_time; + uint64_t dirtyrate =3D 0; + uint64_t dirtyrate_sum =3D 0; + int nvcpu =3D 0; + int i =3D 0; + + CPU_FOREACH(cpu) { + nvcpu++; + } + + dirty_pages =3D g_malloc0(sizeof(*dirty_pages) * nvcpu); + + dirtyrate_global_dirty_log_start(); + + CPU_FOREACH(cpu) { + record_dirtypages(cpu, true); + } + + DirtyStat.method.vcpu.nvcpu =3D nvcpu; + if (last_method !=3D CALC_DIRTY_RING) { + DirtyStat.method.vcpu.rates =3D + g_malloc0(sizeof(DirtyRateVcpu) * nvcpu); + } + + start_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + DirtyStat.start_time =3D start_time / 1000; + + msec =3D config.sample_period_seconds * 1000; + msec =3D set_sample_page_period(msec, start_time); + DirtyStat.calc_time =3D msec / 1000; + + CPU_FOREACH(cpu) { + record_dirtypages(cpu, false); + } + + dirtyrate_global_dirty_log_stop(); + + for (i =3D 0; i < DirtyStat.method.vcpu.nvcpu; i++) { + dirtyrate =3D do_calculate_dirtyrate_vcpu(i); + DirtyStat.method.vcpu.rates[i].id =3D i; + DirtyStat.method.vcpu.rates[i].dirty_rate =3D dirtyrate; + dirtyrate_sum +=3D dirtyrate; + } + + DirtyStat.dirty_rate =3D dirtyrate_sum / DirtyStat.method.vcpu.nvcpu; + g_free(dirty_pages); +} + +static void calculate_dirtyrate_sample_vm(struct DirtyRateConfig config) { struct RamblockDirtyInfo *block_dinfo =3D NULL; int block_count =3D 0; int64_t msec =3D 0; int64_t initial_time; =20 - rcu_register_thread(); rcu_read_lock(); initial_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); if (!record_ramblock_hash_info(&block_dinfo, config, &block_count)) { @@ -364,13 +545,24 @@ static void calculate_dirtyrate(struct DirtyRateConfi= g config) if (!compare_page_hash_info(block_dinfo, block_count)) { goto out; } - update_dirtyrate(msec); =20 out: rcu_read_unlock(); free_ramblock_dirty_info(block_dinfo, block_count); - rcu_unregister_thread(); +} + +static void calculate_dirtyrate(struct DirtyRateConfig config) +{ + if (config.per_vcpu) { + calculate_dirtyrate_vcpu(config); + last_method =3D CALC_DIRTY_RING; + } else { + calculate_dirtyrate_sample_vm(config); + last_method =3D CALC_SAMPLE_PAGES; + } + + trace_dirtyrate_calculate(DirtyStat.dirty_rate); } =20 void *get_dirtyrate_thread(void *arg) @@ -379,6 +571,8 @@ void *get_dirtyrate_thread(void *arg) int ret; int64_t start_time; =20 + rcu_register_thread(); + ret =3D dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_UNSTA= RTED, DIRTY_RATE_STATUS_MEASURING); if (ret =3D=3D -1) { @@ -396,6 +590,8 @@ void *get_dirtyrate_thread(void *arg) if (ret =3D=3D -1) { error_report("change dirtyrate state failed."); } + + rcu_unregister_thread(); return NULL; } =20 diff --git a/migration/trace-events b/migration/trace-events index 860c4f4025..4c5a658665 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -330,6 +330,11 @@ get_ramblock_vfn_hash(const char *idstr, uint64_t vfn,= uint32_t crc) "ramblock n calc_page_dirty_rate(const char *idstr, uint32_t new_crc, uint32_t old_crc= ) "ramblock name: %s, new crc: %" PRIu32 ", old crc: %" PRIu32 skip_sample_ramblock(const char *idstr, uint64_t ramblock_size) "ramblock = name: %s, ramblock size: %" PRIu64 find_page_matched(const char *idstr) "ramblock %s addr or size changed" +dirtyrate_calculate(int64_t dirtyrate) "dirty rate: %" PRIi64 +dirtyrate_do_calculate_vcpu(int idx, uint64_t pages, int64_t seconds) "vcp= u[%d]: dirty %"PRIu64 " pages in %"PRIi64 " seconds" +dirtyrate_powerdown_callback(void) "" +dirtyrate_dirty_log_start(void) "" +dirtyrate_dirty_log_stop(void) "" =20 # block.c migration_block_init_shared(const char *blk_device_name) "Start migration = for %s with shared base image" --=20 2.18.2