From nobody Mon Feb 9 16:47:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1637654356902445.28902781405463; Mon, 22 Nov 2021 23:59:16 -0800 (PST) Received: from localhost ([::1]:43066 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mpQhu-0005sf-8P for importer@patchew.org; Tue, 23 Nov 2021 02:59:14 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51856) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mpQfo-0004GC-Gg for qemu-devel@nongnu.org; Tue, 23 Nov 2021 02:57:05 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.221]:57932 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mpQfl-0004dE-DL for qemu-devel@nongnu.org; Tue, 23 Nov 2021 02:57:04 -0500 Received: from clientip-182.150.57.243 (unknown [172.18.0.188]) by chinatelecom.cn (HERMES) with SMTP id 518982800AC; Tue, 23 Nov 2021 15:56:50 +0800 (CST) Received: from ([172.18.0.188]) by app0023 with ESMTP id 58e39fd84de143b384d555d0e5735492 for qemu-devel@nongnu.org; Tue, 23 Nov 2021 15:56:57 CST HMM_SOURCE_IP: 172.18.0.188:55476.606466348 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn X-Transaction-ID: 58e39fd84de143b384d555d0e5735492 X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.188 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Subject: [PATCH v4 1/3] migration/dirtyrate: implement vCPU dirtyrate calculation periodically Date: Tue, 23 Nov 2021 15:56:34 +0800 Message-Id: <844e5b8d6839e6e27010c52fd36d07e7d3d93224.1637653303.git.huangy81@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=42.123.76.221; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Hildenbrand , Hyman , Juan Quintela , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Markus Armbruster , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZM-MESSAGEID: 1637654360624100003 From: Hyman Huang(=E9=BB=84=E5=8B=87) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty restraint. Implement thread for calculate dirtyrate periodly, which will be used for dirty restraint. Add dirtylimit.h to introduce the util function for dirty limit implementation. Signed-off-by: Hyman Huang(=E9=BB=84=E5=8B=87) --- include/exec/memory.h | 5 +- include/sysemu/dirtylimit.h | 44 ++++++++++++++ migration/dirtyrate.c | 139 ++++++++++++++++++++++++++++++++++++++++= ---- migration/dirtyrate.h | 2 + 4 files changed, 179 insertions(+), 11 deletions(-) create mode 100644 include/sysemu/dirtylimit.h diff --git a/include/exec/memory.h b/include/exec/memory.h index 20f1b27..606bec8 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -69,7 +69,10 @@ static inline void fuzz_dma_read_cb(size_t addr, /* Dirty tracking enabled because measuring dirty rate */ #define GLOBAL_DIRTY_DIRTY_RATE (1U << 1) =20 -#define GLOBAL_DIRTY_MASK (0x3) +/* Dirty tracking enabled because dirty limit */ +#define GLOBAL_DIRTY_LIMIT (1U << 2) + +#define GLOBAL_DIRTY_MASK (0x7) =20 extern unsigned int global_dirty_tracking; =20 diff --git a/include/sysemu/dirtylimit.h b/include/sysemu/dirtylimit.h new file mode 100644 index 0000000..49298a2 --- /dev/null +++ b/include/sysemu/dirtylimit.h @@ -0,0 +1,44 @@ +/* + * dirty limit helper functions + * + * Copyright (c) 2021 CHINA TELECOM CO.,LTD. + * + * Authors: + * Hyman Huang(=E9=BB=84=E5=8B=87) + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ +#ifndef QEMU_DIRTYRLIMIT_H +#define QEMU_DIRTYRLIMIT_H + +#define DIRTYLIMIT_CALC_PERIOD_TIME_S 15 /* 15s */ + +/** + * dirtylimit_calc_current: + * + * get current dirty page rate for specified vCPU. + */ +int64_t dirtylimit_calc_current(int cpu_index); + +/** + * dirtylimit_calc: + * + * start dirty page rate calculation thread. + */ +void dirtylimit_calc(void); + +/** + * dirtylimit_calc_quit: + * + * quit dirty page rate calculation thread. + */ +void dirtylimit_calc_quit(void); + +/** + * dirtylimit_calc_state_init: + * + * initialize dirty page rate calculation state. + */ +void dirtylimit_calc_state_init(int max_cpus); +#endif diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index d65e744..d370a21 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -27,6 +27,7 @@ #include "qapi/qmp/qdict.h" #include "sysemu/kvm.h" #include "sysemu/runstate.h" +#include "sysemu/dirtylimit.h" #include "exec/memory.h" =20 /* @@ -46,6 +47,134 @@ static struct DirtyRateStat DirtyStat; static DirtyRateMeasureMode dirtyrate_mode =3D DIRTY_RATE_MEASURE_MODE_PAGE_SAMPLING; =20 +#define DIRTYLIMIT_CALC_TIME_MS 1000 /* 1000ms */ + +struct { + DirtyRatesData data; + int64_t period; + bool quit; +} *dirtylimit_calc_state; + +static void dirtylimit_global_dirty_log_start(void) +{ + qemu_mutex_lock_iothread(); + memory_global_dirty_log_start(GLOBAL_DIRTY_LIMIT); + qemu_mutex_unlock_iothread(); +} + +static void dirtylimit_global_dirty_log_stop(void) +{ + qemu_mutex_lock_iothread(); + memory_global_dirty_log_sync(); + memory_global_dirty_log_stop(GLOBAL_DIRTY_LIMIT); + qemu_mutex_unlock_iothread(); +} + +static inline void record_dirtypages(DirtyPageRecord *dirty_pages, + CPUState *cpu, bool start) +{ + if (start) { + dirty_pages[cpu->cpu_index].start_pages =3D cpu->dirty_pages; + } else { + dirty_pages[cpu->cpu_index].end_pages =3D cpu->dirty_pages; + } +} + +static void dirtylimit_calc_func(void) +{ + CPUState *cpu; + DirtyPageRecord *dirty_pages; + int64_t start_time, end_time, calc_time; + DirtyRateVcpu rate; + int i =3D 0; + + dirty_pages =3D g_malloc0(sizeof(*dirty_pages) * + dirtylimit_calc_state->data.nvcpu); + + dirtylimit_global_dirty_log_start(); + + CPU_FOREACH(cpu) { + record_dirtypages(dirty_pages, cpu, true); + } + + start_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + g_usleep(DIRTYLIMIT_CALC_TIME_MS * 1000); + end_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + calc_time =3D end_time - start_time; + + dirtylimit_global_dirty_log_stop(); + + CPU_FOREACH(cpu) { + record_dirtypages(dirty_pages, cpu, false); + } + + for (i =3D 0; i < dirtylimit_calc_state->data.nvcpu; i++) { + uint64_t increased_dirty_pages =3D + dirty_pages[i].end_pages - dirty_pages[i].start_pages; + uint64_t memory_size_MB =3D + (increased_dirty_pages * TARGET_PAGE_SIZE) >> 20; + int64_t dirtyrate =3D (memory_size_MB * 1000) / calc_time; + + rate.id =3D i; + rate.dirty_rate =3D dirtyrate; + dirtylimit_calc_state->data.rates[i] =3D rate; + + trace_dirtyrate_do_calculate_vcpu(i, + dirtylimit_calc_state->data.rates[i].dirty_rate); + } +} + +static void *dirtylimit_calc_thread(void *opaque) +{ + rcu_register_thread(); + + while (!qatomic_read(&dirtylimit_calc_state->quit)) { + dirtylimit_calc_func(); + sleep(dirtylimit_calc_state->period); + } + + rcu_unregister_thread(); + return NULL; +} + +int64_t dirtylimit_calc_current(int cpu_index) +{ + DirtyRateVcpu *rates =3D dirtylimit_calc_state->data.rates; + + return qatomic_read(&rates[cpu_index].dirty_rate); +} + +void dirtylimit_calc(void) +{ + if (unlikely(qatomic_read(&dirtylimit_calc_state->quit))) { + qatomic_set(&dirtylimit_calc_state->quit, 0); + QemuThread thread; + qemu_thread_create(&thread, "dirtylimit-calc", + dirtylimit_calc_thread, + NULL, QEMU_THREAD_DETACHED); + } +} + +void dirtylimit_calc_quit(void) +{ + qatomic_set(&dirtylimit_calc_state->quit, 1); +} + +void dirtylimit_calc_state_init(int max_cpus) +{ + dirtylimit_calc_state =3D + g_malloc0(sizeof(*dirtylimit_calc_state)); + + dirtylimit_calc_state->data.nvcpu =3D max_cpus; + dirtylimit_calc_state->data.rates =3D + g_malloc0(sizeof(DirtyRateVcpu) * max_cpus); + + dirtylimit_calc_state->period =3D + DIRTYLIMIT_CALC_PERIOD_TIME_S; + + dirtylimit_calc_state->quit =3D true; +} + static int64_t set_sample_page_period(int64_t msec, int64_t initial_time) { int64_t current_time; @@ -396,16 +525,6 @@ static bool compare_page_hash_info(struct RamblockDirt= yInfo *info, return true; } =20 -static inline void record_dirtypages(DirtyPageRecord *dirty_pages, - CPUState *cpu, bool start) -{ - if (start) { - dirty_pages[cpu->cpu_index].start_pages =3D cpu->dirty_pages; - } else { - dirty_pages[cpu->cpu_index].end_pages =3D cpu->dirty_pages; - } -} - static void dirtyrate_global_dirty_log_start(void) { qemu_mutex_lock_iothread(); diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h index 69d4c5b..e96acdc 100644 --- a/migration/dirtyrate.h +++ b/migration/dirtyrate.h @@ -70,6 +70,8 @@ typedef struct VcpuStat { DirtyRateVcpu *rates; /* array of dirty rate for each vcpu */ } VcpuStat; =20 +typedef struct VcpuStat DirtyRatesData; + /* * Store calculation statistics for each measure. */ --=20 1.8.3.1