From nobody Tue Oct 28 12:10:48 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1515046199567537.8133135812744; Wed, 3 Jan 2018 22:09:59 -0800 (PST) Received: from localhost ([::1]:47486 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eWyj0-0002IF-Pm for importer@patchew.org; Thu, 04 Jan 2018 01:09:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36949) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eWybW-0003nC-Kz for qemu-devel@nongnu.org; Thu, 04 Jan 2018 01:02:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eWybV-0004GV-E7 for qemu-devel@nongnu.org; Thu, 04 Jan 2018 01:02:14 -0500 Received: from mail-pf0-x230.google.com ([2607:f8b0:400e:c00::230]:45414) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eWybV-0004G1-8x for qemu-devel@nongnu.org; Thu, 04 Jan 2018 01:02:13 -0500 Received: by mail-pf0-x230.google.com with SMTP id u19so365592pfa.12 for ; Wed, 03 Jan 2018 22:02:13 -0800 (PST) Received: from localhost.localdomain (120.236.201.35.bc.googleusercontent.com. [35.201.236.120]) by smtp.gmail.com with ESMTPSA id w5sm6214775pfi.74.2018.01.03.22.02.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 03 Jan 2018 22:02:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=o9YWNPx3CTT5V9kJXFl72B413c+NQAN7NhavooxbAVM=; b=ZVTidTRF/i6WHxmN/S2+LAlzpUMljHTipM97xSQqF/FJRO5i7i8Ii7udTI5lLnuaXs 0sgUhK4mdlpPbQsMjVWIC1zwu3xDFSJNmdJO1gS6lpJ2xHMKm9GwBuJn9ZS9PcURN6JQ 9DCBU95yY8P76+h0yOrgiBms6UgJRv7zg5siMQFWnLttvn2IOMM2LYctVUT9mjwbwN5I F8wsCMmpIt5Q1xUk1PBKALOP1N5kJneBr0aD3r8ivUW7RMVheEvzC6UBWBrnNYF895Yz XWaz4qewSYNOO3sgMJvOU+clVn7b118K1GW3mOrEveVSM3bItJLB31iemRFfOFcIZhVj Ql6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=o9YWNPx3CTT5V9kJXFl72B413c+NQAN7NhavooxbAVM=; b=cGFZHGAzrsPV90ruPlJEQMQ/iGzQHqywukxuhrlMIzZXmoTwuh1YyynlzotiTeuLEJ MCIDq5Jsc1b9T8GAXkRm8ZivcXR13PMUqM6+U5FZsTLq2H0GveHGLpLJV7CEo0doNiEa IcNm+r6Q48sqVTmlDwZ96qU2GPoruuaEggcUXIYLrEJNgBNW+6C/2tEix697kRAoVfvz 3uygGj8ZqSRQQM6fGtSNY2uAVojTi/sK37KHTZ7AeuBrh/T51UkKqgtC3Y1frMWsotv/ QZsJygESB7pyBPFbFk9hmPmiOsiH32p3DvNCvY2L1+5vVDTzJyEZ9R/ue+lOOJgVl33A tr1w== X-Gm-Message-State: AKGB3mJ8xv223KRU2eYdMpcbYvVAG6NfYyjLfjKoWBBMKhvU2hebuwd0 UnUR2bkJbiQ9W5qK+m3NWKNqtcoQ X-Google-Smtp-Source: ACJfBovPwaVW556LZoEQaeD14hc8mtwXczhANc3qy8WQqnJDwzCQPbdGnoM2Pu00ouw8hioQpDEg0g== X-Received: by 10.101.101.65 with SMTP id a1mr3155682pgw.289.1515045731915; Wed, 03 Jan 2018 22:02:11 -0800 (PST) From: Zhang Chen To: qemu devel Date: Thu, 4 Jan 2018 14:01:08 +0800 Message-Id: <1515045675-6993-10-git-send-email-zhangckid@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1515045675-6993-1-git-send-email-zhangckid@gmail.com> References: <1515045675-6993-1-git-send-email-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::230 Subject: [Qemu-devel] [PATCH RESEND V3 09/16] COLO: Flush memory data from ram cache X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: zhanghailiang During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkp= oint time. So, the content of SVM's RAM cache will always be same with PVM's mem= ory after checkpoint. Instead of flushing all content of PVM's RAM cache into SVM's MEMORY, we do this in a more efficient way: Only flush any page that dirtied by PVM since last checkpoint. In this way, we can ensure SVM's memory same with PVM's. Besides, we must ensure flush RAM cache before load device state. Cc: Juan Quintela Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 39 +++++++++++++++++++++++++++++++++++++++ migration/trace-events | 2 ++ 2 files changed, 41 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 388333d..23c67e0 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2908,6 +2908,40 @@ static bool postcopy_is_running(void) return ps >=3D POSTCOPY_INCOMING_LISTENING && ps < POSTCOPY_INCOMING_E= ND; } =20 +/* + * Flush content of RAM cache into SVM's memory. + * Only flush the pages that be dirtied by PVM or SVM or both. + */ +static void colo_flush_ram_cache(void) +{ + RAMBlock *block =3D NULL; + void *dst_host; + void *src_host; + unsigned long offset =3D 0; + + trace_colo_flush_ram_cache_begin(ram_state->migration_dirty_pages); + rcu_read_lock(); + block =3D QLIST_FIRST_RCU(&ram_list.blocks); + + while (block) { + offset =3D migration_bitmap_find_dirty(ram_state, block, offset); + migration_bitmap_clear_dirty(ram_state, block, offset); + + if (offset << TARGET_PAGE_BITS >=3D block->used_length) { + offset =3D 0; + block =3D QLIST_NEXT_RCU(block, next); + } else { + dst_host =3D block->host + (offset << TARGET_PAGE_BITS); + src_host =3D block->colo_cache + (offset << TARGET_PAGE_BITS); + memcpy(dst_host, src_host, TARGET_PAGE_SIZE); + } + } + + rcu_read_unlock(); + trace_colo_flush_ram_cache_end(); + assert(ram_state->migration_dirty_pages =3D=3D 0); +} + static int ram_load(QEMUFile *f, void *opaque, int version_id) { int flags =3D 0, ret =3D 0, invalid_flags =3D 0; @@ -2920,6 +2954,7 @@ static int ram_load(QEMUFile *f, void *opaque, int ve= rsion_id) bool postcopy_running =3D postcopy_is_running(); /* ADVISE is earlier, it shows the source has the postcopy capability = on */ bool postcopy_advised =3D postcopy_is_advised(); + bool need_flush =3D false; =20 seq_iter++; =20 @@ -3095,6 +3130,10 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) wait_for_decompress_done(); rcu_read_unlock(); trace_ram_load_complete(ret, seq_iter); + + if (!ret && migration_incoming_in_colo_state() && need_flush) { + colo_flush_ram_cache(); + } return ret; } =20 diff --git a/migration/trace-events b/migration/trace-events index d4738c8..69261a0 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -78,6 +78,8 @@ ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRI= x64 " %x" ram_postcopy_send_discard_bitmap(void) "" ram_save_page(const char *rbname, uint64_t offset, void *host) "%s: offset= : 0x%" PRIx64 " host: %p" ram_save_queue_pages(const char *rbname, size_t start, size_t len) "%s: st= art: 0x%zx len: 0x%zx" +colo_flush_ram_cache_begin(uint64_t dirty_pages) "dirty_pages %" PRIu64 +colo_flush_ram_cache_end(void) "" =20 # migration/migration.c await_return_path_close_on_source_close(void) "" --=20 2.7.4