From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929048625359.7032900057188; Wed, 18 Jul 2018 08:50:48 -0700 (PDT) Received: from localhost ([::1]:37295 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoj1-0002OZ-7f for importer@patchew.org; Wed, 18 Jul 2018 11:50:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52979) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hr-K2 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000Hg-IP for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36366) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000FB-39 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoac-0006bu-Vg; Wed, 18 Jul 2018 18:42:07 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:44 +0300 Message-Id: <20180718154200.26777-2-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 01/17] migration: add background snapshot capability X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The capability is used for the background vmstate saving using the migration infrastructure. Background vmstate saving means that the majority of vmstate (RAM) is saved in the background when VM's vCPUS are running. This helps to reduce the VM downtime on VM snapshotting. Signed-off-by: Denis Plotnikov --- migration/migration.c | 35 +++++++++++++++++++++++++++++++++++ migration/migration.h | 1 + qapi/migration.json | 6 +++++- 3 files changed, 41 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index d780601f0c..87096d23ef 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -697,6 +697,12 @@ static bool migrate_caps_check(bool *cap_list, return false; } =20 + if (cap_list[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT]) { + error_setg(errp, "Postcopy is not compatible " + "with background snapshot"); + return false; + } + /* This check is reasonably expensive, so only when it's being * set the first time, also it's only the destination that needs * special support. @@ -711,6 +717,26 @@ static bool migrate_caps_check(bool *cap_list, } } =20 + if (cap_list[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT]) { + if (cap_list[MIGRATION_CAPABILITY_RELEASE_RAM]) { + error_setg(errp, "Background snapshot is not compatible " + "with release ram capability"); + return false; + } + + if (cap_list[MIGRATION_CAPABILITY_COMPRESS]) { + error_setg(errp, "Background snapshot is not " + "currently compatible with compression"); + return false; + } + + if (cap_list[MIGRATION_CAPABILITY_XBZRLE]) { + error_setg(errp, "Background snapshot is not " + "currently compatible with XBZLRE"); + return false; + } + } + return true; } =20 @@ -1635,6 +1661,15 @@ bool migrate_use_block_incremental(void) return s->parameters.block_incremental; } =20 +bool migrate_background_snapshot(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHO= T]; +} + /* migration thread support */ /* * Something bad happened to the RP stream, mark an error diff --git a/migration/migration.h b/migration/migration.h index 663415fe48..21babbc008 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -201,6 +201,7 @@ int migrate_compress_level(void); int migrate_compress_threads(void); int migrate_decompress_threads(void); bool migrate_use_events(void); +bool migrate_background_snapshot(void); =20 /* Sending on the return path - generic and then for each message type */ void migrate_send_rp_shut(MigrationIncomingState *mis, diff --git a/qapi/migration.json b/qapi/migration.json index 03f57c9616..eb5f52b7ff 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -352,12 +352,16 @@ # # @x-multifd: Use more than one fd for migration (since 2.11) # +# @background-snapshot: Using migration infrastructure makes VM snapshot +# saving its RAM in background. This reduces VM downtime. (since = 2.12) +# # Since: 1.2 ## { 'enum': 'MigrationCapability', 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks', 'compress', 'events', 'postcopy-ram', 'x-colo', 'release-ram', - 'block', 'return-path', 'pause-before-switchover', 'x-multifd' = ] } + 'block', 'return-path', 'pause-before-switchover', 'x-multifd', + 'background-snapshot' ] } =20 ## # @MigrationCapabilityStatus: --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929171028340.59960249197616; Wed, 18 Jul 2018 08:52:51 -0700 (PDT) Received: from localhost ([::1]:37310 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffokz-00043K-RM for importer@patchew.org; Wed, 18 Jul 2018 11:52:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52963) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hi-G7 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000Gk-Ay for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36376) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000F8-2c for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoad-0006bu-4V; Wed, 18 Jul 2018 18:42:07 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:45 +0300 Message-Id: <20180718154200.26777-3-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 02/17] bitops: add some atomic versions of bitmap operations X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" 1. test bit 2. test and set bit Signed-off-by: Denis Plotnikov Reviewed-by: Peter Xu --- include/qemu/bitops.h | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/include/qemu/bitops.h b/include/qemu/bitops.h index 3f0926cf40..72afcfaec5 100644 --- a/include/qemu/bitops.h +++ b/include/qemu/bitops.h @@ -94,6 +94,21 @@ static inline int test_and_set_bit(long nr, unsigned lon= g *addr) return (old & mask) !=3D 0; } =20 +/** + * test_and_set_bit_atomic - Set a bit atomically and return its old value + * @nr: Bit to set + * @addr: Address to count from + */ +static inline int test_and_set_bit_atomic(long nr, unsigned long *addr) +{ + unsigned long mask =3D BIT_MASK(nr); + unsigned long *p =3D addr + BIT_WORD(nr); + unsigned long old; + + old =3D atomic_fetch_or(p, mask); + return (old & mask) !=3D 0; +} + /** * test_and_clear_bit - Clear a bit and return its old value * @nr: Bit to clear @@ -134,6 +149,16 @@ static inline int test_bit(long nr, const unsigned lon= g *addr) return 1UL & (addr[BIT_WORD(nr)] >> (nr & (BITS_PER_LONG-1))); } =20 +/** + * test_bit_atomic - Determine whether a bit is set atomicallly + * @nr: bit number to test + * @addr: Address to start counting from + */ +static inline int test_bit_atomic(long nr, const unsigned long *addr) +{ + long valid_nr =3D nr & (BITS_PER_LONG - 1); + return 1UL & (atomic_read(&addr[BIT_WORD(nr)]) >> valid_nr); +} /** * find_last_bit - find the last set bit in a memory region * @addr: The address to start the search at --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929219900125.61342038055716; Wed, 18 Jul 2018 08:53:39 -0700 (PDT) Received: from localhost ([::1]:37313 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffolc-0004WB-Qv for importer@patchew.org; Wed, 18 Jul 2018 11:53:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52958) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003He-Db for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000IK-NB for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36380) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000F9-AA for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoad-0006bu-9h; Wed, 18 Jul 2018 18:42:07 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:46 +0300 Message-Id: <20180718154200.26777-4-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 03/17] threads: add infrastructure to process sigsegv X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Allows to define sigsegv handler temporary for all threads. This is useful to implement copy-on-write logic while linux usefaultfd doesn't support write-protected faults. In the future, switch to using WP userfaultfd when it's available. Signed-off-by: Denis Plotnikov --- include/qemu/thread.h | 5 ++++ util/qemu-thread-posix.c | 52 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index 9910f49b3a..d6fed833fa 100644 --- a/include/qemu/thread.h +++ b/include/qemu/thread.h @@ -210,4 +210,9 @@ void qemu_lockcnt_inc_and_unlock(QemuLockCnt *lockcnt); */ unsigned qemu_lockcnt_count(QemuLockCnt *lockcnt); =20 + +typedef void (*sigsegv_handler)(int signum, siginfo_t *siginfo, void *sigc= tx); +void sigsegv_user_handler_set(sigsegv_handler handler); +void sigsegv_user_handler_reset(void); + #endif diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c index 7306475899..5424b7106d 100644 --- a/util/qemu-thread-posix.c +++ b/util/qemu-thread-posix.c @@ -489,6 +489,47 @@ static void qemu_thread_set_name(QemuThread *thread, c= onst char *name) #endif } =20 +static sigsegv_handler sigsegv_user_handler; + +void sigsegv_user_handler_set(sigsegv_handler handler) +{ + assert(handler); + atomic_set(&sigsegv_user_handler, handler); +} + +static sigsegv_handler sigsegv_user_handler_get(void) +{ + return atomic_read(&sigsegv_user_handler); +} + +void sigsegv_user_handler_reset(void) +{ + atomic_set(&sigsegv_user_handler, NULL); +} + +static void sigsegv_default_handler(int signum, siginfo_t *siginfo, void *= sigctx) +{ + sigsegv_handler handler =3D sigsegv_user_handler_get(); + + if (!handler) { + /* + * remove the sigsegv handler if it's not set by user + * this will lead to re-raising the error without a handler + * and exiting from the program with "Segmentation fault" + */ + int err; + struct sigaction act; + memset(&act, 0, sizeof(act)); + act.sa_flags =3D SA_RESETHAND; + err =3D sigaction(SIGSEGV, &act, NULL); + if (err) { + error_exit(err, __func__); + } + } else { + handler(signum, siginfo, sigctx); + } +} + void qemu_thread_create(QemuThread *thread, const char *name, void *(*start_routine)(void*), void *arg, int mode) @@ -496,14 +537,25 @@ void qemu_thread_create(QemuThread *thread, const cha= r *name, sigset_t set, oldset; int err; pthread_attr_t attr; + struct sigaction act; =20 err =3D pthread_attr_init(&attr); if (err) { error_exit(err, __func__); } =20 + memset(&act, 0, sizeof(act)); + act.sa_flags =3D SA_SIGINFO; + act.sa_sigaction =3D sigsegv_default_handler; + err =3D sigaction(SIGSEGV, &act, NULL); + if (err) { + error_exit(err, __func__); + } + /* Leave signal handling to the iothread. */ sigfillset(&set); + /* ...all but SIGSEGV */ + sigdelset(&set, SIGSEGV); pthread_sigmask(SIG_SETMASK, &set, &oldset); err =3D pthread_create(&thread->thread, &attr, start_routine, arg); if (err) --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531928861498669.1139862087138; Wed, 18 Jul 2018 08:47:41 -0700 (PDT) Received: from localhost ([::1]:37280 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffog0-00088S-8K for importer@patchew.org; Wed, 18 Jul 2018 11:47:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52976) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hp-Jn for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000Gx-DP for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36354) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000FA-33 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoad-0006bu-Dy; Wed, 18 Jul 2018 18:42:07 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:47 +0300 Message-Id: <20180718154200.26777-5-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 04/17] background snapshot: make a dedicated type for ram block list X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The type will be used later to hold a list of ram blocks which memory content to be used for the vm state saving when making a background snapshot. Signed-off-by: Denis Plotnikov --- include/exec/ramlist.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/include/exec/ramlist.h b/include/exec/ramlist.h index 2e2ac6cb99..e0231d3bec 100644 --- a/include/exec/ramlist.h +++ b/include/exec/ramlist.h @@ -44,11 +44,13 @@ typedef struct { unsigned long *blocks[]; } DirtyMemoryBlocks; =20 +typedef QLIST_HEAD(, RAMBlock) RamBlockList; + typedef struct RAMList { QemuMutex mutex; RAMBlock *mru_block; /* RCU-enabled, writes protected by the ramlist lock. */ - QLIST_HEAD(, RAMBlock) blocks; + RamBlockList blocks; DirtyMemoryBlocks *dirty_memory[DIRTY_MEMORY_NUM]; uint32_t version; QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers; --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531928766751729.6962820436454; Wed, 18 Jul 2018 08:46:06 -0700 (PDT) Received: from localhost ([::1]:37275 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoeT-0006rL-N6 for importer@patchew.org; Wed, 18 Jul 2018 11:46:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52985) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Ht-M3 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000IW-OY for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36382) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000FC-BY for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoad-0006bu-LD; Wed, 18 Jul 2018 18:42:07 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:48 +0300 Message-Id: <20180718154200.26777-6-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 05/17] ram: extend the data structures for background snapshotting X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Denis Plotnikov --- include/exec/ram_addr.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h index 6cbc02aa0f..5b403d537d 100644 --- a/include/exec/ram_addr.h +++ b/include/exec/ram_addr.h @@ -36,6 +36,8 @@ struct RAMBlock { char idstr[256]; /* RCU-enabled, writes protected by the ramlist lock */ QLIST_ENTRY(RAMBlock) next; + /* blocks used for background snapshot */ + QLIST_ENTRY(RAMBlock) bgs_next; QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers; int fd; size_t page_size; @@ -49,6 +51,11 @@ struct RAMBlock { unsigned long *unsentmap; /* bitmap of already received pages in postcopy */ unsigned long *receivedmap; + /* The following 2 are for background snapshot */ + /* Pages currently being copied */ + unsigned long *touched_map; + /* Pages has been copied already */ + unsigned long *copied_map; }; =20 static inline bool offset_in_ramblock(RAMBlock *b, ram_addr_t offset) --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929050296276.65786878556594; Wed, 18 Jul 2018 08:50:50 -0700 (PDT) Received: from localhost ([::1]:37297 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoj3-0002Rk-5p for importer@patchew.org; Wed, 18 Jul 2018 11:50:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52965) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hk-GB for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000H8-FR for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36374) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000F6-2a for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoad-0006bu-Qb; Wed, 18 Jul 2018 18:42:07 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:49 +0300 Message-Id: <20180718154200.26777-7-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 06/17] background snapshot: add helpers to manage a copy of ram block list X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The VM ramblock list may be changed during the snapshotting. We want to make sure that we have the same ramblock list as it was at the time of snapshot beginning. So, we create a copy of the list at the beginning of the snapshotting and use it during the process. Signed-off-by: Denis Plotnikov --- migration/ram.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++ migration/ram.h | 6 ++++++ 2 files changed, 57 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 021d583b9b..4399c31606 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1508,6 +1508,57 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss, return pages; } =20 +/* BackGround Snapshot Blocks */ +static RamBlockList bgs_blocks; + +static RamBlockList *ram_bgs_block_list_get(void) +{ + return &bgs_blocks; +} + +void ram_block_list_create(void) +{ + RAMBlock *block =3D NULL; + RamBlockList *block_list =3D ram_bgs_block_list_get(); + + qemu_mutex_lock_ramlist(); + QLIST_FOREACH(block, &ram_list.blocks, next) { + memory_region_ref(block->mr); + QLIST_INSERT_HEAD(block_list, block, bgs_next); + } + qemu_mutex_unlock_ramlist(); +} + +void ram_block_list_destroy(void) +{ + RAMBlock *next, *block; + RamBlockList *block_list =3D ram_bgs_block_list_get(); + + QLIST_FOREACH_SAFE(block, block_list, bgs_next, next) { + QLIST_REMOVE(block, bgs_next); + memory_region_unref(block->mr); + } +} + +RAMBlock *ram_bgs_block_find(uint8_t *address, ram_addr_t *page_offset) +{ + RAMBlock *block =3D NULL; + RamBlockList *block_list =3D ram_bgs_block_list_get(); + + QLIST_FOREACH(block, block_list, bgs_next) { + /* This case append when the block is not mapped. */ + if (block->host =3D=3D NULL) { + continue; + } + + if (address - block->host < block->max_length) { + *page_offset =3D (address - block->host) & TARGET_PAGE_MASK; + return block; + } + } + + return NULL; +} /** * ram_find_and_save_block: finds a dirty page and sends it to f * diff --git a/migration/ram.h b/migration/ram.h index 64d81e9f1d..385579cae9 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -31,6 +31,7 @@ =20 #include "qemu-common.h" #include "exec/cpu-common.h" +#include "exec/ramlist.h" =20 extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; @@ -61,5 +62,10 @@ void ram_handle_compressed(void *host, uint8_t ch, uint6= 4_t size); int ramblock_recv_bitmap_test(RAMBlock *rb, void *host_addr); void ramblock_recv_bitmap_set(RAMBlock *rb, void *host_addr); void ramblock_recv_bitmap_set_range(RAMBlock *rb, void *host_addr, size_t = nr); +/* For background snapshot */ +void ram_block_list_create(void); +void ram_block_list_destroy(void); + +RAMBlock *ram_bgs_block_find(uint8_t *address, ram_addr_t *page_offset); =20 #endif --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929133608805.507886245768; Wed, 18 Jul 2018 08:52:13 -0700 (PDT) Received: from localhost ([::1]:37304 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffokO-0003cl-HQ for importer@patchew.org; Wed, 18 Jul 2018 11:52:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52974) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hn-IH for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000HM-FP for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36368) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000F7-37 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoad-0006bu-Uw; Wed, 18 Jul 2018 18:42:08 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:50 +0300 Message-Id: <20180718154200.26777-8-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 07/17] background snapshot: introduce page buffer X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To limit the amount of memory used by the background snapshot a memory limiter is used which called "page buffer". In fact, it's a memory page counter limiting the page allocation. Currently, the limit of pages is hardcoded but its setter is the matter of the future work. The background snapshot makes a copy of the page before writing it to the snapshot destination, but it doesn't use a preallocated buffer, because the background snapshot employs the migration infrastructure which, in turn, uses madvice to release the buffer. The advice issuing is hard to track, so here we rely on anonymous memory mapping and releasing it by the existing code. Signed-off-by: Denis Plotnikov --- migration/ram.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++ migration/ram.h | 3 ++ 2 files changed, 94 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 4399c31606..27d3403435 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -192,6 +192,16 @@ struct RAMSrcPageRequest { QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; =20 +/* Page buffer used for background snapshot */ +typedef struct RAMPageBuffer { + /* Page buffer capacity in host pages */ + int capacity; + /* Current number of pages in the buffer */ + int used; + /* Event to notify that buffer usage is under capacity */ + QemuEvent used_decreased; +} RAMPageBuffer; + /* State of RAM for migration */ struct RAMState { /* QEMUFile used for this migration */ @@ -230,6 +240,11 @@ struct RAMState { /* Queue of outstanding page requests from the destination */ QemuMutex src_page_req_mutex; QSIMPLEQ_HEAD(src_page_requests, RAMSrcPageRequest) src_page_requests; + /* The following 2 are for background snapshot */ + /* Buffer data to store copies of ram pages while async vm saving */ + RAMPageBuffer page_buffer; + /* Event to notify that a page copying just has finished*/ + QemuEvent page_copying_done; }; typedef struct RAMState RAMState; =20 @@ -1540,6 +1555,52 @@ void ram_block_list_destroy(void) } } =20 +static void ram_page_buffer_decrease_used(void) +{ + qemu_event_reset(&ram_state->page_buffer.used_decreased); + atomic_dec(&ram_state->page_buffer.used); + qemu_event_set(&ram_state->page_buffer.used_decreased); +} + +static void ram_page_buffer_increase_used_wait(void) +{ + int used, *used_ptr; + RAMState *rs =3D ram_state; + used_ptr =3D &rs->page_buffer.used; + do { + used =3D atomic_read(used_ptr); + if (rs->page_buffer.capacity > used) { + if (atomic_cmpxchg(used_ptr, used, used + 1) =3D=3D used) { + return; + } else { + continue; + } + } else { + qemu_event_wait(&rs->page_buffer.used_decreased); + } + } while (true); +} + +void *ram_page_buffer_get(void) +{ + void *page; + ram_page_buffer_increase_used_wait(); + page =3D mmap(0, TARGET_PAGE_SIZE, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + + if (page =3D=3D MAP_FAILED) { + ram_page_buffer_decrease_used(); + page =3D NULL; + } + return page; +} + +int ram_page_buffer_free(void *buffer) +{ + ram_page_buffer_decrease_used(); + return qemu_madvise(buffer, TARGET_PAGE_SIZE, MADV_DONTNEED); +} + RAMBlock *ram_bgs_block_find(uint8_t *address, ram_addr_t *page_offset) { RAMBlock *block =3D NULL; @@ -1559,6 +1620,7 @@ RAMBlock *ram_bgs_block_find(uint8_t *address, ram_ad= dr_t *page_offset) =20 return NULL; } + /** * ram_find_and_save_block: finds a dirty page and sends it to f * @@ -1651,6 +1713,13 @@ static void xbzrle_load_cleanup(void) =20 static void ram_state_cleanup(RAMState **rsp) { + if (migrate_background_snapshot()) { + qemu_event_destroy(&(*rsp)->page_buffer.used_decreased); + /* In case somebody is still waiting for the event */ + qemu_event_set(&(*rsp)->page_copying_done); + qemu_event_destroy(&(*rsp)->page_copying_done); + } + migration_page_queue_free(*rsp); qemu_mutex_destroy(&(*rsp)->bitmap_mutex); qemu_mutex_destroy(&(*rsp)->src_page_req_mutex); @@ -1689,6 +1758,13 @@ static void ram_save_cleanup(void *opaque) block->bmap =3D NULL; g_free(block->unsentmap); block->unsentmap =3D NULL; + + if (migrate_background_snapshot()) { + g_free(block->touched_map); + block->touched_map =3D NULL; + g_free(block->copied_map); + block->copied_map =3D NULL; + } } =20 xbzrle_cleanup(); @@ -1703,6 +1779,10 @@ static void ram_state_reset(RAMState *rs) rs->last_page =3D 0; rs->last_version =3D ram_list.version; rs->ram_bulk_stage =3D true; + + /* page buffer capacity in number of pages */ + rs->page_buffer.capacity =3D 1000; + rs->page_buffer.used =3D 0; } =20 #define MAX_WAIT 50 /* ms, half buffered_file limit */ @@ -2180,6 +2260,11 @@ static int ram_state_init(RAMState **rsp) */ (*rsp)->migration_dirty_pages =3D ram_bytes_total() >> TARGET_PAGE_BIT= S; =20 + if (migrate_background_snapshot()) { + qemu_event_init(&ram_state->page_buffer.used_decreased, false); + qemu_event_init(&ram_state->page_copying_done, false); + } + ram_state_reset(*rsp); =20 return 0; @@ -2196,10 +2281,16 @@ static void ram_list_init_bitmaps(void) pages =3D block->max_length >> TARGET_PAGE_BITS; block->bmap =3D bitmap_new(pages); bitmap_set(block->bmap, 0, pages); + if (migrate_postcopy_ram()) { block->unsentmap =3D bitmap_new(pages); bitmap_set(block->unsentmap, 0, pages); } + + if (migrate_background_snapshot()) { + block->touched_map =3D bitmap_new(pages); + block->copied_map =3D bitmap_new(pages); + } } } } diff --git a/migration/ram.h b/migration/ram.h index 385579cae9..7c802c1d7f 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -68,4 +68,7 @@ void ram_block_list_destroy(void); =20 RAMBlock *ram_bgs_block_find(uint8_t *address, ram_addr_t *page_offset); =20 +void *ram_page_buffer_get(void); +int ram_page_buffer_free(void *buffer); + #endif --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929311024244.76611157598984; Wed, 18 Jul 2018 08:55:11 -0700 (PDT) Received: from localhost ([::1]:37319 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffonF-0005nI-Vt for importer@patchew.org; Wed, 18 Jul 2018 11:55:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52962) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hh-FU for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000IB-LD for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36386) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000FN-8h for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoae-0006bu-1f; Wed, 18 Jul 2018 18:42:08 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:51 +0300 Message-Id: <20180718154200.26777-9-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 08/17] migration: add helpers to change VM memory protection rights X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Denis Plotnikov --- migration/ram.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++++ migration/ram.h | 3 +++ 2 files changed, 57 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 27d3403435..ce3dead932 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1555,6 +1555,60 @@ void ram_block_list_destroy(void) } } =20 +static int mem_protect(void *addr, uint64_t length, int prot) +{ + int ret =3D mprotect(addr, length, prot); + + if (ret < 0) { + error_report("%s: Can't change protection on ram block at %p", + __func__, addr); + } + + return ret; +} + +static int ram_set_ro(void *addr, uint64_t length) +{ + return mem_protect(addr, length, PROT_READ); +} + +static int ram_set_rw(void *addr, uint64_t length) +{ + return mem_protect(addr, length, PROT_READ | PROT_WRITE); +} + +int ram_block_list_set_readonly(void) +{ + RAMBlock *block =3D NULL; + RamBlockList *block_list =3D ram_bgs_block_list_get(); + int ret =3D 0; + + QLIST_FOREACH(block, block_list, bgs_next) { + ret =3D ram_set_ro(block->host, block->used_length); + if (ret) { + break; + } + } + + return ret; +} + +int ram_block_list_set_writable(void) +{ + RAMBlock *block =3D NULL; + RamBlockList *block_list =3D ram_bgs_block_list_get(); + int ret =3D 0; + + QLIST_FOREACH(block, block_list, bgs_next) { + ret =3D ram_set_rw(block->host, block->used_length); + if (ret) { + break; + } + } + + return ret; +} + static void ram_page_buffer_decrease_used(void) { qemu_event_reset(&ram_state->page_buffer.used_decreased); diff --git a/migration/ram.h b/migration/ram.h index 7c802c1d7f..4c463597a5 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -71,4 +71,7 @@ RAMBlock *ram_bgs_block_find(uint8_t *address, ram_addr_t= *page_offset); void *ram_page_buffer_get(void); int ram_page_buffer_free(void *buffer); =20 +int ram_block_list_set_readonly(void); +int ram_block_list_set_writable(void); + #endif --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531928647489451.45537557813657; Wed, 18 Jul 2018 08:44:07 -0700 (PDT) Received: from localhost ([::1]:37255 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffocY-0004WK-Br for importer@patchew.org; Wed, 18 Jul 2018 11:44:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52977) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hq-KB for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000Hp-JN for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36390) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000FW-3V for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoae-0006bu-7O; Wed, 18 Jul 2018 18:42:08 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:52 +0300 Message-Id: <20180718154200.26777-10-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 09/17] background snapshot: extend RAM request for holding a page copy pointer X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This pointer is going to be used to transfer a memory. Once the memory page is copied the content the snapshot interested in is saved for writing and we can make the page writable again. Signed-off-by: Denis Plotnikov --- migration/ram.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index ce3dead932..dc7dfe0726 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -188,6 +188,7 @@ struct RAMSrcPageRequest { RAMBlock *rb; hwaddr offset; hwaddr len; + void *page_copy; =20 QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; @@ -265,6 +266,8 @@ struct PageSearchStatus { unsigned long page; /* Set once we wrap around */ bool complete_round; + /* Pointer to the cached page */ + void *page_copy; }; typedef struct PageSearchStatus PageSearchStatus; =20 --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929463638520.7720162089546; Wed, 18 Jul 2018 08:57:43 -0700 (PDT) Received: from localhost ([::1]:37340 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffopi-0007iy-FT for importer@patchew.org; Wed, 18 Jul 2018 11:57:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52967) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hl-Hm for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000I3-Kl for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36394) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000Fb-3F for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoae-0006bu-D8; Wed, 18 Jul 2018 18:42:08 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:53 +0300 Message-Id: <20180718154200.26777-11-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 10/17] background snapshots: adapt the page queueing code for using page copies X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The background snapshot uses memeory page copying to seal the page memory content. The patch adapts the migration infrastructure to save copies of the pages. Signed-off-by: Denis Plotnikov --- migration/migration.c | 2 +- migration/ram.c | 59 ++++++++++++++++++++++++++++++++----------- migration/ram.h | 3 ++- 3 files changed, 47 insertions(+), 17 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 87096d23ef..131d0904e4 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1716,7 +1716,7 @@ static void migrate_handle_rp_req_pages(MigrationStat= e *ms, const char* rbname, return; } =20 - if (ram_save_queue_pages(rbname, start, len)) { + if (ram_save_queue_pages(NULL, rbname, start, len, NULL)) { mark_source_rp_bad(ms); } } diff --git a/migration/ram.c b/migration/ram.c index dc7dfe0726..3c4ccd85b4 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -976,7 +976,12 @@ static int ram_save_page(RAMState *rs, PageSearchStatu= s *pss, bool last_stage) RAMBlock *block =3D pss->block; ram_addr_t offset =3D pss->page << TARGET_PAGE_BITS; =20 - p =3D block->host + offset; + if (pss->page_copy) { + p =3D pss->page_copy; + } else { + p =3D block->host + offset; + } + trace_ram_save_page(block->idstr, (uint64_t)offset, p); =20 /* In doubt sent page as normal */ @@ -1003,13 +1008,18 @@ static int ram_save_page(RAMState *rs, PageSearchSt= atus *pss, bool last_stage) } else { pages =3D save_zero_page(rs, block, offset, p); if (pages > 0) { - /* Must let xbzrle know, otherwise a previous (now 0'd) cached - * page would be stale - */ - xbzrle_cache_zero_page(rs, current_addr); - ram_release_pages(block->idstr, offset, pages); + if (pss->page_copy) { + qemu_madvise(p, TARGET_PAGE_SIZE, MADV_DONTNEED); + } else { + /* Must let xbzrle know, otherwise a previous (now 0'd) ca= ched + * page would be stale + */ + xbzrle_cache_zero_page(rs, current_addr); + ram_release_pages(block->idstr, offset, pages); + } } else if (!rs->ram_bulk_stage && - !migration_in_postcopy() && migrate_use_xbzrle()) { + !migration_in_postcopy() && migrate_use_xbzrle() && + !migrate_background_snapshot()) { pages =3D save_xbzrle_page(rs, &p, current_addr, block, offset, last_stage); if (!last_stage) { @@ -1026,9 +1036,10 @@ static int ram_save_page(RAMState *rs, PageSearchSta= tus *pss, bool last_stage) ram_counters.transferred +=3D save_page_header(rs, rs->f, block, offset | RAM_SAVE_FLAG_PAGE= ); if (send_async) { - qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, - migrate_release_ram() & - migration_in_postcopy()); + bool may_free =3D migrate_background_snapshot() || + (migrate_release_ram() && + migration_in_postcopy()); + qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, may_free); } else { qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); } @@ -1269,7 +1280,8 @@ static bool find_dirty_block(RAMState *rs, PageSearch= Status *pss, bool *again) * @rs: current RAM state * @offset: used to return the offset within the RAMBlock */ -static RAMBlock *unqueue_page(RAMState *rs, ram_addr_t *offset) +static RAMBlock *unqueue_page(RAMState *rs, ram_addr_t *offset, + void **page_copy) { RAMBlock *block =3D NULL; =20 @@ -1279,10 +1291,14 @@ static RAMBlock *unqueue_page(RAMState *rs, ram_add= r_t *offset) QSIMPLEQ_FIRST(&rs->src_page_requests); block =3D entry->rb; *offset =3D entry->offset; + *page_copy =3D entry->page_copy; =20 if (entry->len > TARGET_PAGE_SIZE) { entry->len -=3D TARGET_PAGE_SIZE; entry->offset +=3D TARGET_PAGE_SIZE; + if (entry->page_copy) { + entry->page_copy +=3D TARGET_PAGE_SIZE / sizeof(void *); + } } else { memory_region_unref(block->mr); QSIMPLEQ_REMOVE_HEAD(&rs->src_page_requests, next_req); @@ -1309,9 +1325,10 @@ static bool get_queued_page(RAMState *rs, PageSearch= Status *pss) RAMBlock *block; ram_addr_t offset; bool dirty; + void *page_copy; =20 do { - block =3D unqueue_page(rs, &offset); + block =3D unqueue_page(rs, &offset, &page_copy); /* * We're sending this page, and since it's postcopy nothing else * will dirty it, and we must make sure it doesn't get sent again @@ -1349,6 +1366,7 @@ static bool get_queued_page(RAMState *rs, PageSearchS= tatus *pss) */ pss->block =3D block; pss->page =3D offset >> TARGET_PAGE_BITS; + pss->page_copy =3D page_copy; } =20 return !!block; @@ -1386,17 +1404,25 @@ static void migration_page_queue_free(RAMState *rs) * * @rbname: Name of the RAMBLock of the request. NULL means the * same that last one. + * @block: RAMBlock to use. block and rbname have mutualy exclusive + * semantic with higher priority of the block. * @start: starting address from the start of the RAMBlock * @len: length (in bytes) to send + * @page_copy: the address the page should be written from */ -int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t = len) +int ram_save_queue_pages(RAMBlock *block, const char *rbname, + ram_addr_t start, ram_addr_t len, void *page_copy) { RAMBlock *ramblock; RAMState *rs =3D ram_state; =20 ram_counters.postcopy_requests++; + rcu_read_lock(); - if (!rbname) { + + if (block) { + ramblock =3D block; + } else if (!rbname) { /* Reuse last RAMBlock */ ramblock =3D rs->last_req_rb; =20 @@ -1431,6 +1457,7 @@ int ram_save_queue_pages(const char *rbname, ram_addr= _t start, ram_addr_t len) new_entry->rb =3D ramblock; new_entry->offset =3D start; new_entry->len =3D len; + new_entry->page_copy =3D page_copy; =20 memory_region_ref(ramblock->mr); qemu_mutex_lock(&rs->src_page_req_mutex); @@ -1468,7 +1495,8 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss, * xbzrle can do better than compression. */ if (migrate_use_compression() && - (rs->ram_bulk_stage || !migrate_use_xbzrle())) { + (rs->ram_bulk_stage || !migrate_use_xbzrle()) && + !migrate_background_snapshot()) { res =3D ram_save_compressed_page(rs, pss, last_stage); } else { res =3D ram_save_page(rs, pss, last_stage); @@ -1706,6 +1734,7 @@ static int ram_find_and_save_block(RAMState *rs, bool= last_stage) pss.block =3D rs->last_seen_block; pss.page =3D rs->last_page; pss.complete_round =3D false; + pss.page_copy =3D NULL; =20 if (!pss.block) { pss.block =3D QLIST_FIRST_RCU(&ram_list.blocks); diff --git a/migration/ram.h b/migration/ram.h index 4c463597a5..c3679ba65e 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -46,7 +46,8 @@ int multifd_load_setup(void); int multifd_load_cleanup(Error **errp); =20 uint64_t ram_pagesize_summary(void); -int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t = len); +int ram_save_queue_pages(RAMBlock *block, const char *rbname, + ram_addr_t start, ram_addr_t len, void *page_copy= ); void acct_update_position(QEMUFile *f, size_t size, bool zero); void ram_debug_dump_bitmap(unsigned long *todump, bool expected, unsigned long pages); --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929278702158.42723332666094; Wed, 18 Jul 2018 08:54:38 -0700 (PDT) Received: from localhost ([::1]:37316 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffomj-0005LD-C0 for importer@patchew.org; Wed, 18 Jul 2018 11:54:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52973) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hm-IY for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000H6-Db for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36398) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000Fn-3K for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoae-0006bu-Hb; Wed, 18 Jul 2018 18:42:08 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:54 +0300 Message-Id: <20180718154200.26777-12-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 11/17] background snapshot: add a memory page copying function X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It's the only function making a memory page copy. It supports multithreading semantics ensuring that the page is copied by one thread only and releasing the copied page from write protection. Signed-off-by: Denis Plotnikov --- migration/ram.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++++ migration/ram.h | 1 + 2 files changed, 57 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 3c4ccd85b4..10b6fdf23e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1706,6 +1706,62 @@ RAMBlock *ram_bgs_block_find(uint8_t *address, ram_a= ddr_t *page_offset) return NULL; } =20 +/** + * ram_copy_page: make a page copy + * + * Used in the background snapshot to make a copy of a memeory page. + * Ensures that the memeory page is copied only once. + * When a page copy is done, restores read/write access to the memory + * page. + * If a page is being copied by another thread, wait until the copying + * ends and exit. + * + * Returns: + * -1 - on error + * 0 - the page wasn't copied by the function call + * 1 - the page has been copied + * + * @block: RAM block to use + * @page_nr: the page number to copy + * @page_copy: the pointer to return a page copy + * + */ +int ram_copy_page(RAMBlock *block, unsigned long page_nr, + void **page_copy) +{ + void *host_page; + + if (test_and_set_bit_atomic(page_nr, block->touched_map)) { + while (!test_bit_atomic(page_nr, block->copied_map)) { + /* the page is being copied -- wait for the end of the copying + * and check once again + */ + qemu_event_wait(&ram_state->page_copying_done); + } + return 0; + } + + *page_copy =3D ram_page_buffer_get(); + if (!*page_copy) { + return -1; + } + + host_page =3D block->host + (page_nr << TARGET_PAGE_BITS); + memcpy(*page_copy, host_page, TARGET_PAGE_SIZE); + + if (ram_set_rw(host_page, TARGET_PAGE_SIZE)) { + ram_page_buffer_free(*page_copy); + *page_copy =3D NULL; + return -1; + } + + set_bit_atomic(page_nr, block->copied_map); + qemu_event_set(&ram_state->page_copying_done); + qemu_event_reset(&ram_state->page_copying_done); + + return 1; +} + /** * ram_find_and_save_block: finds a dirty page and sends it to f * diff --git a/migration/ram.h b/migration/ram.h index c3679ba65e..76ab0a3377 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -75,4 +75,5 @@ int ram_page_buffer_free(void *buffer); int ram_block_list_set_readonly(void); int ram_block_list_set_writable(void); =20 +int ram_copy_page(RAMBlock *block, unsigned long page_nr, void **page_copy= ); #endif --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929003161638.7788292464469; Wed, 18 Jul 2018 08:50:03 -0700 (PDT) Received: from localhost ([::1]:37291 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoiI-0001Qr-4M for importer@patchew.org; Wed, 18 Jul 2018 11:50:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52975) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Ho-Jp for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000HX-Hj for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36402) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000Fs-3E for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoae-0006bu-KH; Wed, 18 Jul 2018 18:42:08 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:55 +0300 Message-Id: <20180718154200.26777-13-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 12/17] ram: add background snapshot support in ram page saving part of migration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Unify the function saving the ram pages for using in both the migration and the background snapshots. Signed-off-by: Denis Plotnikov --- migration/ram.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 10b6fdf23e..b1623e96e7 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1803,11 +1803,28 @@ static int ram_find_and_save_block(RAMState *rs, bo= ol last_stage) if (!found) { /* priority queue empty, so just search for something dirty */ found =3D find_dirty_block(rs, &pss, &again); + + if (found && migrate_background_snapshot()) { + /* make a copy of the page and + * pass it to the page search status */ + int ret; + ret =3D ram_copy_page(pss.block, pss.page, &pss.page_copy); + if (ret =3D=3D 0) { + found =3D false; + pages =3D 0; + } else if (ret < 0) { + return ret; + } + } } =20 if (found) { pages =3D ram_save_host_page(rs, &pss, last_stage); } + + if (pss.page_copy) { + ram_page_buffer_decrease_used(); + } } while (!pages && again); =20 rs->last_seen_block =3D pss.block; --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531928865684378.4401547800313; Wed, 18 Jul 2018 08:47:45 -0700 (PDT) Received: from localhost ([::1]:37283 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffog4-0008Az-NW for importer@patchew.org; Wed, 18 Jul 2018 11:47:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52982) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hs-Km for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000HV-HU for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36406) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000Fu-4c for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoae-0006bu-Ma; Wed, 18 Jul 2018 18:42:08 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:56 +0300 Message-Id: <20180718154200.26777-14-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 13/17] background snapshot: add write-protected page access handler function X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The handler does all the necessary operations to save the page being accessed to the snapshot file(stream). Signed-off-by: Denis Plotnikov --- migration/ram.c | 43 +++++++++++++++++++++++++++++++++++++++++++ migration/ram.h | 1 + 2 files changed, 44 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index b1623e96e7..04a4bf708d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1762,6 +1762,49 @@ int ram_copy_page(RAMBlock *block, unsigned long pag= e_nr, return 1; } =20 +/** + * ram_process_page_fault + * + * Used in the background snapshot to queue the copy of the memory + * page for writing. + * + * Returns: + * 0 > - on error + * 0 - success + * + * @address: guest address + * + */ +int ram_process_page_fault(void *address) +{ + int ret; + void *page_copy =3D NULL; + unsigned long page_nr; + ram_addr_t offset; + + RAMBlock *block =3D ram_bgs_block_find(address, &offset); + + if (!block) { + return -1; + } + + page_nr =3D offset >> TARGET_PAGE_BITS; + + ret =3D ram_copy_page(block, page_nr, &page_copy); + + if (ret < 0) { + return ret; + } else if (ret > 0) { + if (ram_save_queue_pages(block, NULL, offset, + TARGET_PAGE_SIZE, page_copy)) { + ram_page_buffer_free(page_copy); + return -1; + } + } + + return 0; +} + /** * ram_find_and_save_block: finds a dirty page and sends it to f * diff --git a/migration/ram.h b/migration/ram.h index 76ab0a3377..2b565ce620 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -76,4 +76,5 @@ int ram_block_list_set_readonly(void); int ram_block_list_set_writable(void); =20 int ram_copy_page(RAMBlock *block, unsigned long page_nr, void **page_copy= ); +int ram_process_page_fault(void *address); #endif --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531928816741662.034016801552; Wed, 18 Jul 2018 08:46:56 -0700 (PDT) Received: from localhost ([::1]:37278 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffofH-0007c4-ET for importer@patchew.org; Wed, 18 Jul 2018 11:46:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52964) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hj-GD for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000IZ-O5 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36410) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000G1-8L for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoae-0006bu-PQ; Wed, 18 Jul 2018 18:42:08 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:57 +0300 Message-Id: <20180718154200.26777-15-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 14/17] kvm: add failed memeory access exit reason X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The patch allows qemu to be aware of how to read kvm failed memeory access. This is a temporary patch. It sould be removed on importing the kvm failed memeory access exit from the linux branch. Signed-off-by: Denis Plotnikov --- linux-headers/linux/kvm.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h index d92c9b2f0e..ea63b681d8 100644 --- a/linux-headers/linux/kvm.h +++ b/linux-headers/linux/kvm.h @@ -235,6 +235,7 @@ struct kvm_hyperv_exit { #define KVM_EXIT_S390_STSI 25 #define KVM_EXIT_IOAPIC_EOI 26 #define KVM_EXIT_HYPERV 27 +#define KVM_EXIT_FAIL_MEM_ACCESS 28 =20 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -392,6 +393,10 @@ struct kvm_run { } eoi; /* KVM_EXIT_HYPERV */ struct kvm_hyperv_exit hyperv; + /* KVM_EXIT_FAIL_MEM_ACCESS */ + struct { + __u64 hva; + } fail_mem_access; /* Fix the size of the union. */ char padding[256]; }; --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531928687946944.6595714520377; Wed, 18 Jul 2018 08:44:47 -0700 (PDT) Received: from localhost ([::1]:37262 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffodC-00052j-T7 for importer@patchew.org; Wed, 18 Jul 2018 11:44:46 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52959) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoal-0003Hf-Dy for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000IQ-O2 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:36414) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000GD-D0 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoae-0006bu-Uh; Wed, 18 Jul 2018 18:42:09 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:58 +0300 Message-Id: <20180718154200.26777-16-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 15/17] kvm: add vCPU failed memeory access processing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Is done with support of the KVM patch returning the faulting address. Signed-off-by: Denis Plotnikov --- target/i386/kvm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 3ac5302bc5..55b8860d1a 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -45,6 +45,8 @@ #include "hw/pci/msi.h" #include "hw/pci/msix.h" #include "migration/blocker.h" +#include "migration/savevm.h" +#include "migration/ram.h" #include "exec/memattrs.h" #include "trace.h" =20 @@ -3130,6 +3132,18 @@ static bool host_supports_vmx(void) return ecx & CPUID_EXT_VMX; } =20 +static int kvm_handle_fail_mem_access(CPUState *cpu) +{ + struct kvm_run *run =3D cpu->kvm_run; + int ret =3D ram_process_page_fault((void *)run->fail_mem_access.hva); + + if (ret >=3D 0) { + cpu_resume(cpu); + } + + return ret; +} + #define VMX_INVALID_GUEST_STATE 0x80000021 =20 int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) @@ -3188,6 +3202,9 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run= *run) ioapic_eoi_broadcast(run->eoi.vector); ret =3D 0; break; + case KVM_EXIT_FAIL_MEM_ACCESS: + ret =3D kvm_handle_fail_mem_access(cs); + break; default: fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason); ret =3D -1; --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929349657188.04627870688626; Wed, 18 Jul 2018 08:55:49 -0700 (PDT) Received: from localhost ([::1]:37328 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffons-0006KJ-LT for importer@patchew.org; Wed, 18 Jul 2018 11:55:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52996) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoam-0003Hw-1d for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000Ig-Re for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:16 -0400 Received: from relay.sw.ru ([185.231.240.75]:36418) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000GK-HT for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoaf-0006bu-3I; Wed, 18 Jul 2018 18:42:09 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:41:59 +0300 Message-Id: <20180718154200.26777-17-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 16/17] migration: move the device state saving logic to a separate function X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The logic being split will be reused by the background snapshot. Signed-off-by: Denis Plotnikov --- migration/savevm.c | 91 +++++++++++++++++++++++++--------------------- migration/savevm.h | 2 + 2 files changed, 52 insertions(+), 41 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index f202c3de3a..36074ec9de 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1107,51 +1107,15 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *= f) qemu_fflush(f); } =20 -int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, - bool inactivate_disks) +int qemu_savevm_state_save(QEMUFile *f, bool inactivate_disks, + bool send_eof) { - QJSON *vmdesc; int vmdesc_len; SaveStateEntry *se; int ret; - bool in_postcopy =3D migration_in_postcopy(); - - trace_savevm_state_complete_precopy(); + QJSON *vmdesc =3D qjson_new(); =20 cpu_synchronize_all_states(); - - QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || - (in_postcopy && se->ops->has_postcopy && - se->ops->has_postcopy(se->opaque)) || - (in_postcopy && !iterable_only) || - !se->ops->save_live_complete_precopy) { - continue; - } - - if (se->ops && se->ops->is_active) { - if (!se->ops->is_active(se->opaque)) { - continue; - } - } - trace_savevm_section_start(se->idstr, se->section_id); - - save_section_header(f, se, QEMU_VM_SECTION_END); - - ret =3D se->ops->save_live_complete_precopy(f, se->opaque); - trace_savevm_section_end(se->idstr, se->section_id, ret); - save_section_footer(f, se); - if (ret < 0) { - qemu_file_set_error(f, ret); - return -1; - } - } - - if (iterable_only) { - return 0; - } - - vmdesc =3D qjson_new(); json_prop_int(vmdesc, "page_size", qemu_target_page_size()); json_start_array(vmdesc, "devices"); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { @@ -1193,8 +1157,8 @@ int qemu_savevm_state_complete_precopy(QEMUFile *f, b= ool iterable_only, return ret; } } - if (!in_postcopy) { - /* Postcopy stream will still be going */ + + if (send_eof) { qemu_put_byte(f, QEMU_VM_EOF); } =20 @@ -1213,6 +1177,51 @@ int qemu_savevm_state_complete_precopy(QEMUFile *f, = bool iterable_only, return 0; } =20 +int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, + bool inactivate_disks) +{ + SaveStateEntry *se; + int ret; + bool in_postcopy =3D migration_in_postcopy(); + + trace_savevm_state_complete_precopy(); + + cpu_synchronize_all_states(); + + QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (!se->ops || + (in_postcopy && se->ops->has_postcopy && + se->ops->has_postcopy(se->opaque)) || + (in_postcopy && !iterable_only) || + !se->ops->save_live_complete_precopy) { + continue; + } + + if (se->ops && se->ops->is_active) { + if (!se->ops->is_active(se->opaque)) { + continue; + } + } + trace_savevm_section_start(se->idstr, se->section_id); + + save_section_header(f, se, QEMU_VM_SECTION_END); + + ret =3D se->ops->save_live_complete_precopy(f, se->opaque); + trace_savevm_section_end(se->idstr, se->section_id, ret); + save_section_footer(f, se); + if (ret < 0) { + qemu_file_set_error(f, ret); + return -1; + } + } + + if (iterable_only) { + return 0; + } + + return qemu_savevm_state_save(f, inactivate_disks, !in_postcopy); +} + /* Give an estimate of the amount left to be transferred, * the result is split into the amount for units that can and * for units that can't do postcopy. diff --git a/migration/savevm.h b/migration/savevm.h index 295c4a1f2c..0e8d7aecf8 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -40,6 +40,8 @@ int qemu_savevm_state_complete_precopy(QEMUFile *f, bool = iterable_only, void qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size, uint64_t *res_non_postcopiable, uint64_t *res_postcopiable); +int qemu_savevm_state_save(QEMUFile *f, bool inactivate_disks, + bool send_eof); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); void qemu_savevm_send_open_return_path(QEMUFile *f); int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len); --=20 2.17.0 From nobody Thu May 2 01:38:05 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531929433090953.9110782360444; Wed, 18 Jul 2018 08:57:13 -0700 (PDT) Received: from localhost ([::1]:37332 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffopE-0007Jd-26 for importer@patchew.org; Wed, 18 Jul 2018 11:57:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52999) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ffoam-0003Hx-4w for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ffoag-0000Im-SR for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:16 -0400 Received: from relay.sw.ru ([185.231.240.75]:36422) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ffoag-0000GP-K9 for qemu-devel@nongnu.org; Wed, 18 Jul 2018 11:42:10 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=dptest2.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1ffoaf-0006bu-5y; Wed, 18 Jul 2018 18:42:09 +0300 From: Denis Plotnikov To: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com Date: Wed, 18 Jul 2018 18:42:00 +0300 Message-Id: <20180718154200.26777-18-dplotnikov@virtuozzo.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180718154200.26777-1-dplotnikov@virtuozzo.com> References: <20180718154200.26777-1-dplotnikov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v1 17/17] background snapshot: enable background snapshot X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The patch enables to save vmstate to a migration thread in the background: ram is being saved while vCPUs are running. This is done to reduce downtime on vm snapshotting: the majority of vmstate is ram, the rest of devices consumes only a few MB of memory on a typical vm. By this moment, there were no means to make a snapshot without full vm stopping. From now, one can save a vm state having the vm stopped only for a couple of seconds to save devices' states. Then the vm resumed and ram is written without the vm full stopping. To use async vm state it's needed to enable "background-snapshot" migration capability and start the migration Signed-off-by: Denis Plotnikov --- migration/migration.c | 103 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 101 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 131d0904e4..935e5b6f2e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2188,6 +2188,99 @@ bool migrate_colo_enabled(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_X_COLO]; } =20 +static void background_snapshot_sigsegv_handler(int unused0, siginfo_t *in= fo, + void *unused1) +{ + if (ram_process_page_fault(info->si_addr)) { + assert(false); + } +} + +static void *background_snapshot_thread(void *opaque) +{ + MigrationState *m =3D opaque; + QIOChannelBuffer *bioc; + QEMUFile *fb; + int res =3D 0; + + rcu_register_thread(); + + qemu_file_set_rate_limit(m->to_dst_file, INT64_MAX); + + qemu_mutex_lock_iothread(); + vm_stop(RUN_STATE_PAUSED); + + qemu_savevm_state_header(m->to_dst_file); + qemu_mutex_unlock_iothread(); + qemu_savevm_state_setup(m->to_dst_file); + qemu_mutex_lock_iothread(); + + migrate_set_state(&m->state, MIGRATION_STATUS_SETUP, + MIGRATION_STATUS_ACTIVE); + + /* save the device state to the temporary buffer */ + bioc =3D qio_channel_buffer_new(4096); + qio_channel_set_name(QIO_CHANNEL(bioc), "vmstate-buffer"); + fb =3D qemu_fopen_channel_output(QIO_CHANNEL(bioc)); + object_unref(OBJECT(bioc)); + + ram_block_list_create(); + ram_block_list_set_readonly(); + + if (global_state_store()) { + goto exit; + } + + if (qemu_savevm_state_save(fb, false, true) < 0) { + migrate_set_state(&m->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_FAILED); + goto exit; + } + + sigsegv_user_handler_set(background_snapshot_sigsegv_handler); + + vm_start(); + qemu_mutex_unlock_iothread(); + + while (!res) { + res =3D qemu_savevm_state_iterate(m->to_dst_file, false); + + if (res < 0 || qemu_file_get_error(m->to_dst_file)) { + migrate_set_state(&m->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_FAILED); + goto exit; + } + } + + /* + * By this moment we have RAM saved into the stream + * The next step is to flush the device state right after the + * RAM saved. The rest of device state is stored in + * the temporary buffer. Do flush the buffer into the stream. + */ + qemu_put_buffer(m->to_dst_file, bioc->data, bioc->usage); + qemu_fflush(m->to_dst_file); + + if (qemu_file_get_error(m->to_dst_file)) { + migrate_set_state(&m->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_FAILED); + goto exit; + } + + migrate_set_state(&m->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_COMPLETED); +exit: + ram_block_list_set_writable(); + ram_block_list_destroy(); + sigsegv_user_handler_reset(); + qemu_fclose(fb); + qemu_mutex_lock_iothread(); + qemu_savevm_state_cleanup(); + qemu_mutex_unlock_iothread(); + rcu_unregister_thread(); + return NULL; +} + /* * Master migration thread on the source VM. * It drives the migration and pumps the data down the outgoing channel. @@ -2402,8 +2495,14 @@ void migrate_fd_connect(MigrationState *s) migrate_fd_cleanup(s); return; } - qemu_thread_create(&s->thread, "live_migration", migration_thread, s, - QEMU_THREAD_JOINABLE); + if (migrate_background_snapshot()) { + qemu_thread_create(&s->thread, "bg_snapshot", + background_snapshot_thread, s, + QEMU_THREAD_JOINABLE); + } else { + qemu_thread_create(&s->thread, "live_migration", migration_thread,= s, + QEMU_THREAD_JOINABLE); + } s->migration_thread_running =3D true; } =20 --=20 2.17.0