From nobody Tue May 14 15:26:24 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1683598985; cv=none; d=zohomail.com; s=zohoarc; b=DWwdNTLgTcl/7oTh05L2P5BIfYbC7gNloPEY1Az9t2sImLOSNTVrEy50WzinnG9Xz1qEr5uenJalfq2sdK0S6GX6iLyLYNXLuGJ7mlbaAYWNEzUgVIOOtpamEEfpiuv6VRnGkitv+Xu1JsC4KjZVh/tZ7D+iPzff2/Sc9VB2dnY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1683598985; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=74O2mS3O8kXuvpwKt080JavKpQqyyxkE6lEkvMJMdFU=; b=leDHfF6obvxsMQQmJIAgOvQYxayGBzO3yzv/cRtvrBd41VTDvN4HjlxtGjq06Gu5LPUz/FSB2iDCzfAv1nnEPjEYXuG9tQ2Rg4BylUpprDTDyLSw3sMmuvM/TDOhkAwcJZ6rWPJMDt0UzRkEMBPmpLdEuoJ6BW4V8KdqTEzVshE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1683598985252182.40480203661582; Mon, 8 May 2023 19:23:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pwCzB-0004Ee-K9; Mon, 08 May 2023 22:21:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwCzA-0004E4-0f for qemu-devel@nongnu.org; Mon, 08 May 2023 22:21:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwCz8-0001wU-62 for qemu-devel@nongnu.org; Mon, 08 May 2023 22:21:51 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-39-Y5lL0Qo9Nx6PLUsrWP8UjA-1; Mon, 08 May 2023 22:21:46 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B23493C10ECE; Tue, 9 May 2023 02:21:45 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-118.bne.redhat.com [10.64.54.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 90598492B01; Tue, 9 May 2023 02:21:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683598908; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=74O2mS3O8kXuvpwKt080JavKpQqyyxkE6lEkvMJMdFU=; b=V1m0VI+yZCviQ3o/PDVZxbyb3SqAiDq5sLEnIsGVp+s6OL3DzWL61p2plhzGv0V3ECdP8N aX2LVAs3Yc08i26Q49wbJtmcFvxNj5Jq1/pQEr8tM3pA3WOkR77YEZS5UbanbVBjzOavWb SOiWn1OS5rshdqa6s1MmfuxuJD2OS64= X-MC-Unique: Y5lL0Qo9Nx6PLUsrWP8UjA-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, pbonzini@redhat.com, peter.maydell@linaro.org, peterx@redhat.com, david@redhat.com, philmd@linaro.org, mst@redhat.com, cohuck@redhat.com, quintela@redhat.com, dgilbert@redhat.com, maz@kernel.org, zhenyzha@redhat.com, shan.gavin@gmail.com Subject: [PATCH v3 1/4] migration: Add last stage indicator to global dirty log Date: Tue, 9 May 2023 12:21:19 +1000 Message-Id: <20230509022122.20888-2-gshan@redhat.com> In-Reply-To: <20230509022122.20888-1-gshan@redhat.com> References: <20230509022122.20888-1-gshan@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1683598986484100002 Content-Type: text/plain; charset="utf-8" The global dirty log synchronization is used when KVM and dirty ring are enabled. There is a particularity for ARM64 where the backup bitmap is used to track dirty pages in non-running-vcpu situations. It means the dirty ring works with the combination of ring buffer and backup bitmap. The dirty bits in the backup bitmap needs to collected in the last stage of live migration. In order to identify the last stage of live migration and pass it down, an extra parameter is added to the relevant functions and callbacks. This last stage indicator isn't used until the dirty ring is enabled in the subsequent patches. No functional change intended. Signed-off-by: Gavin Shan Reviewed-by: Peter Xu Tested-by: Zhenyu Zhang --- accel/kvm/kvm-all.c | 2 +- include/exec/memory.h | 7 +++++-- migration/dirtyrate.c | 4 ++-- migration/ram.c | 20 ++++++++++---------- softmmu/memory.c | 10 +++++----- 5 files changed, 23 insertions(+), 20 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index cf3a88d90e..870abad826 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1563,7 +1563,7 @@ static void kvm_log_sync(MemoryListener *listener, kvm_slots_unlock(); } =20 -static void kvm_log_sync_global(MemoryListener *l) +static void kvm_log_sync_global(MemoryListener *l, bool last_stage) { KVMMemoryListener *kml =3D container_of(l, KVMMemoryListener, listener= ); KVMState *s =3D kvm_state; diff --git a/include/exec/memory.h b/include/exec/memory.h index e45ce6061f..df01b0ef8a 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -934,8 +934,11 @@ struct MemoryListener { * its @log_sync must be NULL. Vice versa. * * @listener: The #MemoryListener. + * @last_stage: The last stage to synchronize the log during migration. + * The caller should gurantee that the synchronization with true for + * @last_stage is triggered for once after all VCPUs have been stopped. */ - void (*log_sync_global)(MemoryListener *listener); + void (*log_sync_global)(MemoryListener *listener, bool last_stage); =20 /** * @log_clear: @@ -2423,7 +2426,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *= mr, * * Synchronizes the dirty page log for all address spaces. */ -void memory_global_dirty_log_sync(void); +void memory_global_dirty_log_sync(bool last_stage); =20 /** * memory_global_dirty_log_sync: synchronize the dirty log for all memory diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 180ba38c7a..486085a9cf 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -102,7 +102,7 @@ void global_dirty_log_change(unsigned int flag, bool st= art) static void global_dirty_log_sync(unsigned int flag, bool one_shot) { qemu_mutex_lock_iothread(); - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(false); if (one_shot) { memory_global_dirty_log_stop(flag); } @@ -554,7 +554,7 @@ static void calculate_dirtyrate_dirty_bitmap(struct Dir= tyRateConfig config) * skip it unconditionally and start dirty tracking * from 2'round of log sync */ - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(false); =20 /* * reset page protect manually and unconditionally. diff --git a/migration/ram.c b/migration/ram.c index 5e7bf20ca5..6154e4f18b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1204,7 +1204,7 @@ static void migration_trigger_throttle(RAMState *rs) } } =20 -static void migration_bitmap_sync(RAMState *rs) +static void migration_bitmap_sync(RAMState *rs, bool last_stage) { RAMBlock *block; int64_t end_time; @@ -1216,7 +1216,7 @@ static void migration_bitmap_sync(RAMState *rs) } =20 trace_migration_bitmap_sync_start(); - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(last_stage); =20 qemu_mutex_lock(&rs->bitmap_mutex); WITH_RCU_READ_LOCK_GUARD() { @@ -1251,7 +1251,7 @@ static void migration_bitmap_sync(RAMState *rs) } } =20 -static void migration_bitmap_sync_precopy(RAMState *rs) +static void migration_bitmap_sync_precopy(RAMState *rs, bool last_stage) { Error *local_err =3D NULL; =20 @@ -1264,7 +1264,7 @@ static void migration_bitmap_sync_precopy(RAMState *r= s) local_err =3D NULL; } =20 - migration_bitmap_sync(rs); + migration_bitmap_sync(rs, last_stage); =20 if (precopy_notify(PRECOPY_NOTIFY_AFTER_BITMAP_SYNC, &local_err)) { error_report_err(local_err); @@ -2924,7 +2924,7 @@ void ram_postcopy_send_discard_bitmap(MigrationState = *ms) RCU_READ_LOCK_GUARD(); =20 /* This should be our last sync, the src is now paused */ - migration_bitmap_sync(rs); + migration_bitmap_sync(rs, false); =20 /* Easiest way to make sure we don't resume in the middle of a host-pa= ge */ rs->pss[RAM_CHANNEL_PRECOPY].last_sent_block =3D NULL; @@ -3115,7 +3115,7 @@ static void ram_init_bitmaps(RAMState *rs) /* We don't use dirty log with background snapshots */ if (!migrate_background_snapshot()) { memory_global_dirty_log_start(GLOBAL_DIRTY_MIGRATION); - migration_bitmap_sync_precopy(rs); + migration_bitmap_sync_precopy(rs, false); } } qemu_mutex_unlock_ramlist(); @@ -3439,7 +3439,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) =20 WITH_RCU_READ_LOCK_GUARD() { if (!migration_in_postcopy()) { - migration_bitmap_sync_precopy(rs); + migration_bitmap_sync_precopy(rs, true); } =20 ram_control_before_iterate(f, RAM_CONTROL_FINISH); @@ -3513,7 +3513,7 @@ static void ram_state_pending_exact(void *opaque, uin= t64_t *must_precopy, if (!migration_in_postcopy() && remaining_size < s->threshold_size) { qemu_mutex_lock_iothread(); WITH_RCU_READ_LOCK_GUARD() { - migration_bitmap_sync_precopy(rs); + migration_bitmap_sync_precopy(rs, false); } qemu_mutex_unlock_iothread(); remaining_size =3D rs->migration_dirty_pages * TARGET_PAGE_SIZE; @@ -3923,7 +3923,7 @@ void colo_incoming_start_dirty_log(void) qemu_mutex_lock_iothread(); qemu_mutex_lock_ramlist(); =20 - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(false); WITH_RCU_READ_LOCK_GUARD() { RAMBLOCK_FOREACH_NOT_IGNORED(block) { ramblock_sync_dirty_bitmap(ram_state, block); @@ -4218,7 +4218,7 @@ void colo_flush_ram_cache(void) void *src_host; unsigned long offset =3D 0; =20 - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(false); WITH_RCU_READ_LOCK_GUARD() { RAMBLOCK_FOREACH_NOT_IGNORED(block) { ramblock_sync_dirty_bitmap(ram_state, block); diff --git a/softmmu/memory.c b/softmmu/memory.c index b7b3386e9d..342c121514 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -2253,7 +2253,7 @@ void memory_region_set_dirty(MemoryRegion *mr, hwaddr= addr, * If memory region `mr' is NULL, do global sync. Otherwise, sync * dirty bitmap for the specified memory region. */ -static void memory_region_sync_dirty_bitmap(MemoryRegion *mr) +static void memory_region_sync_dirty_bitmap(MemoryRegion *mr, bool last_st= age) { MemoryListener *listener; AddressSpace *as; @@ -2283,7 +2283,7 @@ static void memory_region_sync_dirty_bitmap(MemoryReg= ion *mr) * is to do a global sync, because we are not capable to * sync in a finer granularity. */ - listener->log_sync_global(listener); + listener->log_sync_global(listener, last_stage); trace_memory_region_sync_dirty(mr ? mr->name : "(all)", listen= er->name, 1); } } @@ -2347,7 +2347,7 @@ DirtyBitmapSnapshot *memory_region_snapshot_and_clear= _dirty(MemoryRegion *mr, { DirtyBitmapSnapshot *snapshot; assert(mr->ram_block); - memory_region_sync_dirty_bitmap(mr); + memory_region_sync_dirty_bitmap(mr, false); snapshot =3D cpu_physical_memory_snapshot_and_clear_dirty(mr, addr, si= ze, client); memory_global_after_dirty_log_sync(); return snapshot; @@ -2873,9 +2873,9 @@ bool memory_region_present(MemoryRegion *container, h= waddr addr) return mr && mr !=3D container; } =20 -void memory_global_dirty_log_sync(void) +void memory_global_dirty_log_sync(bool last_stage) { - memory_region_sync_dirty_bitmap(NULL); + memory_region_sync_dirty_bitmap(NULL, last_stage); } =20 void memory_global_after_dirty_log_sync(void) --=20 2.23.0 From nobody Tue May 14 15:26:24 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1683598967; cv=none; d=zohomail.com; s=zohoarc; b=YOp9v+LhlgV1xilCvYdgAhe0OVRwan4JFw9uhDTYzo3zddY5Ej514GtFbmA/srTTXahFf6Lei4nMrXTyR3cxz4sZAmuXIc4CxGOTfDsfzLTw7CIxdX7vxApAjPQ+cZ1YU2ku90Ne3PKTDWcVUPpW5OsbrNHfaRonrmT2vn0WvDc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1683598967; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=IS1+bURvSYSv/+e3clB+KHrM4sbKl7OvnNf/nWpIgCw=; b=m7461Xd5Q2OeL1Uq6laGl4Yk6Ww9RO8SpzPoqOTXG4rshX6IG8pNh2GBg3TCOA0zyEUTv9PZShaz2Wdc/ggdooETR8crihj/zy2F/2HICz8InfD1zm72pG3Tc7agyJSF6PZ4mYrzyADS/TVBnfADyY9NzQ2XKZIs+dq7QR3b4pE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1683598967458905.8037231796535; Mon, 8 May 2023 19:22:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pwCzQ-0004IQ-HW; Mon, 08 May 2023 22:22:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwCzM-0004GH-ID for qemu-devel@nongnu.org; Mon, 08 May 2023 22:22:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwCzL-0001yP-0g for qemu-devel@nongnu.org; Mon, 08 May 2023 22:22:04 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-458-Vu38PGrZMQyS1pbegxX8xg-1; Mon, 08 May 2023 22:21:51 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0D42D1C0A585; Tue, 9 May 2023 02:21:51 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-118.bne.redhat.com [10.64.54.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3A42D492B00; Tue, 9 May 2023 02:21:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683598922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IS1+bURvSYSv/+e3clB+KHrM4sbKl7OvnNf/nWpIgCw=; b=IsTncCdCdIBUwDkMYCDb8tmSxwFXL7k+JGBzQC1dHqARQKj9K2JiqdXCt1PF8D8kG3AHf0 QiPDwY0BHxEplJgRZVqmh8bPa5K7SYV0/uX5lLm4MBNY4YrPhFpND5nhtiSfyLW1E44IN+ ZXP+HvUtsesb2Y5a8CHzVlnKRmrBO3o= X-MC-Unique: Vu38PGrZMQyS1pbegxX8xg-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, pbonzini@redhat.com, peter.maydell@linaro.org, peterx@redhat.com, david@redhat.com, philmd@linaro.org, mst@redhat.com, cohuck@redhat.com, quintela@redhat.com, dgilbert@redhat.com, maz@kernel.org, zhenyzha@redhat.com, shan.gavin@gmail.com Subject: [PATCH v3 2/4] kvm: Synchronize the backup bitmap in the last stage Date: Tue, 9 May 2023 12:21:20 +1000 Message-Id: <20230509022122.20888-3-gshan@redhat.com> In-Reply-To: <20230509022122.20888-1-gshan@redhat.com> References: <20230509022122.20888-1-gshan@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1683598968323100001 Content-Type: text/plain; charset="utf-8" In the last stage of live migration or memory slot removal, the backup bitmap needs to be synchronized when it has been enabled. Signed-off-by: Gavin Shan Reviewed-by: Peter Xu Tested-by: Zhenyu Zhang --- accel/kvm/kvm-all.c | 11 +++++++++++ include/sysemu/kvm_int.h | 1 + 2 files changed, 12 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 870abad826..c3aaabf304 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1361,6 +1361,10 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, */ if (kvm_state->kvm_dirty_ring_size) { kvm_dirty_ring_reap_locked(kvm_state, NULL); + if (kvm_state->kvm_dirty_ring_with_bitmap) { + kvm_slot_sync_dirty_pages(mem); + kvm_slot_get_dirty_log(kvm_state, mem); + } } else { kvm_slot_get_dirty_log(kvm_state, mem); } @@ -1582,6 +1586,12 @@ static void kvm_log_sync_global(MemoryListener *l, b= ool last_stage) mem =3D &kml->slots[i]; if (mem->memory_size && mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { kvm_slot_sync_dirty_pages(mem); + + if (s->kvm_dirty_ring_with_bitmap && last_stage && + kvm_slot_get_dirty_log(s, mem)) { + kvm_slot_sync_dirty_pages(mem); + } + /* * This is not needed by KVM_GET_DIRTY_LOG because the * ioctl will unconditionally overwrite the whole region. @@ -3710,6 +3720,7 @@ static void kvm_accel_instance_init(Object *obj) s->kernel_irqchip_split =3D ON_OFF_AUTO_AUTO; /* KVM dirty ring is by default off */ s->kvm_dirty_ring_size =3D 0; + s->kvm_dirty_ring_with_bitmap =3D false; s->notify_vmexit =3D NOTIFY_VMEXIT_OPTION_RUN; s->notify_window =3D 0; s->xen_version =3D 0; diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index a641c974ea..511b42bde5 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -115,6 +115,7 @@ struct KVMState } *as; uint64_t kvm_dirty_ring_bytes; /* Size of the per-vcpu dirty ring */ uint32_t kvm_dirty_ring_size; /* Number of dirty GFNs per ring */ + bool kvm_dirty_ring_with_bitmap; struct KVMDirtyRingReaper reaper; NotifyVmexitOption notify_vmexit; uint32_t notify_window; --=20 2.23.0 From nobody Tue May 14 15:26:24 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1683598959; cv=none; d=zohomail.com; s=zohoarc; b=KDGIJC+yWF3rcupm9ToxvOvxNSaJJxZhcScznyKGw1EatrwTv5PqnlslgYO2TZamA+8vWIpA8wjYcFV3+wlpPiAfRUYmHZgy2W2QVOwSqG2FfuHV98UEE0jwVAHlhm1J6/tL2ejNOu2vIIhXu1gnYHh1kJwt3C4slowZoLMs4tM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1683598959; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dLq5iqfZvdwlN8H0aKrQpli7JcXdiwyOEHKYl3gEayQ=; b=FuAPkVmt6guv5Uem+ONnaCoMieh86X8BNBxB9n3oW37IcGllqIsImJyS9CbP/vIuo7H/RBuFk5Z++R/YhBIT24p2NVJj/TujDEtAEqvLtqTEOOvy3erbw4RHxaxUcNnkMBERcPfYXoui9WXyKA8H2T/mGr+tNTYXC/hXhEp4xyc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1683598959548245.60480733245413; Mon, 8 May 2023 19:22:39 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pwCzM-0004GA-66; Mon, 08 May 2023 22:22:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwCzK-0004FI-UY for qemu-devel@nongnu.org; Mon, 08 May 2023 22:22:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwCzJ-0001xm-1c for qemu-devel@nongnu.org; Mon, 08 May 2023 22:22:02 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-31-1Toaw5ZaPrSis9Tq_jJA5A-1; Mon, 08 May 2023 22:21:56 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 753F91C05129; Tue, 9 May 2023 02:21:56 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-118.bne.redhat.com [10.64.54.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9E8E6492B00; Tue, 9 May 2023 02:21:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683598920; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dLq5iqfZvdwlN8H0aKrQpli7JcXdiwyOEHKYl3gEayQ=; b=Nf2LZJyWMjMUXaWZfoo02Pud84BrSKghG+sEkRwZ2eypNtZxyZP23IE//BMiluYBLodNuB hUZ6kBEYjfi7FlkKD494a1hBayN+9gowGKnm5b22/boQIv7SkOxLZFWgBCTwuE19FT8y2H BTWrSC/EpjPd/H8Iid+zq4YGSkLgRC0= X-MC-Unique: 1Toaw5ZaPrSis9Tq_jJA5A-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, pbonzini@redhat.com, peter.maydell@linaro.org, peterx@redhat.com, david@redhat.com, philmd@linaro.org, mst@redhat.com, cohuck@redhat.com, quintela@redhat.com, dgilbert@redhat.com, maz@kernel.org, zhenyzha@redhat.com, shan.gavin@gmail.com Subject: [PATCH v3 3/4] kvm: Add helper kvm_dirty_ring_init() Date: Tue, 9 May 2023 12:21:21 +1000 Message-Id: <20230509022122.20888-4-gshan@redhat.com> In-Reply-To: <20230509022122.20888-1-gshan@redhat.com> References: <20230509022122.20888-1-gshan@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1683598961161100003 Content-Type: text/plain; charset="utf-8" Due to multiple capabilities associated with the dirty ring for different architectures: KVM_CAP_DIRTY_{LOG_RING, LOG_RING_ACQ_REL} for x86 and arm64 separately. There will be more to be done in order to support the dirty ring for arm64. Lets add helper kvm_dirty_ring_init() to enable the dirty ring. With this, the code looks a bit clean. No functional change intended. Signed-off-by: Gavin Shan Reviewed-by: Peter Xu Tested-by: Zhenyu Zhang --- accel/kvm/kvm-all.c | 76 ++++++++++++++++++++++++++++----------------- 1 file changed, 47 insertions(+), 29 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index c3aaabf304..5d0de9d0a8 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1462,6 +1462,50 @@ static int kvm_dirty_ring_reaper_init(KVMState *s) return 0; } =20 +static int kvm_dirty_ring_init(KVMState *s) +{ + uint32_t ring_size =3D s->kvm_dirty_ring_size; + uint64_t ring_bytes =3D ring_size * sizeof(struct kvm_dirty_gfn); + int ret; + + s->kvm_dirty_ring_size =3D 0; + s->kvm_dirty_ring_bytes =3D 0; + + /* Bail if the dirty ring size isn't specified */ + if (!ring_size) { + return 0; + } + + /* + * Read the max supported pages. Fall back to dirty logging mode + * if the dirty ring isn't supported. + */ + ret =3D kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING); + if (ret <=3D 0) { + warn_report("KVM dirty ring not available, using bitmap method"); + return 0; + } + + if (ring_bytes > ret) { + error_report("KVM dirty ring size %" PRIu32 " too big " + "(maximum is %ld). Please use a smaller value.", + ring_size, (long)ret / sizeof(struct kvm_dirty_gfn)); + return -EINVAL; + } + + ret =3D kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING, 0, ring_bytes); + if (ret) { + error_report("Enabling of KVM dirty ring failed: %s. " + "Suggested minimum value is 1024.", strerror(-ret)); + return -EIO; + } + + s->kvm_dirty_ring_size =3D ring_size; + s->kvm_dirty_ring_bytes =3D ring_bytes; + + return 0; +} + static void kvm_region_add(MemoryListener *listener, MemoryRegionSection *section) { @@ -2531,35 +2575,9 @@ static int kvm_init(MachineState *ms) * Enable KVM dirty ring if supported, otherwise fall back to * dirty logging mode */ - if (s->kvm_dirty_ring_size > 0) { - uint64_t ring_bytes; - - ring_bytes =3D s->kvm_dirty_ring_size * sizeof(struct kvm_dirty_gf= n); - - /* Read the max supported pages */ - ret =3D kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING); - if (ret > 0) { - if (ring_bytes > ret) { - error_report("KVM dirty ring size %" PRIu32 " too big " - "(maximum is %ld). Please use a smaller valu= e.", - s->kvm_dirty_ring_size, - (long)ret / sizeof(struct kvm_dirty_gfn)); - ret =3D -EINVAL; - goto err; - } - - ret =3D kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING, 0, ring_b= ytes); - if (ret) { - error_report("Enabling of KVM dirty ring failed: %s. " - "Suggested minimum value is 1024.", strerror(= -ret)); - goto err; - } - - s->kvm_dirty_ring_bytes =3D ring_bytes; - } else { - warn_report("KVM dirty ring not available, using bitmap metho= d"); - s->kvm_dirty_ring_size =3D 0; - } + ret =3D kvm_dirty_ring_init(s); + if (ret < 0) { + goto err; } =20 /* --=20 2.23.0 From nobody Tue May 14 15:26:24 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1683598996; cv=none; d=zohomail.com; s=zohoarc; b=H9En0AzCObjhMsjVCP9YlwDQT83Gc4y8HHFNi3W9Fi6JSXWlX3TSP2bIF59rR1VOAeI8ljLh3nZRbtj+OagVnBT3TdqQWL+gkda0B7Zf12vYBPIjY7A+7bUegXVSZhXBZ7/Csb1KpBF/amZHeKsQ9DylIdqCpvu082oxERmoMgo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1683598996; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KWghLOz3HpWxoe11c4O6AK1yng47haWuN7Nrkqj0J7g=; b=SnK3d30H1kLUqXQeDpBP1eaMrhGzTUU3UT0r1bUP+SB6aGmO7wAHcgT6oTgks7MGz/+cxOFSLeELKhpyYG64lndNuN4USefdJhuaBbtLfwkO1a9xCvqrXZmWtAhWOrxO+pkFIDtdFM3gobiWTctpiO94nr0H6Lb4ModuKrBQ7nc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1683598996153793.7364049629414; Mon, 8 May 2023 19:23:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pwCzW-0004Ji-1D; Mon, 08 May 2023 22:22:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwCzT-0004JH-AC for qemu-devel@nongnu.org; Mon, 08 May 2023 22:22:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwCzR-0001zT-Sl for qemu-devel@nongnu.org; Mon, 08 May 2023 22:22:11 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-258-i6s3r3AgO8-NiEpbQGVeSg-1; Mon, 08 May 2023 22:22:03 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C4432185A790; Tue, 9 May 2023 02:22:01 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-118.bne.redhat.com [10.64.54.118]) by smtp.corp.redhat.com (Postfix) with ESMTP id F1320492B00; Tue, 9 May 2023 02:21:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683598929; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KWghLOz3HpWxoe11c4O6AK1yng47haWuN7Nrkqj0J7g=; b=U9/un7XGra23ts0R00405R5/2Ti17fAPWxp7eK8OFkq8t990gWg62RWEsYYNGp3x71eZrw LR1tnC7ZagDDtjzWZVdC3fdMC181RzwadDXi838PzHPPwCQHYkA9B1e45h9AKo/TOyd7KJ eaUfpypLcRuwJQBYsUjDJVhw+FZ7v8A= X-MC-Unique: i6s3r3AgO8-NiEpbQGVeSg-1 From: Gavin Shan To: qemu-arm@nongnu.org Cc: qemu-devel@nongnu.org, pbonzini@redhat.com, peter.maydell@linaro.org, peterx@redhat.com, david@redhat.com, philmd@linaro.org, mst@redhat.com, cohuck@redhat.com, quintela@redhat.com, dgilbert@redhat.com, maz@kernel.org, zhenyzha@redhat.com, shan.gavin@gmail.com Subject: [PATCH v3 4/4] kvm: Enable dirty ring for arm64 Date: Tue, 9 May 2023 12:21:22 +1000 Message-Id: <20230509022122.20888-5-gshan@redhat.com> In-Reply-To: <20230509022122.20888-1-gshan@redhat.com> References: <20230509022122.20888-1-gshan@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1683598996431100001 Content-Type: text/plain; charset="utf-8" arm64 has different capability from x86 to enable the dirty ring, which is KVM_CAP_DIRTY_LOG_RING_ACQ_REL. Besides, arm64 also needs the backup bitmap extension (KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP) when 'kvm-arm-gicv3' or 'arm-its-kvm' device is enabled. Here the extension is always enabled and the unnecessary overhead to do the last stage of dirty log synchronizat= ion when those two devices aren't used is introduced, but the overhead should be very small and acceptable. The benefit is cover future cases where those two devices are used without modifying the code. Signed-off-by: Gavin Shan Reviewed-by: Juan Quintela Tested-by: Zhenyu Zhang Reviewed-by: Peter Xu --- accel/kvm/kvm-all.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 5d0de9d0a8..7679f397ae 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1466,6 +1466,7 @@ static int kvm_dirty_ring_init(KVMState *s) { uint32_t ring_size =3D s->kvm_dirty_ring_size; uint64_t ring_bytes =3D ring_size * sizeof(struct kvm_dirty_gfn); + unsigned int capability =3D KVM_CAP_DIRTY_LOG_RING; int ret; =20 s->kvm_dirty_ring_size =3D 0; @@ -1480,7 +1481,12 @@ static int kvm_dirty_ring_init(KVMState *s) * Read the max supported pages. Fall back to dirty logging mode * if the dirty ring isn't supported. */ - ret =3D kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING); + ret =3D kvm_vm_check_extension(s, capability); + if (ret <=3D 0) { + capability =3D KVM_CAP_DIRTY_LOG_RING_ACQ_REL; + ret =3D kvm_vm_check_extension(s, capability); + } + if (ret <=3D 0) { warn_report("KVM dirty ring not available, using bitmap method"); return 0; @@ -1493,13 +1499,26 @@ static int kvm_dirty_ring_init(KVMState *s) return -EINVAL; } =20 - ret =3D kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING, 0, ring_bytes); + ret =3D kvm_vm_enable_cap(s, capability, 0, ring_bytes); if (ret) { error_report("Enabling of KVM dirty ring failed: %s. " "Suggested minimum value is 1024.", strerror(-ret)); return -EIO; } =20 + /* Enable the backup bitmap if it is supported */ + ret =3D kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP); + if (ret > 0) { + ret =3D kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP, 0= ); + if (ret) { + error_report("Enabling of KVM dirty ring's backup bitmap faile= d: " + "%s. ", strerror(-ret)); + return -EIO; + } + + s->kvm_dirty_ring_with_bitmap =3D true; + } + s->kvm_dirty_ring_size =3D ring_size; s->kvm_dirty_ring_bytes =3D ring_bytes; =20 --=20 2.23.0