From nobody Mon Feb 9 13:46:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1610350590; cv=none; d=zohomail.com; s=zohoarc; b=EPB9OXNWBsjYtB83K6TPMPmr2yXOXxPGnDFuQjS9tL5s5InVkRpyw9rcNuT4T7IR+LmqKugQ47WlyM1a+NY7IrWNpUwFJhOSKNI5AwQ0/R+wRWDpYXBp9Tzl2CEobo+h2tAcnQdBH4WoAt7tIvWJ/nH8SGSewXVELOEMG7M6kFY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1610350590; h=Content-Type:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=PkP8r5TaCIqpy7JThV0Png2eo7C+MtaxTrv4ORO4zp8=; b=eDMbXPjEpbrsqxqczeI8+cz76ZYcxlG00f3CcWhJGYtMVihLmHgWPvwTfXJcf8WvK8xu7rB/gq20z0Ve4BAU3BhrZdihKwBaSJUxs4RNPc3TwHXj5YGU2T1CDY2vV0t7MgSff6yRR3I0Vhf1x/yv/D34qCBHyRapTciXWJ9bWTk= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1610350590428840.9707282397261; Sun, 10 Jan 2021 23:36:30 -0800 (PST) Received: from localhost ([::1]:35892 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kyrkb-0006Kp-8N for importer@patchew.org; Mon, 11 Jan 2021 02:36:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:51196) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kyrjF-0005gM-Ae; Mon, 11 Jan 2021 02:35:05 -0500 Received: from szxga07-in.huawei.com ([45.249.212.35]:2631) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kyrjB-0005Kf-S3; Mon, 11 Jan 2021 02:35:04 -0500 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.58]) by szxga07-in.huawei.com (SkyGuard) with ESMTP id 4DDlnS18f9z7Sdj; Mon, 11 Jan 2021 15:33:48 +0800 (CST) Received: from DESKTOP-5IS4806.china.huawei.com (10.174.184.42) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.498.0; Mon, 11 Jan 2021 15:34:40 +0800 From: Keqian Zhu To: Kirti Wankhede , Alex Williamson , , Subject: [PATCH] vfio/migrate: Move switch of dirty tracking into vfio_memory_listener Date: Mon, 11 Jan 2021 15:34:39 +0800 Message-ID: <20210111073439.20236-1-zhukeqian1@huawei.com> X-Mailer: git-send-email 2.8.4.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.184.42] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.35; envelope-from=zhukeqian1@huawei.com; helo=szxga07-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Andrew Jones , Eduardo Habkost , wanghaibin.wang@huawei.com, jiangkunkun@huawei.com, "Dr . David Alan Gilbert" , Peter Xu , Stefan Hajnoczi , Igor Mammedov , Zenghui Yu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For now the switch of vfio dirty page tracking is integrated into the vfio_save_handler, it causes some problems [1]. The object of dirty tracking is guest memory, but the object of the vfio_save_handler is device state. This mixed logic produces unnecessary coupling and conflicts: 1. Coupling: Their saving granule is different (perVM vs perDevice). vfio will enable dirty_page_tracking for each devices, actually once is enough. 2. Conflicts: The ram_save_setup() traverses all memory_listeners to execute their log_start() and log_sync() hooks to get the first round dirty bitmap, which is used by the bulk stage of ram saving. However, it can't get dirty bitmap from vfio, as @savevm_ram_handlers is registered before @vfio_save_handler. Move the switch of vfio dirty_page_tracking into vfio_memory_listener can solve above problems. Besides, Do not require devices in SAVING state for vfio_sync_dirty_bitmap(). [1] https://www.spinics.net/lists/kvm/msg229967.html Reported-by: Zenghui Yu Signed-off-by: Keqian Zhu --- hw/vfio/common.c | 53 +++++++++++++++++++++++++++++++++++++-------- hw/vfio/migration.c | 35 ------------------------------ 2 files changed, 44 insertions(+), 44 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 6ff1daa763..9128cd7ee1 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -311,7 +311,7 @@ bool vfio_mig_active(void) return true; } =20 -static bool vfio_devices_all_saving(VFIOContainer *container) +static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) { VFIOGroup *group; VFIODevice *vbasedev; @@ -329,13 +329,8 @@ static bool vfio_devices_all_saving(VFIOContainer *con= tainer) return false; } =20 - if (migration->device_state & VFIO_DEVICE_STATE_SAVING) { - if ((vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_= AUTO_OFF) - && (migration->device_state & VFIO_DEVICE_STATE_RUNNIN= G)) { - return false; - } - continue; - } else { + if ((vbasedev->pre_copy_dirty_page_tracking =3D=3D ON_OFF_AUTO= _OFF) + && (migration->device_state & VFIO_DEVICE_STATE_RUNNING)) { return false; } } @@ -987,6 +982,44 @@ static void vfio_listener_region_del(MemoryListener *l= istener, } } =20 +static void vfio_set_dirty_page_tracking(VFIOContainer *container, bool st= art) +{ + int ret; + struct vfio_iommu_type1_dirty_bitmap dirty =3D { + .argsz =3D sizeof(dirty), + }; + + if (start) { + dirty.flags =3D VFIO_IOMMU_DIRTY_PAGES_FLAG_START; + } else { + dirty.flags =3D VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP; + } + + ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, &dirty); + if (ret) { + error_report("Failed to set dirty tracking flag 0x%x errno: %d", + dirty.flags, errno); + } +} + +static void vfio_listener_log_start(MemoryListener *listener, + MemoryRegionSection *section, + int old, int new) +{ + VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + + vfio_set_dirty_page_tracking(container, true); +} + +static void vfio_listener_log_stop(MemoryListener *listener, + MemoryRegionSection *section, + int old, int new) +{ + VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + + vfio_set_dirty_page_tracking(container, false); +} + static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, uint64_t size, ram_addr_t ram_addr) { @@ -1128,7 +1161,7 @@ static void vfio_listerner_log_sync(MemoryListener *l= istener, return; } =20 - if (vfio_devices_all_saving(container)) { + if (vfio_devices_all_dirty_tracking(container)) { vfio_sync_dirty_bitmap(container, section); } } @@ -1136,6 +1169,8 @@ static void vfio_listerner_log_sync(MemoryListener *l= istener, static const MemoryListener vfio_memory_listener =3D { .region_add =3D vfio_listener_region_add, .region_del =3D vfio_listener_region_del, + .log_start =3D vfio_listener_log_start, + .log_stop =3D vfio_listener_log_stop, .log_sync =3D vfio_listerner_log_sync, }; =20 diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 00daa50ed8..c0f646823a 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -395,40 +395,10 @@ static int vfio_load_device_config_state(QEMUFile *f,= void *opaque) return qemu_file_get_error(f); } =20 -static int vfio_set_dirty_page_tracking(VFIODevice *vbasedev, bool start) -{ - int ret; - VFIOMigration *migration =3D vbasedev->migration; - VFIOContainer *container =3D vbasedev->group->container; - struct vfio_iommu_type1_dirty_bitmap dirty =3D { - .argsz =3D sizeof(dirty), - }; - - if (start) { - if (migration->device_state & VFIO_DEVICE_STATE_SAVING) { - dirty.flags =3D VFIO_IOMMU_DIRTY_PAGES_FLAG_START; - } else { - return -EINVAL; - } - } else { - dirty.flags =3D VFIO_IOMMU_DIRTY_PAGES_FLAG_STOP; - } - - ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, &dirty); - if (ret) { - error_report("Failed to set dirty tracking flag 0x%x errno: %d", - dirty.flags, errno); - return -errno; - } - return ret; -} - static void vfio_migration_cleanup(VFIODevice *vbasedev) { VFIOMigration *migration =3D vbasedev->migration; =20 - vfio_set_dirty_page_tracking(vbasedev, false); - if (migration->region.mmaps) { vfio_region_unmap(&migration->region); } @@ -469,11 +439,6 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) return ret; } =20 - ret =3D vfio_set_dirty_page_tracking(vbasedev, true); - if (ret) { - return ret; - } - qemu_put_be64(f, VFIO_MIG_FLAG_END_OF_STATE); =20 ret =3D qemu_file_get_error(f); --=20 2.19.1