From nobody Sat May 18 19:24:39 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1670863866; cv=none; d=zohomail.com; s=zohoarc; b=VYCeBLXdRqSZvKOjXbA3c3IXwJBf05GTW6s0yKd4huLINpvCUgksMjiLbmWD7lEYHbxAgc9mTViRYMcGbZIJzw03tiqCYf5SzLrVje4nt+opNruBEIhYogaKMCLiZL8Ke1n58IuhAtNVnYmxZNnUYmwTHEaOBXYAowilfbXlivs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1670863866; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0u/ggCD2dpzlHztgUVICOjgPHyTnEFx12/17GnxGqa0=; b=hijE9NgaCegfEuFAj7Ur1lc38rAV33eA1mJd8dY3S16pbrYUkW4oThNbM31y3s7otq8ViGGXmaUHG51l8PRda/P8UeiNh0S1I0sLvNGECe+PWlgQo3+S+7KPJjjCDZ9Qqfaxg5U9Bwz/ot1dUDyQ/20HsE2Bccd8QInsDe3ewIg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1670863866234585.6214412488235; Mon, 12 Dec 2022 08:51:06 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p4m0k-0002F2-0W; Mon, 12 Dec 2022 11:50:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p4m0V-00027P-Ry for qemu-devel@nongnu.org; Mon, 12 Dec 2022 11:50:24 -0500 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p4m0T-0002y7-UI for qemu-devel@nongnu.org; Mon, 12 Dec 2022 11:50:23 -0500 Received: by mail-pj1-x1036.google.com with SMTP id u5so338307pjy.5 for ; Mon, 12 Dec 2022 08:50:17 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id x21-20020a170902ea9500b0018f69009f3esm3012125plb.284.2022.12.12.08.50.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 08:50:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0u/ggCD2dpzlHztgUVICOjgPHyTnEFx12/17GnxGqa0=; b=ch+4PeWlXRtcybXSgMcUtBhLUcnMOGnIb7H6NisLQXo9Dm1+7U7upvRGNYX8NF1f9p yUYJQoQVz/5aaczI6BUPfoSPe+R1GCo8CwmbhUCJWVR4eDivkkuxvoq7lNTkABuyMUCo cnwHXWUZZQ5XAQKC3fkOQvnifEQKjX2HSUHMM6qOMDVGUp3X85f1APVe8gfhSSfFekaq tS9ZIMGeSUlH1CIj2hXw115Zw/lFc9Hxco2mXmVLfJz+hGKqgUTOzcDlf6TowEZ5l2Co PlV6yfzBCIowJ2fLL+sVUf8MemcPGl2c6BGfeBkcU7XXBOFrn00NErbysI7UrQgf0VGd 7BRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0u/ggCD2dpzlHztgUVICOjgPHyTnEFx12/17GnxGqa0=; b=ToYDU85PeYCGFpBD/XY4CdBxM6icxvr3IGosiL7MIPWz2sq6t9hC6S+pwQNa7OJsGC kI1zO8JTTIZhsV2odEUN0cSOIYV91JyKazdnpeXeC3As80QKoptlt/fUDwndxhV5HMjC f6VcJIWDmaPfP3Tm7VlvfRFBRzzVqWhJwiLXxJBgZK2bRvMzFG/rTYFsGIRFmhykS+60 qUq3kX8EEpcWwT+4h9oyo/FaMEHh4LNHALy1Cw4EAfOn7V5FsdS8Oi9UJiie4UdLjqTQ hNDw/5+1Ak1jkvZyO6V0O9Cqmk9fK4qchGsdhFC+JhsiumJFEktFzV1wKdkNcf9HwwOw v3/Q== X-Gm-Message-State: ANoB5pmhqWEnZJv27WE424VKwQdKYIO1CLa7wMJJCaEhxavsKpENdQnL ZQkPaHkupnfQEmxD/4i4KQ+eoSoC2BL6CRvi X-Google-Smtp-Source: AA0mqf4z9maOoX5LlcSApKYQDLtCee9WRKcD7ieQ4yxFf9hQY5Mqmo54AWBVlXyHbxS0e2QTQgwQ/w== X-Received: by 2002:a17:902:c3c3:b0:18d:6138:e4f6 with SMTP id j3-20020a170902c3c300b0018d6138e4f6mr13467483plj.29.1670863816390; Mon, 12 Dec 2022 08:50:16 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, peterx@redhat.com, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v2 1/3] memory: add depth assert in address_space_to_flatview Date: Tue, 13 Dec 2022 00:49:40 +0800 Message-Id: <20221212164942.3614611-2-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221212164942.3614611-1-xuchuangxclwt@bytedance.com> References: <20221212164942.3614611-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x1036.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance-com.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1670863867776100003 Content-Type: text/plain; charset="utf-8" Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 +++++++++ softmmu/memory.c | 1 - 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index 91f8a2395a..b43cd46084 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1069,8 +1069,17 @@ struct FlatView { MemoryRegion *root; }; =20 +static unsigned memory_region_transaction_depth; + static inline FlatView *address_space_to_flatview(AddressSpace *as) { + /* + * Before using any flatview, sanity check we're not during a memory + * region transaction or the map can be invalid. Note that this can + * also be called during commit phase of memory transaction, but that + * should also only happen when the depth decreases to 0 first. + */ + assert(memory_region_transaction_depth =3D=3D 0); return qatomic_rcu_read(&as->current_map); } =20 diff --git a/softmmu/memory.c b/softmmu/memory.c index bc0be3f62c..f177c40cd8 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -37,7 +37,6 @@ =20 //#define DEBUG_UNASSIGNED =20 -static unsigned memory_region_transaction_depth; static bool memory_region_update_pending; static bool ioeventfd_update_pending; unsigned int global_dirty_tracking; --=20 2.20.1 From nobody Sat May 18 19:24:39 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1670863899; cv=none; d=zohomail.com; s=zohoarc; b=m8vOz2T+WmWBBThNNjqZxkJ5G3R4WUvZrnx6aRHPs0iL3TDhOKiETJIb3xciH0kt4PBLLJuXEBRGSkjnrB+Lue/7jSzuABFKURmz7veUvw5CTBMalrOUxmWE7ymsvvxXRmIQWKEvBM/EJz3udv/DnbIgHeEG/mH6yKG5s/RV+CA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1670863899; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DHygLy2vPOJV/Fky9RB+7iOYpBea60FaHFYeg5SQFYY=; b=PDGSxk07M6DgOwXRq+RxXA6r3BgE9ASR5aJ92SkEzzVLCLp+7Ast+UsFFwFxGOPCxct0zVfBR/j3Y4T8LoK2rnGsHbqU1d5AZI90hD+Xdby7nXlQnwwrkH8rBqNwpxnjbbFJ+ERzomRA3JQeaXLj73kqYz7bcpZNGdKFXZrhhI8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1670863899420227.4473645035962; Mon, 12 Dec 2022 08:51:39 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p4m0l-0002HU-9E; Mon, 12 Dec 2022 11:50:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p4m0c-0002Cu-Rf for qemu-devel@nongnu.org; Mon, 12 Dec 2022 11:50:34 -0500 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p4m0Y-00032C-4y for qemu-devel@nongnu.org; Mon, 12 Dec 2022 11:50:27 -0500 Received: by mail-pl1-x629.google.com with SMTP id g10so12651328plo.11 for ; Mon, 12 Dec 2022 08:50:25 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id x21-20020a170902ea9500b0018f69009f3esm3012125plb.284.2022.12.12.08.50.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 08:50:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DHygLy2vPOJV/Fky9RB+7iOYpBea60FaHFYeg5SQFYY=; b=OJjcFdwH7d262/CzFZShlW9lXF/StOZiII2rtlfXMNcBlSZBrZaKNloahfYxS6JPR5 VarIZ52qRG4K1iJAG6WgsSf/2gzPv6GyXOW4ApIWDBQdSAt2zmJK6GwVz3NhZ1rQ5WTa 3yndoCO/E23S7oITbxEeuo69xUyCeGXQCaeUiMtV7XZXtaM6jhZd36KqLoEPu7/T1ON0 4WMvHoqou0nLZt4mS5lixHKVCkuXMp4IksN3USuGLHvIDCJ2T2qUGb9yn12aqP4/D9h0 mALbnkD82y6wMfJrwKhyOu3KzoPefXqDzknVJdEIo+gnisiMhtbstHlJfBah2GAvN6vD VIKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DHygLy2vPOJV/Fky9RB+7iOYpBea60FaHFYeg5SQFYY=; b=ihUTqYABfX5qOKTVFYARdHsIilcRmKog4/Vv4tyyNJP4fjFwEZgEWHD33zblpPJFUq qGWihvE+ZJw+DycWoZQumCVJia4M+cMH70kLM9GfyU8vQbh9pLEz0c/02x3SGEl758yN P17Ac+PlnEdHF+b46xrrmIfOxC449EGl5QEsTM1Wi9CuVj6yP1I077vub6oXoscCJ8yI eWhcJG6mlPq1O7g1v0n5mQeg+Y4t4Q0WuRVJxUuDi4CLNEbIfuxubl2zWyP3mv4aXo0N eZEmRzJ8e8V1WiYR+xu3rYRq39yir5BHIcChoE3JUSmKt4WemN/2TEy7TmYkTTqawopL FPXQ== X-Gm-Message-State: ANoB5pla10l9qz1MrzuTFUN/AfsHH41b7KmqalW01+DjZvMEYoQMFnG0 o3Kpmt4RodTrpT1K7PUCxE7UjU0LchU8vUcj X-Google-Smtp-Source: AA0mqf7E2gghMP4NSRj6ptdVIqtHGsGFp7J//xbb07TCmGgs8/RT9yDWrYbXrcQkYoV6X2dasRyAkQ== X-Received: by 2002:a17:902:edc5:b0:187:3a79:6038 with SMTP id q5-20020a170902edc500b001873a796038mr18934469plk.18.1670863823071; Mon, 12 Dec 2022 08:50:23 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, peterx@redhat.com, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v2 2/3] virtio: support delay of checks in virtio_load() Date: Tue, 13 Dec 2022 00:49:41 +0800 Message-Id: <20221212164942.3614611-3-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221212164942.3614611-1-xuchuangxclwt@bytedance.com> References: <20221212164942.3614611-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x629.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance-com.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1670863900045100001 Content-Type: text/plain; charset="utf-8" Delay checks in virtio_load() to avoid possible address_space_to_flatview()= call during memory region's begin/commit. Signed-off-by: Chuang Xu --- hw/virtio/virtio.c | 33 ++++++++++++++++++++++----------- include/sysemu/sysemu.h | 1 + softmmu/globals.c | 3 +++ 3 files changed, 26 insertions(+), 11 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index eb6347ab5d..3e3fa2a89d 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -33,6 +33,7 @@ #include "hw/virtio/virtio-access.h" #include "sysemu/dma.h" #include "sysemu/runstate.h" +#include "sysemu/sysemu.h" #include "standard-headers/linux/virtio_ids.h" #include "standard-headers/linux/vhost_types.h" #include "standard-headers/linux/virtio_blk.h" @@ -3642,8 +3643,20 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f, int= version_id) vdev->start_on_kick =3D true; } =20 + if (vdc->post_load) { + ret =3D vdc->post_load(vdev); + if (ret) { + return ret; + } + } + + return 0; +} + +static void virtio_load_check_delay(VirtIODevice *vdev) +{ RCU_READ_LOCK_GUARD(); - for (i =3D 0; i < num; i++) { + for (int i =3D 0; i < VIRTIO_QUEUE_MAX; i++) { if (vdev->vq[i].vring.desc) { uint16_t nheads; =20 @@ -3696,19 +3709,12 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f, in= t version_id) i, vdev->vq[i].vring.num, vdev->vq[i].last_avail_idx, vdev->vq[i].used_idx); - return -1; + abort(); } } } =20 - if (vdc->post_load) { - ret =3D vdc->post_load(vdev); - if (ret) { - return ret; - } - } - - return 0; + return; } =20 void virtio_cleanup(VirtIODevice *vdev) @@ -4158,7 +4164,12 @@ static void virtio_memory_listener_commit(MemoryList= ener *listener) if (vdev->vq[i].vring.num =3D=3D 0) { break; } - virtio_init_region_cache(vdev, i); + + if (migration_enable_load_check_delay) { + virtio_load_check_delay(vdev); + } else { + virtio_init_region_cache(vdev, i); + } } } =20 diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h index 6a7a31e64d..0523091445 100644 --- a/include/sysemu/sysemu.h +++ b/include/sysemu/sysemu.h @@ -12,6 +12,7 @@ extern int only_migratable; extern const char *qemu_name; extern QemuUUID qemu_uuid; extern bool qemu_uuid_set; +extern bool migration_enable_load_check_delay; =20 const char *qemu_get_vm_name(void); =20 diff --git a/softmmu/globals.c b/softmmu/globals.c index 527edbefdd..1bd8f6c978 100644 --- a/softmmu/globals.c +++ b/softmmu/globals.c @@ -65,3 +65,6 @@ bool qemu_uuid_set; uint32_t xen_domid; enum xen_mode xen_mode =3D XEN_EMULATE; bool xen_domid_restrict; + +bool migration_enable_load_check_delay; + --=20 2.20.1 From nobody Sat May 18 19:24:39 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1670863874; cv=none; d=zohomail.com; s=zohoarc; b=IOsMnFaHPJjP8HLxzmUfsUxqpSu233G0pp+vptISRXSAXhcgmapaV0XkCvH2NhLSfrpgM2oXx7a7oh4tXO+EvxKyzmbSOd1A8FLlLsSPF363fsAnmyR+9lSm3ZYIHSR5IK8qnjb0V/w3iKlGpxr5xi2FURPep43zHE9Yw4bXA1M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1670863874; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GJg5zSq9kdDfSLar49onFW0ftLsHi0KtswppGKlIvwI=; b=fHouGeIfjYdIM4+YrCty+bMZvkVqHVPvXNo5/wiCwcFpcpLx5K3RGUOX0rNw6hhS2qCrryXITKxoP4EAPmfsr4msrh+5uaFMcQ+jOUDW8n4IaGrZAq9JRWyTvyKumhmwf7ct6hpGpYEMvYKXpC7xhzOJEu+4n8uIuRKZRUKnHYY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1670863874898898.5033420725656; Mon, 12 Dec 2022 08:51:14 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p4m0n-0002K5-Cf; Mon, 12 Dec 2022 11:51:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p4m0e-0002DE-3k for qemu-devel@nongnu.org; Mon, 12 Dec 2022 11:50:34 -0500 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p4m0c-000380-K7 for qemu-devel@nongnu.org; Mon, 12 Dec 2022 11:50:31 -0500 Received: by mail-pl1-x630.google.com with SMTP id jl24so12659654plb.8 for ; Mon, 12 Dec 2022 08:50:28 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id x21-20020a170902ea9500b0018f69009f3esm3012125plb.284.2022.12.12.08.50.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 08:50:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GJg5zSq9kdDfSLar49onFW0ftLsHi0KtswppGKlIvwI=; b=TDL8Y/CKGjF6zhHvVjWrt44P+M/z9U/ZmV2+yCq3sMXNSzzwZX06uuLrwJhoOymRD9 WOxIssj+1+yp6np8uHYnc9JYKNzSqudKH/Z82fGMaTy3O+g3Argwg9EkUIUSG91D0X1L q1EiPVy4xJE8bZjkze2IlHtqleTSOF+Yq+pD9vDhNdQTYKH/2gzsDKyKW18+WSqtYY5U 3eE11VoOVshl+HtHXZ1kiDMnf0RNYnrjtdVhAqc4Drutj+eA1TZo6cTdQqW47xJW4WXz xTnU2M0FdXi82pKeT7JepRYlAqvvdPZHUe7kX+1PhPzIKHDHCiM38xWhaxnpX900AjIT MXFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GJg5zSq9kdDfSLar49onFW0ftLsHi0KtswppGKlIvwI=; b=ZYhCJiR0rrEOkfvzWAq5Umh1uh4dNvYqlkFwjEEZp7/RaDFAtzSgMlxtrTXAe/5lSa nAVavbfUtCQ6zjFSAzbsLS+y6ZXf4cW//7b3CRXKGRSdM0shw13IWZnNGYDg6Xti3fRT d8Crn/7Djqtparx7DRk3ZHcTu5KV81A3+Ox9rtfwkohXLahElKZAxTSSgYtOdlN0AVtR W+zVjWO8hTgVxOhvRak1SGtwaY4hqbL6saJ8K97gFGGgkXEm4AkmPz8Iar0q7YjaTIAu Gf1lNLn4eMT432bbfDccCc/2JAFteKJLqnq44SAGXC3jUcz0WYxR0+kbepJXWoFF6nut CGqw== X-Gm-Message-State: ANoB5pkPPNZrSRoW/1tfgaqhp2H9mjOb5EoKnBe8t8akF8zK73ZvebZH LRX1rdpzNMO9fcRUfCuuiKeiOvQ1CeIaVT36 X-Google-Smtp-Source: AA0mqf7/sLvMcCKZENMSJTjW6NapWh4CF2xdHQDbqU8cUjfiL5Y9DJ2uE+/33jtg2HCP8GryFSs1+A== X-Received: by 2002:a17:903:240d:b0:185:441f:70b1 with SMTP id e13-20020a170903240d00b00185441f70b1mr17605959plo.54.1670863827708; Mon, 12 Dec 2022 08:50:27 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, peterx@redhat.com, zhouyibo@bytedance.com, Chuang Xu Subject: [RFC v2 3/3] migration: reduce time of loading non-iterable vmstate Date: Tue, 13 Dec 2022 00:49:42 +0800 Message-Id: <20221212164942.3614611-4-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221212164942.3614611-1-xuchuangxclwt@bytedance.com> References: <20221212164942.3614611-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x630.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance-com.20210112.gappssmtp.com) X-ZM-MESSAGEID: 1670863875795100003 Content-Type: text/plain; charset="utf-8" The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the time is spent committing memory region changes repeatedly. This patch packs all the changes to memory region during the period of loading non-iterable vmstate in a single memory transaction. With the increase of devices, this patch will greatly improve the performance. Here are the test results: test vm info: - 32 CPUs 128GB RAM - 8 16-queue vhost-net device - 16 4-queue vhost-user-blk device. time of loading non-iterable vmstate before about 210 ms after about 40 ms Signed-off-by: Chuang Xu --- migration/savevm.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..68a7a99b79 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2617,6 +2617,9 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationInco= mingState *mis) uint8_t section_type; int ret =3D 0; =20 + /* call memory_region_transaction_begin() before loading vmstate */ + memory_region_transaction_begin(); + retry: while (true) { section_type =3D qemu_get_byte(f); @@ -2684,6 +2687,16 @@ out: goto retry; } } + + /* + * call memory_region_transaction_commit() after loading non-iterable + * vmstate, make sure the migration_enable_load_check_delay flag is + * true during commit. + */ + migration_enable_load_check_delay =3D true; + memory_region_transaction_commit(); + migration_enable_load_check_delay =3D false; + return ret; } =20 --=20 2.20.1