From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619695886; cv=none; d=zohomail.com; s=zohoarc; b=kU1iAjfTBOCwf0QB8XIa9Eyf7G8hU5+FNglZJE9WM5FSj7kHijoHzPhbnjfnDYoSZ1OfqWRPsWjcANyXXRn7EkW3b2YcEK9NRoia2VoYXadSdMdZb1zfcn1sZZ9q/wwqqUyUb18Kuo8o04HT2g5adnE5E4XxC1Px4lO2RZgBrV0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619695886; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=C6Z9iLYgXWQ95p6aANNNRUmSBrphlRfsCtYiZznnWR4=; b=B7NH1BFWetqxhljD4sS+/MkGW9r0eWHK9O5vvVmZWbuAgrWX34sxiwYkBZl36MHE2f2mwKpOB2hgODQsodT3E1KuRD8kf4Fk7KkmsBj3CuFYQlgwBRPcIKyvKd2iyJkO2v+VUhLMrOQSk3TEgib9q8wcIwPWCAv4i+Gc4AGRTl4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1619695886643608.093831042804; Thu, 29 Apr 2021 04:31:26 -0700 (PDT) Received: from localhost ([::1]:59182 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lc4tA-0002oJ-T0 for importer@patchew.org; Thu, 29 Apr 2021 07:31:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58470) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4pV-0007hx-D7 for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:27:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:22773) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4pT-0003LK-M5 for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:27:37 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-289-x7_5uFYPNPOejntptz2e3w-1; Thu, 29 Apr 2021 07:27:31 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 67A808042AD; Thu, 29 Apr 2021 11:27:30 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id B3FCE69513; Thu, 29 Apr 2021 11:27:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695655; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C6Z9iLYgXWQ95p6aANNNRUmSBrphlRfsCtYiZznnWR4=; b=AGl5EdeD32Rt29qfmIEEnzvTeE/Z6t41dDLXrEgBEiZ61g0vUYxtZpbS1DxeAu9NSKsHwA /WKn0yPJhYWD5mNOW4tPoEdlWPK4n6C8n6Sv7hH8WTXub7vqT99g+xqb9KyxsgIdB4zTFt LQsqHYxp2VkJ+94nUVIOtYoVbM4wNnU= X-MC-Unique: x7_5uFYPNPOejntptz2e3w-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v5 01/10] util: vfio-helpers: Factor out and fix processing of existing ram blocks Date: Thu, 29 Apr 2021 13:26:59 +0200 Message-Id: <20210429112708.12291-2-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela , Alex Williamson , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Factor it out into common code when a new notifier is registered, just as done with the memory region notifier. This keeps logic about how to process existing ram blocks at a central place. Just like when adding a new ram block, we have to register the max_length. Ram blocks are only "fake resized". All memory (max_length) is mapped. Print the warning from inside qemu_vfio_ram_block_added(). Reviewed-by: Peter Xu Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- hw/core/numa.c | 14 ++++++++++++++ include/exec/cpu-common.h | 1 + softmmu/physmem.c | 5 +++++ util/vfio-helpers.c | 29 ++++++++--------------------- 4 files changed, 28 insertions(+), 21 deletions(-) diff --git a/hw/core/numa.c b/hw/core/numa.c index 68cee65f61..7f08c27a6d 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -803,9 +803,23 @@ void query_numa_node_mem(NumaNodeMem node_mem[], Machi= neState *ms) } } =20 +static int ram_block_notify_add_single(RAMBlock *rb, void *opaque) +{ + const ram_addr_t max_size =3D qemu_ram_get_max_length(rb); + void *host =3D qemu_ram_get_host_addr(rb); + RAMBlockNotifier *notifier =3D opaque; + + if (host) { + notifier->ram_block_added(notifier, host, max_size); + } + return 0; +} + void ram_block_notifier_add(RAMBlockNotifier *n) { QLIST_INSERT_HEAD(&ram_list.ramblock_notifiers, n, next); + /* Notify about all existing ram blocks. */ + qemu_ram_foreach_block(ram_block_notify_add_single, n); } =20 void ram_block_notifier_remove(RAMBlockNotifier *n) diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index 5a0a2d93e0..ccabed4003 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -57,6 +57,7 @@ const char *qemu_ram_get_idstr(RAMBlock *rb); void *qemu_ram_get_host_addr(RAMBlock *rb); ram_addr_t qemu_ram_get_offset(RAMBlock *rb); ram_addr_t qemu_ram_get_used_length(RAMBlock *rb); +ram_addr_t qemu_ram_get_max_length(RAMBlock *rb); bool qemu_ram_is_shared(RAMBlock *rb); bool qemu_ram_is_uf_zeroable(RAMBlock *rb); void qemu_ram_set_uf_zeroable(RAMBlock *rb); diff --git a/softmmu/physmem.c b/softmmu/physmem.c index 85034d9c11..bd2c0dc4ec 100644 --- a/softmmu/physmem.c +++ b/softmmu/physmem.c @@ -1697,6 +1697,11 @@ ram_addr_t qemu_ram_get_used_length(RAMBlock *rb) return rb->used_length; } =20 +ram_addr_t qemu_ram_get_max_length(RAMBlock *rb) +{ + return rb->max_length; +} + bool qemu_ram_is_shared(RAMBlock *rb) { return rb->flags & RAM_SHARED; diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index 97dfa3fd57..92b9565797 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -463,8 +463,14 @@ static void qemu_vfio_ram_block_added(RAMBlockNotifier= *n, void *host, size_t size) { QEMUVFIOState *s =3D container_of(n, QEMUVFIOState, ram_notifier); + int ret; + trace_qemu_vfio_ram_block_added(s, host, size); - qemu_vfio_dma_map(s, host, size, false, NULL); + ret =3D qemu_vfio_dma_map(s, host, size, false, NULL); + if (ret) { + error_report("qemu_vfio_dma_map(%p, %zu) failed: %s", host, size, + strerror(-ret)); + } } =20 static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, @@ -477,33 +483,14 @@ static void qemu_vfio_ram_block_removed(RAMBlockNotif= ier *n, } } =20 -static int qemu_vfio_init_ramblock(RAMBlock *rb, void *opaque) -{ - void *host_addr =3D qemu_ram_get_host_addr(rb); - ram_addr_t length =3D qemu_ram_get_used_length(rb); - int ret; - QEMUVFIOState *s =3D opaque; - - if (!host_addr) { - return 0; - } - ret =3D qemu_vfio_dma_map(s, host_addr, length, false, NULL); - if (ret) { - fprintf(stderr, "qemu_vfio_init_ramblock: failed %p %" PRId64 "\n", - host_addr, (uint64_t)length); - } - return 0; -} - static void qemu_vfio_open_common(QEMUVFIOState *s) { qemu_mutex_init(&s->lock); s->ram_notifier.ram_block_added =3D qemu_vfio_ram_block_added; s->ram_notifier.ram_block_removed =3D qemu_vfio_ram_block_removed; - ram_block_notifier_add(&s->ram_notifier); s->low_water_mark =3D QEMU_VFIO_IOVA_MIN; s->high_water_mark =3D QEMU_VFIO_IOVA_MAX; - qemu_ram_foreach_block(qemu_vfio_init_ramblock, s); + ram_block_notifier_add(&s->ram_notifier); } =20 /** --=20 2.30.2 From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619695694; cv=none; d=zohomail.com; s=zohoarc; b=UUpBXl/M5n7f7qypzBAMpd3eAVReZk3Oy5vKeP6LSTvDfBQNm+BwerjtXy4k4v2vaiWb7riXlymObgs95yanaOjCjF2cCaQAefj1PHF+Gm8Pm8Rar8ZVR46eSIEwd5vySnC5ToQ7018mgyA0B9sMNNHgX0MvxRUI4B4z041VoEc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619695694; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UNce5xZGYwSY6mhN845V4iwjhXo75SIJ2plRmjN8C/4=; b=Prx95ib9JznyakjcriF/Ab103lbBTI8DK/473yvgM/vcuwh57Qzxqcen+YjunQbrjYm5hZ7/YTlR5NiXR/erftU1k6b1Uyap8nxuHWcfJ0c+1+pKBxiGnk4jToPjiESQY2OnJlE3lJf/mBHr8dTMd1LYy/O1RZrmW/q4ilwtoCg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 161969569452450.86757689218132; Thu, 29 Apr 2021 04:28:14 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.119887.226664 (Exim 4.92) (envelope-from ) id 1lc4pm-000734-EX; Thu, 29 Apr 2021 11:27:54 +0000 Received: by outflank-mailman (output) from mailman id 119887.226664; Thu, 29 Apr 2021 11:27:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lc4pm-00072x-Be; Thu, 29 Apr 2021 11:27:54 +0000 Received: by outflank-mailman (input) for mailman id 119887; Thu, 29 Apr 2021 11:27:53 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lc4pl-00072n-RD for xen-devel@lists.xenproject.org; Thu, 29 Apr 2021 11:27:53 +0000 Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id cbbfa78a-5ce5-47ee-9caa-5fa2bdec6dff; Thu, 29 Apr 2021 11:27:52 +0000 (UTC) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-405-5Pvl26hHO4OBYFbRkMiEmQ-1; Thu, 29 Apr 2021 07:27:50 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6DF1A8042A6; Thu, 29 Apr 2021 11:27:48 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id C705F67CC4; Thu, 29 Apr 2021 11:27:30 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cbbfa78a-5ce5-47ee-9caa-5fa2bdec6dff DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695672; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UNce5xZGYwSY6mhN845V4iwjhXo75SIJ2plRmjN8C/4=; b=Bz4fI4OIZeB35QXYouiHWX0Ha/gcpjWhn8KOejU4ODk3AWYXiybUdWHM5QkEqy3LXHJIiV LKPwD9lw4NK4JBNnwQ5xAfHPA7BXxGxxKFJgLWkGkHiqBGX5NuPy3JHU1UBBhEHMiprRrw wFqzk7zEhztABPZ5siNmFmJO4YycitU= X-MC-Unique: 5Pvl26hHO4OBYFbRkMiEmQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Cc: David Hildenbrand , Eduardo Habkost , Marcel Apfelbaum , "Michael S. Tsirkin" , Paolo Bonzini , Richard Henderson , Juan Quintela , "Dr. David Alan Gilbert" , Peter Xu , Alex Williamson , Paul Durrant , xen-devel@lists.xenproject.org, haxm-team@intel.com, Stefano Stabellini , Anthony Perard , Wenchao Wang , Colin Xu Subject: [PATCH v5 02/10] numa: Teach ram block notifiers about resizeable ram blocks Date: Thu, 29 Apr 2021 13:27:00 +0200 Message-Id: <20210429112708.12291-3-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Ram block notifiers are currently not aware of resizes. To properly handle resizes during migration, we want to teach ram block notifiers about resizeable ram. Introduce the basic infrastructure but keep using max_size in the existing notifiers. Supply the max_size when adding and removing ram blocks. Also, notify on resizes. Acked-by: Paul Durrant Reviewed-by: Peter Xu Cc: xen-devel@lists.xenproject.org Cc: haxm-team@intel.com Cc: Paul Durrant Cc: Stefano Stabellini Cc: Anthony Perard Cc: Wenchao Wang Cc: Colin Xu Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost --- hw/core/numa.c | 22 +++++++++++++++++----- hw/i386/xen/xen-mapcache.c | 7 ++++--- include/exec/ramlist.h | 13 +++++++++---- softmmu/physmem.c | 12 ++++++++++-- target/i386/hax/hax-mem.c | 5 +++-- target/i386/sev.c | 18 ++++++++++-------- util/vfio-helpers.c | 16 ++++++++-------- 7 files changed, 61 insertions(+), 32 deletions(-) diff --git a/hw/core/numa.c b/hw/core/numa.c index 7f08c27a6d..921bf86ab4 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -806,11 +806,12 @@ void query_numa_node_mem(NumaNodeMem node_mem[], Mach= ineState *ms) static int ram_block_notify_add_single(RAMBlock *rb, void *opaque) { const ram_addr_t max_size =3D qemu_ram_get_max_length(rb); + const ram_addr_t size =3D qemu_ram_get_used_length(rb); void *host =3D qemu_ram_get_host_addr(rb); RAMBlockNotifier *notifier =3D opaque; =20 if (host) { - notifier->ram_block_added(notifier, host, max_size); + notifier->ram_block_added(notifier, host, size, max_size); } return 0; } @@ -827,20 +828,31 @@ void ram_block_notifier_remove(RAMBlockNotifier *n) QLIST_REMOVE(n, next); } =20 -void ram_block_notify_add(void *host, size_t size) +void ram_block_notify_add(void *host, size_t size, size_t max_size) { RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_added(notifier, host, size); + notifier->ram_block_added(notifier, host, size, max_size); } } =20 -void ram_block_notify_remove(void *host, size_t size) +void ram_block_notify_remove(void *host, size_t size, size_t max_size) { RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_removed(notifier, host, size); + notifier->ram_block_removed(notifier, host, size, max_size); + } +} + +void ram_block_notify_resize(void *host, size_t old_size, size_t new_size) +{ + RAMBlockNotifier *notifier; + + QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { + if (notifier->ram_block_resized) { + notifier->ram_block_resized(notifier, host, old_size, new_size= ); + } } } diff --git a/hw/i386/xen/xen-mapcache.c b/hw/i386/xen/xen-mapcache.c index 5b120ed44b..d6dcea65d1 100644 --- a/hw/i386/xen/xen-mapcache.c +++ b/hw/i386/xen/xen-mapcache.c @@ -169,7 +169,8 @@ static void xen_remap_bucket(MapCacheEntry *entry, =20 if (entry->vaddr_base !=3D NULL) { if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) { - ram_block_notify_remove(entry->vaddr_base, entry->size); + ram_block_notify_remove(entry->vaddr_base, entry->size, + entry->size); } if (munmap(entry->vaddr_base, entry->size) !=3D 0) { perror("unmap fails"); @@ -211,7 +212,7 @@ static void xen_remap_bucket(MapCacheEntry *entry, } =20 if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) { - ram_block_notify_add(vaddr_base, size); + ram_block_notify_add(vaddr_base, size, size); } =20 entry->vaddr_base =3D vaddr_base; @@ -452,7 +453,7 @@ static void xen_invalidate_map_cache_entry_unlocked(uin= t8_t *buffer) } =20 pentry->next =3D entry->next; - ram_block_notify_remove(entry->vaddr_base, entry->size); + ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size); if (munmap(entry->vaddr_base, entry->size) !=3D 0) { perror("unmap fails"); exit(-1); diff --git a/include/exec/ramlist.h b/include/exec/ramlist.h index 26704aa3b0..ece6497ee2 100644 --- a/include/exec/ramlist.h +++ b/include/exec/ramlist.h @@ -65,15 +65,20 @@ void qemu_mutex_lock_ramlist(void); void qemu_mutex_unlock_ramlist(void); =20 struct RAMBlockNotifier { - void (*ram_block_added)(RAMBlockNotifier *n, void *host, size_t size); - void (*ram_block_removed)(RAMBlockNotifier *n, void *host, size_t size= ); + void (*ram_block_added)(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size); + void (*ram_block_removed)(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size); + void (*ram_block_resized)(RAMBlockNotifier *n, void *host, size_t old_= size, + size_t new_size); QLIST_ENTRY(RAMBlockNotifier) next; }; =20 void ram_block_notifier_add(RAMBlockNotifier *n); void ram_block_notifier_remove(RAMBlockNotifier *n); -void ram_block_notify_add(void *host, size_t size); -void ram_block_notify_remove(void *host, size_t size); +void ram_block_notify_add(void *host, size_t size, size_t max_size); +void ram_block_notify_remove(void *host, size_t size, size_t max_size); +void ram_block_notify_resize(void *host, size_t old_size, size_t new_size); =20 void ram_block_dump(Monitor *mon); =20 diff --git a/softmmu/physmem.c b/softmmu/physmem.c index bd2c0dc4ec..d6463f203d 100644 --- a/softmmu/physmem.c +++ b/softmmu/physmem.c @@ -1810,6 +1810,7 @@ static int memory_try_enable_merging(void *addr, size= _t len) */ int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error **errp) { + const ram_addr_t oldsize =3D block->used_length; const ram_addr_t unaligned_size =3D newsize; =20 assert(block); @@ -1846,6 +1847,11 @@ int qemu_ram_resize(RAMBlock *block, ram_addr_t news= ize, Error **errp) return -EINVAL; } =20 + /* Notify before modifying the ram block and touching the bitmaps. */ + if (block->host) { + ram_block_notify_resize(block->host, oldsize, newsize); + } + cpu_physical_memory_clear_dirty_range(block->offset, block->used_lengt= h); block->used_length =3D newsize; cpu_physical_memory_set_dirty_range(block->offset, block->used_length, @@ -2013,7 +2019,8 @@ static void ram_block_add(RAMBlock *new_block, Error = **errp, bool shared) qemu_madvise(new_block->host, new_block->max_length, QEMU_MADV_DONTFORK); } - ram_block_notify_add(new_block->host, new_block->max_length); + ram_block_notify_add(new_block->host, new_block->used_length, + new_block->max_length); } } =20 @@ -2192,7 +2199,8 @@ void qemu_ram_free(RAMBlock *block) } =20 if (block->host) { - ram_block_notify_remove(block->host, block->max_length); + ram_block_notify_remove(block->host, block->used_length, + block->max_length); } =20 qemu_mutex_lock_ramlist(); diff --git a/target/i386/hax/hax-mem.c b/target/i386/hax/hax-mem.c index 35495f5e82..8d44edbffd 100644 --- a/target/i386/hax/hax-mem.c +++ b/target/i386/hax/hax-mem.c @@ -293,7 +293,8 @@ static MemoryListener hax_memory_listener =3D { .priority =3D 10, }; =20 -static void hax_ram_block_added(RAMBlockNotifier *n, void *host, size_t si= ze) +static void hax_ram_block_added(RAMBlockNotifier *n, void *host, size_t si= ze, + size_t max_size) { /* * We must register each RAM block with the HAXM kernel module, or @@ -304,7 +305,7 @@ static void hax_ram_block_added(RAMBlockNotifier *n, vo= id *host, size_t size) * host physical pages for the RAM block as part of this registration * process, hence the name hax_populate_ram(). */ - if (hax_populate_ram((uint64_t)(uintptr_t)host, size) < 0) { + if (hax_populate_ram((uint64_t)(uintptr_t)host, max_size) < 0) { fprintf(stderr, "HAX failed to populate RAM\n"); abort(); } diff --git a/target/i386/sev.c b/target/i386/sev.c index 72b9e2ab40..9cfe5afe31 100644 --- a/target/i386/sev.c +++ b/target/i386/sev.c @@ -181,7 +181,8 @@ sev_set_guest_state(SevGuestState *sev, SevState new_st= ate) } =20 static void -sev_ram_block_added(RAMBlockNotifier *n, void *host, size_t size) +sev_ram_block_added(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size) { int r; struct kvm_enc_region range; @@ -198,19 +199,20 @@ sev_ram_block_added(RAMBlockNotifier *n, void *host, = size_t size) } =20 range.addr =3D (__u64)(unsigned long)host; - range.size =3D size; + range.size =3D max_size; =20 - trace_kvm_memcrypt_register_region(host, size); + trace_kvm_memcrypt_register_region(host, max_size); r =3D kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_REG_REGION, &range); if (r) { error_report("%s: failed to register region (%p+%#zx) error '%s'", - __func__, host, size, strerror(errno)); + __func__, host, max_size, strerror(errno)); exit(1); } } =20 static void -sev_ram_block_removed(RAMBlockNotifier *n, void *host, size_t size) +sev_ram_block_removed(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size) { int r; struct kvm_enc_region range; @@ -227,13 +229,13 @@ sev_ram_block_removed(RAMBlockNotifier *n, void *host= , size_t size) } =20 range.addr =3D (__u64)(unsigned long)host; - range.size =3D size; + range.size =3D max_size; =20 - trace_kvm_memcrypt_unregister_region(host, size); + trace_kvm_memcrypt_unregister_region(host, max_size); r =3D kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_UNREG_REGION, &range); if (r) { error_report("%s: failed to unregister region (%p+%#zx)", - __func__, host, size); + __func__, host, max_size); } } =20 diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index 92b9565797..911115b86e 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -459,26 +459,26 @@ fail_container: return ret; } =20 -static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, - void *host, size_t size) +static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, void *host, + size_t size, size_t max_size) { QEMUVFIOState *s =3D container_of(n, QEMUVFIOState, ram_notifier); int ret; =20 - trace_qemu_vfio_ram_block_added(s, host, size); - ret =3D qemu_vfio_dma_map(s, host, size, false, NULL); + trace_qemu_vfio_ram_block_added(s, host, max_size); + ret =3D qemu_vfio_dma_map(s, host, max_size, false, NULL); if (ret) { - error_report("qemu_vfio_dma_map(%p, %zu) failed: %s", host, size, + error_report("qemu_vfio_dma_map(%p, %zu) failed: %s", host, max_si= ze, strerror(-ret)); } } =20 -static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, - void *host, size_t size) +static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, void *host, + size_t size, size_t max_size) { QEMUVFIOState *s =3D container_of(n, QEMUVFIOState, ram_notifier); if (host) { - trace_qemu_vfio_ram_block_removed(s, host, size); + trace_qemu_vfio_ram_block_removed(s, host, max_size); qemu_vfio_dma_unmap(s, host); } } --=20 2.30.2 From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619695789; cv=none; d=zohomail.com; s=zohoarc; b=JVjpjx8E7BmP6m9Pglc4+XIgMxao8irL6mjEa+OMn8M2/xbabTF8evBsog572sTZM4XtshZ4kQHAgAa7uYlWz37X1mxc6qqxckRiDDusUMQDWaoq03eowgUWKJWwCB4xvKvKgJjeMyx5MNYcNN8QJ1WKXCka3S6bARHSW+UwsZQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619695789; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PtJ3WkYq/6WfkYFyId9CQxf2bqSHIctEmPP8mwkYZ2c=; b=kEYkbdPL/jH3DsXlajZCoU/ukxO+vlZGuB/or6q+FPDNKlTpVa57tprDAHZcnBwPo4Nb40Cm4qrFjl1C92Ly0+icnsRN77516WBLg401nN4eRGzKAJ8WvC3A2+wQHYSUMxwqeDluNGCkS1jx8W3a9n9EfjMPxg6krpVaoA2r5Bo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1619695789052436.6278448892351; Thu, 29 Apr 2021 04:29:49 -0700 (PDT) Received: from localhost ([::1]:54446 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lc4rb-0000kA-UC for importer@patchew.org; Thu, 29 Apr 2021 07:29:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58534) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4pq-00080w-KC for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:27:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:20333) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4pp-0003YO-1F for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:27:58 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-468-kJwPIqPFPAiRWld1n55FqA-1; Thu, 29 Apr 2021 07:27:52 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5987B6D500; Thu, 29 Apr 2021 11:27:51 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id CC782679EB; Thu, 29 Apr 2021 11:27:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695676; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PtJ3WkYq/6WfkYFyId9CQxf2bqSHIctEmPP8mwkYZ2c=; b=I0QDb8MYW/G4qS9p+HSQDk1Uh3WdPcF05gm9/d4gXsOQFBgqXaA3+MwlZs+HY9/CFy5wY3 zK0CY0wFIRMbEiZVOfayTWV/cHOshyvVoJw3zG7SzA6cn418mjLxBKbGiXHmxHUfIH8f5C ME22SOaSmkbtIFJ5fHH+qwpavzr9UNo= X-MC-Unique: kJwPIqPFPAiRWld1n55FqA-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v5 03/10] numa: Make all callbacks of ram block notifiers optional Date: Thu, 29 Apr 2021 13:27:01 +0200 Message-Id: <20210429112708.12291-4-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela , Alex Williamson , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Let's make add/remove optional. We want to introduce a RAM block notifier for RAM migration that is only interested in resize events. Reviewed-by: Peter Xu Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost --- hw/core/numa.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/core/numa.c b/hw/core/numa.c index 921bf86ab4..64b9334050 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -819,8 +819,11 @@ static int ram_block_notify_add_single(RAMBlock *rb, v= oid *opaque) void ram_block_notifier_add(RAMBlockNotifier *n) { QLIST_INSERT_HEAD(&ram_list.ramblock_notifiers, n, next); + /* Notify about all existing ram blocks. */ - qemu_ram_foreach_block(ram_block_notify_add_single, n); + if (n->ram_block_added) { + qemu_ram_foreach_block(ram_block_notify_add_single, n); + } } =20 void ram_block_notifier_remove(RAMBlockNotifier *n) @@ -833,7 +836,9 @@ void ram_block_notify_add(void *host, size_t size, size= _t max_size) RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_added(notifier, host, size, max_size); + if (notifier->ram_block_added) { + notifier->ram_block_added(notifier, host, size, max_size); + } } } =20 @@ -842,7 +847,9 @@ void ram_block_notify_remove(void *host, size_t size, s= ize_t max_size) RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_removed(notifier, host, size, max_size); + if (notifier->ram_block_removed) { + notifier->ram_block_removed(notifier, host, size, max_size); + } } } =20 --=20 2.30.2 From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619696243; cv=none; d=zohomail.com; s=zohoarc; b=bRDYr2VSnR0Pmaa6XDj1Q7Uh4r59oGN6P294vnx1jDLnJfNacNHyhhBn8oHAwBr5ED9Lc85TmxSikgbtLr+0uTqbTHKpIn6QwLfJdPWxpnxIqaZ4bu6OxdxwTngJ4xN90RIaTEx2LfY0g2GsdLvASbwSCSnb1aiQssgfq0FiH8I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619696243; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7PW44Lst8XBAAdn5kkQIDnWPWN8/Rz050wyZ5Uvo4x0=; b=ROzYJ518BuvDo2qY4sYxdo2ScprA3RdBaOxl1XPN791Q3YihrpwN7ftd4SDws6JE8JD3NnEqnDgtLlgcneSzE7wua0DnJVqMmrvdAHSnOXO/8gTe4c613L08MS67sylEoQ7GiZ5He4n8oFnpNeyFjCPBHTHwIX6aehzH0zwcZY8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1619696243273883.1799178533873; Thu, 29 Apr 2021 04:37:23 -0700 (PDT) Received: from localhost ([::1]:39204 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lc4yr-0006Uj-Nb for importer@patchew.org; Thu, 29 Apr 2021 07:37:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58612) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4q7-0008Vz-1X for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:20371) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4q5-0003ii-4i for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:14 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-588-1UbaSJttMsa2-hkT9Sr_yQ-1; Thu, 29 Apr 2021 07:28:10 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9C4A36D50F; Thu, 29 Apr 2021 11:28:09 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id B7E4D67879; Thu, 29 Apr 2021 11:27:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695692; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7PW44Lst8XBAAdn5kkQIDnWPWN8/Rz050wyZ5Uvo4x0=; b=ShSts4c9gVTIE/P9kXT0lR7cqjV8+Lh+0OAMfVZUPbtGbuOfPSLdME6LtoLsLPnm48Px6j 2JFr4YgckJokSDgmpZWgT7MLrXPkbgujlipWooQYo85dwQDTjA5TKPGG6/XJ4zyXU6TwL0 voXOK4oG5b6bCdB7nFq9t7O0Qble02I= X-MC-Unique: 1UbaSJttMsa2-hkT9Sr_yQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v5 04/10] migration/ram: Handle RAM block resizes during precopy Date: Thu, 29 Apr 2021 13:27:02 +0200 Message-Id: <20210429112708.12291-5-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URG_BIZ=0.573 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela , Alex Williamson , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Resizing while migrating is dangerous and does not work as expected. The whole migration code works on the usable_length of ram blocks and does not expect this to change at random points in time. In the case of precopy, the ram block size must not change on the source, after syncing the RAM block list in ram_save_setup(), so as long as the guest is still running on the source. Resizing can be trigger *after* (but not during) a reset in ACPI code by the guest - hw/arm/virt-acpi-build.c:acpi_ram_update() - hw/i386/acpi-build.c:acpi_ram_update() Use the ram block notifier to get notified about resizes. Let's simply cancel migration and indicate the reason. We'll continue running on the source. No harm done. Update the documentation. Postcopy will be handled separately. Reviewed-by: Peter Xu Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost --- include/exec/memory.h | 10 ++++++---- migration/migration.c | 9 +++++++-- migration/migration.h | 1 + migration/ram.c | 31 +++++++++++++++++++++++++++++++ softmmu/physmem.c | 5 +++-- 5 files changed, 48 insertions(+), 8 deletions(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index 5728a681b2..c8b9088924 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -131,7 +131,7 @@ typedef struct IOMMUTLBEvent { #define RAM_SHARED (1 << 1) =20 /* Only a portion of RAM (used_length) is actually used, and migrated. - * This used_length size can change across reboots. + * Resizing RAM while migrating can result in the migration being canceled. */ #define RAM_RESIZEABLE (1 << 2) =20 @@ -955,7 +955,9 @@ void memory_region_init_ram_shared_nomigrate(MemoryRegi= on *mr, * RAM. Accesses into the region will * modify memory directly. Only an in= itial * portion of this RAM is actually use= d. - * The used size can change across reb= oots. + * Changing the size while migrating + * can result in the migration being + * canceled. * * @mr: the #MemoryRegion to be initialized. * @owner: the object that tracks the region's reference count @@ -1586,8 +1588,8 @@ void *memory_region_get_ram_ptr(MemoryRegion *mr); =20 /* memory_region_ram_resize: Resize a RAM region. * - * Only legal before guest might have detected the memory size: e.g. on - * incoming migration, or right after reset. + * Resizing RAM while migrating can result in the migration being canceled. + * Care has to be taken if the guest might have already detected the memor= y. * * @mr: a memory region created with @memory_region_init_resizeable_ram. * @newsize: the new size the region diff --git a/migration/migration.c b/migration/migration.c index 8ca034136b..2dea8e2fc6 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -223,13 +223,18 @@ void migration_object_init(void) dirty_bitmap_mig_init(); } =20 +void migration_cancel(void) +{ + migrate_fd_cancel(current_migration); +} + void migration_shutdown(void) { /* * Cancel the current migration - that will (eventually) * stop the migration using this structure */ - migrate_fd_cancel(current_migration); + migration_cancel(); object_unref(OBJECT(current_migration)); =20 /* @@ -2310,7 +2315,7 @@ void qmp_migrate(const char *uri, bool has_blk, bool = blk, =20 void qmp_migrate_cancel(Error **errp) { - migrate_fd_cancel(migrate_get_current()); + migration_cancel(); } =20 void qmp_migrate_continue(MigrationStatus state, Error **errp) diff --git a/migration/migration.h b/migration/migration.h index db6708326b..f7b388d718 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -375,5 +375,6 @@ int foreach_not_ignored_block(RAMBlockIterFunc func, vo= id *opaque); void migration_make_urgent_request(void); void migration_consume_urgent_request(void); bool migration_rate_limit(void); +void migration_cancel(void); =20 #endif diff --git a/migration/ram.c b/migration/ram.c index 4682f3625c..195fabbab0 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -53,6 +53,7 @@ #include "block.h" #include "sysemu/sysemu.h" #include "sysemu/cpu-throttle.h" +#include "sysemu/runstate.h" #include "savevm.h" #include "qemu/iov.h" #include "multifd.h" @@ -4138,8 +4139,38 @@ static SaveVMHandlers savevm_ram_handlers =3D { .resume_prepare =3D ram_resume_prepare, }; =20 +static void ram_mig_ram_block_resized(RAMBlockNotifier *n, void *host, + size_t old_size, size_t new_size) +{ + ram_addr_t offset; + RAMBlock *rb =3D qemu_ram_block_from_host(host, false, &offset); + Error *err =3D NULL; + + if (ramblock_is_ignored(rb)) { + return; + } + + if (!migration_is_idle()) { + /* + * Precopy code on the source cannot deal with the size of RAM blo= cks + * changing at random points in time - especially after sending the + * RAM block sizes in the migration stream, they must no longer ch= ange. + * Abort and indicate a proper reason. + */ + error_setg(&err, "RAM block '%s' resized during precopy.", rb->ids= tr); + migrate_set_error(migrate_get_current(), err); + error_free(err); + migration_cancel(); + } +} + +static RAMBlockNotifier ram_mig_ram_notifier =3D { + .ram_block_resized =3D ram_mig_ram_block_resized, +}; + void ram_mig_init(void) { qemu_mutex_init(&XBZRLE.lock); register_savevm_live("ram", 0, 4, &savevm_ram_handlers, &ram_state); + ram_block_notifier_add(&ram_mig_ram_notifier); } diff --git a/softmmu/physmem.c b/softmmu/physmem.c index d6463f203d..4834003d47 100644 --- a/softmmu/physmem.c +++ b/softmmu/physmem.c @@ -1801,8 +1801,9 @@ static int memory_try_enable_merging(void *addr, size= _t len) return qemu_madvise(addr, len, QEMU_MADV_MERGEABLE); } =20 -/* Only legal before guest might have detected the memory size: e.g. on - * incoming migration, or right after reset. +/* + * Resizing RAM while migrating can result in the migration being canceled. + * Care has to be taken if the guest might have already detected the memor= y. * * As memory core doesn't know how is memory accessed, it is up to * resize callback to update device state and/or add assertions to detect --=20 2.30.2 From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619696356; cv=none; d=zohomail.com; s=zohoarc; b=Twzj62aJaP4QADIJ01h/JXDceTKGSmc6IUSN9s6RKt6FA1KF2Yk6h8l2OG1NInQX2iNhZlKhJJ6aNQ8v4abieVymkv2zJMM8Dt8CO6JO82MqsyRyYyk0f8E4AmIqpEtBae0+Aok3E3Xcqjat4kmfgajSjuVdCy0toTDxBhjiiJE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619696356; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nq+3IeRzoepyEJ0CosTis2dz02HyPiq+/3zoaugBJJ8=; b=eD8EXD9/s042goLJi/ZGtnjx/MigaTvy8zHPMC5Nr2bhetVCKXx1mMpu8DLSAxCRkyMKU6O/NTA6JQLARtPHxYkq1+hPoc6os4nF23O1w/0BnKJOEdGpdisVRgnSGthyEgnQb5VnhQb6wBGB8NAuMrHOkmRmP1xEFUnnlOyw1FM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1619696356177398.37734504051593; Thu, 29 Apr 2021 04:39:16 -0700 (PDT) Received: from localhost ([::1]:41432 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lc50l-0007U5-4L for importer@patchew.org; Thu, 29 Apr 2021 07:39:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58636) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4q9-0000Ai-RV for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28627) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4q8-0003km-6N for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:17 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-33-kbDAFACiOna2dFJuuTsvxw-1; Thu, 29 Apr 2021 07:28:13 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7C993803622; Thu, 29 Apr 2021 11:28:12 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0730E17177; Thu, 29 Apr 2021 11:28:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695695; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nq+3IeRzoepyEJ0CosTis2dz02HyPiq+/3zoaugBJJ8=; b=Fx+lnElSeUJnlwtYx2L57uzIoHqr6reH2viNszTJHsQvz2FPvGPT7sb7Tr4shIIR41F8T5 rDXgB+6TCxTwDAzSho6ydF1YfCfyK+5YN4Fv2ESMcvDIkCe9y3azxO+HUZscOEQk8zPp7a laFj5yU5k3srzVHH3r9ciYfinZcUZ/4= X-MC-Unique: kbDAFACiOna2dFJuuTsvxw-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v5 05/10] exec: Relax range check in ram_block_discard_range() Date: Thu, 29 Apr 2021 13:27:03 +0200 Message-Id: <20210429112708.12291-6-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela , Alex Williamson , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We want to make use of ram_block_discard_range() in the RAM block resize callback when growing a RAM block, *before* used_length is changed. Let's relax the check. As RAM blocks always mmap the whole max_length area, we cannot corrupt unrelated data. Reviewed-by: Peter Xu Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost --- softmmu/physmem.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/softmmu/physmem.c b/softmmu/physmem.c index 4834003d47..4e587de150 100644 --- a/softmmu/physmem.c +++ b/softmmu/physmem.c @@ -3503,7 +3503,7 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t st= art, size_t length) goto err; } =20 - if ((start + length) <=3D rb->used_length) { + if ((start + length) <=3D rb->max_length) { bool need_madvise, need_fallocate; if (!QEMU_IS_ALIGNED(length, rb->page_size)) { error_report("ram_block_discard_range: Unaligned length: %zx", @@ -3570,7 +3570,7 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t st= art, size_t length) } else { error_report("ram_block_discard_range: Overrun block '%s' (%" PRIu= 64 "/%zx/" RAM_ADDR_FMT")", - rb->idstr, start, length, rb->used_length); + rb->idstr, start, length, rb->max_length); } =20 err: --=20 2.30.2 From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619696518; cv=none; d=zohomail.com; s=zohoarc; b=Ff/IcgfCi0Mx/Xja/8JdcXkUKGvNvrQseppBE+i535c4lzYuF1fbiUDOONrEQNS6cQ5lMGEesYaUstc2su1zMWIHOou/hIRGA+hv87LM+rpKhJdG4zMC1G3jme3H2UNABFhRky14yH0YkI280FaHQFA7PxuJsHuKQFWkdhsJSyA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619696518; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DtLOv/uoqVwUIB8NBbmGCi38t/hatIqKLXB2OxF8Pnc=; b=K164wmWNDx7bxNEo+U7sqDOoaHrKb5WGRBppfjQgzIle1PlkLYs+OfsnGtc2BV0bEOH/NpMdABAfqN8DvL+RteVy7sqFxGLXmQOz3kcKEJ42fcOds9SQi1s6vnaCAEnxXB8rINMxuFDLa3v6LueqGutMWnB21Lil14f4nvItjvc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1619696518953110.55145401185405; Thu, 29 Apr 2021 04:41:58 -0700 (PDT) Received: from localhost ([::1]:43658 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lc53N-0008Vh-Lx for importer@patchew.org; Thu, 29 Apr 2021 07:41:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58716) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4qW-0000U1-54 for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:36057) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4qL-0003tX-17 for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:38 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-519-bDmoSFEOP3Krp0vO7V4fIg-1; Thu, 29 Apr 2021 07:28:26 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9679F8049D4; Thu, 29 Apr 2021 11:28:25 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id DB178679E7; Thu, 29 Apr 2021 11:28:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695708; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DtLOv/uoqVwUIB8NBbmGCi38t/hatIqKLXB2OxF8Pnc=; b=CxUuxQONHEmBZkugqTuvM1WmhZWNS0yvztHPSZ/FmIspzv4Fbp2Jexbl8R451JyU0HknUU Aw7o3zri540gvVmWzMqFMU5xQvmunG45na3Dg7CwLtqgcysxUah0w6qhTkTBq/nwtOjdro KCyTH2/XqYrncQvuqiJ1vM39uv2ArtY= X-MC-Unique: bDmoSFEOP3Krp0vO7V4fIg-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v5 06/10] migration/ram: Discard RAM when growing RAM blocks after ram_postcopy_incoming_init() Date: Thu, 29 Apr 2021 13:27:04 +0200 Message-Id: <20210429112708.12291-7-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela , Alex Williamson , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" In case we grow our RAM after ram_postcopy_incoming_init() (e.g., when synchronizing the RAM block state with the migration source), the resized part would not get discarded. Let's perform that when being notified about a resize while postcopy has been advised, but is not listening yet. With precopy, the process is as following: 1. VM created - RAM blocks are created 2. Incomming migration started - Postcopy is advised - All pages in RAM blocks are discarded 3. Precopy starts - RAM blocks are resized to match the size on the migration source. - RAM pages from precopy stream are loaded - Uffd handler is registered, postcopy starts listening 4. Guest started, postcopy running - Pagefaults get resolved, pages get placed Reviewed-by: Peter Xu Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost --- migration/ram.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 195fabbab0..693851d7a0 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4142,6 +4142,7 @@ static SaveVMHandlers savevm_ram_handlers =3D { static void ram_mig_ram_block_resized(RAMBlockNotifier *n, void *host, size_t old_size, size_t new_size) { + PostcopyState ps =3D postcopy_state_get(); ram_addr_t offset; RAMBlock *rb =3D qemu_ram_block_from_host(host, false, &offset); Error *err =3D NULL; @@ -4162,6 +4163,35 @@ static void ram_mig_ram_block_resized(RAMBlockNotifi= er *n, void *host, error_free(err); migration_cancel(); } + + switch (ps) { + case POSTCOPY_INCOMING_ADVISE: + /* + * Update what ram_postcopy_incoming_init()->init_range() does at = the + * time postcopy was advised. Syncing RAM blocks with the source w= ill + * result in RAM resizes. + */ + if (old_size < new_size) { + if (ram_discard_range(rb->idstr, old_size, new_size - old_size= )) { + error_report("RAM block '%s' discard of resized RAM failed= ", + rb->idstr); + } + } + break; + case POSTCOPY_INCOMING_NONE: + case POSTCOPY_INCOMING_RUNNING: + case POSTCOPY_INCOMING_END: + /* + * Once our guest is running, postcopy does no longer care about + * resizes. When growing, the new memory was not available on the + * source, no handler needed. + */ + break; + default: + error_report("RAM block '%s' resized during postcopy state: %d", + rb->idstr, ps); + exit(-1); + } } =20 static RAMBlockNotifier ram_mig_ram_notifier =3D { --=20 2.30.2 From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619695844; cv=none; d=zohomail.com; s=zohoarc; b=gHoVeUaO+swy59rqwccrQHac4ZIk59te4WbjrL4IF/f9By9/yQXJxwaVM0a3UfLzyblsTjt4b/7pF+lank78CpMXd/2s66rp0fCH72YB0N+aN6I/PE2XsMTT2RDV3v0lGaT6WAZlrh3DZ8toLoE1cdN3XSvewm5M0O6R6mlWdbA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619695844; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7uBPvYn3+AdNBAv5CiUJ7cT6Cug/y9zuONBG/GzcdC4=; b=HAhi+5XmtX8kZnbSshH6ZJQDWMRn4DA2RshVCilw5+amJ4W6Fn5SLt9IVCjp+RWeIVypuq69pGG9MRrLda2m6NVlOmKU7dDH26y9TwK1uZKT5CVHiGY2pN/7s9RyuPNgpVt6GhZ7hSytbbXLkq1nQMhuJ/tOEJNmAsZkbueckSo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1619695844615768.8401914376082; Thu, 29 Apr 2021 04:30:44 -0700 (PDT) Received: from localhost ([::1]:57838 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lc4sV-00029B-6r for importer@patchew.org; Thu, 29 Apr 2021 07:30:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58750) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4qd-0000Wd-W5 for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:33738) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4qU-0003wP-ES for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:47 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-183-w-XCiFWiPtugpz8Aic6nfw-1; Thu, 29 Apr 2021 07:28:33 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2F1538049CB; Thu, 29 Apr 2021 11:28:32 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id 00C5F679E7; Thu, 29 Apr 2021 11:28:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695716; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7uBPvYn3+AdNBAv5CiUJ7cT6Cug/y9zuONBG/GzcdC4=; b=bCFO0etn2+ENXs8gmpYAZOjX20gyLO4o2RXg5lu+ZwTJKjTq/oEkVN+mwl1VuO0jnWBJh3 Is9CXU6i5BmPz6P0lWRa5Az0D73GEf8Ht2nWK37atcdg6LG+aGizYrq5JGzueWg8D2pfOF oVqA4MpN0jZLrxiEFDfgNHfBXBGJxkE= X-MC-Unique: w-XCiFWiPtugpz8Aic6nfw-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v5 07/10] migration/ram: Simplify host page handling in ram_load_postcopy() Date: Thu, 29 Apr 2021 13:27:05 +0200 Message-Id: <20210429112708.12291-8-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela , Alex Williamson , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Add two new helper functions. This will come in come handy once we want to handle ram block resizes while postcopy is active. Note that ram_block_from_stream() will already print proper errors. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost --- migration/ram.c | 55 ++++++++++++++++++++++++++++--------------------- 1 file changed, 32 insertions(+), 23 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 693851d7a0..72df5228ee 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3121,6 +3121,20 @@ static inline void *host_from_ram_block_offset(RAMBl= ock *block, return block->host + offset; } =20 +static void *host_page_from_ram_block_offset(RAMBlock *block, + ram_addr_t offset) +{ + /* Note: Explicitly no check against offset_in_ramblock(). */ + return (void *)QEMU_ALIGN_DOWN((uintptr_t)block->host + offset, + block->page_size); +} + +static ram_addr_t host_page_offset_from_ram_block_offset(RAMBlock *block, + ram_addr_t offset) +{ + return ((uintptr_t)block->host + offset) & (block->page_size - 1); +} + static inline void *colo_cache_from_block_offset(RAMBlock *block, ram_addr_t offset, bool record_bitmap) { @@ -3524,13 +3538,12 @@ static int ram_load_postcopy(QEMUFile *f) MigrationIncomingState *mis =3D migration_incoming_get_current(); /* Temporary page that is later 'placed' */ void *postcopy_host_page =3D mis->postcopy_tmp_page; - void *this_host =3D NULL; + void *host_page =3D NULL; bool all_zero =3D true; int target_pages =3D 0; =20 while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { ram_addr_t addr; - void *host =3D NULL; void *page_buffer =3D NULL; void *place_source =3D NULL; RAMBlock *block =3D NULL; @@ -3555,9 +3568,12 @@ static int ram_load_postcopy(QEMUFile *f) if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE)) { block =3D ram_block_from_stream(f, flags); + if (!block) { + ret =3D -EINVAL; + break; + } =20 - host =3D host_from_ram_block_offset(block, addr); - if (!host) { + if (!offset_in_ramblock(block, addr)) { error_report("Illegal RAM offset " RAM_ADDR_FMT, addr); ret =3D -EINVAL; break; @@ -3575,19 +3591,17 @@ static int ram_load_postcopy(QEMUFile *f) * of a host page in one chunk. */ page_buffer =3D postcopy_host_page + - ((uintptr_t)host & (block->page_size - 1)); + host_page_offset_from_ram_block_offset(block, ad= dr); + /* If all TP are zero then we can optimise the place */ if (target_pages =3D=3D 1) { - this_host =3D (void *)QEMU_ALIGN_DOWN((uintptr_t)host, - block->page_size); - } else { + host_page =3D host_page_from_ram_block_offset(block, addr); + } else if (host_page !=3D host_page_from_ram_block_offset(bloc= k, + addr))= { /* not the 1st TP within the HP */ - if (QEMU_ALIGN_DOWN((uintptr_t)host, block->page_size) != =3D - (uintptr_t)this_host) { - error_report("Non-same host page %p/%p", - host, this_host); - ret =3D -EINVAL; - break; - } + error_report("Non-same host page %p/%p", host_page, + host_page_from_ram_block_offset(block, addr)); + ret =3D -EINVAL; + break; } =20 /* @@ -3666,16 +3680,11 @@ static int ram_load_postcopy(QEMUFile *f) } =20 if (!ret && place_needed) { - /* This gets called at the last target page in the host page */ - void *place_dest =3D (void *)QEMU_ALIGN_DOWN((uintptr_t)host, - block->page_size); - if (all_zero) { - ret =3D postcopy_place_page_zero(mis, place_dest, - block); + ret =3D postcopy_place_page_zero(mis, host_page, block); } else { - ret =3D postcopy_place_page(mis, place_dest, - place_source, block); + ret =3D postcopy_place_page(mis, host_page, place_source, + block); } place_needed =3D false; target_pages =3D 0; --=20 2.30.2 From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619696027; cv=none; d=zohomail.com; s=zohoarc; b=NTijQ4Z/yg5X8KPU3vUhW6HREZ8+jNBKjXdxhZ89K+/cd8XbMFVAjp44pqI6iXi2kBqPUDvjzCkqAQpB0Vf3NJ/p+dHFnMC9HVBOQbM9fcb8ZEuwqa09yWaRFjIeT7WWn3Vtl5uWFC/+vJ7YaZ633SXwqGPWqWk5XQRjoyilAJg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619696027; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OD3byHF/qXMVsJmYoEoajd2ySkwEjyveLb2xvi/YObU=; b=lx3zrWWp5MS44qVHiai+bMzZYLdu3f3iPa4QsnQG61uD5KRYuCMgXfBVzJupFbP6+dWLMzq2oDKYB3M3lKl3/1GAS8Wgc9FKaOUr4YJ5PbnthIWNd+7ZXAkkHkWaP3PyT22O0qRF1BlvaUHWTUZd0f9QlGDqT+MH14ylPf06PpE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1619696027304258.65784311973607; Thu, 29 Apr 2021 04:33:47 -0700 (PDT) Received: from localhost ([::1]:34022 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lc4vS-00047e-52 for importer@patchew.org; Thu, 29 Apr 2021 07:33:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58800) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4qh-0000e6-Po for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:20051) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4qf-00040h-Iv for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:28:51 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-262-SxrB0BEWMRSsemympi3tKA-1; Thu, 29 Apr 2021 07:28:46 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CE4B11005E5F; Thu, 29 Apr 2021 11:28:45 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E4F461D31; Thu, 29 Apr 2021 11:28:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695729; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OD3byHF/qXMVsJmYoEoajd2ySkwEjyveLb2xvi/YObU=; b=MuomOXP9MRg7GNf/u/r0dXXqIVvdyZDywFYRHyd8mb4phFIRxkullV65bJ5c2YnRKW+Ne2 rlPKaIy5QrPCFUVueh5/U5QIHiI6P18TvaEPritGLuIQTfOmcBf9mYjWXaHJ6B+zcUpGZs CmpC4D6R4x77Sh+wPcsJHU9BnKWXqik= X-MC-Unique: SxrB0BEWMRSsemympi3tKA-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v5 08/10] migration/ram: Handle RAM block resizes during postcopy Date: Thu, 29 Apr 2021 13:27:06 +0200 Message-Id: <20210429112708.12291-9-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela , Alex Williamson , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Resizing while migrating is dangerous and does not work as expected. The whole migration code works with the usable_length of a ram block and does not expect this value to change at random points in time. In the case of postcopy, relying on used_length is racy as soon as the guest is running. Also, when used_length changes we might leave the uffd handler registered for some memory regions, reject valid pages when migrating and fail when sending the recv bitmap to the source. Resizing can be trigger *after* (but not during) a reset in ACPI code by the guest - hw/arm/virt-acpi-build.c:acpi_ram_update() - hw/i386/acpi-build.c:acpi_ram_update() Let's remember the original used_length in a separate variable and use it in relevant postcopy code. Make sure to update it when we resize during precopy, when synchronizing the RAM block sizes with the source. Reviewed-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost --- include/exec/ramblock.h | 10 ++++++++++ migration/postcopy-ram.c | 15 ++++++++++++--- migration/ram.c | 11 +++++++++-- 3 files changed, 31 insertions(+), 5 deletions(-) diff --git a/include/exec/ramblock.h b/include/exec/ramblock.h index 07d50864d8..664701b759 100644 --- a/include/exec/ramblock.h +++ b/include/exec/ramblock.h @@ -59,6 +59,16 @@ struct RAMBlock { */ unsigned long *clear_bmap; uint8_t clear_bmap_shift; + + /* + * RAM block length that corresponds to the used_length on the migrati= on + * source (after RAM block sizes were synchronized). Especially, after + * starting to run the guest, used_length and postcopy_length can diff= er. + * Used to register/unregister uffd handlers and as the size of the re= ceived + * bitmap. Receiving any page beyond this length will bail out, as it + * could not have been valid on the source. + */ + ram_addr_t postcopy_length; }; #endif #endif diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index ab482adef1..2e9697bdd2 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -17,6 +17,7 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/rcu.h" #include "exec/target_page.h" #include "migration.h" #include "qemu-file.h" @@ -30,6 +31,7 @@ #include "qemu/error-report.h" #include "trace.h" #include "hw/boards.h" +#include "exec/ramblock.h" =20 /* Arbitrary limit on size of each discard command, * keeps them around ~200 bytes @@ -452,6 +454,13 @@ static int init_range(RAMBlock *rb, void *opaque) ram_addr_t length =3D qemu_ram_get_used_length(rb); trace_postcopy_init_range(block_name, host_addr, offset, length); =20 + /* + * Save the used_length before running the guest. In case we have to + * resize RAM blocks when syncing RAM block sizes from the source duri= ng + * precopy, we'll update it manually via the ram block notifier. + */ + rb->postcopy_length =3D length; + /* * We need the whole of RAM to be truly empty for postcopy, so things * like ROMs and any data tables built during init must be zero'd @@ -474,7 +483,7 @@ static int cleanup_range(RAMBlock *rb, void *opaque) const char *block_name =3D qemu_ram_get_idstr(rb); void *host_addr =3D qemu_ram_get_host_addr(rb); ram_addr_t offset =3D qemu_ram_get_offset(rb); - ram_addr_t length =3D qemu_ram_get_used_length(rb); + ram_addr_t length =3D rb->postcopy_length; MigrationIncomingState *mis =3D opaque; struct uffdio_range range_struct; trace_postcopy_cleanup_range(block_name, host_addr, offset, length); @@ -580,7 +589,7 @@ static int nhp_range(RAMBlock *rb, void *opaque) const char *block_name =3D qemu_ram_get_idstr(rb); void *host_addr =3D qemu_ram_get_host_addr(rb); ram_addr_t offset =3D qemu_ram_get_offset(rb); - ram_addr_t length =3D qemu_ram_get_used_length(rb); + ram_addr_t length =3D rb->postcopy_length; trace_postcopy_nhp_range(block_name, host_addr, offset, length); =20 /* @@ -624,7 +633,7 @@ static int ram_block_enable_notify(RAMBlock *rb, void *= opaque) struct uffdio_register reg_struct; =20 reg_struct.range.start =3D (uintptr_t)qemu_ram_get_host_addr(rb); - reg_struct.range.len =3D qemu_ram_get_used_length(rb); + reg_struct.range.len =3D rb->postcopy_length; reg_struct.mode =3D UFFDIO_REGISTER_MODE_MISSING; =20 /* Now tell our userfault_fd that it's responsible for this area */ diff --git a/migration/ram.c b/migration/ram.c index 72df5228ee..6d4d13d345 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -243,7 +243,7 @@ int64_t ramblock_recv_bitmap_send(QEMUFile *file, return -1; } =20 - nbits =3D block->used_length >> TARGET_PAGE_BITS; + nbits =3D block->postcopy_length >> TARGET_PAGE_BITS; =20 /* * Make sure the tmp bitmap buffer is big enough, e.g., on 32bit @@ -3573,7 +3573,13 @@ static int ram_load_postcopy(QEMUFile *f) break; } =20 - if (!offset_in_ramblock(block, addr)) { + /* + * Relying on used_length is racy and can result in false posi= tives. + * We might place pages beyond used_length in case RAM was shr= unk + * while in postcopy, which is fine - trying to place via + * UFFDIO_COPY/UFFDIO_ZEROPAGE will never segfault. + */ + if (!block->host || addr >=3D block->postcopy_length) { error_report("Illegal RAM offset " RAM_ADDR_FMT, addr); ret =3D -EINVAL; break; @@ -4186,6 +4192,7 @@ static void ram_mig_ram_block_resized(RAMBlockNotifie= r *n, void *host, rb->idstr); } } + rb->postcopy_length =3D new_size; break; case POSTCOPY_INCOMING_NONE: case POSTCOPY_INCOMING_RUNNING: --=20 2.30.2 From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619696670; cv=none; d=zohomail.com; s=zohoarc; b=m6TasGnoH5h23M/o/2iJ0863cC4Ag1DJQcDKCY1iy/XtTnDChUX7iAZpJgnMXdIBDjYNmRghHA28+amr71eyVzl0xFmMWE1DGVy5JQFWgUduEjBAVlXwq3FqPjsY3d+ZkaBNU1V2gzR79eLuXsLBUCBkm72xUoO+ixeoh1lmHKQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619696670; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kAwwnZs2To7cMUKg45elufUxRzYsBFQyMnkBZTeGzJY=; b=SPPwmtNKb/O4BSoOfkbqQygd/YR0MkLTQOaKJ2CE+npk7oGLxqRIdGfEet7CImxzUZuSbIgE0YpyX1vySbSi7dc8HbA4HuS3Zs6o7fNF2xm+C6uOParuLai69kzvoyF/b6ZnLMFIl1T+G0Wuz3V98X7iwJhP6qyOtZo6KkHo1Ck= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1619696670176792.265420260357; Thu, 29 Apr 2021 04:44:30 -0700 (PDT) Received: from localhost ([::1]:46454 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lc55p-0001K4-01 for importer@patchew.org; Thu, 29 Apr 2021 07:44:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58868) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4qx-0000jB-1d for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:29:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:20357) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4qm-000444-Ar for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:29:05 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-483-G6h8zzf4OHCfHwnZMwzXAQ-1; Thu, 29 Apr 2021 07:28:53 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6D3C3107ACE3; Thu, 29 Apr 2021 11:28:52 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id 48A6467CC4; Thu, 29 Apr 2021 11:28:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695735; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kAwwnZs2To7cMUKg45elufUxRzYsBFQyMnkBZTeGzJY=; b=L4frkaGfceM1Vxt3nKexudaX66qq/+gRwO1cOiQsB9kHCswmT4X8M01z6BJxgZoTucjSsh GtTpsNXjswtBSV8P+vAjeNU0+rk5jFnllN5EnIypJrth/2Nn8Rim4o7mWIAfGzMbShXIoI QzGAQgTIIphDss3isQua7hfIFQrfhQU= X-MC-Unique: G6h8zzf4OHCfHwnZMwzXAQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v5 09/10] migration/multifd: Print used_length of memory block Date: Thu, 29 Apr 2021 13:27:07 +0200 Message-Id: <20210429112708.12291-10-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela , Alex Williamson , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We actually want to print the used_length, against which we check. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost --- migration/multifd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/multifd.c b/migration/multifd.c index a6677c45c8..0a4803cfcc 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -361,7 +361,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams= *p, Error **errp) if (offset > (block->used_length - qemu_target_page_size())) { error_setg(errp, "multifd: offset too long %" PRIu64 " (max " RAM_ADDR_FMT ")", - offset, block->max_length); + offset, block->used_length); return -1; } p->pages->iov[i].iov_base =3D block->host + offset; --=20 2.30.2 From nobody Sun Feb 8 06:33:51 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619696798; cv=none; d=zohomail.com; s=zohoarc; b=OAdLaH18L0IuYChPSekLZtKwc6LyOmQJgGlTYz7XA2+0+90mwK9FNNLFKcusuasUIkqKpNIMrmcQo4uu9n8GO5D2+CcWVD5nnJhBf2Xpr6msX0DpyTBlgT84tkFs4pXGp+IQmiNlTBMQntWbb+QlUGGfKFzNWJ0tsFBJQCi2Y8k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619696798; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cXn1k9gv4nKfQz0VnAQey6Yg58NDvhOR0rpeqfjrWbI=; b=ly+UpRXSldfLLuOwOHhspEoqP0HN5VSZqTOp4SujbR0H/LaV3xhjw/YX2O5xBokv//63tBzGordptdf5Iq0cV+Frliz/dq4dcrjMFnoV9btOK47RBBEVAq01YEU1sTmMUAWmO/3X5VbgplcRP3FnSC3IWppvLUCfLj0CK6b+0Ys= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1619696798784540.4957937666284; Thu, 29 Apr 2021 04:46:38 -0700 (PDT) Received: from localhost ([::1]:48758 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lc57t-0002Oo-Ni for importer@patchew.org; Thu, 29 Apr 2021 07:46:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58902) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4r7-0000qi-JZ for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:29:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24549) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lc4r5-0004Bc-3Q for qemu-devel@nongnu.org; Thu, 29 Apr 2021 07:29:17 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-118-5WtEvihzPaGLBaEQDmGTpQ-1; Thu, 29 Apr 2021 07:29:12 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C2C991006C85; Thu, 29 Apr 2021 11:29:11 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-50.ams2.redhat.com [10.36.114.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB1A5679E7; Thu, 29 Apr 2021 11:28:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619695754; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cXn1k9gv4nKfQz0VnAQey6Yg58NDvhOR0rpeqfjrWbI=; b=gbXO1GHuYsFe+6SBjVWc3E3pVgXVN7ACY3EGQlushYIBGjxTy3E6plBBwdoVdm9I8KUCiq tavknnGadflIdf5okw8N9nrk00p60e5fXFlxxbE5/hrZtMRsF1681KAVQ6bBZrvJDYmY3A oxYgundRiG9kBFoilQ50Cpfv+74jb1g= X-MC-Unique: 5WtEvihzPaGLBaEQDmGTpQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v5 10/10] migration/ram: Use offset_in_ramblock() in range checks Date: Thu, 29 Apr 2021 13:27:08 +0200 Message-Id: <20210429112708.12291-11-david@redhat.com> In-Reply-To: <20210429112708.12291-1-david@redhat.com> References: <20210429112708.12291-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S. Tsirkin" , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela , Alex Williamson , Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We never read or write beyond the used_length of memory blocks when migrating. Make this clearer by using offset_in_ramblock() consistently. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: David Hildenbrand Acked-by: Eduardo Habkost --- migration/ram.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 6d4d13d345..1814c6cf03 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1368,8 +1368,8 @@ static bool find_dirty_block(RAMState *rs, PageSearch= Status *pss, bool *again) *again =3D false; return false; } - if ((((ram_addr_t)pss->page) << TARGET_PAGE_BITS) - >=3D pss->block->used_length) { + if (!offset_in_ramblock(pss->block, + ((ram_addr_t)pss->page) << TARGET_PAGE_BITS)) { /* Didn't find anything in this RAM Block */ pss->page =3D 0; pss->block =3D QLIST_NEXT_RCU(pss->block, next); @@ -1894,7 +1894,7 @@ int ram_save_queue_pages(const char *rbname, ram_addr= _t start, ram_addr_t len) rs->last_req_rb =3D ramblock; } trace_ram_save_queue_pages(ramblock->idstr, start, len); - if (start + len > ramblock->used_length) { + if (!offset_in_ramblock(ramblock, start + len - 1)) { error_report("%s request overrun start=3D" RAM_ADDR_FMT " len=3D" RAM_ADDR_FMT " blocklen=3D" RAM_ADDR_FMT, __func__, start, len, ramblock->used_length); @@ -3739,8 +3739,8 @@ void colo_flush_ram_cache(void) while (block) { offset =3D migration_bitmap_find_dirty(ram_state, block, offse= t); =20 - if (((ram_addr_t)offset) << TARGET_PAGE_BITS - >=3D block->used_length) { + if (!offset_in_ramblock(block, + ((ram_addr_t)offset) << TARGET_PAGE_BI= TS)) { offset =3D 0; block =3D QLIST_NEXT_RCU(block, next); } else { --=20 2.30.2