From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459360; cv=none; d=zohomail.com; s=zohoarc; b=IU+QIEpV9kLeBlkoAor+Mo3wz759G+gPFLX+Ie5xzksERjlLh8UvetB7jBIWiOtWC8Sqd1onFv9RGV8yWqu2qi8NYRb4xJAbBqyVWFtVUEoPvP5W0Ncsdkr+K5sIXaq61lJT3XRlzgH0vEbYJIFtkHd4sRiYFpFxTDGuNqY9rxs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459360; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=MniKsTyZcYYg4+0248McbTqEYXfCaUzKL5fDumZzcL8=; b=HBfeGt2cJDw4bkVUWDuRWu2kExlb9KCQG5GWV76qvPdFxrhPIaxVMCnZ9IK3SC+9LIPWbqbU6YUo9SsQaYPnZqKe1g/xvdY0Knn4YLVykUrq0vqtMbGMpciBCTKM5mae31c1+E8uO9zJ9mwf1CRljOUywilPUZzDLr+DjQLBN44= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459360779267.86280069702013; Tue, 21 Apr 2020 01:56:00 -0700 (PDT) Received: from localhost ([::1]:54108 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQohD-0001Ma-GT for importer@patchew.org; Tue, 21 Apr 2020 04:55:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57162) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofG-0007Cx-A9 for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:53:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQof5-000059-Mh for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:53:58 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:20315 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQof5-0008Sk-A3 for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:53:47 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-126-3VdHCkBzMbSh5N1tzltISA-1; Tue, 21 Apr 2020 04:53:41 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CB8A1190D37F; Tue, 21 Apr 2020 08:53:40 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id A5DD710027BA; Tue, 21 Apr 2020 08:53:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459225; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MniKsTyZcYYg4+0248McbTqEYXfCaUzKL5fDumZzcL8=; b=UWDQDLYWciWpFad6/CwwTYxSxRVsUKynVM3IpmP/HntHWOleRP/lnou1gaifVaCmrYzwKS nFDP8khDWlM+wFv0OVXPTyT3U1D1x/qcxR9VM5xXUh7HqJGJqx9FqnFZXA+JhA6lVWmB/p UYZThsW4Yy6khC3t/MDu3mTFSW8xOCc= X-MC-Unique: 3VdHCkBzMbSh5N1tzltISA-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 01/13] util: vfio-helpers: Factor out and fix processing of existing ram blocks Date: Tue, 21 Apr 2020 10:52:48 +0200 Message-Id: <20200421085300.7734-2-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=david@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 03:31:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Stefan Hajnoczi , Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Alex Williamson , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Factor it out into common code when a new notifier is registered, just as done with the memory region notifier. This allows us to have the logic about how to process existing ram blocks at a central place (which will be extended soon). Just like when adding a new ram block, we have to register the max_length for now. We don't have a way to get notified about resizes yet, and some memory would not be mapped when growing the ram block. Note: Currently, ram blocks are only "fake resized". All memory (max_length) is accessible. We can get rid of a bunch of functions in stubs/ram-block.c . Print the warning from inside qemu_vfio_ram_block_added(). Reviewed-by: Peter Xu Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Alex Williamson Cc: Stefan Hajnoczi Cc: Peter Xu Signed-off-by: David Hildenbrand --- exec.c | 5 +++++ hw/core/numa.c | 14 ++++++++++++++ include/exec/cpu-common.h | 1 + util/vfio-helpers.c | 29 ++++++++--------------------- 4 files changed, 28 insertions(+), 21 deletions(-) diff --git a/exec.c b/exec.c index 2874bb5088..4f804347a6 100644 --- a/exec.c +++ b/exec.c @@ -1966,6 +1966,11 @@ ram_addr_t qemu_ram_get_used_length(RAMBlock *rb) return rb->used_length; } =20 +ram_addr_t qemu_ram_get_max_length(RAMBlock *rb) +{ + return rb->max_length; +} + bool qemu_ram_is_shared(RAMBlock *rb) { return rb->flags & RAM_SHARED; diff --git a/hw/core/numa.c b/hw/core/numa.c index 316bc50d75..dc5e5b4046 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -854,9 +854,23 @@ void query_numa_node_mem(NumaNodeMem node_mem[], Machi= neState *ms) } } =20 +static int ram_block_notify_add_single(RAMBlock *rb, void *opaque) +{ + const ram_addr_t max_size =3D qemu_ram_get_max_length(rb); + void *host =3D qemu_ram_get_host_addr(rb); + RAMBlockNotifier *notifier =3D opaque; + + if (host) { + notifier->ram_block_added(notifier, host, max_size); + } + return 0; +} + void ram_block_notifier_add(RAMBlockNotifier *n) { QLIST_INSERT_HEAD(&ram_list.ramblock_notifiers, n, next); + /* Notify about all existing ram blocks. */ + qemu_ram_foreach_block(ram_block_notify_add_single, n); } =20 void ram_block_notifier_remove(RAMBlockNotifier *n) diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index b47e5630e7..09decb8d93 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -59,6 +59,7 @@ const char *qemu_ram_get_idstr(RAMBlock *rb); void *qemu_ram_get_host_addr(RAMBlock *rb); ram_addr_t qemu_ram_get_offset(RAMBlock *rb); ram_addr_t qemu_ram_get_used_length(RAMBlock *rb); +ram_addr_t qemu_ram_get_max_length(RAMBlock *rb); bool qemu_ram_is_shared(RAMBlock *rb); bool qemu_ram_is_uf_zeroable(RAMBlock *rb); void qemu_ram_set_uf_zeroable(RAMBlock *rb); diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index ddd9a96e76..260570ae19 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -376,8 +376,14 @@ static void qemu_vfio_ram_block_added(RAMBlockNotifier= *n, void *host, size_t size) { QEMUVFIOState *s =3D container_of(n, QEMUVFIOState, ram_notifier); + int ret; + trace_qemu_vfio_ram_block_added(s, host, size); - qemu_vfio_dma_map(s, host, size, false, NULL); + ret =3D qemu_vfio_dma_map(s, host, size, false, NULL); + if (ret) { + error_report("qemu_vfio_dma_map(%p, %zu) failed: %s", host, size, + strerror(-ret)); + } } =20 static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, @@ -390,33 +396,14 @@ static void qemu_vfio_ram_block_removed(RAMBlockNotif= ier *n, } } =20 -static int qemu_vfio_init_ramblock(RAMBlock *rb, void *opaque) -{ - void *host_addr =3D qemu_ram_get_host_addr(rb); - ram_addr_t length =3D qemu_ram_get_used_length(rb); - int ret; - QEMUVFIOState *s =3D opaque; - - if (!host_addr) { - return 0; - } - ret =3D qemu_vfio_dma_map(s, host_addr, length, false, NULL); - if (ret) { - fprintf(stderr, "qemu_vfio_init_ramblock: failed %p %" PRId64 "\n", - host_addr, (uint64_t)length); - } - return 0; -} - static void qemu_vfio_open_common(QEMUVFIOState *s) { qemu_mutex_init(&s->lock); s->ram_notifier.ram_block_added =3D qemu_vfio_ram_block_added; s->ram_notifier.ram_block_removed =3D qemu_vfio_ram_block_removed; - ram_block_notifier_add(&s->ram_notifier); s->low_water_mark =3D QEMU_VFIO_IOVA_MIN; s->high_water_mark =3D QEMU_VFIO_IOVA_MAX; - qemu_ram_foreach_block(qemu_vfio_init_ramblock, s); + ram_block_notifier_add(&s->ram_notifier); } =20 /** --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459448; cv=none; d=zohomail.com; s=zohoarc; b=ivpjcdlNlBkz78g2Yz3gu/7Y97rZRn6Fo//j1tJpXzJoUztlzVK+OTjcadIfBRgTe8zEiYcstCfOdsJmNlIZVJCWy5lqfZqTtXLpNltjTDLB4OKj8l6r1+YgdZfs5Uzshxk+0oDD479zXc/EM3AaTgqTX5yVozeSX0fQdQ62woU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459448; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SjRWKBNyfsv6ykwqaQYS3rdIlYVkKXEACYGSeD/NfqE=; b=id/hvB4L32aPw2oHtALYfxodHDqdTQuVakVGuqDWUnLEIEY5OOrwbiJ6xjoGLH/McAMQc1YQN7Wywsnd0lfr+oZoK1ZHt5b/W+mqCX/P2oRrr4OTVw9cyy3YOgpR2dHl5V7W/Xi+w1uYd0ZBJpCyzMt7HN/1ee+eek6/42RsSEY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459448843724.7414148485342; Tue, 21 Apr 2020 01:57:28 -0700 (PDT) Received: from localhost ([::1]:54124 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQoid-0003Le-Hl for importer@patchew.org; Tue, 21 Apr 2020 04:57:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57228) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofP-0007Uv-MA for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQof7-00009Z-LI for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:07 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:55656 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQof7-00008L-8Z for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:53:49 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-194-T_WkhvcOPQa0PGQNtIHyHg-1; Tue, 21 Apr 2020 04:53:46 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E84921922023; Tue, 21 Apr 2020 08:53:45 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 436211059102; Tue, 21 Apr 2020 08:53:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SjRWKBNyfsv6ykwqaQYS3rdIlYVkKXEACYGSeD/NfqE=; b=UJPO3eHfgjeYObwg7ZtE8JQ80kVgo3XV2VA3c6YvhpP2IPOVtzeuxpztAKuQmdXLn/vpho IszbGH9m9Nk+a3x8dTxm1QCVR7vEusMixk/e16v3P9lnlz6R7uDwt4xtUsiQKYicZ5TMSx sF1RxKsm+eGDwAqW7iuv8aHxTwLhq5U= X-MC-Unique: T_WkhvcOPQa0PGQNtIHyHg-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 02/13] stubs/ram-block: Remove stubs that are no longer needed Date: Tue, 21 Apr 2020 10:52:49 +0200 Message-Id: <20200421085300.7734-3-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=david@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 03:31:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Current code no longer needs these stubs to compile. Let's just remove them. Reviewed-by: Peter Xu Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Peter Xu Signed-off-by: David Hildenbrand --- stubs/ram-block.c | 20 -------------------- 1 file changed, 20 deletions(-) diff --git a/stubs/ram-block.c b/stubs/ram-block.c index 73c0a3ee08..10855b52dd 100644 --- a/stubs/ram-block.c +++ b/stubs/ram-block.c @@ -2,21 +2,6 @@ #include "exec/ramlist.h" #include "exec/cpu-common.h" =20 -void *qemu_ram_get_host_addr(RAMBlock *rb) -{ - return 0; -} - -ram_addr_t qemu_ram_get_offset(RAMBlock *rb) -{ - return 0; -} - -ram_addr_t qemu_ram_get_used_length(RAMBlock *rb) -{ - return 0; -} - void ram_block_notifier_add(RAMBlockNotifier *n) { } @@ -24,8 +9,3 @@ void ram_block_notifier_add(RAMBlockNotifier *n) void ram_block_notifier_remove(RAMBlockNotifier *n) { } - -int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque) -{ - return 0; -} --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459274; cv=none; d=zohomail.com; s=zohoarc; b=CB+Ngl7XHT6q0l56TPWntCqMXlUmL9P9yMzZMgJXcYTObrwH7QR3hfR22z31MjM6tbogiqdWMrqhpjx+o/FoGp4VhA8rU5Muzm3uX9Fmivurxq7GHXhS6KDvqKU9mwHW4omGLy4xEK3TxP7ATXMcvuUARODm18Kcy+jAhnhZlK0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459274; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=1rQk9nLhjoKU4Y84lvzoPJz3nfGTEDpCxX8N/2k68SI=; b=jcvcaToMjhcihyx8y+BuMP5dY4Ms+DHVHDDtaJhn92ZlpHSq9xLhl35PbpoQodQErUzXxh7LwQe2OYxt7aZEuwWFjFKy9K+/MjkaiZAl2MCeixSvT4w4opID95wkEoTU+jgF9MA3gvNMVqfQ3EG4L4vUDWRB84eiE4xBOwfw9yA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1587459274271956.0013920299945; Tue, 21 Apr 2020 01:54:34 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jQofJ-0006fM-7G; Tue, 21 Apr 2020 08:54:01 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jQofH-0006fH-Nt for xen-devel@lists.xenproject.org; Tue, 21 Apr 2020 08:53:59 +0000 Received: from us-smtp-delivery-1.mimecast.com (unknown [207.211.31.120]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id a34639b0-83ad-11ea-9118-12813bfff9fa; Tue, 21 Apr 2020 08:53:58 +0000 (UTC) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-263-jqpXaHuMO16QPOc7WagJGg-1; Tue, 21 Apr 2020 04:53:56 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ADD5E107ACC7; Tue, 21 Apr 2020 08:53:54 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3FBB510027A9; Tue, 21 Apr 2020 08:53:46 +0000 (UTC) X-Inumbo-ID: a34639b0-83ad-11ea-9118-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459237; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1rQk9nLhjoKU4Y84lvzoPJz3nfGTEDpCxX8N/2k68SI=; b=K+0wwCFdseKsjQ8s3LhNJxNmrRu0BzB/y9U7Drzaqzv7HQb2NpVbueoZqrh0mjWgNdqqH9 OmSfo750i7EORuPjd6e+Ev9F/GoMgjZ/tg1zk5gMCeUCqNhGB2xxN3z2wWcH8Vx0QzZzRJ rBfSdruvGJ6TyN7jzS9ChfE7cdx2CE8= X-MC-Unique: jqpXaHuMO16QPOc7WagJGg-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 03/13] numa: Teach ram block notifiers about resizeable ram blocks Date: Tue, 21 Apr 2020 10:52:50 +0200 Message-Id: <20200421085300.7734-4-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paul Durrant , Igor Mammedov , Marcel Apfelbaum , xen-devel@lists.xenproject.org, Anthony Perard , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Ram block notifiers are currently not aware of resizes. Especially to handle resizes during migration, but also to implement actually resizeable ram blocks (make everything between used_length and max_length inaccessible), we want to teach ram block notifiers about resizeable ram. Introduce the basic infrastructure but keep using max_size in the existing notifiers. Supply the max_size when adding and removing ram blocks. Also, notify on resizes. Acked-by: Paul Durrant Reviewed-by: Peter Xu Cc: Richard Henderson Cc: Paolo Bonzini Cc: "Dr. David Alan Gilbert" Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Stefano Stabellini Cc: Anthony Perard Cc: Paul Durrant Cc: "Michael S. Tsirkin" Cc: xen-devel@lists.xenproject.org Cc: Igor Mammedov Signed-off-by: David Hildenbrand --- exec.c | 11 +++++++++-- hw/core/numa.c | 22 +++++++++++++++++----- hw/i386/xen/xen-mapcache.c | 7 ++++--- include/exec/ramlist.h | 13 +++++++++---- target/i386/hax-mem.c | 5 +++-- target/i386/sev.c | 18 ++++++++++-------- util/vfio-helpers.c | 16 ++++++++-------- 7 files changed, 60 insertions(+), 32 deletions(-) diff --git a/exec.c b/exec.c index 4f804347a6..83304e51c6 100644 --- a/exec.c +++ b/exec.c @@ -2115,6 +2115,11 @@ int qemu_ram_resize(RAMBlock *block, ram_addr_t news= ize, Error **errp) return -EINVAL; } =20 + /* Notify before modifying the ram block and touching the bitmaps. */ + if (block->host) { + ram_block_notify_resize(block->host, block->used_length, newsize); + } + cpu_physical_memory_clear_dirty_range(block->offset, block->used_lengt= h); block->used_length =3D newsize; cpu_physical_memory_set_dirty_range(block->offset, block->used_length, @@ -2281,7 +2286,8 @@ static void ram_block_add(RAMBlock *new_block, Error = **errp, bool shared) qemu_madvise(new_block->host, new_block->max_length, QEMU_MADV_DONTFORK); } - ram_block_notify_add(new_block->host, new_block->max_length); + ram_block_notify_add(new_block->host, new_block->used_length, + new_block->max_length); } } =20 @@ -2461,7 +2467,8 @@ void qemu_ram_free(RAMBlock *block) } =20 if (block->host) { - ram_block_notify_remove(block->host, block->max_length); + ram_block_notify_remove(block->host, block->used_length, + block->max_length); } =20 qemu_mutex_lock_ramlist(); diff --git a/hw/core/numa.c b/hw/core/numa.c index dc5e5b4046..fe6ca5c50d 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -857,11 +857,12 @@ void query_numa_node_mem(NumaNodeMem node_mem[], Mach= ineState *ms) static int ram_block_notify_add_single(RAMBlock *rb, void *opaque) { const ram_addr_t max_size =3D qemu_ram_get_max_length(rb); + const ram_addr_t size =3D qemu_ram_get_used_length(rb); void *host =3D qemu_ram_get_host_addr(rb); RAMBlockNotifier *notifier =3D opaque; =20 if (host) { - notifier->ram_block_added(notifier, host, max_size); + notifier->ram_block_added(notifier, host, size, max_size); } return 0; } @@ -878,20 +879,31 @@ void ram_block_notifier_remove(RAMBlockNotifier *n) QLIST_REMOVE(n, next); } =20 -void ram_block_notify_add(void *host, size_t size) +void ram_block_notify_add(void *host, size_t size, size_t max_size) { RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_added(notifier, host, size); + notifier->ram_block_added(notifier, host, size, max_size); } } =20 -void ram_block_notify_remove(void *host, size_t size) +void ram_block_notify_remove(void *host, size_t size, size_t max_size) { RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_removed(notifier, host, size); + notifier->ram_block_removed(notifier, host, size, max_size); + } +} + +void ram_block_notify_resize(void *host, size_t old_size, size_t new_size) +{ + RAMBlockNotifier *notifier; + + QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { + if (notifier->ram_block_resized) { + notifier->ram_block_resized(notifier, host, old_size, new_size= ); + } } } diff --git a/hw/i386/xen/xen-mapcache.c b/hw/i386/xen/xen-mapcache.c index 5b120ed44b..d6dcea65d1 100644 --- a/hw/i386/xen/xen-mapcache.c +++ b/hw/i386/xen/xen-mapcache.c @@ -169,7 +169,8 @@ static void xen_remap_bucket(MapCacheEntry *entry, =20 if (entry->vaddr_base !=3D NULL) { if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) { - ram_block_notify_remove(entry->vaddr_base, entry->size); + ram_block_notify_remove(entry->vaddr_base, entry->size, + entry->size); } if (munmap(entry->vaddr_base, entry->size) !=3D 0) { perror("unmap fails"); @@ -211,7 +212,7 @@ static void xen_remap_bucket(MapCacheEntry *entry, } =20 if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) { - ram_block_notify_add(vaddr_base, size); + ram_block_notify_add(vaddr_base, size, size); } =20 entry->vaddr_base =3D vaddr_base; @@ -452,7 +453,7 @@ static void xen_invalidate_map_cache_entry_unlocked(uin= t8_t *buffer) } =20 pentry->next =3D entry->next; - ram_block_notify_remove(entry->vaddr_base, entry->size); + ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size); if (munmap(entry->vaddr_base, entry->size) !=3D 0) { perror("unmap fails"); exit(-1); diff --git a/include/exec/ramlist.h b/include/exec/ramlist.h index bc4faa1b00..293c0ddabe 100644 --- a/include/exec/ramlist.h +++ b/include/exec/ramlist.h @@ -65,15 +65,20 @@ void qemu_mutex_lock_ramlist(void); void qemu_mutex_unlock_ramlist(void); =20 struct RAMBlockNotifier { - void (*ram_block_added)(RAMBlockNotifier *n, void *host, size_t size); - void (*ram_block_removed)(RAMBlockNotifier *n, void *host, size_t size= ); + void (*ram_block_added)(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size); + void (*ram_block_removed)(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size); + void (*ram_block_resized)(RAMBlockNotifier *n, void *host, size_t old_= size, + size_t new_size); QLIST_ENTRY(RAMBlockNotifier) next; }; =20 void ram_block_notifier_add(RAMBlockNotifier *n); void ram_block_notifier_remove(RAMBlockNotifier *n); -void ram_block_notify_add(void *host, size_t size); -void ram_block_notify_remove(void *host, size_t size); +void ram_block_notify_add(void *host, size_t size, size_t max_size); +void ram_block_notify_remove(void *host, size_t size, size_t max_size); +void ram_block_notify_resize(void *host, size_t old_size, size_t new_size); =20 void ram_block_dump(Monitor *mon); =20 diff --git a/target/i386/hax-mem.c b/target/i386/hax-mem.c index 6bb5a24917..454d7fb212 100644 --- a/target/i386/hax-mem.c +++ b/target/i386/hax-mem.c @@ -293,7 +293,8 @@ static MemoryListener hax_memory_listener =3D { .priority =3D 10, }; =20 -static void hax_ram_block_added(RAMBlockNotifier *n, void *host, size_t si= ze) +static void hax_ram_block_added(RAMBlockNotifier *n, void *host, size_t si= ze, + size_t max_size) { /* * We must register each RAM block with the HAXM kernel module, or @@ -304,7 +305,7 @@ static void hax_ram_block_added(RAMBlockNotifier *n, vo= id *host, size_t size) * host physical pages for the RAM block as part of this registration * process, hence the name hax_populate_ram(). */ - if (hax_populate_ram((uint64_t)(uintptr_t)host, size) < 0) { + if (hax_populate_ram((uint64_t)(uintptr_t)host, max_size) < 0) { fprintf(stderr, "HAX failed to populate RAM\n"); abort(); } diff --git a/target/i386/sev.c b/target/i386/sev.c index 846018a12d..65d852adf8 100644 --- a/target/i386/sev.c +++ b/target/i386/sev.c @@ -129,7 +129,8 @@ sev_set_guest_state(SevState new_state) } =20 static void -sev_ram_block_added(RAMBlockNotifier *n, void *host, size_t size) +sev_ram_block_added(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size) { int r; struct kvm_enc_region range; @@ -146,19 +147,20 @@ sev_ram_block_added(RAMBlockNotifier *n, void *host, = size_t size) } =20 range.addr =3D (__u64)(unsigned long)host; - range.size =3D size; + range.size =3D max_size; =20 - trace_kvm_memcrypt_register_region(host, size); + trace_kvm_memcrypt_register_region(host, max_size); r =3D kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_REG_REGION, &range); if (r) { error_report("%s: failed to register region (%p+%#zx) error '%s'", - __func__, host, size, strerror(errno)); + __func__, host, max_size, strerror(errno)); exit(1); } } =20 static void -sev_ram_block_removed(RAMBlockNotifier *n, void *host, size_t size) +sev_ram_block_removed(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size) { int r; struct kvm_enc_region range; @@ -175,13 +177,13 @@ sev_ram_block_removed(RAMBlockNotifier *n, void *host= , size_t size) } =20 range.addr =3D (__u64)(unsigned long)host; - range.size =3D size; + range.size =3D max_size; =20 - trace_kvm_memcrypt_unregister_region(host, size); + trace_kvm_memcrypt_unregister_region(host, max_size); r =3D kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_UNREG_REGION, &range); if (r) { error_report("%s: failed to unregister region (%p+%#zx)", - __func__, host, size); + __func__, host, max_size); } } =20 diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index 260570ae19..9ec01bfe26 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -372,26 +372,26 @@ fail_container: return ret; } =20 -static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, - void *host, size_t size) +static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, void *host, + size_t size, size_t max_size) { QEMUVFIOState *s =3D container_of(n, QEMUVFIOState, ram_notifier); int ret; =20 - trace_qemu_vfio_ram_block_added(s, host, size); - ret =3D qemu_vfio_dma_map(s, host, size, false, NULL); + trace_qemu_vfio_ram_block_added(s, host, max_size); + ret =3D qemu_vfio_dma_map(s, host, max_size, false, NULL); if (ret) { - error_report("qemu_vfio_dma_map(%p, %zu) failed: %s", host, size, + error_report("qemu_vfio_dma_map(%p, %zu) failed: %s", host, max_si= ze, strerror(-ret)); } } =20 -static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, - void *host, size_t size) +static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, void *host, + size_t size, size_t max_size) { QEMUVFIOState *s =3D container_of(n, QEMUVFIOState, ram_notifier); if (host) { - trace_qemu_vfio_ram_block_removed(s, host, size); + trace_qemu_vfio_ram_block_removed(s, host, max_size); qemu_vfio_dma_unmap(s, host); } } --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459473; cv=none; d=zohomail.com; s=zohoarc; b=YzAAJKHcGLOnmi+NbjQ4ZZNCZB/7ziYpsgSvHg465vabxt3d8hoC60A8IXLpSRDnJ7vk+tzju4pB6KYZGIMU1EO5+Zfx82laoWMBDJsXsEGolHBp3qrcBuyYR6Ikh6nwrr4kAsG3nXj1PZvDsQSQEzNOwjImSYZd4B2WPkbk/SI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459473; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lpGQhFLuleVEynnu5N5oJ+Xtp+TcToWLSl/iP1Dq6v4=; b=RxYpoQtUVzbx0shdt2eI+A+bA+p1jixuzg2qdmbTWTRd4eo64lCCaDqnRPtLWkXyIpUqe7YCVkVDHF6LW7sMHWoC2rDczZH8pTT86X93v+e5/VlNRfY8MsNFsfqWTRgGanadRCqBAIbt7NYojDGqpcGIZVUEqFHcHBQg7osE37w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459473787490.2991452641155; Tue, 21 Apr 2020 01:57:53 -0700 (PDT) Received: from localhost ([::1]:54126 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQoj2-0004As-Iz for importer@patchew.org; Tue, 21 Apr 2020 04:57:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57182) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofK-0007Jk-5o for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQofJ-0000cd-In for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:01 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:36059 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQofJ-0000ZH-4A for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:01 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-211-lAgRNmmDNbOCbbGGsDKSqw-1; Tue, 21 Apr 2020 04:53:58 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 462621084432; Tue, 21 Apr 2020 08:53:57 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 04A9F1036D27; Tue, 21 Apr 2020 08:53:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459240; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lpGQhFLuleVEynnu5N5oJ+Xtp+TcToWLSl/iP1Dq6v4=; b=WCnmYKDT3kSXylRU0Mt9GLPTmfPWsjDyxh5RW4cNEan+c6Ygv4OzHG+PFggJR0KzASb8Q+ TEi0yPbQpx+lk12n0i2i3WAa9DlLxqWSGwdm4MMsy0K60pNJwQbeJ2q50ceFITV3fftAdK UkFfnPfaUYiivVfpZqkwg1sC7yd4J0o= X-MC-Unique: lAgRNmmDNbOCbbGGsDKSqw-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 04/13] numa: Make all callbacks of ram block notifiers optional Date: Tue, 21 Apr 2020 10:52:51 +0200 Message-Id: <20200421085300.7734-5-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=david@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 04:54:00 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Let's make add/remove optional. We want to introduce a RAM block notifier for RAM migration, that's only interested in resizes. Reviewed-by: Peter Xu Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Peter Xu Signed-off-by: David Hildenbrand --- hw/core/numa.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/core/numa.c b/hw/core/numa.c index fe6ca5c50d..37ce175e13 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -870,8 +870,11 @@ static int ram_block_notify_add_single(RAMBlock *rb, v= oid *opaque) void ram_block_notifier_add(RAMBlockNotifier *n) { QLIST_INSERT_HEAD(&ram_list.ramblock_notifiers, n, next); + /* Notify about all existing ram blocks. */ - qemu_ram_foreach_block(ram_block_notify_add_single, n); + if (n->ram_block_added) { + qemu_ram_foreach_block(ram_block_notify_add_single, n); + } } =20 void ram_block_notifier_remove(RAMBlockNotifier *n) @@ -884,7 +887,9 @@ void ram_block_notify_add(void *host, size_t size, size= _t max_size) RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_added(notifier, host, size, max_size); + if (notifier->ram_block_added) { + notifier->ram_block_added(notifier, host, size, max_size); + } } } =20 @@ -893,7 +898,9 @@ void ram_block_notify_remove(void *host, size_t size, s= ize_t max_size) RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_removed(notifier, host, size, max_size); + if (notifier->ram_block_removed) { + notifier->ram_block_removed(notifier, host, size, max_size); + } } } =20 --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459439; cv=none; d=zohomail.com; s=zohoarc; b=C8KyfpDywM37VdPV8bS2UdCS0zjba9QsdDXEl80VP1N10v52WEwt9FxnJolvmp+uiMYW/DE2H+VVNzhmp/MXz8LiDWpeDqb7REvD8yZqhU5VDg6zeeDsG7wb+g498jMKVyOOMc5S2q7oi9ThuyExb6gTMe1erVZPg9zaeO/yJUY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459439; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0BUY+7AyduGiq5cIOvaaefFtBjl39KV3uCDOFF0Z35g=; b=UKy/3hCfquYXzu1fmhds4JPazsgLtTGB39A2CWDcxG//C8nw9durW4Uwng1Q76GVMaYa0drrkbPPt7qFjl/lQPJWqg5xdJSvmZz+6TABNDxnI2XEWytKQ0iNIPnyptGHAYjH6XXh+CZo6GQngyJSLzk1a0A1PngRkd6iUiIQeXQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459439161348.88965455235166; Tue, 21 Apr 2020 01:57:19 -0700 (PDT) Received: from localhost ([::1]:54122 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQoiT-0002zY-M6 for importer@patchew.org; Tue, 21 Apr 2020 04:57:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57210) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofN-0007R0-84 for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQofM-0000kO-GI for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:04 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:20851 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQofM-0000iE-1i for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:04 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-442-yY0SrPaZNQWrccMZDPpslQ-1; Tue, 21 Apr 2020 04:54:01 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 68A15107ACC7; Tue, 21 Apr 2020 08:54:00 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 97F1A100E7E3; Tue, 21 Apr 2020 08:53:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459243; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0BUY+7AyduGiq5cIOvaaefFtBjl39KV3uCDOFF0Z35g=; b=TFXt8OHuzoS5Bt5k/r8fqn+tnwf4ZVkrLsECreP7jQ32vLlNEfjfi2Ve1A0XgZlThfxZx/ 6dctE3xWL/ynJehVcs8vhz+iiNoLcCaYAM7DOnqWqQ+6trwF+3o/dHd+JFdJPj0ogVg2WI r3GLomT61L2wY8GxsX2+ANiQYYAqTiU= X-MC-Unique: yY0SrPaZNQWrccMZDPpslQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 05/13] migration/ram: Handle RAM block resizes during precopy Date: Tue, 21 Apr 2020 10:52:52 +0200 Message-Id: <20200421085300.7734-6-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=david@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 03:31:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , Richard Henderson , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Shannon Zhao , Igor Mammedov , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Resizing while migrating is dangerous and does not work as expected. The whole migration code works on the usable_length of ram blocks and does not expect this to change at random points in time. In the case of precopy, the ram block size must not change on the source, after syncing the RAM block list in ram_save_setup(), so as long as the guest is still running on the source. Resizing can be trigger *after* (but not during) a reset in ACPI code by the guest - hw/arm/virt-acpi-build.c:acpi_ram_update() - hw/i386/acpi-build.c:acpi_ram_update() Use the ram block notifier to get notified about resizes. Let's simply cancel migration and indicate the reason. We'll continue running on the source. No harm done. Update the documentation. Postcopy will be handled separately. Reviewed-by: Peter Xu Cc: "Dr. David Alan Gilbert" Cc: Juan Quintela Cc: Eduardo Habkost Cc: Paolo Bonzini Cc: Igor Mammedov Cc: "Michael S. Tsirkin" Cc: Richard Henderson Cc: Shannon Zhao Cc: Alex Benn=C3=A9e Cc: Peter Xu Signed-off-by: David Hildenbrand --- exec.c | 5 +++-- include/exec/memory.h | 10 ++++++---- migration/migration.c | 9 +++++++-- migration/migration.h | 1 + migration/ram.c | 31 +++++++++++++++++++++++++++++++ 5 files changed, 48 insertions(+), 8 deletions(-) diff --git a/exec.c b/exec.c index 83304e51c6..78dfab6ec6 100644 --- a/exec.c +++ b/exec.c @@ -2070,8 +2070,9 @@ static int memory_try_enable_merging(void *addr, size= _t len) return qemu_madvise(addr, len, QEMU_MADV_MERGEABLE); } =20 -/* Only legal before guest might have detected the memory size: e.g. on - * incoming migration, or right after reset. +/* + * Resizing RAM while migrating can result in the migration being canceled. + * Care has to be taken if the guest might have already detected the memor= y. * * As memory core doesn't know how is memory accessed, it is up to * resize callback to update device state and/or add assertions to detect diff --git a/include/exec/memory.h b/include/exec/memory.h index e000bd2f97..23fe94d382 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -113,7 +113,7 @@ typedef struct IOMMUNotifier IOMMUNotifier; #define RAM_SHARED (1 << 1) =20 /* Only a portion of RAM (used_length) is actually used, and migrated. - * This used_length size can change across reboots. + * Resizing RAM while migrating can result in the migration being canceled. */ #define RAM_RESIZEABLE (1 << 2) =20 @@ -843,7 +843,9 @@ void memory_region_init_ram_shared_nomigrate(MemoryRegi= on *mr, * RAM. Accesses into the region will * modify memory directly. Only an in= itial * portion of this RAM is actually use= d. - * The used size can change across reb= oots. + * Changing the size while migrating + * can result in the migration being + * canceled. * * @mr: the #MemoryRegion to be initialized. * @owner: the object that tracks the region's reference count @@ -1464,8 +1466,8 @@ void *memory_region_get_ram_ptr(MemoryRegion *mr); =20 /* memory_region_ram_resize: Resize a RAM region. * - * Only legal before guest might have detected the memory size: e.g. on - * incoming migration, or right after reset. + * Resizing RAM while migrating can result in the migration being canceled. + * Care has to be taken if the guest might have already detected the memor= y. * * @mr: a memory region created with @memory_region_init_resizeable_ram. * @newsize: the new size the region diff --git a/migration/migration.c b/migration/migration.c index 187ac0410c..8cc06488ed 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -181,13 +181,18 @@ void migration_object_init(void) } } =20 +void migration_cancel(void) +{ + migrate_fd_cancel(current_migration); +} + void migration_shutdown(void) { /* * Cancel the current migration - that will (eventually) * stop the migration using this structure */ - migrate_fd_cancel(current_migration); + migration_cancel(); object_unref(OBJECT(current_migration)); } =20 @@ -2066,7 +2071,7 @@ void qmp_migrate(const char *uri, bool has_blk, bool = blk, =20 void qmp_migrate_cancel(Error **errp) { - migrate_fd_cancel(migrate_get_current()); + migration_cancel(); } =20 void qmp_migrate_continue(MigrationStatus state, Error **errp) diff --git a/migration/migration.h b/migration/migration.h index 507284e563..69d995a4b1 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -346,5 +346,6 @@ int foreach_not_ignored_block(RAMBlockIterFunc func, vo= id *opaque); void migration_make_urgent_request(void); void migration_consume_urgent_request(void); bool migration_rate_limit(void); +void migration_cancel(void); =20 #endif diff --git a/migration/ram.c b/migration/ram.c index 04f13feb2e..d587ecbd92 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -52,6 +52,7 @@ #include "migration/colo.h" #include "block.h" #include "sysemu/sysemu.h" +#include "sysemu/runstate.h" #include "savevm.h" #include "qemu/iov.h" #include "multifd.h" @@ -3752,8 +3753,38 @@ static SaveVMHandlers savevm_ram_handlers =3D { .resume_prepare =3D ram_resume_prepare, }; =20 +static void ram_mig_ram_block_resized(RAMBlockNotifier *n, void *host, + size_t old_size, size_t new_size) +{ + ram_addr_t offset; + Error *err =3D NULL; + RAMBlock *rb =3D qemu_ram_block_from_host(host, false, &offset); + + if (ramblock_is_ignored(rb)) { + return; + } + + if (!migration_is_idle()) { + /* + * Precopy code on the source cannot deal with the size of RAM blo= cks + * changing at random points in time - especially after sending the + * RAM block sizes to the migration stream, they must no longer ch= ange. + * Abort and indicate a proper reason. + */ + error_setg(&err, "RAM block '%s' resized during precopy.", rb->ids= tr); + migrate_set_error(migrate_get_current(), err); + error_free(err); + migration_cancel(); + } +} + +static RAMBlockNotifier ram_mig_ram_notifier =3D { + .ram_block_resized =3D ram_mig_ram_block_resized, +}; + void ram_mig_init(void) { qemu_mutex_init(&XBZRLE.lock); register_savevm_live("ram", 0, 4, &savevm_ram_handlers, &ram_state); + ram_block_notifier_add(&ram_mig_ram_notifier); } --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459579; cv=none; d=zohomail.com; s=zohoarc; b=kkYl+iadU1H+KxJnm/qelHmoErgAoBKaxXd2l3qSnxIH7KhNHs0rbSJnGXIsoKUsbHJCOdTUJv2ytHsJ6Cu31CkYukdhAJpDXxielCMEy4NtMLPtgUCPdrV5NiDCmAP/GQTOYKuDx4gDuTGzfRmS0+o9WS0Em6yGZOPpVBIDEdw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459579; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5O9IOf18Q5tNrCWkix6XYufNCtn6qQgs8x9W8pVhcUE=; b=C8LLtmtLxudNuESNkjLm0oW/EkLCJ2cNf9AzbG3Ty34mwGeYF1r4lptJPYDWBdfz/EHMwk74zTieVXAbpBdSNM0PCgk5Uyf+ZKlk8hf4jS92SRk2k3+CzyfEiZCF3/bhustqxb0+4Qy/TTH35h0Z6DcRzvTeR/qm3IezVMyz+1s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459579975669.3726593717197; Tue, 21 Apr 2020 01:59:39 -0700 (PDT) Received: from localhost ([::1]:54148 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQokj-0006ti-Mv for importer@patchew.org; Tue, 21 Apr 2020 04:59:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57284) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofX-0007jj-S5 for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQofX-000141-9A for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:15 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:44822 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQofW-00012T-TZ for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:14 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-381-Ts3c5l0TND6KWfg2ZNNJDQ-1; Tue, 21 Apr 2020 04:54:07 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2099B8017FC; Tue, 21 Apr 2020 08:54:06 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id B421D1001DDE; Tue, 21 Apr 2020 08:54:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459253; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5O9IOf18Q5tNrCWkix6XYufNCtn6qQgs8x9W8pVhcUE=; b=e3H21JNO3kk8boxKJePyz6T90YP7j5v/mbz1QLds+S/NYpmemgyEz+c0wpYVEl929ULcJH InvxF3+BOhtk+ErrnHSphjrPzikh+p8n2A2O9pXHdgNxEaTEnLQ8Ng3ljdCbt5IWuunNB5 3Lz5u72VC2g52OUvEfEwhdF3FjyJd4k= X-MC-Unique: Ts3c5l0TND6KWfg2ZNNJDQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 06/13] exec: Relax range check in ram_block_discard_range() Date: Tue, 21 Apr 2020 10:52:53 +0200 Message-Id: <20200421085300.7734-7-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=david@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 01:28:51 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We want to make use of ram_block_discard_range() in the RAM block resize callback when growing a RAM block, *before* used_length is changed. Let's relax the check. We always have a reserved mapping for the whole max_length, so we cannot corrupt unrelated data. Reviewed-by: Peter Xu Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Peter Xu Signed-off-by: David Hildenbrand --- exec.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/exec.c b/exec.c index 78dfab6ec6..591c3b0e32 100644 --- a/exec.c +++ b/exec.c @@ -3887,7 +3887,7 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t st= art, size_t length) goto err; } =20 - if ((start + length) <=3D rb->used_length) { + if ((start + length) <=3D rb->max_length) { bool need_madvise, need_fallocate; if (!QEMU_IS_ALIGNED(length, rb->page_size)) { error_report("ram_block_discard_range: Unaligned length: %zx", @@ -3954,7 +3954,7 @@ int ram_block_discard_range(RAMBlock *rb, uint64_t st= art, size_t length) } else { error_report("ram_block_discard_range: Overrun block '%s' (%" PRIu= 64 "/%zx/" RAM_ADDR_FMT")", - rb->idstr, start, length, rb->used_length); + rb->idstr, start, length, rb->max_length); } =20 err: --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459534; cv=none; d=zohomail.com; s=zohoarc; b=WRG02cfAdMSlL2/Wqs5akyhcuIolj3FrCi5iwEPPXpx4SYJoFNekDdD+i9RhnUs15yr/UNnp6NON4+hEBcDO5yUs/cf52sOUg8DZqFylj8r2tN5Vx0Hjp4LYGesRwj8kKxthFkUsy28SQpeJoOi0KsHkkR0Yz9AZtDhOzm/C3tg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459534; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PiYdaufXH/OcHgyja+9iT9AeZgtrLODnqhTcI1EwYAg=; b=hYhsdbrk8wa/cnMU86esZWOuK0CavKAxFgbo997ONu0scYIdER7hfEpMaRwMqnFDZdet/xlfx1U+oFPsqHRd0/EyXYtvXiwnSdJH1i0CSbq9XYlA0dy3hne3UtBqRZlVhyjCqmgJ4BxWW7DrIpEDTdwOu9gf1U+aLwQZAzLgyic= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459534649207.08352927465955; Tue, 21 Apr 2020 01:58:54 -0700 (PDT) Received: from localhost ([::1]:54144 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQok1-0005hI-7C for importer@patchew.org; Tue, 21 Apr 2020 04:58:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57310) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofb-0007pX-B7 for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQofa-00019k-PB for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:19 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:44980 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQofa-00018q-CR for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:18 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-65-5B3mA3g4NdOKDmIDStIK1A-1; Tue, 21 Apr 2020 04:54:16 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 33CC9107ACCC; Tue, 21 Apr 2020 08:54:15 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 71064100EBB9; Tue, 21 Apr 2020 08:54:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PiYdaufXH/OcHgyja+9iT9AeZgtrLODnqhTcI1EwYAg=; b=gG+LTR4bX9tgPrdAYc/8B9XrTlsowMs+h0ITKJN4oEh18b1Anks+B47KDz3QSqhUZRpsMT iZ6Q0kmRjAV22bbH18DHs/GxvSMTvJRp61Y5sLklgzElOQVsbup3z3NainkQWI9dp+/he5 0z2D8oxAqAYXED2HVHvJvP6qOhhOzJk= X-MC-Unique: 5B3mA3g4NdOKDmIDStIK1A-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 07/13] migration/ram: Discard RAM when growing RAM blocks after ram_postcopy_incoming_init() Date: Tue, 21 Apr 2020 10:52:54 +0200 Message-Id: <20200421085300.7734-8-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=david@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 04:54:00 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" In case we grow our RAM after ram_postcopy_incoming_init() (e.g., when synchronizing the RAM block state with the migration source), the resized part would not get discarded. Let's perform that when being notified about a resize while postcopy has been advised, but is not listening yet. With precopy, the process is as following: 1. VM created - RAM blocks are created 2. Incomming migration started - Postcopy is advised - All pages in RAM blocks are discarded 3. Precopy starts - RAM blocks are resized to match the size on the migration source. - RAM pages from precopy stream are loaded - Uffd handler is registered, postcopy starts listening 4. Guest started, postcopy running - Pagefaults get resolved, pages get placed Reviewed-by: Peter Xu Cc: "Dr. David Alan Gilbert" Cc: Juan Quintela Cc: Peter Xu Signed-off-by: David Hildenbrand --- migration/ram.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index d587ecbd92..2704d32def 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3756,6 +3756,7 @@ static SaveVMHandlers savevm_ram_handlers =3D { static void ram_mig_ram_block_resized(RAMBlockNotifier *n, void *host, size_t old_size, size_t new_size) { + PostcopyState ps =3D postcopy_state_get(); ram_addr_t offset; Error *err =3D NULL; RAMBlock *rb =3D qemu_ram_block_from_host(host, false, &offset); @@ -3776,6 +3777,35 @@ static void ram_mig_ram_block_resized(RAMBlockNotifi= er *n, void *host, error_free(err); migration_cancel(); } + + switch (ps) { + case POSTCOPY_INCOMING_ADVISE: + /* + * Update what ram_postcopy_incoming_init()->init_range() does at = the + * time postcopy was advised. Syncing RAM blocks with the source w= ill + * result in RAM resizes. + */ + if (old_size < new_size) { + if (ram_discard_range(rb->idstr, old_size, new_size - old_size= )) { + error_report("RAM block '%s' discard of resized RAM failed= ", + rb->idstr); + } + } + break; + case POSTCOPY_INCOMING_NONE: + case POSTCOPY_INCOMING_RUNNING: + case POSTCOPY_INCOMING_END: + /* + * Once our guest is running, postcopy does no longer care about + * resizes. When growing, the new memory was not available on the + * source, no handler needed. + */ + break; + default: + error_report("RAM block '%s' resized during postcopy state: %d", + rb->idstr, ps); + exit(-1); + } } =20 static RAMBlockNotifier ram_mig_ram_notifier =3D { --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459686; cv=none; d=zohomail.com; s=zohoarc; b=DSt8QLZSbBESwoxGG85uqZl7rf29OwHqS/wdk7BYgcdIwkeE1CVqZBsdWmkNcYHJoQ/9XSKPkr1gFXLr8X8mQ7jujRXQzyqz82qZlH4n7Xq/TEGXgJC0jVSIpb+zB3r2u57kwhCDNITtsW/muYOmy64dCh4dVhfx3PjUgx5hDC0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459686; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QWNorBYfar3WMb9elUru/Lr93DUggyUyRqR6DMBbnEc=; b=eurPPzbr6TmeNkRuBu+9ujzPBViDOMR/LXQ0cIOCms5oG2h2sqWeOQKfv+sZrh16KBN9gnwuxzlY9bEEU1MCUo95q53U0wZQuyAGOrxkah+eZLSPMx2lLlTQpcSiGDCf/BmzfN9gX7b+ojPj5M7DnkcPk1sT66CGw3+gxNaCjXo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459686572703.1037598344276; Tue, 21 Apr 2020 02:01:26 -0700 (PDT) Received: from localhost ([::1]:54190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQomT-0001Id-80 for importer@patchew.org; Tue, 21 Apr 2020 05:01:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57362) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofk-0007wt-1s for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQofi-0001JF-1J for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:27 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:32099 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQofh-0001Ee-8C for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:25 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-101-NZd9qpQsMIuspO-QoxqIFw-1; Tue, 21 Apr 2020 04:54:20 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3BF1E802560; Tue, 21 Apr 2020 08:54:19 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 82FDB1001B30; Tue, 21 Apr 2020 08:54:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459262; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QWNorBYfar3WMb9elUru/Lr93DUggyUyRqR6DMBbnEc=; b=UDyGBDUVP6wV86Ft82lGbHXeh0vBJEIO3AC/1HGcu/e9KKPVJG539f7sDx9nBQZSNB3k5Q 7bZY0hx90Mulsssw/J/uJA6JWL2lQjSUhYduyqeg3sO+Sy9DMR6Uz/SqVJgWatQyd0EhgB tXh8qE+YVaywdGI/Wu7S5m6u5hPr0Is= X-MC-Unique: NZd9qpQsMIuspO-QoxqIFw-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 08/13] migration/ram: Simplify host page handling in ram_load_postcopy() Date: Tue, 21 Apr 2020 10:52:55 +0200 Message-Id: <20200421085300.7734-9-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=david@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 04:54:00 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Add two new helper functions. This will in come handy once we want to handle ram block resizes while postcopy is active. Note that ram_block_from_stream() will already print proper errors. Reviewed-by: Dr. David Alan Gilbert Cc: "Dr. David Alan Gilbert" Cc: Juan Quintela Cc: Peter Xu Signed-off-by: David Hildenbrand --- migration/ram.c | 54 ++++++++++++++++++++++++++++--------------------- 1 file changed, 31 insertions(+), 23 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 2704d32def..2a2165b478 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2743,6 +2743,20 @@ static inline void *host_from_ram_block_offset(RAMBl= ock *block, return block->host + offset; } =20 +static void *host_page_from_ram_block_offset(RAMBlock *block, + ram_addr_t offset) +{ + /* Note: Explicitly no check against offset_in_ramblock(). */ + return (void *)QEMU_ALIGN_DOWN((uintptr_t)block->host + offset, + block->page_size); +} + +static ram_addr_t host_page_offset_from_ram_block_offset(RAMBlock *block, + ram_addr_t offset) +{ + return ((uintptr_t)block->host + offset) & (block->page_size - 1); +} + static inline void *colo_cache_from_block_offset(RAMBlock *block, ram_addr_t offset, bool record_bitmap) { @@ -3135,13 +3149,12 @@ static int ram_load_postcopy(QEMUFile *f) MigrationIncomingState *mis =3D migration_incoming_get_current(); /* Temporary page that is later 'placed' */ void *postcopy_host_page =3D mis->postcopy_tmp_page; - void *this_host =3D NULL; + void *host_page =3D NULL; bool all_zero =3D false; int target_pages =3D 0; =20 while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { ram_addr_t addr; - void *host =3D NULL; void *page_buffer =3D NULL; void *place_source =3D NULL; RAMBlock *block =3D NULL; @@ -3167,9 +3180,12 @@ static int ram_load_postcopy(QEMUFile *f) if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE)) { block =3D ram_block_from_stream(f, flags); + if (!block) { + ret =3D -EINVAL; + break; + } =20 - host =3D host_from_ram_block_offset(block, addr); - if (!host) { + if (!offset_in_ramblock(block, addr)) { error_report("Illegal RAM offset " RAM_ADDR_FMT, addr); ret =3D -EINVAL; break; @@ -3187,21 +3203,18 @@ static int ram_load_postcopy(QEMUFile *f) * of a host page in one chunk. */ page_buffer =3D postcopy_host_page + - ((uintptr_t)host & (block->page_size - 1)); + host_page_offset_from_ram_block_offset(block, ad= dr); /* If all TP are zero then we can optimise the place */ if (target_pages =3D=3D 1) { all_zero =3D true; - this_host =3D (void *)QEMU_ALIGN_DOWN((uintptr_t)host, - block->page_size); - } else { + host_page =3D host_page_from_ram_block_offset(block, addr); + } else if (host_page !=3D host_page_from_ram_block_offset(bloc= k, + addr))= { /* not the 1st TP within the HP */ - if (QEMU_ALIGN_DOWN((uintptr_t)host, block->page_size) != =3D - (uintptr_t)this_host) { - error_report("Non-same host page %p/%p", - host, this_host); - ret =3D -EINVAL; - break; - } + error_report("Non-same host page %p/%p", host_page, + host_page_from_ram_block_offset(block, addr)); + ret =3D -EINVAL; + break; } =20 /* @@ -3281,16 +3294,11 @@ static int ram_load_postcopy(QEMUFile *f) } =20 if (!ret && place_needed) { - /* This gets called at the last target page in the host page */ - void *place_dest =3D (void *)QEMU_ALIGN_DOWN((uintptr_t)host, - block->page_size); - if (all_zero) { - ret =3D postcopy_place_page_zero(mis, place_dest, - block); + ret =3D postcopy_place_page_zero(mis, host_page, block); } else { - ret =3D postcopy_place_page(mis, place_dest, - place_source, block); + ret =3D postcopy_place_page(mis, host_page, place_source, + block); } } } --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459779; cv=none; d=zohomail.com; s=zohoarc; b=jdFMvGkxmMz7DK4vJWCakWZisTXGLSDh7v/M94WHrucd1dXB6RSZfokuX4VMBNLaz5bknZ8gQXF0m4poi9UpRQgfLbGLFHpTgiM1o+kXwCt+zuvy+gzQaQLDJYb3uSkkoGHqHOsxs5XmH/kcf7HWr/gS0T3qdjR490qDO1IsyjY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459779; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ek2o7jkqrCOq3tnDF+D8qeQ7NVH7+0m81v+G2i6NfUc=; b=JbnuFm2Y0vHM/m0CSHHIjLPFQxseQ+ZO0LE9kWixGNFbruhw+dPDoaBZUkEvpHZ9CB1YE+8oToGhmcL+GB4D9IXY1E+SIE/sJiv9kK2SbZTB8oIWDynXJ4VvmertsU01295DfZPkqhGYIiBk1y/MlJcds0RIqjQq7GTTD+mlbe8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459779883746.31443488271; Tue, 21 Apr 2020 02:02:59 -0700 (PDT) Received: from localhost ([::1]:54226 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQony-0003CP-Ke for importer@patchew.org; Tue, 21 Apr 2020 05:02:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57372) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofl-0007xP-OU for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQofk-0001M2-9E for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:29 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:21030 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQofj-0001KI-Qj for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-11-UuzGtCelO5upyQeRtMmEEQ-1; Tue, 21 Apr 2020 04:54:25 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3714E802562; Tue, 21 Apr 2020 08:54:24 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8BA431001B30; Tue, 21 Apr 2020 08:54:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ek2o7jkqrCOq3tnDF+D8qeQ7NVH7+0m81v+G2i6NfUc=; b=Oz4m39nKJIHaENGHNX8xgcHjwKmuPQ6Ayl+QPjuZ61eOHTd/Y312Aw7qpGoLhX/IRn+YEf 7KB1LEG9NPWpZX1iJYqZ81wwPnvOVxGaAOZUcp8WRMMacnsDoFmslq+ZOu2OeM+k8O/2UT DvEamkt5fdIi+pLIAKfBA3xxdYjenlM= X-MC-Unique: UuzGtCelO5upyQeRtMmEEQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 09/13] migration/ram: Consolidate variable reset after placement in ram_load_postcopy() Date: Tue, 21 Apr 2020 10:52:56 +0200 Message-Id: <20200421085300.7734-10-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=david@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 04:54:00 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Let's consolidate resetting the variables. Cc: "Dr. David Alan Gilbert" Cc: Juan Quintela Cc: Peter Xu Signed-off-by: David Hildenbrand Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 2a2165b478..7eca3165c8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3150,7 +3150,7 @@ static int ram_load_postcopy(QEMUFile *f) /* Temporary page that is later 'placed' */ void *postcopy_host_page =3D mis->postcopy_tmp_page; void *host_page =3D NULL; - bool all_zero =3D false; + bool all_zero =3D true; int target_pages =3D 0; =20 while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { @@ -3176,7 +3176,6 @@ static int ram_load_postcopy(QEMUFile *f) addr &=3D TARGET_PAGE_MASK; =20 trace_ram_load_postcopy_loop((uint64_t)addr, flags); - place_needed =3D false; if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE)) { block =3D ram_block_from_stream(f, flags); @@ -3204,9 +3203,7 @@ static int ram_load_postcopy(QEMUFile *f) */ page_buffer =3D postcopy_host_page + host_page_offset_from_ram_block_offset(block, ad= dr); - /* If all TP are zero then we can optimise the place */ if (target_pages =3D=3D 1) { - all_zero =3D true; host_page =3D host_page_from_ram_block_offset(block, addr); } else if (host_page !=3D host_page_from_ram_block_offset(bloc= k, addr))= { @@ -3223,7 +3220,6 @@ static int ram_load_postcopy(QEMUFile *f) */ if (target_pages =3D=3D (block->page_size / TARGET_PAGE_SIZE))= { place_needed =3D true; - target_pages =3D 0; } place_source =3D postcopy_host_page; } @@ -3300,6 +3296,10 @@ static int ram_load_postcopy(QEMUFile *f) ret =3D postcopy_place_page(mis, host_page, place_source, block); } + place_needed =3D false; + target_pages =3D 0; + /* Assume we have a zero page until we detect something differ= ent */ + all_zero =3D true; } } =20 --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459894; cv=none; d=zohomail.com; s=zohoarc; b=DsRsLyUs7d80MmmQN+2yotUORLMujYtjzdWBvdeRj7IU4dBqMUAEDAV927+Cdd7SJvmXIvh8GkV2TNotlldXqqJHZJ1G/NX3NdxSXjRsVxiTvJBS1uaT7aYb+QeBWBlQCXr1dYVs3elvkP7feqPq8SQJ+qIjPXXpcjD30ufzBCs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459894; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=J9A5ZoTaU0j1lTzJPOgWHNA44PHHOKwC0K9ma1Qy9y8=; b=f98igS9VQr2qyqQeM9z6yM81uKL3ajMoK9LA1BGnDfmVy6CWTVgzziCIvRnl1tP05Bmnvopco+qP4PsXT5Ie6ugU3v6bOQDO9qSOwbyGOwgtz24pdmS4cuxK836vyFpQIG6kIa+Pln48bGUjLxhYgBfGX+oFr6RlVDJTWCCj5i0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459894166770.0012856073876; Tue, 21 Apr 2020 02:04:54 -0700 (PDT) Received: from localhost ([::1]:54246 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQopo-00053e-He for importer@patchew.org; Tue, 21 Apr 2020 05:04:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57406) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofr-00081r-Cw for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQofn-0001SV-FN for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:35 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:51130 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQofn-0001QY-2r for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:31 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-337-fHfnHR-uOMSlolH7VsE3aw-1; Tue, 21 Apr 2020 04:54:28 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 500A413FB; Tue, 21 Apr 2020 08:54:27 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 88E041001B30; Tue, 21 Apr 2020 08:54:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459270; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J9A5ZoTaU0j1lTzJPOgWHNA44PHHOKwC0K9ma1Qy9y8=; b=CQGhlJ5upbLIPBhlWy5dPuazIHSM/jd0w6K8QWEaBAOpnZ44ULygV9lpNp/aB42nQEhVJd eKRF6ncFiPTydvXkZO8Hmeaz+lSJFp43t+IQLfT3X+kvccFwn9MBlKMb+q21KZv02bHtk9 3t0TJ7UsVBPlFrrgrp0GjrTfBcMGnXc= X-MC-Unique: fHfnHR-uOMSlolH7VsE3aw-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 10/13] migration/ram: Handle RAM block resizes during postcopy Date: Tue, 21 Apr 2020 10:52:57 +0200 Message-Id: <20200421085300.7734-11-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=david@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 01:28:51 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , Richard Henderson , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Shannon Zhao , Igor Mammedov , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Resizing while migrating is dangerous and does not work as expected. The whole migration code works on the usable_length of ram blocks and does not expect this to change at random points in time. In the case of postcopy, relying on used_length is racy as soon as the guest is running. Also, when used_length changes we might leave the uffd handler registered for some memory regions, reject valid pages when migrating and fail when sending the recv bitmap to the source. Resizing can be trigger *after* (but not during) a reset in ACPI code by the guest - hw/arm/virt-acpi-build.c:acpi_ram_update() - hw/i386/acpi-build.c:acpi_ram_update() Let's remember the original used_length in a separate variable and use it in relevant postcopy code. Make sure to update it when we resize during precopy, when synchronizing the RAM block sizes with the source. Reviewed-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Cc: "Dr. David Alan Gilbert" Cc: Juan Quintela Cc: Eduardo Habkost Cc: Paolo Bonzini Cc: Igor Mammedov Cc: "Michael S. Tsirkin" Cc: Richard Henderson Cc: Shannon Zhao Cc: Alex Benn=C3=A9e Cc: Peter Xu Signed-off-by: David Hildenbrand --- include/exec/ramblock.h | 10 ++++++++++ migration/postcopy-ram.c | 15 ++++++++++++--- migration/ram.c | 11 +++++++++-- 3 files changed, 31 insertions(+), 5 deletions(-) diff --git a/include/exec/ramblock.h b/include/exec/ramblock.h index 07d50864d8..664701b759 100644 --- a/include/exec/ramblock.h +++ b/include/exec/ramblock.h @@ -59,6 +59,16 @@ struct RAMBlock { */ unsigned long *clear_bmap; uint8_t clear_bmap_shift; + + /* + * RAM block length that corresponds to the used_length on the migrati= on + * source (after RAM block sizes were synchronized). Especially, after + * starting to run the guest, used_length and postcopy_length can diff= er. + * Used to register/unregister uffd handlers and as the size of the re= ceived + * bitmap. Receiving any page beyond this length will bail out, as it + * could not have been valid on the source. + */ + ram_addr_t postcopy_length; }; #endif #endif diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index a36402722b..c68caf4e42 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -17,6 +17,7 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/rcu.h" #include "exec/target_page.h" #include "migration.h" #include "qemu-file.h" @@ -31,6 +32,7 @@ #include "qemu/error-report.h" #include "trace.h" #include "hw/boards.h" +#include "exec/ramblock.h" =20 /* Arbitrary limit on size of each discard command, * keeps them around ~200 bytes @@ -456,6 +458,13 @@ static int init_range(RAMBlock *rb, void *opaque) ram_addr_t length =3D qemu_ram_get_used_length(rb); trace_postcopy_init_range(block_name, host_addr, offset, length); =20 + /* + * Save the used_length before running the guest. In case we have to + * resize RAM blocks when syncing RAM block sizes from the source duri= ng + * precopy, we'll update it manually via the ram block notifier. + */ + rb->postcopy_length =3D length; + /* * We need the whole of RAM to be truly empty for postcopy, so things * like ROMs and any data tables built during init must be zero'd @@ -478,7 +487,7 @@ static int cleanup_range(RAMBlock *rb, void *opaque) const char *block_name =3D qemu_ram_get_idstr(rb); void *host_addr =3D qemu_ram_get_host_addr(rb); ram_addr_t offset =3D qemu_ram_get_offset(rb); - ram_addr_t length =3D qemu_ram_get_used_length(rb); + ram_addr_t length =3D rb->postcopy_length; MigrationIncomingState *mis =3D opaque; struct uffdio_range range_struct; trace_postcopy_cleanup_range(block_name, host_addr, offset, length); @@ -600,7 +609,7 @@ static int nhp_range(RAMBlock *rb, void *opaque) const char *block_name =3D qemu_ram_get_idstr(rb); void *host_addr =3D qemu_ram_get_host_addr(rb); ram_addr_t offset =3D qemu_ram_get_offset(rb); - ram_addr_t length =3D qemu_ram_get_used_length(rb); + ram_addr_t length =3D rb->postcopy_length; trace_postcopy_nhp_range(block_name, host_addr, offset, length); =20 /* @@ -644,7 +653,7 @@ static int ram_block_enable_notify(RAMBlock *rb, void *= opaque) struct uffdio_register reg_struct; =20 reg_struct.range.start =3D (uintptr_t)qemu_ram_get_host_addr(rb); - reg_struct.range.len =3D qemu_ram_get_used_length(rb); + reg_struct.range.len =3D rb->postcopy_length; reg_struct.mode =3D UFFDIO_REGISTER_MODE_MISSING; =20 /* Now tell our userfault_fd that it's responsible for this area */ diff --git a/migration/ram.c b/migration/ram.c index 7eca3165c8..f03bc84692 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -244,7 +244,7 @@ int64_t ramblock_recv_bitmap_send(QEMUFile *file, return -1; } =20 - nbits =3D block->used_length >> TARGET_PAGE_BITS; + nbits =3D block->postcopy_length >> TARGET_PAGE_BITS; =20 /* * Make sure the tmp bitmap buffer is big enough, e.g., on 32bit @@ -3184,7 +3184,13 @@ static int ram_load_postcopy(QEMUFile *f) break; } =20 - if (!offset_in_ramblock(block, addr)) { + /* + * Relying on used_length is racy and can result in false posi= tives. + * We might place pages beyond used_length in case RAM was shr= unk + * while in postcopy, which is fine - trying to place via + * UFFDIO_COPY/UFFDIO_ZEROPAGE will never segfault. + */ + if (!block->host || addr >=3D block->postcopy_length) { error_report("Illegal RAM offset " RAM_ADDR_FMT, addr); ret =3D -EINVAL; break; @@ -3799,6 +3805,7 @@ static void ram_mig_ram_block_resized(RAMBlockNotifie= r *n, void *host, rb->idstr); } } + rb->postcopy_length =3D new_size; break; case POSTCOPY_INCOMING_NONE: case POSTCOPY_INCOMING_RUNNING: --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459551; cv=none; d=zohomail.com; s=zohoarc; b=Lsv7NDw5hvc2v6JjWnQNbFkSqG8HQiXJMdeh22CobVwCobarL15BXQ8yMHbdvm/uxXjORItOPVkyPeoB8w66rWMkiY0CWytO7XSCuGrsLxQ1aYw+KvvV8+5vFQakGwzJuY9kyNcKCeMReXZyKFIosyAu89SdGC2+NKqX/f4B11k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459551; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=6SjjE1hP+kC4ESv3vq5BAN1iuJe4xS50e87aSvh0BdU=; b=UzTe9UjbRVmSYTWU7HK1Mjv1o6DgSQ0im62kXn8y7TIBSDScuZT68+C983mKGxHf1JSe27SPanEi/KjDtpkjYB7PC4Y3vnrbfqmyI/Dh1h4tiZtBwv6n+PfzSADVeLLL29Jl1V6sKSXhKtDgZaTspz/xnW+jn4L+p+EQsfesj6I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459551484688.7275534584372; Tue, 21 Apr 2020 01:59:11 -0700 (PDT) Received: from localhost ([::1]:54146 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQokI-0006AP-Av for importer@patchew.org; Tue, 21 Apr 2020 04:59:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57426) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQofw-0008AR-9v for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQofv-0001hR-SX for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:40 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:20553 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQofv-0001g1-Gp for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:39 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-245-BPPl0aU-Nea36dh5mCMAFg-1; Tue, 21 Apr 2020 04:54:37 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2016B8017FD; Tue, 21 Apr 2020 08:54:36 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9CDA81036D27; Tue, 21 Apr 2020 08:54:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6SjjE1hP+kC4ESv3vq5BAN1iuJe4xS50e87aSvh0BdU=; b=gK+4LDJOGJFenr1w2Md0Si2vkkg+zSNvEqtg8uH5CLae57cuq1eQZnpbpt9ulg/ksK7k0I fRyuxAkFnawm/yQ0n2JBFmiIDiYcRoEtEXS/9yzaDXZ1NVMFDnSJp5bPP3LX7h7qxw88NM YF1VHgREeOc+72WBIYhOAatg5rlL67w= X-MC-Unique: BPPl0aU-Nea36dh5mCMAFg-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 11/13] migration/multifd: Print used_length of memory block Date: Tue, 21 Apr 2020 10:52:58 +0200 Message-Id: <20200421085300.7734-12-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=david@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 04:54:00 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We actually want to print the used_length, against which we check. Reviewed-by: Dr. David Alan Gilbert Cc: "Dr. David Alan Gilbert" Cc: Juan Quintela Cc: Peter Xu Signed-off-by: David Hildenbrand --- migration/multifd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/multifd.c b/migration/multifd.c index cb6a4a3ab8..4f65c871a9 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -356,7 +356,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams= *p, Error **errp) if (offset > (block->used_length - qemu_target_page_size())) { error_setg(errp, "multifd: offset too long %" PRIu64 " (max " RAM_ADDR_FMT ")", - offset, block->max_length); + offset, block->used_length); return -1; } p->pages->iov[i].iov_base =3D block->host + offset; --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459658; cv=none; d=zohomail.com; s=zohoarc; b=ZJCflDLI9R5WIHvgUq3Ml9wmpjIPin9MWHtqoirL2lNqj0HcmgMfnX115fnvg3Eb2LfHrNUCwWl4nMRKgBgKWBRDaWTjyjLKEqoP92FFG8f7u8qGqrKQb1ClvxSKR/hCmyGZ8ufPWmAwKBJB1gXal794RM2TVj7N29XnBvSSOso= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459658; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UFjeKFlJklAtpOcDzb7yTpLlFHzQoOTiWe6EZYuztUg=; b=cozerIABhhN9vsPJdHtLoKvnic8s0Y+kDWiPYWf1VpaXXMVcr60dVGzdZLSHnFHO248dhbGaFZJlAqVapA+h39HAo/jqG+BaI+G8H1ATQ0SC4gCDg7tPlWnBnDu7W4ibvHyrSW6HCSiu8F7B9Uf4xMTk2D+JVTSGeC8qqJgHTjc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459658750344.82331515858687; Tue, 21 Apr 2020 02:00:58 -0700 (PDT) Received: from localhost ([::1]:54180 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQom1-0000VH-25 for importer@patchew.org; Tue, 21 Apr 2020 05:00:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57450) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQog0-0008Dd-Bp for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQofy-0001ov-Om for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:44 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:37110 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQofy-0001n1-CB for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:42 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-486-FfDCX3zTOR-bFjIVI02klg-1; Tue, 21 Apr 2020 04:54:39 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 82EAC1084437; Tue, 21 Apr 2020 08:54:38 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B90C1001B30; Tue, 21 Apr 2020 08:54:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459281; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UFjeKFlJklAtpOcDzb7yTpLlFHzQoOTiWe6EZYuztUg=; b=YriV46qIJNAofEP913YDe1Lhi7xzCor+NxYCEUyM5QPoz119pui0DHQ+3Pd7S/40nNHBhp eM8oFZfVFidsuFetv8fO4eKfPO/Il5KK35/OOdacW3Fi0NwnbQyWS5z6hLPagWCmTuYnN3 F2AaHpQXduqOtr8/tJF7U4j+iZicHts= X-MC-Unique: FfDCX3zTOR-bFjIVI02klg-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 12/13] migration/ram: Use offset_in_ramblock() in range checks Date: Tue, 21 Apr 2020 10:52:59 +0200 Message-Id: <20200421085300.7734-13-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=david@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 03:31:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We never read or write beyond the used_length of memory blocks when migrating. Make this clearer by using offset_in_ramblock() consistently. Reviewed-by: Dr. David Alan Gilbert Cc: "Dr. David Alan Gilbert" Cc: Juan Quintela Cc: Peter Xu Signed-off-by: David Hildenbrand --- migration/ram.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index f03bc84692..2a8cd962df 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1318,8 +1318,8 @@ static bool find_dirty_block(RAMState *rs, PageSearch= Status *pss, bool *again) *again =3D false; return false; } - if ((((ram_addr_t)pss->page) << TARGET_PAGE_BITS) - >=3D pss->block->used_length) { + if (!offset_in_ramblock(pss->block, + ((ram_addr_t)pss->page) << TARGET_PAGE_BITS)) { /* Didn't find anything in this RAM Block */ pss->page =3D 0; pss->block =3D QLIST_NEXT_RCU(pss->block, next); @@ -1523,7 +1523,7 @@ int ram_save_queue_pages(const char *rbname, ram_addr= _t start, ram_addr_t len) rs->last_req_rb =3D ramblock; } trace_ram_save_queue_pages(ramblock->idstr, start, len); - if (start+len > ramblock->used_length) { + if (!offset_in_ramblock(ramblock, start + len - 1)) { error_report("%s request overrun start=3D" RAM_ADDR_FMT " len=3D" RAM_ADDR_FMT " blocklen=3D" RAM_ADDR_FMT, __func__, start, len, ramblock->used_length); @@ -3349,8 +3349,8 @@ static void colo_flush_ram_cache(void) while (block) { offset =3D migration_bitmap_find_dirty(ram_state, block, offse= t); =20 - if (((ram_addr_t)offset) << TARGET_PAGE_BITS - >=3D block->used_length) { + if (!offset_in_ramblock(block, + ((ram_addr_t)offset) << TARGET_PAGE_BI= TS)) { offset =3D 0; block =3D QLIST_NEXT_RCU(block, next); } else { --=20 2.25.1 From nobody Sat May 18 02:20:45 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1587459636; cv=none; d=zohomail.com; s=zohoarc; b=NVQ1FkMI9dhQQ3LqnBlgWgspqghAIAjDqefDVxuNlZdX2WvyuHAsQ+ApfE+yW6wNiJKbl3REDtoVTG6AOBxH3Mwdx5bbBIFlOIxcrzGds9E2ZtNGa4oB87udUZp3oC74nmGPRQ8Sr5r42GksbN3EGyoq18KpYdC8WVQHYlHaFqs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1587459636; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0Molmob3j2KDeb6RSfU0OQ3FgemMU/laKc44xE63wEM=; b=anhjSQEKU9QFGMW6xXeIU1VgTm68JpH5jiX0duhs6T61EmZZ35JKBig5G63k3+O7HKTt4e4COxNX9zToED7vx4GC8TnFAY9OhGwcDO/8ctZWXWbK5Ls0Wx7r7wOj6aVtisJD7iVNI5Vfzo4RoQS4HXbgWsRaw70jtQZbzBWbCAI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1587459636734736.8980455326116; Tue, 21 Apr 2020 02:00:36 -0700 (PDT) Received: from localhost ([::1]:54172 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQole-0008TE-0r for importer@patchew.org; Tue, 21 Apr 2020 05:00:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57488) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jQog8-0008In-0v for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jQog6-00025A-8t for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:51 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:46381 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jQog4-0001yL-4w for qemu-devel@nongnu.org; Tue, 21 Apr 2020 04:54:48 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-258-5CLb4CiWM1yXRyBwlZcgLQ-1; Tue, 21 Apr 2020 04:54:44 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A2A7F107ACC7; Tue, 21 Apr 2020 08:54:43 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-245.ams2.redhat.com [10.36.113.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD32C1001B30; Tue, 21 Apr 2020 08:54:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1587459286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0Molmob3j2KDeb6RSfU0OQ3FgemMU/laKc44xE63wEM=; b=gGR7i4H9/BAHHcgf0i4o1nAavtw6mv5Zv2S9yFvRNJmzt/OCfZ1XtA56TuuH3agzfD7VyQ 6Vfcs1q8MA1bnQhBhxdXBnbWxbTaS3mtfiL2khfaktPQGN/l2kp4M66RfxaQh1jMLFJtM4 YNDT1nVLWTf1OFh4l0GtzuhlVlpMTDo= X-MC-Unique: 5CLb4CiWM1yXRyBwlZcgLQ-1 From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v4 13/13] migration/ram: Tolerate partially changed mappings in postcopy code Date: Tue, 21 Apr 2020 10:53:00 +0200 Message-Id: <20200421085300.7734-14-david@redhat.com> In-Reply-To: <20200421085300.7734-1-david@redhat.com> References: <20200421085300.7734-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=david@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/21 03:31:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrea Arcangeli , Eduardo Habkost , Juan Quintela , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" When we partially change mappings (esp., mmap over parts of an existing mmap like qemu_ram_remap() does) where we have a userfaultfd handler registered, the handler will implicitly be unregistered from the parts that changed. Trying to place pages onto mappings where there is no longer a handler registered will fail. Let's make sure that any waiter is woken up - we have to do that manually. Let's also document how UFFDIO_UNREGISTER will handle this scenario. This is mainly a preparation for RAM blocks with resizable allcoations, where the mapping of the invalid RAM range will change. The source will keep sending pages that are outside of the new (shrunk) RAM size. We have to treat these pages like they would have been migrated, but can essentially simply drop the content (ignore the placement error). Keep printing a warning when we hit EINVAL, to avoid hiding other (programming) issues. ENOENT is unique. Reviewed-by: Peter Xu Cc: "Dr. David Alan Gilbert" Cc: Juan Quintela Cc: Peter Xu Cc: Andrea Arcangeli Signed-off-by: David Hildenbrand --- migration/postcopy-ram.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index c68caf4e42..f39c6304de 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -506,6 +506,14 @@ static int cleanup_range(RAMBlock *rb, void *opaque) range_struct.start =3D (uintptr_t)host_addr; range_struct.len =3D length; =20 + /* + * In case the mapping was partially changed since we enabled userfault + * (e.g., via qemu_ram_remap()), the userfaultfd handler was already r= emoved + * for the mappings that changed. Unregistering will, however, still w= ork + * and ignore mappings without a registered handler. There could only = be + * an issue if we would suddenly encounter a mapping that's incompatib= le + * with UFFD - which cannot happen within a single RAM block. + */ if (ioctl(mis->userfault_fd, UFFDIO_UNREGISTER, &range_struct)) { error_report("%s: userfault unregister %s", __func__, strerror(err= no)); =20 @@ -1180,6 +1188,17 @@ int postcopy_ram_incoming_setup(MigrationIncomingSta= te *mis) return 0; } =20 +static int qemu_ufd_wake_ioctl(int userfault_fd, void *host_addr, + uint64_t pagesize) +{ + struct uffdio_range range =3D { + .start =3D (uint64_t)(uintptr_t)host_addr, + .len =3D pagesize, + }; + + return ioctl(userfault_fd, UFFDIO_WAKE, &range); +} + static int qemu_ufd_copy_ioctl(int userfault_fd, void *host_addr, void *from_addr, uint64_t pagesize, RAMBloc= k *rb) { @@ -1198,6 +1217,26 @@ static int qemu_ufd_copy_ioctl(int userfault_fd, voi= d *host_addr, zero_struct.mode =3D 0; ret =3D ioctl(userfault_fd, UFFDIO_ZEROPAGE, &zero_struct); } + + /* + * When the mapping gets partially changed (e.g., qemu_ram_remap()) be= fore + * we try to place a page, the userfaultfd handler will be removed for= the + * changed mappings and placing pages will fail. We can safely ignore = this, + * because mappings that changed on the destination don't need data fr= om the + * source (e.g., qemu_ram_remap()). Wake up any waiter waiting for tha= t page + * (unlikely but possible). Waking up waiters is always possible, even + * without a registered userfaultfd handler. + * + * Old kernels report EINVAL, new kernels report ENOENT in case there = is + * no longer a userfaultfd handler for a mapping. + */ + if (ret && (errno =3D=3D ENOENT || errno =3D=3D EINVAL)) { + if (errno =3D=3D EINVAL) { + warn_report("%s: Failed to place page %p. Waking up any waiter= s.", + __func__, host_addr); + } + ret =3D qemu_ufd_wake_ioctl(userfault_fd, host_addr, pagesize); + } if (!ret) { ramblock_recv_bitmap_set_range(rb, host_addr, pagesize / qemu_target_page_size()); --=20 2.25.1