From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731474; cv=none; d=zohomail.com; s=zohoarc; b=CMwTqpAwNpUtZR6k3fMT6E0GNsAhMD9xN3YXl/zyal9qK2d8USbupvzwgEzig5d68YHc7uN0xJXaD33jFxwY131VRQGCcPWPmWvke9WRj3T8vNIEEfUtHrnot7qK2Qn4EWIF2WmrdF2CdaSheJfO9uNtclEAI3xLkQzIpjm/Kx8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731474; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=4tyAthjyAJfKR2wxZiLS098ns/1BTHiWpLuviegbRVQ=; b=EukoKim31NXx74MAl3swJAWthTo5R+a3vhduu2BzhIXrjZd7j8uu9n/Ial72rs+Ysv0if1kDRDzYQIAw4t5valMOxBWNFgHikFGTJkISuilh5ZD19vu8TKoEyCcCcZ+HZLHNcuSBe/zIc/ejOytk2ReYm2qEDoMC2tF8sdujgQw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731474692408.46228782903006; Mon, 6 Feb 2023 16:57:54 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCIL-0002fy-Tc; Mon, 06 Feb 2023 19:57:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIH-0002f3-4g for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIF-0003ZL-H1 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:08 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-655-6Rcrl1TjM66tyeSAaNYYpg-1; Mon, 06 Feb 2023 19:57:02 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BE38A800B24; Tue, 7 Feb 2023 00:57:01 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id AEF9F492C3C; Tue, 7 Feb 2023 00:56:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731426; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4tyAthjyAJfKR2wxZiLS098ns/1BTHiWpLuviegbRVQ=; b=dc9TfAsNAPS/CG1MC9DNUuspR45gIGvejLg2KWdlqfqsx7uzR1FX8s+Y0RdPOT37vrqBZ6 OSW+r8rPvevQ/e52ebY2QCTD+jSxH8EEaE7va8Uc4YsfRwGCm+ma6b9jprgO7j4EnpKQ79 SOC6KKNZPQfHWHlqQc1/c9HDHgi0oF0= X-MC-Unique: 6Rcrl1TjM66tyeSAaNYYpg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu , qemu-stable@nongnu.org Subject: [PULL 01/30] migration: Fix migration crash when target psize larger than host Date: Tue, 7 Feb 2023 01:56:21 +0100 Message-Id: <20230207005650.1810-2-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731476760100005 Content-Type: text/plain; charset="utf-8" From: Peter Xu Commit d9e474ea56 overlooked the case where the target psize is even larger than the host psize. One example is Alpha has 8K page size and migration will start to crash the source QEMU when running Alpha migration on x86. Fix it by detecting that case and set host start/end just to cover the single page to be migrated. This will slightly optimize the common case where host psize equals to guest psize so we don't even need to do the roundups, but that's trivial. Cc: qemu-stable@nongnu.org Reported-by: Thomas Huth Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1456 Fixes: d9e474ea56 ("migration: Teach PSS about host page") Signed-off-by: Peter Xu Reviewed-by: Thomas Huth Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/ram.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 334309f1c6..68a45338e3 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2319,8 +2319,25 @@ static void pss_host_page_prepare(PageSearchStatus *= pss) size_t guest_pfns =3D qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BIT= S; =20 pss->host_page_sending =3D true; - pss->host_page_start =3D ROUND_DOWN(pss->page, guest_pfns); - pss->host_page_end =3D ROUND_UP(pss->page + 1, guest_pfns); + if (guest_pfns <=3D 1) { + /* + * This covers both when guest psize =3D=3D host psize, or when gu= est + * has larger psize than the host (guest_pfns=3D=3D0). + * + * For the latter, we always send one whole guest page per + * iteration of the host page (example: an Alpha VM on x86 host + * will have guest psize 8K while host psize 4K). + */ + pss->host_page_start =3D pss->page; + pss->host_page_end =3D pss->page + 1; + } else { + /* + * The host page spans over multiple guest pages, we send them + * within the same host page iteration. + */ + pss->host_page_start =3D ROUND_DOWN(pss->page, guest_pfns); + pss->host_page_end =3D ROUND_UP(pss->page + 1, guest_pfns); + } } =20 /* --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731547; cv=none; d=zohomail.com; s=zohoarc; b=mnAe2OFgj/6EnfBHRJV4/Tb+b6CgXTEg1mK0zLVVoqsMwM+ZSMdLJXsbINYdDpmqgoYUGB0VmNW6fMMGpuzA4tRokrlqSn911A5fH1ltYatzlq+xsJGJw+d8dVDOABLDltGXc6MJt854ZaJ8GEUNwzs9rZ1SEokIxSge7ROuoRA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731547; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZH6YddLEeOQzAr9SjLkB0f5BGyw0/sKDCslYwuvDfrs=; b=DB8Jk598Jz8ZZg/pvvc/USprAAeqZCHqLhPRns9iwqFYfsAnzGLEAaAR/93WswSFT0I7gMcQq8HEvDpI04mSTbAVdzNlCdjq1oyUm+eYMshfz+mD734Ntyd8Uv8Ru6wVzbeQojhOlxGzPj1u3Mo2x5zo9Krc1hOnzphVFHDv/lM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731547485337.65819152176243; Mon, 6 Feb 2023 16:59:07 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCIb-0002kO-PU; Mon, 06 Feb 2023 19:57:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIa-0002ip-Iw for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIO-0003aZ-Uo for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:28 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-9-wgPjX4k_NKeHkIKwMhKEaA-1; Mon, 06 Feb 2023 19:57:08 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D4939101A52E; Tue, 7 Feb 2023 00:57:06 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0FA28492C3C; Tue, 7 Feb 2023 00:57:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731433; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZH6YddLEeOQzAr9SjLkB0f5BGyw0/sKDCslYwuvDfrs=; b=S/Z6Jf/ex5IUpVyLbjIPbXCH0P9KI0oJhFYUpxMeMQ8lGqPxm1Kwi+pRzpPWWQz2UNXpIF ZqanskrwEtl9xzhn+YIY/W7ZoIBWdKiX60/Uwtmo4xG13oKjaRPDIFJl5iem54Q9nccLOt u8WHXEs/2aU1lIra6N5bJdejHIe8uoA= X-MC-Unique: wgPjX4k_NKeHkIKwMhKEaA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman Subject: [PULL 02/30] migration: No save_live_pending() method uses the QEMUFile parameter Date: Tue, 7 Feb 2023 01:56:22 +0100 Message-Id: <20230207005650.1810-3-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731549056100005 Content-Type: text/plain; charset="utf-8" So remove it everywhere. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- include/migration/register.h | 2 +- migration/savevm.h | 2 +- hw/s390x/s390-stattrib.c | 2 +- hw/vfio/migration.c | 2 +- migration/block-dirty-bitmap.c | 2 +- migration/block.c | 2 +- migration/migration.c | 2 +- migration/ram.c | 2 +- migration/savevm.c | 4 ++-- 9 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index c1dcff0f90..6ca71367af 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -46,7 +46,7 @@ typedef struct SaveVMHandlers { =20 /* This runs outside the iothread lock! */ int (*save_setup)(QEMUFile *f, void *opaque); - void (*save_live_pending)(QEMUFile *f, void *opaque, + void (*save_live_pending)(void *opaque, uint64_t threshold_size, uint64_t *res_precopy_only, uint64_t *res_compatible, diff --git a/migration/savevm.h b/migration/savevm.h index 6461342cb4..524cf12f25 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -40,7 +40,7 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, bool inactivate_disks); -void qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size, +void qemu_savevm_state_pending(uint64_t max_size, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only); diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index 9eda1c3b2a..a553a1e850 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -182,7 +182,7 @@ static int cmma_save_setup(QEMUFile *f, void *opaque) return 0; } =20 -static void cmma_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, +static void cmma_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index c74453e0b5..b2125c7607 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -456,7 +456,7 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } =20 -static void vfio_save_pending(QEMUFile *f, void *opaque, +static void vfio_save_pending(void *opaque, uint64_t threshold_size, uint64_t *res_precopy_only, uint64_t *res_compatible, diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 15127d489a..c27ef9b033 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -762,7 +762,7 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void= *opaque) return 0; } =20 -static void dirty_bitmap_save_pending(QEMUFile *f, void *opaque, +static void dirty_bitmap_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy_only, uint64_t *res_compatible, diff --git a/migration/block.c b/migration/block.c index 5da15a62de..47852b8d58 100644 --- a/migration/block.c +++ b/migration/block.c @@ -863,7 +863,7 @@ static int block_save_complete(QEMUFile *f, void *opaqu= e) return 0; } =20 -static void block_save_pending(QEMUFile *f, void *opaque, uint64_t max_siz= e, +static void block_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) diff --git a/migration/migration.c b/migration/migration.c index 56859d5869..5e2c891845 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3781,7 +3781,7 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) uint64_t pending_size, pend_pre, pend_compat, pend_post; bool in_postcopy =3D s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE; =20 - qemu_savevm_state_pending(s->to_dst_file, s->threshold_size, &pend_pre, + qemu_savevm_state_pending(s->threshold_size, &pend_pre, &pend_compat, &pend_post); pending_size =3D pend_pre + pend_compat + pend_post; =20 diff --git a/migration/ram.c b/migration/ram.c index 68a45338e3..389739f162 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3409,7 +3409,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) return 0; } =20 -static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, +static void ram_save_pending(void *opaque, uint64_t max_size, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) diff --git a/migration/savevm.c b/migration/savevm.c index a783789430..5e4bccb966 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1472,7 +1472,7 @@ flush: * the result is split into the amount for units that can and * for units that can't do postcopy. */ -void qemu_savevm_state_pending(QEMUFile *f, uint64_t threshold_size, +void qemu_savevm_state_pending(uint64_t threshold_size, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) @@ -1493,7 +1493,7 @@ void qemu_savevm_state_pending(QEMUFile *f, uint64_t = threshold_size, continue; } } - se->ops->save_live_pending(f, se->opaque, threshold_size, + se->ops->save_live_pending(se->opaque, threshold_size, res_precopy_only, res_compatible, res_postcopy_only); } --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731474; cv=none; d=zohomail.com; s=zohoarc; b=UwmNi0rvli42UST4MLqn1RnycW7AeGwkQP7upraAtWK+wKTYC7oaiXkPVn2bdpQS8Z1fnBS5M460CYPoHRSsYcw8cdNgkbkM1kHczFxVtZJGOSL/cPMGOcEdN6+nYFTozp5326RdcHBSwcPDLrfqzsCdY9yRi1eMhn2uCgTusL0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731474; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SCjSZLQpgEecPUo0rKgZK+3cB0mkcNSmCHgsd+Zf0V0=; b=mxvB3T26CFyx01fFvkIHUKlZ1TToox7o5HrXjtSyP93FLlLItZmAmaXHIveMhq8P1CP9r0m/5f0786s4fm5QmT5/nPq6NSdA2a1+XZRVyts9dbIFzMpWFhqoOnNwm8cmTTPIhuox5apS/G9Q3EWltTzhDNRlaHN+My1tFEaQM6c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731474779813.0463873775151; Mon, 6 Feb 2023 16:57:54 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCIg-0002n5-2x; Mon, 06 Feb 2023 19:57:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIf-0002mL-1S for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIR-0003as-QF for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:32 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-398-FfRlNWDAOsmjCPDwPs68iA-1; Mon, 06 Feb 2023 19:57:12 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E6D893C025B6; Tue, 7 Feb 2023 00:57:11 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 25B54492C3C; Tue, 7 Feb 2023 00:57:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731436; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SCjSZLQpgEecPUo0rKgZK+3cB0mkcNSmCHgsd+Zf0V0=; b=SFRLV+pDG8mK+hdwdQQuBiBX9zS1dZtObsMCdycpKzJDuoXSq/H03dN9G76yJt/71hWLFO UlhoEuuBXWNXTBWYyvn3lpvP+U6ZpN5JC/iINkHbr9rG9PNV3SCfw+xdIOjaz0oyU2VZMw YGu+F4pakUjwf1uKvb86+4wQemexWs0= X-MC-Unique: FfRlNWDAOsmjCPDwPs68iA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman Subject: [PULL 03/30] migration: Split save_live_pending() into state_pending_* Date: Tue, 7 Feb 2023 01:56:23 +0100 Message-Id: <20230207005650.1810-4-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731476809100006 Content-Type: text/plain; charset="utf-8" We split the function into to: - state_pending_estimate: We estimate the remaining state size without stopping the machine. - state pending_exact: We calculate the exact amount of remaining state. The only "device" that implements different functions for _estimate() and _exact() is ram. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- docs/devel/migration.rst | 18 ++++++++------- docs/devel/vfio-migration.rst | 4 ++-- include/migration/register.h | 19 +++++++++------ migration/savevm.h | 12 ++++++---- hw/s390x/s390-stattrib.c | 11 +++++---- hw/vfio/migration.c | 21 +++++++++-------- migration/block-dirty-bitmap.c | 15 ++++++------ migration/block.c | 13 ++++++----- migration/migration.c | 20 +++++++++++----- migration/ram.c | 35 ++++++++++++++++++++-------- migration/savevm.c | 42 +++++++++++++++++++++++++++------- hw/vfio/trace-events | 2 +- migration/trace-events | 7 +++--- 13 files changed, 143 insertions(+), 76 deletions(-) diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst index 3e9656d8e0..6f65c23b47 100644 --- a/docs/devel/migration.rst +++ b/docs/devel/migration.rst @@ -482,15 +482,17 @@ An iterative device must provide: - A ``load_setup`` function that initialises the data structures on the destination. =20 - - A ``save_live_pending`` function that is called repeatedly and must - indicate how much more data the iterative data must save. The core - migration code will use this to determine when to pause the CPUs - and complete the migration. + - A ``state_pending_exact`` function that indicates how much more + data we must save. The core migration code will use this to + determine when to pause the CPUs and complete the migration. =20 - - A ``save_live_iterate`` function (called after ``save_live_pending`` - when there is significant data still to be sent). It should send - a chunk of data until the point that stream bandwidth limits tell it - to stop. Each call generates one section. + - A ``state_pending_estimate`` function that indicates how much more + data we must save. When the estimated amount is smaller than the + threshold, we call ``state_pending_exact``. + + - A ``save_live_iterate`` function should send a chunk of data until + the point that stream bandwidth limits tell it to stop. Each call + generates one section. =20 - A ``save_live_complete_precopy`` function that must transmit the last section for the device containing any remaining data. diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst index 9ff6163c88..673057c90d 100644 --- a/docs/devel/vfio-migration.rst +++ b/docs/devel/vfio-migration.rst @@ -28,7 +28,7 @@ VFIO implements the device hooks for the iterative approa= ch as follows: * A ``load_setup`` function that sets up the migration region on the destination and sets _RESUMING flag in the VFIO device state. =20 -* A ``save_live_pending`` function that reads pending_bytes from the vendor +* A ``state_pending_exact`` function that reads pending_bytes from the ven= dor driver, which indicates the amount of data that the vendor driver has ye= t to save for the VFIO device. =20 @@ -114,7 +114,7 @@ Live migration save path (RUNNING, _SETUP, _RUNNING|_SAVING) | (RUNNING, _ACTIVE, _RUNNING|_SAVING) - If device is active, get pending_bytes by .save_live_pending() + If device is active, get pending_bytes by .state_pending_exac= t() If total pending_bytes >=3D threshold_size, call .save_live_iter= ate() Data of VFIO device for pre-copy phase is copied Iterate till total pending bytes converge and are less than thresh= old diff --git a/include/migration/register.h b/include/migration/register.h index 6ca71367af..15cf32994d 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -46,11 +46,6 @@ typedef struct SaveVMHandlers { =20 /* This runs outside the iothread lock! */ int (*save_setup)(QEMUFile *f, void *opaque); - void (*save_live_pending)(void *opaque, - uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only); /* Note for save_live_pending: * - res_precopy_only is for data which must be migrated in precopy ph= ase * or in stopped state, in other words - before target vm start @@ -61,8 +56,18 @@ typedef struct SaveVMHandlers { * Sum of res_postcopy_only, res_compatible and res_postcopy_only is t= he * whole amount of pending data. */ - - + /* This estimates the remaining data to transfer */ + void (*state_pending_estimate)(void *opaque, + uint64_t threshold_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only); + /* This calculate the exact remaining data to transfer */ + void (*state_pending_exact)(void *opaque, + uint64_t threshold_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only); LoadStateHandler *load_state; int (*load_setup)(QEMUFile *f, void *opaque); int (*load_cleanup)(void *opaque); diff --git a/migration/savevm.h b/migration/savevm.h index 524cf12f25..5d2cff4411 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -40,10 +40,14 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, bool inactivate_disks); -void qemu_savevm_state_pending(uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only); +void qemu_savevm_state_pending_exact(uint64_t threshold_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only); +void qemu_savevm_state_pending_estimate(uint64_t thershold_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); void qemu_savevm_send_open_return_path(QEMUFile *f); int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len); diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index a553a1e850..8f573ebb10 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -182,10 +182,10 @@ static int cmma_save_setup(QEMUFile *f, void *opaque) return 0; } =20 -static void cmma_save_pending(void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) +static void cmma_state_pending(void *opaque, uint64_t max_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) { S390StAttribState *sas =3D S390_STATTRIB(opaque); S390StAttribClass *sac =3D S390_STATTRIB_GET_CLASS(sas); @@ -371,7 +371,8 @@ static SaveVMHandlers savevm_s390_stattrib_handlers =3D= { .save_setup =3D cmma_save_setup, .save_live_iterate =3D cmma_save_iterate, .save_live_complete_precopy =3D cmma_save_complete, - .save_live_pending =3D cmma_save_pending, + .state_pending_exact =3D cmma_state_pending, + .state_pending_estimate =3D cmma_state_pending, .save_cleanup =3D cmma_save_cleanup, .load_state =3D cmma_load, .is_active =3D cmma_active, diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index b2125c7607..c49ca466d4 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -456,11 +456,11 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } =20 -static void vfio_save_pending(void *opaque, - uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) +static void vfio_state_pending(void *opaque, + uint64_t threshold_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; @@ -473,7 +473,7 @@ static void vfio_save_pending(void *opaque, =20 *res_precopy_only +=3D migration->pending_bytes; =20 - trace_vfio_save_pending(vbasedev->name, *res_precopy_only, + trace_vfio_state_pending(vbasedev->name, *res_precopy_only, *res_postcopy_only, *res_compatible); } =20 @@ -515,9 +515,9 @@ static int vfio_save_iterate(QEMUFile *f, void *opaque) } =20 /* - * Reset pending_bytes as .save_live_pending is not called during save= vm or - * snapshot case, in such case vfio_update_pending() at the start of t= his - * function updates pending_bytes. + * Reset pending_bytes as state_pending* are not called during + * savevm or snapshot case, in such case vfio_update_pending() at + * the start of this function updates pending_bytes. */ migration->pending_bytes =3D 0; trace_vfio_save_iterate(vbasedev->name, data_size); @@ -685,7 +685,8 @@ static int vfio_load_state(QEMUFile *f, void *opaque, i= nt version_id) static SaveVMHandlers savevm_vfio_handlers =3D { .save_setup =3D vfio_save_setup, .save_cleanup =3D vfio_save_cleanup, - .save_live_pending =3D vfio_save_pending, + .state_pending_exact =3D vfio_state_pending, + .state_pending_estimate =3D vfio_state_pending, .save_live_iterate =3D vfio_save_iterate, .save_live_complete_precopy =3D vfio_save_complete_precopy, .save_state =3D vfio_save_state, diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index c27ef9b033..6fac9fb34f 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -762,11 +762,11 @@ static int dirty_bitmap_save_complete(QEMUFile *f, vo= id *opaque) return 0; } =20 -static void dirty_bitmap_save_pending(void *opaque, - uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) +static void dirty_bitmap_state_pending(void *opaque, + uint64_t max_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) { DBMSaveState *s =3D &((DBMState *)opaque)->save; SaveBitmapState *dbms; @@ -784,7 +784,7 @@ static void dirty_bitmap_save_pending(void *opaque, =20 qemu_mutex_unlock_iothread(); =20 - trace_dirty_bitmap_save_pending(pending, max_size); + trace_dirty_bitmap_state_pending(pending); =20 *res_postcopy_only +=3D pending; } @@ -1253,7 +1253,8 @@ static SaveVMHandlers savevm_dirty_bitmap_handlers = =3D { .save_live_complete_postcopy =3D dirty_bitmap_save_complete, .save_live_complete_precopy =3D dirty_bitmap_save_complete, .has_postcopy =3D dirty_bitmap_has_postcopy, - .save_live_pending =3D dirty_bitmap_save_pending, + .state_pending_exact =3D dirty_bitmap_state_pending, + .state_pending_estimate =3D dirty_bitmap_state_pending, .save_live_iterate =3D dirty_bitmap_save_iterate, .is_active_iterate =3D dirty_bitmap_is_active_iterate, .load_state =3D dirty_bitmap_load, diff --git a/migration/block.c b/migration/block.c index 47852b8d58..544e74e9c5 100644 --- a/migration/block.c +++ b/migration/block.c @@ -863,10 +863,10 @@ static int block_save_complete(QEMUFile *f, void *opa= que) return 0; } =20 -static void block_save_pending(void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) +static void block_state_pending(void *opaque, uint64_t max_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) { /* Estimate pending number of bytes to send */ uint64_t pending; @@ -885,7 +885,7 @@ static void block_save_pending(void *opaque, uint64_t m= ax_size, pending =3D BLK_MIG_BLOCK_SIZE; } =20 - trace_migration_block_save_pending(pending); + trace_migration_block_state_pending(pending); /* We don't do postcopy */ *res_precopy_only +=3D pending; } @@ -1020,7 +1020,8 @@ static SaveVMHandlers savevm_block_handlers =3D { .save_setup =3D block_save_setup, .save_live_iterate =3D block_save_iterate, .save_live_complete_precopy =3D block_save_complete, - .save_live_pending =3D block_save_pending, + .state_pending_exact =3D block_state_pending, + .state_pending_estimate =3D block_state_pending, .load_state =3D block_load, .save_cleanup =3D block_migration_cleanup, .is_active =3D block_is_active, diff --git a/migration/migration.c b/migration/migration.c index 5e2c891845..877a6f7011 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3778,15 +3778,23 @@ typedef enum { */ static MigIterateState migration_iteration_run(MigrationState *s) { - uint64_t pending_size, pend_pre, pend_compat, pend_post; + uint64_t pend_pre, pend_compat, pend_post; bool in_postcopy =3D s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE; =20 - qemu_savevm_state_pending(s->threshold_size, &pend_pre, - &pend_compat, &pend_post); - pending_size =3D pend_pre + pend_compat + pend_post; + qemu_savevm_state_pending_estimate(s->threshold_size, &pend_pre, + &pend_compat, &pend_post); + uint64_t pending_size =3D pend_pre + pend_compat + pend_post; =20 - trace_migrate_pending(pending_size, s->threshold_size, - pend_pre, pend_compat, pend_post); + trace_migrate_pending_estimate(pending_size, s->threshold_size, + pend_pre, pend_compat, pend_post); + + if (pend_pre + pend_compat <=3D s->threshold_size) { + qemu_savevm_state_pending_exact(s->threshold_size, &pend_pre, + &pend_compat, &pend_post); + pending_size =3D pend_pre + pend_compat + pend_post; + trace_migrate_pending_exact(pending_size, s->threshold_size, + pend_pre, pend_compat, pend_post); + } =20 if (pending_size && pending_size >=3D s->threshold_size) { /* Still a significant amount to transfer */ diff --git a/migration/ram.c b/migration/ram.c index 389739f162..56ff9cd29d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3409,19 +3409,35 @@ static int ram_save_complete(QEMUFile *f, void *opa= que) return 0; } =20 -static void ram_save_pending(void *opaque, uint64_t max_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) +static void ram_state_pending_estimate(void *opaque, uint64_t max_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) { RAMState **temp =3D opaque; RAMState *rs =3D *temp; - uint64_t remaining_size; =20 - remaining_size =3D rs->migration_dirty_pages * TARGET_PAGE_SIZE; + uint64_t remaining_size =3D rs->migration_dirty_pages * TARGET_PAGE_SI= ZE; =20 - if (!migration_in_postcopy() && - remaining_size < max_size) { + if (migrate_postcopy_ram()) { + /* We can do postcopy, and all the data is postcopiable */ + *res_postcopy_only +=3D remaining_size; + } else { + *res_precopy_only +=3D remaining_size; + } +} + +static void ram_state_pending_exact(void *opaque, uint64_t max_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) +{ + RAMState **temp =3D opaque; + RAMState *rs =3D *temp; + + uint64_t remaining_size =3D rs->migration_dirty_pages * TARGET_PAGE_SI= ZE; + + if (!migration_in_postcopy()) { qemu_mutex_lock_iothread(); WITH_RCU_READ_LOCK_GUARD() { migration_bitmap_sync_precopy(rs); @@ -4577,7 +4593,8 @@ static SaveVMHandlers savevm_ram_handlers =3D { .save_live_complete_postcopy =3D ram_save_complete, .save_live_complete_precopy =3D ram_save_complete, .has_postcopy =3D ram_has_postcopy, - .save_live_pending =3D ram_save_pending, + .state_pending_exact =3D ram_state_pending_exact, + .state_pending_estimate =3D ram_state_pending_estimate, .load_state =3D ram_load, .save_cleanup =3D ram_save_cleanup, .load_setup =3D ram_load_setup, diff --git a/migration/savevm.c b/migration/savevm.c index 5e4bccb966..7f9f770c1e 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1472,10 +1472,10 @@ flush: * the result is split into the amount for units that can and * for units that can't do postcopy. */ -void qemu_savevm_state_pending(uint64_t threshold_size, - uint64_t *res_precopy_only, - uint64_t *res_compatible, - uint64_t *res_postcopy_only) +void qemu_savevm_state_pending_estimate(uint64_t threshold_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) { SaveStateEntry *se; =20 @@ -1485,7 +1485,7 @@ void qemu_savevm_state_pending(uint64_t threshold_siz= e, =20 =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->save_live_pending) { + if (!se->ops || !se->ops->state_pending_exact) { continue; } if (se->ops->is_active) { @@ -1493,9 +1493,35 @@ void qemu_savevm_state_pending(uint64_t threshold_si= ze, continue; } } - se->ops->save_live_pending(se->opaque, threshold_size, - res_precopy_only, res_compatible, - res_postcopy_only); + se->ops->state_pending_exact(se->opaque, threshold_size, + res_precopy_only, res_compatible, + res_postcopy_only); + } +} + +void qemu_savevm_state_pending_exact(uint64_t threshold_size, + uint64_t *res_precopy_only, + uint64_t *res_compatible, + uint64_t *res_postcopy_only) +{ + SaveStateEntry *se; + + *res_precopy_only =3D 0; + *res_compatible =3D 0; + *res_postcopy_only =3D 0; + + QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (!se->ops || !se->ops->state_pending_estimate) { + continue; + } + if (se->ops->is_active) { + if (!se->ops->is_active(se->opaque)) { + continue; + } + } + se->ops->state_pending_estimate(se->opaque, threshold_size, + res_precopy_only, res_compatible, + res_postcopy_only); } } =20 diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 73dffe9e00..52de1c84f8 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -157,7 +157,7 @@ vfio_save_cleanup(const char *name) " (%s)" vfio_save_buffer(const char *name, uint64_t data_offset, uint64_t data_siz= e, uint64_t pending) " (%s) Offset 0x%"PRIx64" size 0x%"PRIx64" pending 0x%= "PRIx64 vfio_update_pending(const char *name, uint64_t pending) " (%s) pending 0x%= "PRIx64 vfio_save_device_config_state(const char *name) " (%s)" -vfio_save_pending(const char *name, uint64_t precopy, uint64_t postcopy, u= int64_t compatible) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64" compati= ble 0x%"PRIx64 +vfio_state_pending(const char *name, uint64_t precopy, uint64_t postcopy, = uint64_t compatible) " (%s) precopy 0x%"PRIx64" postcopy 0x%"PRIx64" compat= ible 0x%"PRIx64 vfio_save_iterate(const char *name, int data_size) " (%s) data_size %d" vfio_save_complete_precopy(const char *name) " (%s)" vfio_load_device_config_state(const char *name) " (%s)" diff --git a/migration/trace-events b/migration/trace-events index 57003edcbd..adb680b0e6 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -150,7 +150,8 @@ migrate_fd_cleanup(void) "" migrate_fd_error(const char *error_desc) "error=3D%s" migrate_fd_cancel(void) "" migrate_handle_rp_req_pages(const char *rbname, size_t start, size_t len) = "in %s at 0x%zx len 0x%zx" -migrate_pending(uint64_t size, uint64_t max, uint64_t pre, uint64_t compat= , uint64_t post) "pending size %" PRIu64 " max %" PRIu64 " (pre =3D %" PRIu= 64 " compat=3D%" PRIu64 " post=3D%" PRIu64 ")" +migrate_pending_exact(uint64_t size, uint64_t max, uint64_t pre, uint64_t = compat, uint64_t post) "exact pending size %" PRIu64 " max %" PRIu64 " (pre= =3D %" PRIu64 " compat=3D%" PRIu64 " post=3D%" PRIu64 ")" +migrate_pending_estimate(uint64_t size, uint64_t max, uint64_t pre, uint64= _t compat, uint64_t post) "estimate pending size %" PRIu64 " max %" PRIu64 = " (pre =3D %" PRIu64 " compat=3D%" PRIu64 " post=3D%" PRIu64 ")" migrate_send_rp_message(int msg_type, uint16_t len) "%d: len %d" migrate_send_rp_recv_bitmap(char *name, int64_t size) "block '%s' size 0x%= "PRIi64 migration_completion_file_err(void) "" @@ -330,7 +331,7 @@ send_bitmap_bits(uint32_t flags, uint64_t start_sector,= uint32_t nr_sectors, uin dirty_bitmap_save_iterate(int in_postcopy) "in postcopy: %d" dirty_bitmap_save_complete_enter(void) "" dirty_bitmap_save_complete_finish(void) "" -dirty_bitmap_save_pending(uint64_t pending, uint64_t max_size) "pending %"= PRIu64 " max: %" PRIu64 +dirty_bitmap_state_pending(uint64_t pending) "pending %" PRIu64 dirty_bitmap_load_complete(void) "" dirty_bitmap_load_bits_enter(uint64_t first_sector, uint32_t nr_sectors) "= chunk: %" PRIu64 " %" PRIu32 dirty_bitmap_load_bits_zeroes(void) "" @@ -355,7 +356,7 @@ migration_block_save_device_dirty(int64_t sector) "Erro= r reading sector %" PRId6 migration_block_flush_blks(const char *action, int submitted, int read_don= e, int transferred) "%s submitted %d read_done %d transferred %d" migration_block_save(const char *mig_stage, int submitted, int transferred= ) "Enter save live %s submitted %d transferred %d" migration_block_save_complete(void) "Block migration completed" -migration_block_save_pending(uint64_t pending) "Enter save live pending %= " PRIu64 +migration_block_state_pending(uint64_t pending) "Enter save live pending = %" PRIu64 =20 # page_cache.c migration_pagecache_init(int64_t max_num_items) "Setting cache buckets to = %" PRId64 --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731548; cv=none; d=zohomail.com; s=zohoarc; b=UYCIE5AktjT/IGqsSHkCW+wgvnsvCqayRl3vJdNn13cIdadlMIKBEaIgVgzg4T6G8mQGvSxi8SnJCF6b9QrSeM7MDBNx46Ow12qQj4KB1OSREs48lsbaRrRrn72m1FQJhkr+UwUen75FljQTMitmA+o4sFn9pyqNU0xpzfARdq4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731548; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=df0XD1zjBX5ehVMrNYF1VciZIgoAaVOXMNkGv86nMxg=; b=hhhs9vgp8VanV65YYZoAA5j83jF1xSgcOUKHLcFu4otuBqpZnZAB0ucsZtzuvQDQZGkU3gnNPLHbwzp1Y94kP3DRYnTHAuERBj44k9vJOBCJYxD0ZR9mQFR3KtwKD54y47zamrcxGDguJWcF2NThp5+fl0aEkChfKkz9SEWfm80= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731548824683.654468002009; Mon, 6 Feb 2023 16:59:08 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCIh-0002s5-Mj; Mon, 06 Feb 2023 19:57:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIg-0002oA-Me for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:34 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIV-0003bv-Kb for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:34 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-572-4_QBF0IHMRKmR34KP1k4tQ-1; Mon, 06 Feb 2023 19:57:18 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0603B2A59561; Tue, 7 Feb 2023 00:57:17 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3876C492C3C; Tue, 7 Feb 2023 00:57:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731443; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=df0XD1zjBX5ehVMrNYF1VciZIgoAaVOXMNkGv86nMxg=; b=QAC8rW2+EWP8XyoChQW+VrHWEvEarGavKZ1w+t6MJBdupMTMvUuxNHSku8XyEKRBCsSWMz 7w33vL3b9k7nLAZUuZGE1ryjwGmvRmIMp3o2HfP/x8lDVwxvtC7dVdk0vhdGHpm9A7HJdf 2EMNBlc/kxveB07HaEqTHiyIRWrKTBY= X-MC-Unique: 4_QBF0IHMRKmR34KP1k4tQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman Subject: [PULL 04/30] migration: Remove unused threshold_size parameter Date: Tue, 7 Feb 2023 01:56:24 +0100 Message-Id: <20230207005650.1810-5-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731551088100011 Content-Type: text/plain; charset="utf-8" Until previous commit, save_live_pending() was used for ram. Now with the split into state_pending_estimate() and state_pending_exact() it is not needed anymore, so remove them. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- include/migration/register.h | 2 -- migration/savevm.h | 6 ++---- hw/s390x/s390-stattrib.c | 2 +- hw/vfio/migration.c | 1 - migration/block-dirty-bitmap.c | 1 - migration/block.c | 2 +- migration/migration.c | 10 ++++------ migration/ram.c | 4 ++-- migration/savevm.c | 11 ++++------- migration/trace-events | 4 ++-- 10 files changed, 16 insertions(+), 27 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index 15cf32994d..b91a0cdbf8 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -58,13 +58,11 @@ typedef struct SaveVMHandlers { */ /* This estimates the remaining data to transfer */ void (*state_pending_estimate)(void *opaque, - uint64_t threshold_size, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only); /* This calculate the exact remaining data to transfer */ void (*state_pending_exact)(void *opaque, - uint64_t threshold_size, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only); diff --git a/migration/savevm.h b/migration/savevm.h index 5d2cff4411..b1901e68d5 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -40,12 +40,10 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only, bool inactivate_disks); -void qemu_savevm_state_pending_exact(uint64_t threshold_size, - uint64_t *res_precopy_only, +void qemu_savevm_state_pending_exact(uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only); -void qemu_savevm_state_pending_estimate(uint64_t thershold_size, - uint64_t *res_precopy_only, +void qemu_savevm_state_pending_estimate(uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index 8f573ebb10..3e32002eab 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -182,7 +182,7 @@ static int cmma_save_setup(QEMUFile *f, void *opaque) return 0; } =20 -static void cmma_state_pending(void *opaque, uint64_t max_size, +static void cmma_state_pending(void *opaque, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index c49ca466d4..b3318f0f20 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -457,7 +457,6 @@ static void vfio_save_cleanup(void *opaque) } =20 static void vfio_state_pending(void *opaque, - uint64_t threshold_size, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index 6fac9fb34f..5a621419d3 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -763,7 +763,6 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void= *opaque) } =20 static void dirty_bitmap_state_pending(void *opaque, - uint64_t max_size, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) diff --git a/migration/block.c b/migration/block.c index 544e74e9c5..29f69025af 100644 --- a/migration/block.c +++ b/migration/block.c @@ -863,7 +863,7 @@ static int block_save_complete(QEMUFile *f, void *opaqu= e) return 0; } =20 -static void block_state_pending(void *opaque, uint64_t max_size, +static void block_state_pending(void *opaque, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) diff --git a/migration/migration.c b/migration/migration.c index 877a6f7011..7cab4b8192 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3781,18 +3781,16 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) uint64_t pend_pre, pend_compat, pend_post; bool in_postcopy =3D s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE; =20 - qemu_savevm_state_pending_estimate(s->threshold_size, &pend_pre, - &pend_compat, &pend_post); + qemu_savevm_state_pending_estimate(&pend_pre, &pend_compat, &pend_post= ); uint64_t pending_size =3D pend_pre + pend_compat + pend_post; =20 - trace_migrate_pending_estimate(pending_size, s->threshold_size, + trace_migrate_pending_estimate(pending_size, pend_pre, pend_compat, pend_post); =20 if (pend_pre + pend_compat <=3D s->threshold_size) { - qemu_savevm_state_pending_exact(s->threshold_size, &pend_pre, - &pend_compat, &pend_post); + qemu_savevm_state_pending_exact(&pend_pre, &pend_compat, &pend_pos= t); pending_size =3D pend_pre + pend_compat + pend_post; - trace_migrate_pending_exact(pending_size, s->threshold_size, + trace_migrate_pending_exact(pending_size, pend_pre, pend_compat, pend_post); } =20 diff --git a/migration/ram.c b/migration/ram.c index 56ff9cd29d..885d7dbf23 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3409,7 +3409,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) return 0; } =20 -static void ram_state_pending_estimate(void *opaque, uint64_t max_size, +static void ram_state_pending_estimate(void *opaque, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) @@ -3427,7 +3427,7 @@ static void ram_state_pending_estimate(void *opaque, = uint64_t max_size, } } =20 -static void ram_state_pending_exact(void *opaque, uint64_t max_size, +static void ram_state_pending_exact(void *opaque, uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) diff --git a/migration/savevm.c b/migration/savevm.c index 7f9f770c1e..e1caa3ea7c 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1472,8 +1472,7 @@ flush: * the result is split into the amount for units that can and * for units that can't do postcopy. */ -void qemu_savevm_state_pending_estimate(uint64_t threshold_size, - uint64_t *res_precopy_only, +void qemu_savevm_state_pending_estimate(uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) { @@ -1483,7 +1482,6 @@ void qemu_savevm_state_pending_estimate(uint64_t thre= shold_size, *res_compatible =3D 0; *res_postcopy_only =3D 0; =20 - QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { if (!se->ops || !se->ops->state_pending_exact) { continue; @@ -1493,14 +1491,13 @@ void qemu_savevm_state_pending_estimate(uint64_t th= reshold_size, continue; } } - se->ops->state_pending_exact(se->opaque, threshold_size, + se->ops->state_pending_exact(se->opaque, res_precopy_only, res_compatible, res_postcopy_only); } } =20 -void qemu_savevm_state_pending_exact(uint64_t threshold_size, - uint64_t *res_precopy_only, +void qemu_savevm_state_pending_exact(uint64_t *res_precopy_only, uint64_t *res_compatible, uint64_t *res_postcopy_only) { @@ -1519,7 +1516,7 @@ void qemu_savevm_state_pending_exact(uint64_t thresho= ld_size, continue; } } - se->ops->state_pending_estimate(se->opaque, threshold_size, + se->ops->state_pending_estimate(se->opaque, res_precopy_only, res_compatible, res_postcopy_only); } diff --git a/migration/trace-events b/migration/trace-events index adb680b0e6..67b65a70ff 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -150,8 +150,8 @@ migrate_fd_cleanup(void) "" migrate_fd_error(const char *error_desc) "error=3D%s" migrate_fd_cancel(void) "" migrate_handle_rp_req_pages(const char *rbname, size_t start, size_t len) = "in %s at 0x%zx len 0x%zx" -migrate_pending_exact(uint64_t size, uint64_t max, uint64_t pre, uint64_t = compat, uint64_t post) "exact pending size %" PRIu64 " max %" PRIu64 " (pre= =3D %" PRIu64 " compat=3D%" PRIu64 " post=3D%" PRIu64 ")" -migrate_pending_estimate(uint64_t size, uint64_t max, uint64_t pre, uint64= _t compat, uint64_t post) "estimate pending size %" PRIu64 " max %" PRIu64 = " (pre =3D %" PRIu64 " compat=3D%" PRIu64 " post=3D%" PRIu64 ")" +migrate_pending_exact(uint64_t size, uint64_t pre, uint64_t compat, uint64= _t post) "exact pending size %" PRIu64 " (pre =3D %" PRIu64 " compat=3D%" P= RIu64 " post=3D%" PRIu64 ")" +migrate_pending_estimate(uint64_t size, uint64_t pre, uint64_t compat, uin= t64_t post) "estimate pending size %" PRIu64 " (pre =3D %" PRIu64 " compat= =3D%" PRIu64 " post=3D%" PRIu64 ")" migrate_send_rp_message(int msg_type, uint16_t len) "%d: len %d" migrate_send_rp_recv_bitmap(char *name, int64_t size) "block '%s' size 0x%= "PRIi64 migration_completion_file_err(void) "" --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731542; cv=none; d=zohomail.com; s=zohoarc; b=k8kAurSmxjCEb61mISNcDMUfVmWxagC7SnFLPOTCvutUb8HQfzC/nGLm0tJEC5nA9jHAl5UOuK5qPRAakrPN7mwtq6lwSBY8iKOSeOH8GoNTCtR+e+ZuwUCKavRUJiZp+SsR+XiUcfMMnW3ZD7iwatB/WHWiHIlzHpXKBHrFnRg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731542; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=d1J7wsb/5vyGRvo/issEjC+pk8cISfHxnGFOzvK/Shs=; b=e9rTIpvBVaPkbeaildth2CJFKeEX5y87OI/RWXjvkQjnVNvzCmxSEgsc3hbh1RqsHk1CI7w7giSDBG4iCj5ZV8hr3j0CUArCs2W889P4QmwluVU693PrZBET4gojoB+s0WMuSgyeSjLJksb8u+NZaZeObJoGuA6Isp4nrZ1hx4o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731542831818.2301478333353; Mon, 6 Feb 2023 16:59:02 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCIl-00031C-VC; Mon, 06 Feb 2023 19:57:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIh-0002ru-IE for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:35 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIZ-0003c8-By for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:35 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-121-KcCxm7hKNk2KRzgUe2mdQw-1; Mon, 06 Feb 2023 19:57:22 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DDC841871D94; Tue, 7 Feb 2023 00:57:21 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B853492C3C; Tue, 7 Feb 2023 00:57:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731445; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d1J7wsb/5vyGRvo/issEjC+pk8cISfHxnGFOzvK/Shs=; b=CfdUsJ2DfXdZqo4reI4lu7Woqg6kpveL41zFf9466LMBS5copIVHSJD8c0Wf5ERiH2axGx IF4CQgg50SquXm9M49jJYTRPAAtrTUoq1iy8zcW6aAtQXxwxQ9A9XmRBpab7DGH5o+y6/w 3Iec9Xm3+rWVZlxuuDahXoevEIggg78= X-MC-Unique: KcCxm7hKNk2KRzgUe2mdQw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman Subject: [PULL 05/30] migration: simplify migration_iteration_run() Date: Tue, 7 Feb 2023 01:56:25 +0100 Message-Id: <20230207005650.1810-6-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731545088100007 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 7cab4b8192..6d4cd8083b 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3794,23 +3794,23 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) pend_pre, pend_compat, pend_post); } =20 - if (pending_size && pending_size >=3D s->threshold_size) { - /* Still a significant amount to transfer */ - if (!in_postcopy && pend_pre <=3D s->threshold_size && - qatomic_read(&s->start_postcopy)) { - if (postcopy_start(s)) { - error_report("%s: postcopy failed to start", __func__); - } - return MIG_ITERATE_SKIP; - } - /* Just another iteration step */ - qemu_savevm_state_iterate(s->to_dst_file, in_postcopy); - } else { + if (!pending_size || pending_size < s->threshold_size) { trace_migration_thread_low_pending(pending_size); migration_completion(s); return MIG_ITERATE_BREAK; } =20 + /* Still a significant amount to transfer */ + if (!in_postcopy && pend_pre <=3D s->threshold_size && + qatomic_read(&s->start_postcopy)) { + if (postcopy_start(s)) { + error_report("%s: postcopy failed to start", __func__); + } + return MIG_ITERATE_SKIP; + } + + /* Just another iteration step */ + qemu_savevm_state_iterate(s->to_dst_file, in_postcopy); return MIG_ITERATE_RESUME; } =20 --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731542; cv=none; d=zohomail.com; s=zohoarc; b=XAcK6v9sxZsq0r2FlTs9fNhcX6U0BHq5vr3KFMSJwFraK1xFjIbcEG/gQjQVoU1H8YuQDGqTUegMvV4vbREynHDcuGv215Op08iYpcwAH9Fykg/2tQ0eoyKS21hyA38xRKxJ8CEfEa5Z8vCbYe6q1tYib4Rj8IaL2VPRZXFM/9s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731542; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=dKZAVW4KhViUgdBggWo5EKxv7CoNmT5xnU8AYti1Uuw=; b=m1fuKlX6XDwJQM99/JDqQ6c2vKRv7/TZwDNY/yAK/ImmPtddfhEYfeKZYYMI+dywemweorEpTofugRZq8dHtEJxaub4y3DQCzaFyVGKblvbSosJobA7h+ZGUtIQ0VdrR1jr6JToY6QOhaom91vPOuaZ4DNDWBVqhYrmaD9Uf9hE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731542712723.3752947832871; Mon, 6 Feb 2023 16:59:02 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCIm-00032x-5z; Mon, 06 Feb 2023 19:57:40 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIk-0002za-5I for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIf-0003dE-BE for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:37 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-220-QTMJkEyKO3-5SxZbFlmD6Q-1; Mon, 06 Feb 2023 19:57:27 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 24BD780D0ED; Tue, 7 Feb 2023 00:57:27 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2E147492C3C; Tue, 7 Feb 2023 00:57:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731452; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dKZAVW4KhViUgdBggWo5EKxv7CoNmT5xnU8AYti1Uuw=; b=PyZt3rmtJoPsjTcjbTwE12X7gZsihKl6qsIt1CRDuOzYOn70/YCHv7HbmUROoWiph6uek/ epHY+92iwWgR5xzpit1IlRF3SkM40UFML1CXuiH4l7CHb1eu9IzBuYpPTz8LgwjgGyCCj3 WIob9vgJIub+vyESrcc9RlhqaUNuXJ0= X-MC-Unique: QTMJkEyKO3-5SxZbFlmD6Q-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 06/30] util/userfaultfd: Add uffd_open() Date: Tue, 7 Feb 2023 01:56:26 +0100 Message-Id: <20230207005650.1810-7-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731543027100001 From: Peter Xu Add a helper to create the uffd handle. Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Juan Quintela Signed-off-by: Peter Xu Signed-off-by: Juan Quintela --- include/qemu/userfaultfd.h | 12 ++++++++++++ migration/postcopy-ram.c | 11 +++++------ tests/qtest/migration-test.c | 4 ++-- util/userfaultfd.c | 13 +++++++++++-- 4 files changed, 30 insertions(+), 10 deletions(-) diff --git a/include/qemu/userfaultfd.h b/include/qemu/userfaultfd.h index 6b74f92792..d764496f0b 100644 --- a/include/qemu/userfaultfd.h +++ b/include/qemu/userfaultfd.h @@ -13,10 +13,20 @@ #ifndef USERFAULTFD_H #define USERFAULTFD_H =20 +#ifdef CONFIG_LINUX + #include "qemu/osdep.h" #include "exec/hwaddr.h" #include =20 +/** + * uffd_open(): Open an userfaultfd handle for current context. + * + * @flags: The flags we want to pass in when creating the handle. + * + * Returns: the uffd handle if >=3D0, or <0 if error happens. + */ +int uffd_open(int flags); int uffd_query_features(uint64_t *features); int uffd_create_fd(uint64_t features, bool non_blocking); void uffd_close_fd(int uffd_fd); @@ -32,4 +42,6 @@ int uffd_wakeup(int uffd_fd, void *addr, uint64_t length); int uffd_read_events(int uffd_fd, struct uffd_msg *msgs, int count); bool uffd_poll_events(int uffd_fd, int tmo); =20 +#endif /* CONFIG_LINUX */ + #endif /* USERFAULTFD_H */ diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index b9a37ef255..0c55df0e52 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -37,6 +37,7 @@ #include "qemu-file.h" #include "yank_functions.h" #include "tls.h" +#include "qemu/userfaultfd.h" =20 /* Arbitrary limit on size of each discard command, * keeps them around ~200 bytes @@ -226,11 +227,9 @@ static bool receive_ufd_features(uint64_t *features) int ufd; bool ret =3D true; =20 - /* if we are here __NR_userfaultfd should exists */ - ufd =3D syscall(__NR_userfaultfd, O_CLOEXEC); + ufd =3D uffd_open(O_CLOEXEC); if (ufd =3D=3D -1) { - error_report("%s: syscall __NR_userfaultfd failed: %s", __func__, - strerror(errno)); + error_report("%s: uffd_open() failed: %s", __func__, strerror(errn= o)); return false; } =20 @@ -375,7 +374,7 @@ bool postcopy_ram_supported_by_host(MigrationIncomingSt= ate *mis) goto out; } =20 - ufd =3D syscall(__NR_userfaultfd, O_CLOEXEC); + ufd =3D uffd_open(O_CLOEXEC); if (ufd =3D=3D -1) { error_report("%s: userfaultfd not available: %s", __func__, strerror(errno)); @@ -1160,7 +1159,7 @@ static int postcopy_temp_pages_setup(MigrationIncomin= gState *mis) int postcopy_ram_incoming_setup(MigrationIncomingState *mis) { /* Open the fd for the kernel to give us userfaults */ - mis->userfault_fd =3D syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK= ); + mis->userfault_fd =3D uffd_open(O_CLOEXEC | O_NONBLOCK); if (mis->userfault_fd =3D=3D -1) { error_report("%s: Failed to open userfault fd: %s", __func__, strerror(errno)); diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 1dd32c9506..109bc8e7b1 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -61,14 +61,14 @@ static bool uffd_feature_thread_id; #if defined(__linux__) && defined(__NR_userfaultfd) && defined(CONFIG_EVEN= TFD) #include #include -#include +#include "qemu/userfaultfd.h" =20 static bool ufd_version_check(void) { struct uffdio_api api_struct; uint64_t ioctl_mask; =20 - int ufd =3D syscall(__NR_userfaultfd, O_CLOEXEC); + int ufd =3D uffd_open(O_CLOEXEC); =20 if (ufd =3D=3D -1) { g_test_message("Skipping test: userfaultfd not available"); diff --git a/util/userfaultfd.c b/util/userfaultfd.c index f1cd6af2b1..4953b3137d 100644 --- a/util/userfaultfd.c +++ b/util/userfaultfd.c @@ -19,6 +19,15 @@ #include #include =20 +int uffd_open(int flags) +{ +#if defined(__NR_userfaultfd) + return syscall(__NR_userfaultfd, flags); +#else + return -EINVAL; +#endif +} + /** * uffd_query_features: query UFFD features * @@ -32,7 +41,7 @@ int uffd_query_features(uint64_t *features) struct uffdio_api api_struct =3D { 0 }; int ret =3D -1; =20 - uffd_fd =3D syscall(__NR_userfaultfd, O_CLOEXEC); + uffd_fd =3D uffd_open(O_CLOEXEC); if (uffd_fd < 0) { trace_uffd_query_features_nosys(errno); return -1; @@ -69,7 +78,7 @@ int uffd_create_fd(uint64_t features, bool non_blocking) uint64_t ioctl_mask =3D BIT(_UFFDIO_REGISTER) | BIT(_UFFDIO_UNREGISTER= ); =20 flags =3D O_CLOEXEC | (non_blocking ? O_NONBLOCK : 0); - uffd_fd =3D syscall(__NR_userfaultfd, flags); + uffd_fd =3D uffd_open(flags); if (uffd_fd < 0) { trace_uffd_create_fd_nosys(errno); return -1; --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731520; cv=none; d=zohomail.com; s=zohoarc; b=Jxxx6iPcNlB+DHV6rem17QzRsjnCw8nW/Yhc2E7fVcx7Bl1H48tLk74nqg5nwYCDcLXryo+yDtET7Ab/iTLEjuHewad+TpVKzZ1s2yeRGxoJcE0PSClN7AjPzbVQWg5z0q0LgiNybCneY8OOLJYrirJHUMayGnnGZ2cnTgAfiRY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731520; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ypsPnyWgNLqIOy/pWtoUGHZ1xhiN89F5ncT+BrUMGVs=; b=UWV+ADqUHU6mVJviMbsG/cjXHv9sMYv9Hj7ga/fjigqpwfNCVLQX8IEg9DTDrLkEjqtdw0/qPxSDSSNibDzlNre/2FYAvTOGcel5V9dgNFEG9wHSoMY/pptHKk90PON9/CiQxxcpCw7X+PTqCYOL8olZD/1n34EQXkHInyLVESU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731520144679.2437171036376; Mon, 6 Feb 2023 16:58:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCIo-00034m-29; Mon, 06 Feb 2023 19:57:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIm-00032v-2h for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:40 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIk-0003ev-Ke for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:39 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-184-r3Vz7fzAPviNaExW0GUCvQ-1; Mon, 06 Feb 2023 19:57:33 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 62E753C025B6; Tue, 7 Feb 2023 00:57:32 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 71C5F492C3C; Tue, 7 Feb 2023 00:57:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731458; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ypsPnyWgNLqIOy/pWtoUGHZ1xhiN89F5ncT+BrUMGVs=; b=Z3b4Y7v1bdZanp0tJOMHE4l7WrOGAPxoZj7rLJUCBu2BUS73OrSWineOELlm6gAchYAvUp gW7HGEQsA3XeJwNWPkIvONukBJF6Gg55Bzjb9vuxn0BlM/s+mawK85HO00ZuBFNrpGCCXj FxYrAXhVDQRHKLKGkEJywqY+HrEOgsU= X-MC-Unique: r3Vz7fzAPviNaExW0GUCvQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , qemu-stable@nongnu.org, Peter Xu Subject: [PULL 07/30] migration/ram: Fix populate_read_range() Date: Tue, 7 Feb 2023 01:56:27 +0100 Message-Id: <20230207005650.1810-8-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731520921100003 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand Unfortunately, commit f7b9dcfbcf44 broke populate_read_range(): the loop end condition is very wrong, resulting in that function not populating the full range. Lets' fix that. Fixes: f7b9dcfbcf44 ("migration/ram: Factor out populating pages readable i= n ram_block_populate_pages()") Cc: qemu-stable@nongnu.org Reviewed-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- migration/ram.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 885d7dbf23..ba228eead4 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1774,13 +1774,15 @@ out: static inline void populate_read_range(RAMBlock *block, ram_addr_t offset, ram_addr_t size) { + const ram_addr_t end =3D offset + size; + /* * We read one byte of each page; this will preallocate page tables if * required and populate the shared zeropage on MAP_PRIVATE anonymous = memory * where no page was populated yet. This might require adaption when * supporting other mappings, like shmem. */ - for (; offset < size; offset +=3D block->page_size) { + for (; offset < end; offset +=3D block->page_size) { char tmp =3D *((char *)block->host + offset); =20 /* Don't optimize the read out */ --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731551; cv=none; d=zohomail.com; s=zohoarc; b=P9091DXWYoW+LmPTLzOMKuKsHlVzs4Vq6AvvBZ+xoEmMB1Sqxqm2lkQX8C0qyOXHcFfHSw5N5QfgMhhVhsv/LY1kaY9LPNzE2o4vXJkjxGQvYcNw3e1YspMezdkQS3btJIHap++rPlsys3Em8jAR1lOjAaTBhOdHOR0oE2tsrro= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731551; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=haPCYqw8dBiTHO9WqDCahmHDPX0ls4553+sPmN0fSgI=; b=cm63GdvyD/Jo7S14ZsMhPrSTojtEDaxR6UkSj4U0tMuh7ZpS9XeXOvwta7gwdq9kO6GhNJPapzUIC4+WlKN8VwX4ZhwWhlyzLTUHSY+BAAziX1PiOhx//llALR8S3c4iBxqqMiv/gwN96oErmUJhsepWD7WErC7GZ6cmRM1VTm4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 167573155189370.02729905220781; Mon, 6 Feb 2023 16:59:11 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCIs-00038D-1y; Mon, 06 Feb 2023 19:57:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIq-000376-Bg for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIo-0003fh-Se for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:44 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-376-9qC4XlEYMheI8RWvxJ3_MA-1; Mon, 06 Feb 2023 19:57:38 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 94B3480D0ED; Tue, 7 Feb 2023 00:57:37 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id A895F492C3C; Tue, 7 Feb 2023 00:57:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731462; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=haPCYqw8dBiTHO9WqDCahmHDPX0ls4553+sPmN0fSgI=; b=OBTqJuY4r0YJBLy/vXFXeWJP8R+6bJ0EOZ0A3qgctguvcrHXnSP8w9S7MUtan0G1iwUKkd CQCOJbPHNSwhd4HlKVcIPundvbpD1NJZDAmmDaumpxTmFWTeXuGe1xgfOsPyrCj2dfbake sX5BRmLIv161oNHDwNjajC+drfGgIs0= X-MC-Unique: 9qC4XlEYMheI8RWvxJ3_MA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , qemu-stable@nongnu.org, Peter Xu Subject: [PULL 08/30] migration/ram: Fix error handling in ram_write_tracking_start() Date: Tue, 7 Feb 2023 01:56:28 +0100 Message-Id: <20230207005650.1810-9-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731553053100019 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand If something goes wrong during uffd_change_protection(), we would miss to unregister uffd-wp and not release our reference. Fix it by performing the uffd_change_protection(true) last. Note that a uffd_change_protection(false) on the recovery path without a prior uffd_change_protection(false) is fine. Fixes: 278e2f551a09 ("migration: support UFFD write fault processing in ram= _save_iterate()") Cc: qemu-stable@nongnu.org Reviewed-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- migration/ram.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ba228eead4..73e5ca93e5 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1896,13 +1896,14 @@ int ram_write_tracking_start(void) block->max_length, UFFDIO_REGISTER_MODE_WP, NULL)) { goto fail; } + block->flags |=3D RAM_UF_WRITEPROTECT; + memory_region_ref(block->mr); + /* Apply UFFD write protection to the block memory range */ if (uffd_change_protection(rs->uffdio_fd, block->host, block->max_length, true, false)) { goto fail; } - block->flags |=3D RAM_UF_WRITEPROTECT; - memory_region_ref(block->mr); =20 trace_ram_write_tracking_ramblock_start(block->idstr, block->page_= size, block->host, block->max_length); --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731495; cv=none; d=zohomail.com; s=zohoarc; b=SJvQ2iiMeMZySNxoMcrr/5Qr81aatuyp/i32GiInG7VzVVcMP30R2RBfzD864V+S9356evgJcgFQ1ij6suH/6ywAsMlCB4YK53qAH0bvEzTMFF8Zg3Ij9C9GmzCfeuAf0hSWj3+wcO+4af2X2rxAS7gNRrvhj/F/dATnSQUpgn0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731495; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qzSBIR6LRz6N4Dl8hXA3eueT1HX6gOU9Tw+HrG3oo/w=; b=k9rRmXQZ3YPk5BRpHdCAf6/OfkYWLGXK+UtGsqg5N2T9Ffl5JG89SiMN1aSuC2dQW5XIpC71jgM9RWLpdDdKwKV9/FvQMfnhg/trtX00peXWOAG2kfllWR8UAh7U5pcwH+Dk2zzrXjKMaTrnlOfs6FO4wFooLgkbhnY7e9v2mjk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731495608631.747417005486; Mon, 6 Feb 2023 16:58:15 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCIw-0003CM-WA; Mon, 06 Feb 2023 19:57:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIv-0003BZ-S2 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:49 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIu-0003hZ-9w for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:49 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-657-GR52TcoOO6qY4ikAJib3BQ-1; Mon, 06 Feb 2023 19:57:43 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9F65A85A588; Tue, 7 Feb 2023 00:57:42 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id DAF28492C3C; Tue, 7 Feb 2023 00:57:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731466; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qzSBIR6LRz6N4Dl8hXA3eueT1HX6gOU9Tw+HrG3oo/w=; b=HK4nUVmS3MeObKUt56U9uuMMKIjvvuraCn4e0QdLQWX6PPO2+Xngf8IJETP10dv7lqO606 7n1WurIt4NQOH68XTYnnqnywBxWm0KcV3HwSoHxK2iti0NthrmqPhoICmJIEQ3iOk/Cn6k JYFxCBQRjt0ZAPuza8QcMR3mJKvcZ/M= X-MC-Unique: GR52TcoOO6qY4ikAJib3BQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 09/30] migration/ram: Don't explicitly unprotect when unregistering uffd-wp Date: Tue, 7 Feb 2023 01:56:29 +0100 Message-Id: <20230207005650.1810-10-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731496886100001 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand When unregistering uffd-wp, older kernels before commit f369b07c86143 ("mm/uffd:reset write protection when unregister with wp-mode") won't clear the uffd-wp PTE bit. When re-registering uffd-wp, the previous uffd-wp PTE bits would trigger again. With above commit, the kernel will clear the uffd-wp PTE bits when unregistering itself. Consequently, we'll clear the uffd-wp PTE bits now twice -- whereby we don't care about clearing them at all: a new background snapshot will re-register uffd-wp and re-protect all memory either way. So let's skip the manual clearing of uffd-wp. If ever relevant, we could clear conditionally in uffd_unregister_memory() -- we just need a way to figure out more recent kernels. Reviewed-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- migration/ram.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 73e5ca93e5..efaae07dd8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1918,12 +1918,6 @@ fail: if ((block->flags & RAM_UF_WRITEPROTECT) =3D=3D 0) { continue; } - /* - * In case some memory block failed to be write-protected - * remove protection and unregister all succeeded RAM blocks - */ - uffd_change_protection(rs->uffdio_fd, block->host, block->max_leng= th, - false, false); uffd_unregister_memory(rs->uffdio_fd, block->host, block->max_leng= th); /* Cleanup flags and remove reference */ block->flags &=3D ~RAM_UF_WRITEPROTECT; @@ -1949,9 +1943,6 @@ void ram_write_tracking_stop(void) if ((block->flags & RAM_UF_WRITEPROTECT) =3D=3D 0) { continue; } - /* Remove protection and unregister all affected RAM blocks */ - uffd_change_protection(rs->uffdio_fd, block->host, block->max_leng= th, - false, false); uffd_unregister_memory(rs->uffdio_fd, block->host, block->max_leng= th); =20 trace_ram_write_tracking_ramblock_stop(block->idstr, block->page_s= ize, --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731722; cv=none; d=zohomail.com; s=zohoarc; b=WZ6xE/58ljQN5ZaGwBe2xTc+07A4mGzCmjt/tUsO3idRfmviU+aO0Fgn8eRYDKsQFIbXd6EAN0kOHyT5N+GUkQzOaexryRgg+CUaOL3S0CskbxiGRqliYOgapPnLgJDpEGAO4jPgybDIWhSHW4k9Nd9tkn7//9X6HBrXrv+xmH0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731722; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=6lIg7aqLoDZba0OtgiOTBlqeFrJWsZnu9N17z0NU7N0=; b=NrlIuFenq7u9wNHOWGpd4ik8R6DX+x6KI3rPS5le2HlMNdqTSbiEA2ut0C58CPX7yNKktHk5/u5RaKIlU8VdoNXsZI1N4UGUVJ/+ScFWw6xKhirPGpyEM0xDSKDKYPPqwnvpN56/efB7bknbFCWJ0FZHV3ILrnjiMJHJQKe7+M8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731722001536.8644366516348; Mon, 6 Feb 2023 17:02:02 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJ1-0003Ji-49; Mon, 06 Feb 2023 19:57:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIz-0003Dj-It for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:53 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCIy-0003iG-5d for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:53 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-9-UoOnwQr8NsuicMww8-sPjg-1; Mon, 06 Feb 2023 19:57:48 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B07DE3C025B6; Tue, 7 Feb 2023 00:57:47 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id E57EA492C3C; Tue, 7 Feb 2023 00:57:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731471; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6lIg7aqLoDZba0OtgiOTBlqeFrJWsZnu9N17z0NU7N0=; b=eGvdoqnZWmIXzd9K8XuAZdIKpQUbQmqlxsJSKucUvRUUMQL1iWVTSM+8GuTFKqqiCJKnN1 Lht3vR5F0DqqNuZqjtHOpNsf2okkFRP9qHVEvCmCzrr+/BWsxqLNpmDwqzktoDMwdh00DN H3MOyXOkSDmtGz3qJncNyhQX87Clj+c= X-MC-Unique: UoOnwQr8NsuicMww8-sPjg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 10/30] migration/ram: Rely on used_length for uffd_change_protection() Date: Tue, 7 Feb 2023 01:56:30 +0100 Message-Id: <20230207005650.1810-11-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731724075100014 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand ram_mig_ram_block_resized() will abort migration (including background snapshots) when resizing a RAMBlock. ram_block_populate_read() will only populate RAM up to used_length, so at least for anonymous memory protecting everything between used_length and max_length won't actually be protected and is just a NOP. So let's only protect everything up to used_length. Note: it still makes sense to register uffd-wp for max_length, such that RAM_UF_WRITEPROTECT is independent of a changing used_length. Reviewed-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- migration/ram.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index efaae07dd8..a6956c9e7d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1901,7 +1901,7 @@ int ram_write_tracking_start(void) =20 /* Apply UFFD write protection to the block memory range */ if (uffd_change_protection(rs->uffdio_fd, block->host, - block->max_length, true, false)) { + block->used_length, true, false)) { goto fail; } =20 --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731510; cv=none; d=zohomail.com; s=zohoarc; b=hJwZu1EynRqGQivZcDxUJ2Bo5yhD/4QPQ0XN5qahxj+qm++UlcGTDHoahRu4HHS5mYhYlf0kMXqrCgJumAHg4uthWvmM3uaYun+z4V8PQUKjj7stw6bj0SQYNPAG11Q5guC5QM5OFuvhlShRjnQsSAVlCIcyf13SAoLDydGNdgE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731510; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DYITrzVxeANIn4myE/xtt+IWMtwojgiIpFXvIFEzIJQ=; b=L8ZdziXT6OdhppY6qOrg2/uCom3i9sqOHTed1l3UJcm0uPF/gpQobdxEHT1xv029tkz5JjdrgR397M0zOeFeJKRXbLkGkF8oeBEthZjt4Tn/5160eaJu8aHXlKvvPGee5LNkExv2D+n3YjgfYrVFBJNz2kgxHb6ktGAUvurhvyY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731510555435.4898958830654; Mon, 6 Feb 2023 16:58:30 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJ6-0003OL-3D; Mon, 06 Feb 2023 19:58:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJ4-0003Nu-Ob for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:58 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJ3-0003jL-2q for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:57:58 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-20-I1wun_KfPuyJwkeiCjBvVA-1; Mon, 06 Feb 2023 19:57:53 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BFB4285A5A3; Tue, 7 Feb 2023 00:57:52 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0216F492C3C; Tue, 7 Feb 2023 00:57:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731476; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DYITrzVxeANIn4myE/xtt+IWMtwojgiIpFXvIFEzIJQ=; b=i53eXb+LE/FkyehmV6fnJlF/jIG9oTtnyLsX/fqeFNIo3e7em0CbzWnwfOodEJoTgIPw4X YuaSJdSmHoTM26aPBVwIeUdCVumx6gefNYsIhHl7jesIGv5Vd2t5Jr/sQXWDujOx2RHgf3 CrgBM994D4lUQiaVKoIPkJcZanJ6/0c= X-MC-Unique: I1wun_KfPuyJwkeiCjBvVA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 11/30] migration/ram: Optimize ram_write_tracking_start() for RamDiscardManager Date: Tue, 7 Feb 2023 01:56:31 +0100 Message-Id: <20230207005650.1810-12-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731510874100003 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand ram_block_populate_read() already optimizes for RamDiscardManager. However, ram_write_tracking_start() will still try protecting discarded memory ranges. Let's optimize, because discarded ranges don't map any pages and (1) For anonymous memory, trying to protect using uffd-wp without a mapped page is ignored by the kernel and consequently a NOP. (2) For shared/file-backed memory, we will fill present page tables in the range with PTE markers. However, we will even allocate page tables just to fill them with unnecessary PTE markers and effectively waste memory. So let's exclude these ranges, just like ram_block_populate_read() already does. Reviewed-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- migration/ram.c | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index a6956c9e7d..7f6d5efe8d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1865,6 +1865,39 @@ void ram_write_tracking_prepare(void) } } =20 +static inline int uffd_protect_section(MemoryRegionSection *section, + void *opaque) +{ + const hwaddr size =3D int128_get64(section->size); + const hwaddr offset =3D section->offset_within_region; + RAMBlock *rb =3D section->mr->ram_block; + int uffd_fd =3D (uintptr_t)opaque; + + return uffd_change_protection(uffd_fd, rb->host + offset, size, true, + false); +} + +static int ram_block_uffd_protect(RAMBlock *rb, int uffd_fd) +{ + assert(rb->flags & RAM_UF_WRITEPROTECT); + + /* See ram_block_populate_read() */ + if (rb->mr && memory_region_has_ram_discard_manager(rb->mr)) { + RamDiscardManager *rdm =3D memory_region_get_ram_discard_manager(r= b->mr); + MemoryRegionSection section =3D { + .mr =3D rb->mr, + .offset_within_region =3D 0, + .size =3D rb->mr->size, + }; + + return ram_discard_manager_replay_populated(rdm, §ion, + uffd_protect_section, + (void *)(uintptr_t)uff= d_fd); + } + return uffd_change_protection(uffd_fd, rb->host, + rb->used_length, true, false); +} + /* * ram_write_tracking_start: start UFFD-WP memory tracking * @@ -1900,8 +1933,7 @@ int ram_write_tracking_start(void) memory_region_ref(block->mr); =20 /* Apply UFFD write protection to the block memory range */ - if (uffd_change_protection(rs->uffdio_fd, block->host, - block->used_length, true, false)) { + if (ram_block_uffd_protect(block, uffd_fd)) { goto fail; } =20 --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731499; cv=none; d=zohomail.com; s=zohoarc; b=cjWl7MSKx8R+bSaIJwYmiZ4O5FSkH6gclV9ZVNJsuTsWMKE5sh9BrD0xGbT9Ot1kOaseDFp0y5hvImsNalDVw9TYVFYWPUaPc/9r1LnAi124YixMp/boFaiA9Bsn1o7Q+oV7RnWfC2BQA8PIqzUGCtZvlVoR3tqvrg8PxjHVWlw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731499; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3H2e5YeH5/21K34UaegkcQcfDpUz2GgM6q4M95PjxcQ=; b=Bg2Nvq+lewnWotNpn+vcDBci1XCExCnWmHxdkOhFUkYooACCYJuWm68HsJHgkaKX+AdrE1OHcj/glLDMB8+cHwzVhYh+wHq8XSgXLc1hgUfzl2wlj4XYUDdEVF9O2zQFIBqWkZIw195lH87PNeM4WPy7a6pus8MOcY20t8HmPlM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731499801377.8943547538456; Mon, 6 Feb 2023 16:58:19 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJD-0003Vr-Gw; Mon, 06 Feb 2023 19:58:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJC-0003Ur-7l for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:06 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJ8-0003jw-J6 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:05 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-271-IqC1i6GfNkqIiDRMOqEECg-1; Mon, 06 Feb 2023 19:57:58 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D02913C025B6; Tue, 7 Feb 2023 00:57:57 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 170CB492C3C; Tue, 7 Feb 2023 00:57:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731481; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3H2e5YeH5/21K34UaegkcQcfDpUz2GgM6q4M95PjxcQ=; b=bJdsunhWe1TO97GM9ANSBYkeWYVrLXOBo3O5bT3KsgtN511G0QAnz7UNCS1tLe9xHiqxxA yJYe9wNZSqeexPGgPqM1hczL/8WuLibfouqtY6m6wXuy0vmEEEPcL2JSQnA/GSIX7+3BCv MuMt2VPpmkbcqrmyBQZe2w61akNj+pI= X-MC-Unique: IqC1i6GfNkqIiDRMOqEECg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 12/30] migration/savevm: Move more savevm handling into vmstate_save() Date: Tue, 7 Feb 2023 01:56:32 +0100 Message-Id: <20230207005650.1810-13-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731500855100003 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand Let's move more code into vmstate_save(), reducing code duplication and preparing for reuse of vmstate_save() in qemu_savevm_state_setup(). We have to move vmstate_save() to make the compiler happy. We'll now also trace from qemu_save_device_state(), triggering the same tracepoints as previously called from qemu_savevm_state_complete_precopy_non_iterable() only. Note that qemu_save_device_state() ignores iterable device state, such as RAM, and consequently doesn't trigger some other trace points (e.g., trace_savevm_state_setup()). Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- migration/savevm.c | 79 ++++++++++++++++++++++------------------------ 1 file changed, 37 insertions(+), 42 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index e1caa3ea7c..3e3631652e 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -898,17 +898,6 @@ static void vmstate_save_old_style(QEMUFile *f, SaveSt= ateEntry *se, } } =20 -static int vmstate_save(QEMUFile *f, SaveStateEntry *se, - JSONWriter *vmdesc) -{ - trace_vmstate_save(se->idstr, se->vmsd ? se->vmsd->name : "(old)"); - if (!se->vmsd) { - vmstate_save_old_style(f, se, vmdesc); - return 0; - } - return vmstate_save_state(f, se->vmsd, se->opaque, vmdesc); -} - /* * Write the header for device section (QEMU_VM_SECTION START/END/PART/FUL= L) */ @@ -942,6 +931,43 @@ static void save_section_footer(QEMUFile *f, SaveState= Entry *se) } } =20 +static int vmstate_save(QEMUFile *f, SaveStateEntry *se, JSONWriter *vmdes= c) +{ + int ret; + + if ((!se->ops || !se->ops->save_state) && !se->vmsd) { + return 0; + } + if (se->vmsd && !vmstate_save_needed(se->vmsd, se->opaque)) { + trace_savevm_section_skip(se->idstr, se->section_id); + return 0; + } + + trace_savevm_section_start(se->idstr, se->section_id); + save_section_header(f, se, QEMU_VM_SECTION_FULL); + if (vmdesc) { + json_writer_start_object(vmdesc, NULL); + json_writer_str(vmdesc, "name", se->idstr); + json_writer_int64(vmdesc, "instance_id", se->instance_id); + } + + trace_vmstate_save(se->idstr, se->vmsd ? se->vmsd->name : "(old)"); + if (!se->vmsd) { + vmstate_save_old_style(f, se, vmdesc); + } else { + ret =3D vmstate_save_state(f, se->vmsd, se->opaque, vmdesc); + if (ret) { + return ret; + } + } + + trace_savevm_section_end(se->idstr, se->section_id, 0); + save_section_footer(f, se); + if (vmdesc) { + json_writer_end_object(vmdesc); + } + return 0; +} /** * qemu_savevm_command_send: Send a 'QEMU_VM_COMMAND' type element with the * command and associated data. @@ -1375,31 +1401,11 @@ int qemu_savevm_state_complete_precopy_non_iterable= (QEMUFile *f, json_writer_int64(vmdesc, "page_size", qemu_target_page_size()); json_writer_start_array(vmdesc, "devices"); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - - if ((!se->ops || !se->ops->save_state) && !se->vmsd) { - continue; - } - if (se->vmsd && !vmstate_save_needed(se->vmsd, se->opaque)) { - trace_savevm_section_skip(se->idstr, se->section_id); - continue; - } - - trace_savevm_section_start(se->idstr, se->section_id); - - json_writer_start_object(vmdesc, NULL); - json_writer_str(vmdesc, "name", se->idstr); - json_writer_int64(vmdesc, "instance_id", se->instance_id); - - save_section_header(f, se, QEMU_VM_SECTION_FULL); ret =3D vmstate_save(f, se, vmdesc); if (ret) { qemu_file_set_error(f, ret); return ret; } - trace_savevm_section_end(se->idstr, se->section_id, 0); - save_section_footer(f, se); - - json_writer_end_object(vmdesc); } =20 if (inactivate_disks) { @@ -1618,21 +1624,10 @@ int qemu_save_device_state(QEMUFile *f) if (se->is_ram) { continue; } - if ((!se->ops || !se->ops->save_state) && !se->vmsd) { - continue; - } - if (se->vmsd && !vmstate_save_needed(se->vmsd, se->opaque)) { - continue; - } - - save_section_header(f, se, QEMU_VM_SECTION_FULL); - ret =3D vmstate_save(f, se, NULL); if (ret) { return ret; } - - save_section_footer(f, se); } =20 qemu_put_byte(f, QEMU_VM_EOF); --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731508; cv=none; d=zohomail.com; s=zohoarc; b=XFST7/R3xPeJGOiWos5DC6CfKyeUNVRaY93NFIlQpdZwwivQL9iUCrPCrxCxRc51I40pqP7PleoKLcJmXjJKQNaSvYpx51FuED12qAFssVnvFjyl0GH53y4pQUYBpv/XzA2ufeLCxxj7xOft2zag6XN7qmdIZgjKz+8bFld1DYk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731508; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XNMcu/nOPKMigMVPfZvKyLlihBT8/C5Yazboh+qWDkU=; b=NhixZ6P72qgeyW1HhMZ4oe6YT/kGFDxjZsNb4pCaGCyMi70RQqAe5QmS06TiIztKy4YtdXx+N3g7T74nqBaFaS2qAOJ4r778YAWMN3Y2t/ePFzmTDDQ6OXZAw1sX8Vx/uv7Kt3V9aYFLrH+4sDpIlvyRX+gY8d3/Eqm9C0b430Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731508772452.9312767914147; Mon, 6 Feb 2023 16:58:28 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJG-0003gC-PN; Mon, 06 Feb 2023 19:58:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJF-0003d3-7c for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJD-0003kf-Fn for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:08 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-149-DbZrM0DIOgCFgksgAvN1Mg-1; Mon, 06 Feb 2023 19:58:03 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E1874885627; Tue, 7 Feb 2023 00:58:02 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 24A22492C3C; Tue, 7 Feb 2023 00:57:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731486; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XNMcu/nOPKMigMVPfZvKyLlihBT8/C5Yazboh+qWDkU=; b=g4/eBN6gyiFllf54N0nfsB9fzT0VxtCkBYUsjZAByS6TpqrTTa/VRJcvg41RXpjCi1GW/C x9EWJ21Z1pfPia/K/+OzTjlO5HxOfKGpsO2yuUqEqn6Uj1z8Co5PI08EVFZaJkOGdr8yxx zcqpnrKZ7udSUxwaE+biY1brkActZnk= X-MC-Unique: DbZrM0DIOgCFgksgAvN1Mg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 13/30] migration/savevm: Prepare vmdesc json writer in qemu_savevm_state_setup() Date: Tue, 7 Feb 2023 01:56:33 +0100 Message-Id: <20230207005650.1810-14-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731510880100004 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand ... and store it in the migration state. This is a preparation for storing selected vmds's already in qemu_savevm_state_setup(). Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- migration/migration.h | 4 ++++ migration/migration.c | 2 ++ migration/savevm.c | 18 ++++++++++++------ 3 files changed, 18 insertions(+), 6 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index ae4ffd3454..66511ce532 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -17,6 +17,7 @@ #include "exec/cpu-common.h" #include "hw/qdev-core.h" #include "qapi/qapi-types-migration.h" +#include "qapi/qmp/json-writer.h" #include "qemu/thread.h" #include "qemu/coroutine_int.h" #include "io/channel.h" @@ -366,6 +367,9 @@ struct MigrationState { * This save hostname when out-going migration starts */ char *hostname; + + /* QEMU_VM_VMDESCRIPTION content filled for all non-iterable devices. = */ + JSONWriter *vmdesc; }; =20 void migrate_set_state(int *state, int old_state, int new_state); diff --git a/migration/migration.c b/migration/migration.c index 6d4cd8083b..c3ad4cd670 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1933,6 +1933,8 @@ static void migrate_fd_cleanup(MigrationState *s) =20 g_free(s->hostname); s->hostname =3D NULL; + json_writer_free(s->vmdesc); + s->vmdesc =3D NULL; =20 qemu_savevm_state_cleanup(); =20 diff --git a/migration/savevm.c b/migration/savevm.c index 3e3631652e..28f88b5521 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -42,7 +42,6 @@ #include "postcopy-ram.h" #include "qapi/error.h" #include "qapi/qapi-commands-migration.h" -#include "qapi/qmp/json-writer.h" #include "qapi/clone-visitor.h" #include "qapi/qapi-builtin-visit.h" #include "qapi/qmp/qerror.h" @@ -1190,10 +1189,16 @@ bool qemu_savevm_state_guest_unplug_pending(void) =20 void qemu_savevm_state_setup(QEMUFile *f) { + MigrationState *ms =3D migrate_get_current(); SaveStateEntry *se; Error *local_err =3D NULL; int ret; =20 + ms->vmdesc =3D json_writer_new(false); + json_writer_start_object(ms->vmdesc, NULL); + json_writer_int64(ms->vmdesc, "page_size", qemu_target_page_size()); + json_writer_start_array(ms->vmdesc, "devices"); + trace_savevm_state_setup(); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { if (!se->ops || !se->ops->save_setup) { @@ -1391,15 +1396,12 @@ int qemu_savevm_state_complete_precopy_non_iterable= (QEMUFile *f, bool in_postcopy, bool inactivate_disks) { - g_autoptr(JSONWriter) vmdesc =3D NULL; + MigrationState *ms =3D migrate_get_current(); + JSONWriter *vmdesc =3D ms->vmdesc; int vmdesc_len; SaveStateEntry *se; int ret; =20 - vmdesc =3D json_writer_new(false); - json_writer_start_object(vmdesc, NULL); - json_writer_int64(vmdesc, "page_size", qemu_target_page_size()); - json_writer_start_array(vmdesc, "devices"); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { ret =3D vmstate_save(f, se, vmdesc); if (ret) { @@ -1434,6 +1436,10 @@ int qemu_savevm_state_complete_precopy_non_iterable(= QEMUFile *f, qemu_put_buffer(f, (uint8_t *)json_writer_get(vmdesc), vmdesc_len); } =20 + /* Free it now to detect any inconsistencies. */ + json_writer_free(vmdesc); + ms->vmdesc =3D NULL; + return 0; } =20 --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731556; cv=none; d=zohomail.com; s=zohoarc; b=CswW9vWS8UkqwdbuVEsdB9wSDHrIwoq89BqMHVohWIx/DnCIUxFhLpFOg7ljvZyWInM/hxPbixZz1Bb4YkcAijkkibc7bjsepWCQdzlQAapfEbL3GCSAi8gQ1yjOdHWH5nVptFfcO5XM3TGaB/ltqAsV9zK0eO8zpffXOET6wCU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731556; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Lw58clu7Y1bzBf2f7fvddUJNRU5/Wbva2ZRRL1vM7CM=; b=KfQBJbmo2ihE1AX+CFd4YSoJOxJhlmIPjUkbVcjy1Goo29vP7RAvGT4iEICRVapqwrfeZ5Nj/rAwgBfpr83QhgMv8YdNgnrHzobOK45CFRjO2y/+vLa3DQ9lIMLZlLGAzonrglbSDF2vt9PUhKD71U2i3/TdKohtYDOtOlDC+2s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731556542258.8031522894347; Mon, 6 Feb 2023 16:59:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJQ-0004Py-AR; Mon, 06 Feb 2023 19:58:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJO-0004DH-Ha for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:18 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJM-0003mD-J6 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:18 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-625-5gYAzHyFMp-7ya_0fhxQiA-1; Mon, 06 Feb 2023 19:58:09 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5FF601871D97; Tue, 7 Feb 2023 00:58:08 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 33CDE492C3C; Tue, 7 Feb 2023 00:58:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731496; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lw58clu7Y1bzBf2f7fvddUJNRU5/Wbva2ZRRL1vM7CM=; b=hAdkXvjy6FK3BAPASg74KJ8bH8POTA3U/zpUm4MfnijNCcDD+QpqWOOh+ay9VV98vJmi73 qESqEg4rpxiwZ0khB/2smJgSn3E2vl1q2d8a1E1gtFja6PLDI/t+B1oi3Wulm24NFjT3ov h6G8+Cds74l1cNtiaAHC9WWCga9maNM= X-MC-Unique: 5gYAzHyFMp-7ya_0fhxQiA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 14/30] migration/savevm: Allow immutable device state to be migrated early (i.e., before RAM) Date: Tue, 7 Feb 2023 01:56:34 +0100 Message-Id: <20230207005650.1810-15-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731557132100003 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand For virtio-mem, we want to have the plugged/unplugged state of memory blocks available before migrating any actual RAM content, and perform sanity checks before touching anything on the destination. This information is immutable on the migration source while migration is active, We want to use this information for proper preallocation support with migration: currently, we don't preallocate memory on the migration target, and especially with hugetlb, we can easily run out of hugetlb pages during RAM migration and will crash (SIGBUS) instead of catching this gracefully via preallocation. Migrating device state via a VMSD before we start iterating is currently impossible: the only approach that would be possible is avoiding a VMSD and migrating state manually during save_setup(), to be restored during load_state(). Let's allow for migrating device state via a VMSD early, during the setup phase in qemu_savevm_state_setup(). To keep it simple, we indicate applicable VMSD's using an "early_setup" flag. Note that only very selected devices (i.e., ones seriously messing with RAM setup) are supposed to make use of such early state migration. While at it, also use a bool for the "unmigratable" member. Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela S Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- include/migration/vmstate.h | 16 +++++++++++++++- migration/savevm.c | 14 ++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index ad24aa1934..64680d824e 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -178,7 +178,21 @@ struct VMStateField { =20 struct VMStateDescription { const char *name; - int unmigratable; + bool unmigratable; + /* + * This VMSD describes something that should be sent during setup phase + * of migration. It plays similar role as save_setup() for explicitly + * registered vmstate entries, so it can be seen as a way to describe + * save_setup() in VMSD structures. + * + * Note that for now, a SaveStateEntry cannot have a VMSD and + * operations (e.g., save_setup()) set at the same time. Consequently, + * save_setup() and a VMSD with early_setup set to true are mutually + * exclusive. For this reason, also early_setup VMSDs are migrated in a + * QEMU_VM_SECTION_FULL section, while save_setup() data is migrated in + * a QEMU_VM_SECTION_START section. + */ + bool early_setup; int version_id; int minimum_version_id; MigrationPriority priority; diff --git a/migration/savevm.c b/migration/savevm.c index 28f88b5521..6d985ad4af 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1201,6 +1201,15 @@ void qemu_savevm_state_setup(QEMUFile *f) =20 trace_savevm_state_setup(); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (se->vmsd && se->vmsd->early_setup) { + ret =3D vmstate_save(f, se, ms->vmdesc); + if (ret) { + qemu_file_set_error(f, ret); + break; + } + continue; + } + if (!se->ops || !se->ops->save_setup) { continue; } @@ -1403,6 +1412,11 @@ int qemu_savevm_state_complete_precopy_non_iterable(= QEMUFile *f, int ret; =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (se->vmsd && se->vmsd->early_setup) { + /* Already saved during qemu_savevm_state_setup(). */ + continue; + } + ret =3D vmstate_save(f, se, vmdesc); if (ret) { qemu_file_set_error(f, ret); --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731671; cv=none; d=zohomail.com; s=zohoarc; b=cbg2hej9wIAa8/o55Ku77/leQUwBdfyHcACNO38vjHrm+waBDo3lb5+jIcYhBT8Y5w6aus3Yyhp6BkAPyOikop/rIvKyIy7RhqqjnVdK8z76077fD5ZehRgZdmK5gtdNi+n6LEx3kgXRFcXFtAlAbtK7IigrUVmdxt+MpcuNgos= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731671; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DenhEVPqZWI8U/oaBiFpU68cmZm9r5bjKt8BUGFm1K4=; b=PNosAgaWMUZ0Q7BZyvN1kVI/eOCrzvPkl6mp6hdJaR09pINBe5Vt6jzE/23xYoBKajF0N4dkZzttdo0hDnNoP/i72KEms+fAteWRqo/euPVv2ercVUEk1P/WjJz6EVb02owo+UyL05zayaxBQudA1iwMgM6co1Mlg1SMaU5DArY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731671538635.0808347516809; Mon, 6 Feb 2023 17:01:11 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJU-0004v8-8d; Mon, 06 Feb 2023 19:58:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJS-0004gz-HV for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:22 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJP-0003mp-IW for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:22 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-266-_RxjEEZOPcyqj1uToBC_bw-1; Mon, 06 Feb 2023 19:58:14 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6B56F800050; Tue, 7 Feb 2023 00:58:13 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id A5354492C3F; Tue, 7 Feb 2023 00:58:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731498; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DenhEVPqZWI8U/oaBiFpU68cmZm9r5bjKt8BUGFm1K4=; b=I2I45uMN5RthrkIWitgwcUJ47Z6wIUM9KT3/OxuhUPCT9i+tV1SDTIjajDPWFrHlJLBOn6 ph4amIz1KWrcoeaA1/4qdQ7RZixy4oN7ipjZI9p6FQjvoxDawmDJHauUlWmy7RFpqiY7Am A2NmsH2LsP/ICclwSDNHGByzF62hDP4= X-MC-Unique: _RxjEEZOPcyqj1uToBC_bw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 15/30] migration/vmstate: Introduce VMSTATE_WITH_TMP_TEST() and VMSTATE_BITMAP_TEST() Date: Tue, 7 Feb 2023 01:56:35 +0100 Message-Id: <20230207005650.1810-16-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731671806100001 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand We'll make use of both next in the context of virtio-mem. Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela S Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- include/migration/vmstate.h | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 64680d824e..28a3b92aa1 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -719,8 +719,9 @@ extern const VMStateInfo vmstate_info_qlist; * '_state' type * That the pointer is right at the start of _tmp_type. */ -#define VMSTATE_WITH_TMP(_state, _tmp_type, _vmsd) { \ +#define VMSTATE_WITH_TMP_TEST(_state, _test, _tmp_type, _vmsd) { \ .name =3D "tmp", \ + .field_exists =3D (_test), \ .size =3D sizeof(_tmp_type) + \ QEMU_BUILD_BUG_ON_ZERO(offsetof(_tmp_type, parent) != =3D 0) + \ type_check_pointer(_state, \ @@ -729,6 +730,9 @@ extern const VMStateInfo vmstate_info_qlist; .info =3D &vmstate_info_tmp, \ } =20 +#define VMSTATE_WITH_TMP(_state, _tmp_type, _vmsd) \ + VMSTATE_WITH_TMP_TEST(_state, NULL, _tmp_type, _vmsd) + #define VMSTATE_UNUSED_BUFFER(_test, _version, _size) { \ .name =3D "unused", \ .field_exists =3D (_test), \ @@ -752,8 +756,9 @@ extern const VMStateInfo vmstate_info_qlist; /* _field_size should be a int32_t field in the _state struct giving the * size of the bitmap _field in bits. */ -#define VMSTATE_BITMAP(_field, _state, _version, _field_size) { \ +#define VMSTATE_BITMAP_TEST(_field, _state, _test, _version, _field_size) = { \ .name =3D (stringify(_field)), \ + .field_exists =3D (_test), \ .version_id =3D (_version), \ .size_offset =3D vmstate_offset_value(_state, _field_size, int32_t),\ .info =3D &vmstate_info_bitmap, \ @@ -761,6 +766,9 @@ extern const VMStateInfo vmstate_info_qlist; .offset =3D offsetof(_state, _field), \ } =20 +#define VMSTATE_BITMAP(_field, _state, _version, _field_size) \ + VMSTATE_BITMAP_TEST(_field, _state, NULL, _version, _field_size) + /* For migrating a QTAILQ. * Target QTAILQ needs be properly initialized. * _type: type of QTAILQ element --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731643; cv=none; d=zohomail.com; s=zohoarc; b=lEWVDR2lqyAb2YMiN0SNXxWBAErgu8sRNslqPikHlW7ClFj4oNqXfB+AN07yJx0Uf3JLJWOMLx1FARgRQGFbGXBRQHSmoh65RjLjwhYS/W29T3js5gObHXCTh/bf2s8628NwLjYSm8HECqZt7O7OVmVgRoZWeOvnlAFTGewHiOA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731643; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=q1LoaRfWiwzrRwpeBFvBBVZ3XpJ9G+LHn1D5QvSQUWI=; b=ObN0gke6Ff2LNc1HhhNkqVj5HDlvGnNiBAmXxEIiKwtq49lVbMYZwbSYIbu3M5clFpL3vySaDEmm48j359MCGbAvnVRzlig7Ravc6Uiln0hJ4tO9AweA2cMrZvs0cMu3UHeQN9qnB37iictxqP2Np25xvO4lupRkinZ8eCoEpPI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731643963662.7522202825146; Mon, 6 Feb 2023 17:00:43 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJf-0006KC-Ip; Mon, 06 Feb 2023 19:58:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJd-0006Er-Sj for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJc-0003q6-7o for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:33 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-324-e2cld422PqqWKO-zCUj-eA-1; Mon, 06 Feb 2023 19:58:25 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A5CAF1C05AE8; Tue, 7 Feb 2023 00:58:24 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id B0A89492C3C; Tue, 7 Feb 2023 00:58:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731511; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q1LoaRfWiwzrRwpeBFvBBVZ3XpJ9G+LHn1D5QvSQUWI=; b=dZvP1lbE8i0P1fgsFABF5Cv9hSvxqF7pixM9DUmF2+XXGw8bf8bKyZWqaL80QVSAIUnGob +qqgW2Upse6EAtCLK9JkT+ZRenQr0hfl2Dqdv09N253RSEOEGyT33dW7uFeqaoYIx2mXCp bRu2DuLE8gsEs3wXEu/0UO/SvZt9z/A= X-MC-Unique: e2cld422PqqWKO-zCUj-eA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 16/30] migration/ram: Factor out check for advised postcopy Date: Tue, 7 Feb 2023 01:56:36 +0100 Message-Id: <20230207005650.1810-17-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731645666100001 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand Let's factor out this check, to be used in virtio-mem context next. While at it, fix a spelling error in a related comment. Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela S Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- include/migration/misc.h | 4 +++- migration/migration.c | 7 +++++++ migration/ram.c | 8 +------- 3 files changed, 11 insertions(+), 8 deletions(-) diff --git a/include/migration/misc.h b/include/migration/misc.h index 465906710d..8b49841016 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -67,8 +67,10 @@ bool migration_has_failed(MigrationState *); /* ...and after the device transmission */ bool migration_in_postcopy_after_devices(MigrationState *); void migration_global_dump(Monitor *mon); -/* True if incomming migration entered POSTCOPY_INCOMING_DISCARD */ +/* True if incoming migration entered POSTCOPY_INCOMING_DISCARD */ bool migration_in_incoming_postcopy(void); +/* True if incoming migration entered POSTCOPY_INCOMING_ADVISE */ +bool migration_incoming_postcopy_advised(void); /* True if background snapshot is active */ bool migration_in_bg_snapshot(void); =20 diff --git a/migration/migration.c b/migration/migration.c index c3ad4cd670..f321e419c7 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2126,6 +2126,13 @@ bool migration_in_incoming_postcopy(void) return ps >=3D POSTCOPY_INCOMING_DISCARD && ps < POSTCOPY_INCOMING_END; } =20 +bool migration_incoming_postcopy_advised(void) +{ + PostcopyState ps =3D postcopy_state_get(); + + return ps >=3D POSTCOPY_INCOMING_ADVISE && ps < POSTCOPY_INCOMING_END; +} + bool migration_in_bg_snapshot(void) { MigrationState *s =3D migrate_get_current(); diff --git a/migration/ram.c b/migration/ram.c index 7f6d5efe8d..b966e148c2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4150,12 +4150,6 @@ int ram_load_postcopy(QEMUFile *f, int channel) return ret; } =20 -static bool postcopy_is_advised(void) -{ - PostcopyState ps =3D postcopy_state_get(); - return ps >=3D POSTCOPY_INCOMING_ADVISE && ps < POSTCOPY_INCOMING_END; -} - static bool postcopy_is_running(void) { PostcopyState ps =3D postcopy_state_get(); @@ -4226,7 +4220,7 @@ static int ram_load_precopy(QEMUFile *f) MigrationIncomingState *mis =3D migration_incoming_get_current(); int flags =3D 0, ret =3D 0, invalid_flags =3D 0, len =3D 0, i =3D 0; /* ADVISE is earlier, it shows the source has the postcopy capability = on */ - bool postcopy_advised =3D postcopy_is_advised(); + bool postcopy_advised =3D migration_incoming_postcopy_advised(); if (!migrate_use_compression()) { invalid_flags |=3D RAM_SAVE_FLAG_COMPRESS_PAGE; } --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731548; cv=none; d=zohomail.com; s=zohoarc; b=TKNFE5vbsCL9FsRcI3A8/yabCuHft7BHh07omxnmjoNnmaw0g5ETfA1pUtw3gcoZe6e4eaUyFKhaZtMA/OBx8yLZsc2F39tGYlLA7QX+AWGCp7YkF/4SJ6r4gfC1KpNUwV9U+DXO7owBuWS67uqwqidlnLHzxdHOwmEQ3jwjOoU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731548; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DTtSRy6DBbNnmyL3bRVSvZHzrLJgaxvatHJOywuEQV8=; b=bpmP99zQ+sFKjJssCnjp1lSS3THYnXFIuKEgpCGQUB8dt0SF2YtD75DOB/0dmhGzpmaPGeljvXcmkWtvYR5YwozdIiLHA2XD++J5W60RxazjolyFYBmxN+zzOt3vDSNtHiFcCsx/CO5f7zPvcOTEJ0JW5/+rhGV6fVTU3qqD2tA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731548018682.8706634172516; Mon, 6 Feb 2023 16:59:08 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJz-00075N-2C; Mon, 06 Feb 2023 19:58:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJl-0006UB-DN for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJj-0003qb-Ay for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:40 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-245-k-UVy5UqOMaXaOqZnGnDfQ-1; Mon, 06 Feb 2023 19:58:31 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DF5531C05AE2; Tue, 7 Feb 2023 00:58:29 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id EBD9F492C3C; Tue, 7 Feb 2023 00:58:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731515; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DTtSRy6DBbNnmyL3bRVSvZHzrLJgaxvatHJOywuEQV8=; b=Q3ksgiuQptnXN07wir6frTRPxWzc3jhC2Vbi9wZDtDdK/niKERUqebIBVxlhA6JIHYU1Fr Io7tM1S9Z76hWCaPcum9dPo7HnOgx4lyQeK7NzMNoYBidr1L6g8Z2MVyEDTuW+b5dgCEWv QLUbf5++E4xLHUxLx7AKvGkYtuZNpjg= X-MC-Unique: k-UVy5UqOMaXaOqZnGnDfQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Michal Privoznik , Peter Xu Subject: [PULL 17/30] virtio-mem: Fail if a memory backend with "prealloc=on" is specified Date: Tue, 7 Feb 2023 01:56:37 +0100 Message-Id: <20230207005650.1810-18-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731549061100006 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand "prealloc=3Don" for the memory backend does not work as expected, as virtio-mem will simply discard all preallocated memory immediately again. In the best case, it's an expensive NOP. In the worst case, it's an unexpected allocation error. Instead, "prealloc=3Don" should be specified for the virtio-mem device only, such that virtio-mem will try preallocating memory before plugging memory dynamically to the guest. Fail if such a memory backend is provided. Tested-by: Michal Privoznik Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela S Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- hw/virtio/virtio-mem.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c index 1ed1f5a4af..02f7b5469a 100644 --- a/hw/virtio/virtio-mem.c +++ b/hw/virtio/virtio-mem.c @@ -772,6 +772,12 @@ static void virtio_mem_device_realize(DeviceState *dev= , Error **errp) error_setg(errp, "'%s' property specifies an unsupported memdev", VIRTIO_MEM_MEMDEV_PROP); return; + } else if (vmem->memdev->prealloc) { + error_setg(errp, "'%s' property specifies a memdev with preallocat= ion" + " enabled: %s. Instead, specify 'prealloc=3Don' for the" + " virtio-mem device. ", VIRTIO_MEM_MEMDEV_PROP, + object_get_canonical_path_component(OBJECT(vmem->memdev= ))); + return; } =20 if ((nb_numa_nodes && vmem->node >=3D nb_numa_nodes) || --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731851; cv=none; d=zohomail.com; s=zohoarc; b=hiOIMwpPom1Ps5XXWHmMOEahBI25T0MAhxU1y4IHEHtoPqw8zCcU26L2p3V4ASEULA4BRlT1ldv5+2I4StlKe5SJ8EjsYOG9okxwuCzeofSjLFfpVJnaH+MdeToquiOldSzO9KJIpNYmHvQy/YI0M5DmNVLWjhBhUvniT5KF0kE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731851; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7arfj3QdjtnFGkIbZuw6cq2c+PvzWVH+SNgQFzzJt5I=; b=cLKeeJblbyzHACpmg3CTFWfpzFL4w1Mnq/UHm0wR6FtgBCk6Yb0SB0AGGlHjoyZDIED3ZViiRW+bpZuJD355GvfYfZfomNOxu/m4P/w4+IqjbiG11paX5ixe8RLINWgzzkfQVV/tt7KoGTTmk5w0UhHNQRz8LGZ+5dC4Ufzb6jw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731851111744.1964703144532; Mon, 6 Feb 2023 17:04:11 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJx-0006sT-Mo; Mon, 06 Feb 2023 19:58:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJn-0006WS-3Q for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJl-0003rV-60 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:42 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-568-0nbTdWibPye5Ci0rNdo41w-1; Mon, 06 Feb 2023 19:58:36 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4698C800B24; Tue, 7 Feb 2023 00:58:35 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2FE20492C3F; Tue, 7 Feb 2023 00:58:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731520; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7arfj3QdjtnFGkIbZuw6cq2c+PvzWVH+SNgQFzzJt5I=; b=UTe6G7OHSbWHdvS4iuN1xNyHtm7VvkjMxKMIlkDp2BfcO4iaDK/aHqPrU9N+d+wRL2zM0b 4ePdV9LNeCBxbx3DK0zmQ0b7MoSaCpRxYTeu8Wz6QrXXCce075EHc5RNrOE09WLawo6n0w R9fvLuo/1bUcaWFsYd7kMOstI/EjH68= X-MC-Unique: 0nbTdWibPye5Ci0rNdo41w-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu Subject: [PULL 18/30] virtio-mem: Migrate immutable properties early Date: Tue, 7 Feb 2023 01:56:38 +0100 Message-Id: <20230207005650.1810-19-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731852986100001 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand The bitmap and the size are immutable while migration is active: see virtio_mem_is_busy(). We can migrate this information early, before migrating any actual RAM content. Further, all information we need for sanity checks is immutable as well. Having this information in place early will, for example, allow for properly preallocating memory before touching these memory locations during RAM migration: this way, we can make sure that all memory was actually preallocated and that any user errors (e.g., insufficient hugetlb pages) can be handled gracefully. In contrast, usable_region_size and requested_size can theoretically still be modified on the source while the VM is running. Keep migrating these properties the usual, late, way. Use a new device property to keep behavior of compat machines unmodified. Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela S Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- include/hw/virtio/virtio-mem.h | 8 ++++++ hw/core/machine.c | 4 ++- hw/virtio/virtio-mem.c | 51 ++++++++++++++++++++++++++++++++-- 3 files changed, 60 insertions(+), 3 deletions(-) diff --git a/include/hw/virtio/virtio-mem.h b/include/hw/virtio/virtio-mem.h index 7745cfc1a3..f15e561785 100644 --- a/include/hw/virtio/virtio-mem.h +++ b/include/hw/virtio/virtio-mem.h @@ -31,6 +31,7 @@ OBJECT_DECLARE_TYPE(VirtIOMEM, VirtIOMEMClass, #define VIRTIO_MEM_BLOCK_SIZE_PROP "block-size" #define VIRTIO_MEM_ADDR_PROP "memaddr" #define VIRTIO_MEM_UNPLUGGED_INACCESSIBLE_PROP "unplugged-inaccessible" +#define VIRTIO_MEM_EARLY_MIGRATION_PROP "x-early-migration" #define VIRTIO_MEM_PREALLOC_PROP "prealloc" =20 struct VirtIOMEM { @@ -74,6 +75,13 @@ struct VirtIOMEM { /* whether to prealloc memory when plugging new blocks */ bool prealloc; =20 + /* + * Whether we migrate properties that are immutable while migration is + * active early, before state of other devices and especially, before + * migrating any RAM content. + */ + bool early_migration; + /* notifiers to notify when "size" changes */ NotifierList size_change_notifiers; =20 diff --git a/hw/core/machine.c b/hw/core/machine.c index f7761baab5..b5cd42cd8c 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -41,7 +41,9 @@ #include "hw/virtio/virtio-pci.h" #include "qom/object_interfaces.h" =20 -GlobalProperty hw_compat_7_2[] =3D {}; +GlobalProperty hw_compat_7_2[] =3D { + { "virtio-mem", "x-early-migration", "false" }, +}; const size_t hw_compat_7_2_len =3D G_N_ELEMENTS(hw_compat_7_2); =20 GlobalProperty hw_compat_7_1[] =3D { diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c index 02f7b5469a..ca37949df8 100644 --- a/hw/virtio/virtio-mem.c +++ b/hw/virtio/virtio-mem.c @@ -31,6 +31,8 @@ #include CONFIG_DEVICES #include "trace.h" =20 +static const VMStateDescription vmstate_virtio_mem_device_early; + /* * We only had legacy x86 guests that did not support * VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE. Other targets don't have legacy gu= ests. @@ -878,6 +880,10 @@ static void virtio_mem_device_realize(DeviceState *dev= , Error **errp) =20 host_memory_backend_set_mapped(vmem->memdev, true); vmstate_register_ram(&vmem->memdev->mr, DEVICE(vmem)); + if (vmem->early_migration) { + vmstate_register(VMSTATE_IF(vmem), VMSTATE_INSTANCE_ID_ANY, + &vmstate_virtio_mem_device_early, vmem); + } qemu_register_reset(virtio_mem_system_reset, vmem); =20 /* @@ -899,6 +905,10 @@ static void virtio_mem_device_unrealize(DeviceState *d= ev) */ memory_region_set_ram_discard_manager(&vmem->memdev->mr, NULL); qemu_unregister_reset(virtio_mem_system_reset, vmem); + if (vmem->early_migration) { + vmstate_unregister(VMSTATE_IF(vmem), &vmstate_virtio_mem_device_ea= rly, + vmem); + } vmstate_unregister_ram(&vmem->memdev->mr, DEVICE(vmem)); host_memory_backend_set_mapped(vmem->memdev, false); virtio_del_queue(vdev, 0); @@ -1015,18 +1025,53 @@ static const VMStateDescription vmstate_virtio_mem_= sanity_checks =3D { }, }; =20 +static bool virtio_mem_vmstate_field_exists(void *opaque, int version_id) +{ + const VirtIOMEM *vmem =3D VIRTIO_MEM(opaque); + + /* With early migration, these fields were already migrated. */ + return !vmem->early_migration; +} + static const VMStateDescription vmstate_virtio_mem_device =3D { .name =3D "virtio-mem-device", .minimum_version_id =3D 1, .version_id =3D 1, .priority =3D MIG_PRI_VIRTIO_MEM, .post_load =3D virtio_mem_post_load, + .fields =3D (VMStateField[]) { + VMSTATE_WITH_TMP_TEST(VirtIOMEM, virtio_mem_vmstate_field_exists, + VirtIOMEMMigSanityChecks, + vmstate_virtio_mem_sanity_checks), + VMSTATE_UINT64(usable_region_size, VirtIOMEM), + VMSTATE_UINT64_TEST(size, VirtIOMEM, virtio_mem_vmstate_field_exis= ts), + VMSTATE_UINT64(requested_size, VirtIOMEM), + VMSTATE_BITMAP_TEST(bitmap, VirtIOMEM, virtio_mem_vmstate_field_ex= ists, + 0, bitmap_size), + VMSTATE_END_OF_LIST() + }, +}; + +/* + * Transfer properties that are immutable while migration is active early, + * such that we have have this information around before migrating any RAM + * content. + * + * Note that virtio_mem_is_busy() makes sure these properties can no longer + * change on the migration source until migration completed. + * + * With QEMU compat machines, we transmit these properties later, via + * vmstate_virtio_mem_device instead -- see virtio_mem_vmstate_field_exist= s(). + */ +static const VMStateDescription vmstate_virtio_mem_device_early =3D { + .name =3D "virtio-mem-device-early", + .minimum_version_id =3D 1, + .version_id =3D 1, + .early_setup =3D true, .fields =3D (VMStateField[]) { VMSTATE_WITH_TMP(VirtIOMEM, VirtIOMEMMigSanityChecks, vmstate_virtio_mem_sanity_checks), - VMSTATE_UINT64(usable_region_size, VirtIOMEM), VMSTATE_UINT64(size, VirtIOMEM), - VMSTATE_UINT64(requested_size, VirtIOMEM), VMSTATE_BITMAP(bitmap, VirtIOMEM, 0, bitmap_size), VMSTATE_END_OF_LIST() }, @@ -1211,6 +1256,8 @@ static Property virtio_mem_properties[] =3D { DEFINE_PROP_ON_OFF_AUTO(VIRTIO_MEM_UNPLUGGED_INACCESSIBLE_PROP, VirtIO= MEM, unplugged_inaccessible, ON_OFF_AUTO_AUTO), #endif + DEFINE_PROP_BOOL(VIRTIO_MEM_EARLY_MIGRATION_PROP, VirtIOMEM, + early_migration, true), DEFINE_PROP_END_OF_LIST(), }; =20 --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731870; cv=none; d=zohomail.com; s=zohoarc; b=TjyYhQoPt6gBTJH0hPIOnZDpTv0m9qvkvzCGO9LRqTr0/gUnBpFvxbd/LUmdHZsc0JM5sU219838EHtoFuFZVj9Ed5QAiqYlUoXBA85fKPkMrDzmD0QXXbT/UgqixznyyPooAUCgVzuPyWpI2SXgWSGRMuo/2OnZLXSISFmCBlg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731870; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=oRlBuILGEMZyGHaqfHlLNBv2WLhGLyPVAUh7puGRWj4=; b=oKUY9rd/phhmqY6D5Z+NNRuMB7Ww9jCgNKZ7l58rc3qIj1iPSzlXITKigOkf1yl3J9fN/V+zEFdUBwhTHBjvssA50qJn8rwebqbHo4aYGF2YNkJLlWN/d+6mhAJ5LeDIY8cNavvAMUmFHI/2A18OsF+wbMHHL6c0hHIo2+Zl/5I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731870515972.9114926267048; Mon, 6 Feb 2023 17:04:30 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJy-0006vV-36; Mon, 06 Feb 2023 19:58:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJu-0006i4-6S for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJr-0003u1-BK for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:49 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-219-P3llwEEHM66CvczKJ-1WSg-1; Mon, 06 Feb 2023 19:58:41 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BD0B13C025B0; Tue, 7 Feb 2023 00:58:40 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8CFC8492C3C; Tue, 7 Feb 2023 00:58:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oRlBuILGEMZyGHaqfHlLNBv2WLhGLyPVAUh7puGRWj4=; b=XZvW2RXhBqfKzv1sFnZZiM8EryeGNbOPT4DtoGiF7E/k5BKaqvgPmSYNzMFc4mzId3Maj2 JM/hiWLHdkD8PfIfQiCIvBmomjk0//0i85OgF02CwCF/87AvBZfwgD19LHUTgiVzsJolKm +L9p48KzHmOjIXxoNwxC0IF6k7R7qMM= X-MC-Unique: P3llwEEHM66CvczKJ-1WSg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Jing Qi , Peter Xu Subject: [PULL 19/30] virtio-mem: Proper support for preallocation with migration Date: Tue, 7 Feb 2023 01:56:39 +0100 Message-Id: <20230207005650.1810-20-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731871073100001 Content-Type: text/plain; charset="utf-8" From: David Hildenbrand Ordinary memory preallocation runs when QEMU starts up and creates the memory backends, before processing the incoming migration stream. With virtio-mem, we don't know which memory blocks to preallocate before migration started. Now that we migrate the virtio-mem bitmap early, before migrating any RAM content, we can safely preallocate memory for all plugged memory blocks before migrating any RAM content. This is especially relevant for the following cases: (1) User errors With hugetlb/files, if we don't have sufficient backend memory available on the migration destination, we'll crash QEMU (SIGBUS) during RAM migration when running out of backend memory. Preallocating memory before actual RAM migration allows for failing gracefully and informing the user about the setup problem. (2) Excluded memory ranges during migration For example, virtio-balloon free page hinting will exclude some pages from getting migrated. In that case, we won't crash during RAM migration, but later, when running the VM on the destination, which is bad. To fix this for new QEMU machines that migrate the bitmap early, preallocate the memory early, before any RAM migration. Warn with old QEMU machines. Getting postcopy right is a bit tricky, but we essentially now implement the same (problematic) preallocation logic as ordinary preallocation: preallocate memory early and discard it again before precopy starts. During ordinary preallocation, discarding of RAM happens when postcopy is advised. As the state (bitmap) is loaded after postcopy was advised but before postcopy starts listening, we have to discard memory we preallocated immediately again ourselves. Note that nothing (not even hugetlb reservations) guarantees for postcopy that backend memory (especially, hugetlb pages) are still free after they were freed ones while discarding RAM. Still, allocating that memory at least once helps catching some basic setup problems. Before this change, trying to restore a VM when insufficient hugetlb pages are around results in the process crashing to to a "Bus error" (SIGBUS). With this change, QEMU fails gracefully: qemu-system-x86_64: qemu_prealloc_mem: preallocating memory failed: Bad a= ddress qemu-system-x86_64: error while loading state for instance 0x0 of device = '0000:00:03.0/virtio-mem-device-early' qemu-system-x86_64: load of migration failed: Cannot allocate memory And we can even introspect the early migration data, including the bitmap: $ ./scripts/analyze-migration.py -f STATEFILE { "ram (2)": { "section sizes": { "0000:00:03.0/mem0": "0x0000000780000000", "0000:00:04.0/mem1": "0x0000000780000000", "pc.ram": "0x0000000100000000", "/rom@etc/acpi/tables": "0x0000000000020000", "pc.bios": "0x0000000000040000", "0000:00:02.0/e1000.rom": "0x0000000000040000", "pc.rom": "0x0000000000020000", "/rom@etc/table-loader": "0x0000000000001000", "/rom@etc/acpi/rsdp": "0x0000000000001000" } }, "0000:00:03.0/virtio-mem-device-early (51)": { "tmp": "00 00 00 01 40 00 00 00 00 00 00 07 80 00 00 00 00 00 00 00 0= 0 20 00 00 00 00 00 00", "size": "0x0000000040000000", "bitmap": "ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff [...] }, "0000:00:04.0/virtio-mem-device-early (53)": { "tmp": "00 00 00 08 c0 00 00 00 00 00 00 07 80 00 00 00 00 00 00 00 0= 0 20 00 00 00 00 00 00", "size": "0x00000001fa400000", "bitmap": "ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff [...] }, [...] Reported-by: Jing Qi Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu Reviewed-by: Michael S. Tsirkin Reviewed-by: Juan Quintela S Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- hw/virtio/virtio-mem.c | 87 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c index ca37949df8..957fe77dc0 100644 --- a/hw/virtio/virtio-mem.c +++ b/hw/virtio/virtio-mem.c @@ -204,6 +204,30 @@ static int virtio_mem_for_each_unplugged_range(const V= irtIOMEM *vmem, void *arg, return ret; } =20 +static int virtio_mem_for_each_plugged_range(const VirtIOMEM *vmem, void *= arg, + virtio_mem_range_cb cb) +{ + unsigned long first_bit, last_bit; + uint64_t offset, size; + int ret =3D 0; + + first_bit =3D find_first_bit(vmem->bitmap, vmem->bitmap_size); + while (first_bit < vmem->bitmap_size) { + offset =3D first_bit * vmem->block_size; + last_bit =3D find_next_zero_bit(vmem->bitmap, vmem->bitmap_size, + first_bit + 1) - 1; + size =3D (last_bit - first_bit + 1) * vmem->block_size; + + ret =3D cb(vmem, arg, offset, size); + if (ret) { + break; + } + first_bit =3D find_next_bit(vmem->bitmap, vmem->bitmap_size, + last_bit + 2); + } + return ret; +} + /* * Adjust the memory section to cover the intersection with the given rang= e. * @@ -938,6 +962,10 @@ static int virtio_mem_post_load(void *opaque, int vers= ion_id) RamDiscardListener *rdl; int ret; =20 + if (vmem->prealloc && !vmem->early_migration) { + warn_report("Proper preallocation with migration requires a newer = QEMU machine"); + } + /* * We started out with all memory discarded and our memory region is m= apped * into an address space. Replay, now that we updated the bitmap. @@ -957,6 +985,64 @@ static int virtio_mem_post_load(void *opaque, int vers= ion_id) return virtio_mem_restore_unplugged(vmem); } =20 +static int virtio_mem_prealloc_range_cb(const VirtIOMEM *vmem, void *arg, + uint64_t offset, uint64_t size) +{ + void *area =3D memory_region_get_ram_ptr(&vmem->memdev->mr) + offset; + int fd =3D memory_region_get_fd(&vmem->memdev->mr); + Error *local_err =3D NULL; + + qemu_prealloc_mem(fd, area, size, 1, NULL, &local_err); + if (local_err) { + error_report_err(local_err); + return -ENOMEM; + } + return 0; +} + +static int virtio_mem_post_load_early(void *opaque, int version_id) +{ + VirtIOMEM *vmem =3D VIRTIO_MEM(opaque); + RAMBlock *rb =3D vmem->memdev->mr.ram_block; + int ret; + + if (!vmem->prealloc) { + return 0; + } + + /* + * We restored the bitmap and verified that the basic properties + * match on source and destination, so we can go ahead and preallocate + * memory for all plugged memory blocks, before actual RAM migration s= tarts + * touching this memory. + */ + ret =3D virtio_mem_for_each_plugged_range(vmem, NULL, + virtio_mem_prealloc_range_cb); + if (ret) { + return ret; + } + + /* + * This is tricky: postcopy wants to start with a clean slate. On + * POSTCOPY_INCOMING_ADVISE, postcopy code discards all (ordinarily + * preallocated) RAM such that postcopy will work as expected later. + * + * However, we run after POSTCOPY_INCOMING_ADVISE -- but before actual + * RAM migration. So let's discard all memory again. This looks like an + * expensive NOP, but actually serves a purpose: we made sure that we + * were able to allocate all required backend memory once. We cannot + * guarantee that the backend memory we will free will remain free + * until we need it during postcopy, but at least we can catch the + * obvious setup issues this way. + */ + if (migration_incoming_postcopy_advised()) { + if (ram_block_discard_range(rb, 0, qemu_ram_get_used_length(rb))) { + return -EBUSY; + } + } + return 0; +} + typedef struct VirtIOMEMMigSanityChecks { VirtIOMEM *parent; uint64_t addr; @@ -1068,6 +1154,7 @@ static const VMStateDescription vmstate_virtio_mem_de= vice_early =3D { .minimum_version_id =3D 1, .version_id =3D 1, .early_setup =3D true, + .post_load =3D virtio_mem_post_load_early, .fields =3D (VMStateField[]) { VMSTATE_WITH_TMP(VirtIOMEM, VirtIOMEMMigSanityChecks, vmstate_virtio_mem_sanity_checks), --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731718; cv=none; d=zohomail.com; s=zohoarc; b=CRtJ/qC8ent4fVjpfF22SoHdnlvQiHIquQe1nBENaGdOp9zDD27iH+wT1LqxfCD6/mSDSixbWev62u6HDaBsz4NVCVjWuqWrQyGQS688uWzR+5Vh+4PaQBvYSiROr2o4Jg6g89PnqH05pfrRMvZ4CEb5Nekq7vALxpz0m8KdLEg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731718; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=S8YJoHwNGFrow4yvZqYD9fJCjU9peOTc1LZupkn3uiI=; b=bYuFF7axAaWCm8H162YHqlqekJjLBdoyYWfMexWGT6TghFNPGIxDlwYCITNzg063C7wJWwzFwA7MNyUXymOzHFcP0QcREF7YsfBGAADCiqHP124V67ysk5QZZURGZJA1Kof8su5hrZAGEOJmUnbIMmNyvqY86qzWZ4aApskwfTM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731718398363.6668231300521; Mon, 6 Feb 2023 17:01:58 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCJz-00079T-Ef; Mon, 06 Feb 2023 19:58:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJx-0006s2-8V for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:53 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCJv-0003uS-Jk for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:58:52 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-70-S-TouU7AOKmBAmOzrg8efA-1; Mon, 06 Feb 2023 19:58:46 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 10DC41871D9A; Tue, 7 Feb 2023 00:58:46 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0EE12492C3C; Tue, 7 Feb 2023 00:58:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731530; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S8YJoHwNGFrow4yvZqYD9fJCjU9peOTc1LZupkn3uiI=; b=iucYo/NMEweBafI2QK6Ny/sSoFTApuIq4a7elPAMfMIFLTnr8tObmHLu1X3XUHQFtmRJrg MNOsJyW2TZiHFTMLKmeZnqQ4TsTALtNWUk41J7ugUMAQ+pHt3FERmhIcGa6P44Kiyvtg4Q zdV8Q0ox5Qajxo6ebiVYsqL6tn1ssLE= X-MC-Unique: S-TouU7AOKmBAmOzrg8efA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Xu , Leonardo Bras Subject: [PULL 20/30] migration: Show downtime during postcopy phase Date: Tue, 7 Feb 2023 01:56:40 +0100 Message-Id: <20230207005650.1810-21-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731720022100003 Content-Type: text/plain; charset="utf-8" From: Peter Xu The downtime should be displayed during postcopy phase because the switchover phase is done. OTOH it's weird to show "expected downtime" which can confuse what does that mean if the switchover has already happened anyway. This is a slight ABI change on QMP, but I assume it shouldn't affect anyone. Reviewed-by: Leonardo Bras Reviewed-by: Juan Quintela Signed-off-by: Peter Xu Signed-off-by: Juan Quintela --- migration/migration.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index f321e419c7..4f4d798d3e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1051,20 +1051,30 @@ bool migration_is_running(int state) } } =20 +static bool migrate_show_downtime(MigrationState *s) +{ + return (s->state =3D=3D MIGRATION_STATUS_COMPLETED) || migration_in_po= stcopy(); +} + static void populate_time_info(MigrationInfo *info, MigrationState *s) { info->has_status =3D true; info->has_setup_time =3D true; info->setup_time =3D s->setup_time; + if (s->state =3D=3D MIGRATION_STATUS_COMPLETED) { info->has_total_time =3D true; info->total_time =3D s->total_time; - info->has_downtime =3D true; - info->downtime =3D s->downtime; } else { info->has_total_time =3D true; info->total_time =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME) - s->start_time; + } + + if (migrate_show_downtime(s)) { + info->has_downtime =3D true; + info->downtime =3D s->downtime; + } else { info->has_expected_downtime =3D true; info->expected_downtime =3D s->expected_downtime; } --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731549187460.0254221942823; Mon, 6 Feb 2023 16:59:09 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCK9-0008PY-DH; Mon, 06 Feb 2023 19:59:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCK7-0008L1-Ur for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:03 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCK2-0003vI-UE for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:03 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-549-jTf_FcQjNK-18fFMoEJD9g-1; Mon, 06 Feb 2023 19:58:52 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7FDBC1C05156; Tue, 7 Feb 2023 00:58:51 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5DDAD492C3C; Tue, 7 Feb 2023 00:58:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731538; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oLTUIKkYKLDqvnCZf7ylBVeUifFGysX8cjMj/PMEy8c=; b=XTONqcRMHqyTdzom8wsfQwOTYIE/yrAKnEGvABqrAqkjH5Y9nnD7LR/AMr+5/wDxfRfvis OdYPOZC+YWAZh+N5jV7rknfP2GBwcdNPMaeqLRCBW0jQOoQLgFWEjfeSZytaU5PbZVvCAB SFmyCBsdV2ZlYG911MZ3TR9h9XYtUVk= X-MC-Unique: jTf_FcQjNK-18fFMoEJD9g-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Fiona Ebner , Zhang Chen Subject: [PULL 21/30] migration/rdma: fix return value for qio_channel_rdma_{readv, writev} Date: Tue, 7 Feb 2023 01:56:41 +0100 Message-Id: <20230207005650.1810-22-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_SPF_HELO_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1675731551173100013 Content-Type: text/plain; charset="utf-8" From: Fiona Ebner upon errors. As the documentation in include/io/channel.h states, only -1 and QIO_CHANNEL_ERR_BLOCK should be returned upon error. Other values have the potential to confuse the call sites. error_setg is used rather than error_setg_errno, because there are certain code paths where -1 (as a non-errno) is propagated up (e.g. starting from qemu_rdma_block_for_wrid or qemu_rdma_post_recv_control) all the way to qio_channel_rdma_{readv,writev}. Similar to a216ec85b7 ("migration/channel-block: fix return value for qio_channel_block_{readv,writev}"). Suggested-by: Zhang Chen Reviewed-by: Juan Quintela Signed-off-by: Fiona Ebner Signed-off-by: Juan Quintela --- migration/rdma.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 94a55dd95b..0ba1668d70 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2785,7 +2785,8 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, rdma =3D qatomic_rcu_read(&rioc->rdmaout); =20 if (!rdma) { - return -EIO; + error_setg(errp, "RDMA control channel output is not set"); + return -1; } =20 CHECK_ERROR_STATE(); @@ -2797,7 +2798,8 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, ret =3D qemu_rdma_write_flush(f, rdma); if (ret < 0) { rdma->error_state =3D ret; - return ret; + error_setg(errp, "qemu_rdma_write_flush returned %d", ret); + return -1; } =20 for (i =3D 0; i < niov; i++) { @@ -2816,7 +2818,8 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, =20 if (ret < 0) { rdma->error_state =3D ret; - return ret; + error_setg(errp, "qemu_rdma_exchange_send returned %d", re= t); + return -1; } =20 data +=3D len; @@ -2867,7 +2870,8 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc, rdma =3D qatomic_rcu_read(&rioc->rdmain); =20 if (!rdma) { - return -EIO; + error_setg(errp, "RDMA control channel input is not set"); + return -1; } =20 CHECK_ERROR_STATE(); @@ -2903,7 +2907,8 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc, =20 if (ret < 0) { rdma->error_state =3D ret; - return ret; + error_setg(errp, "qemu_rdma_exchange_recv returned %d", ret); + return -1; } =20 /* --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731626; cv=none; d=zohomail.com; s=zohoarc; b=AtkvTnhV7oReDgafuA5Rtk+VnJNWILu8wuSVCP6K5HwTXdr7oHdFcrhgKpEkDIugWWCFA0JH+sTc5jQTdOhIRWlfJiMpDMdhaT2Z5tA1wIIExLC2yosWqLz4a29bB1E6kL4ChNngzCkH/MIvXJRcwH7QNv0ayb2pck6jllpYR98= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731626; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=YmJLSmZt2ubSLueF/Ct+KxlmQVT2kWkbO3XpvgWK71g=; b=NlH3ywtl4jkVvchbUs8GnKB03mmm1hJtc6RH1Z2KSQUmD4Z/k4f0LLuyspgTgM3IQP5y8B/WWNqP61QoAGelElPE6Z44ON0QySLgKnTnMqHIJn/Qfr8DYqQ631kk0/g6divKW0Gw+n8Ys7jd6tMyOwiqSrYauOmmbBkDs7iXtz0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731626675679.8230556466183; Mon, 6 Feb 2023 17:00:26 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCKB-00005z-2q; Mon, 06 Feb 2023 19:59:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCK9-0008QD-Iq for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:05 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCK7-0003vw-62 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:05 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-651-YpNNj4d2Ppqct6eKs75ACQ-1; Mon, 06 Feb 2023 19:58:57 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ACA0188564C; Tue, 7 Feb 2023 00:58:56 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id C4DEF492C3C; Tue, 7 Feb 2023 00:58:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YmJLSmZt2ubSLueF/Ct+KxlmQVT2kWkbO3XpvgWK71g=; b=Dc/r2FBlzrICDIShJPevDe0LO9Wxgqq3+xHQyQEIXrl4eDL7jvQc5zUF6aCzr8R8oNHchw VuMM186jG0I+vzVrYQ2bay7+IIFU7BGHM2WRSNOOHk4dLjCiLGE8W/UlczwCF6qUQym2Et lBZ9Y+O8SbYUUOQL8OLPxzdXJqlWCFo= X-MC-Unique: YpNNj4d2Ppqct6eKs75ACQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Maydell , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL 22/30] migration: Add canary to VMSTATE_END_OF_LIST Date: Tue, 7 Feb 2023 01:56:42 +0100 Message-Id: <20230207005650.1810-23-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731627727100003 From: "Dr. David Alan Gilbert" We fairly regularly forget VMSTATE_END_OF_LIST markers off descriptions; given that the current check is only for ->name being NULL, sometimes we get unlucky and the code apparently works and no one spots the error. Explicitly add a flag, VMS_END that should be set, and assert it is set during the traversal. Note: This can't go in until we update the copy of vmstate.h in slirp. Suggested-by: Peter Maydell Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Peter Maydell Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- include/migration/vmstate.h | 7 ++++++- migration/savevm.c | 1 + migration/vmstate.c | 2 ++ 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 28a3b92aa1..084f5e784a 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -147,6 +147,9 @@ enum VMStateFlags { * VMStateField.struct_version_id to tell which version of the * structure we are referencing to use. */ VMS_VSTRUCT =3D 0x8000, + + /* Marker for end of list */ + VMS_END =3D 0x10000 }; =20 typedef enum { @@ -1183,7 +1186,9 @@ extern const VMStateInfo vmstate_info_qlist; VMSTATE_UNUSED_BUFFER(_test, 0, _size) =20 #define VMSTATE_END_OF_LIST() \ - {} + { \ + .flags =3D VMS_END, \ + } =20 int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd, void *opaque, int version_id); diff --git a/migration/savevm.c b/migration/savevm.c index 6d985ad4af..5c3e5b1bb5 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -585,6 +585,7 @@ static void dump_vmstate_vmsd(FILE *out_file, field++; first =3D false; } + assert(field->flags =3D=3D VMS_END); fprintf(out_file, "\n%*s]", indent, ""); } if (vmsd->subsections !=3D NULL) { diff --git a/migration/vmstate.c b/migration/vmstate.c index 924494bda3..83ca4c7d3e 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -154,6 +154,7 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescri= ption *vmsd, } field++; } + assert(field->flags =3D=3D VMS_END); ret =3D vmstate_subsection_load(f, vmsd, opaque); if (ret !=3D 0) { return ret; @@ -408,6 +409,7 @@ int vmstate_save_state_v(QEMUFile *f, const VMStateDesc= ription *vmsd, } field++; } + assert(field->flags =3D=3D VMS_END); =20 if (vmdesc) { json_writer_end_array(vmdesc); --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731589; cv=none; d=zohomail.com; s=zohoarc; b=nS2x3lwzZabRB0TaLrO0AC+2Ais87ZV+g1no8IvgBlAWkVMOYMfk0ZfxHZI5UHplnhO9eOSDP1rGKn97OjpDvwfw4lJr48AGEoo3UNooeO0JD53fgHovvCoX7pe/WtPuq3Y4qBkZ0RE/4jNMHTgzB41KkqqPtuV005pd4YuLhIA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731589; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZKWrS9/I8tLNhFvOvL6wTWGGmGHs1Q+gINxHteToEV0=; b=GUI/aGKJRnFFVA1c1ggDGUvEq0rIDmBg2TbiXsrQc63lulgGTb3ADBDUXslFJKgXqMntMA5dNU3NMX7QQy7XrD9Gz+Qxc1NEr/v5imbxv9Yrlm65G83pvr5zvs138jsIf05G96H49ekppsbB5Y6GDr/YsARLPWNRXChkGeSG7R0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731589220468.6253539602966; Mon, 6 Feb 2023 16:59:49 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCKF-0000Oa-I1; Mon, 06 Feb 2023 19:59:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKE-0000LH-5D for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:10 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKB-0003wz-Ez for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:09 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-655-HF2VcH95PXi_zzGmr5YGsw-1; Mon, 06 Feb 2023 19:59:03 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B74B1101A521; Tue, 7 Feb 2023 00:59:01 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id F2A94492C3C; Tue, 7 Feb 2023 00:58:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731546; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZKWrS9/I8tLNhFvOvL6wTWGGmGHs1Q+gINxHteToEV0=; b=IcIDyyj5fJDNmTz3hEfIJ3P7zzp8pUqH6A+mcB8GWEpPbuIBb2B/eYzmi/gCwnEkRnviQL C3ezcdt1iKVgIKRftwD6hn+y09l4zQTLZtVOYX/AKUJdsv/E+BnKVvyMTY80q4QGU1mcGX zVqI3E/gmV7QQ2WZyDKt/IuRjD/qA7o= X-MC-Unique: HF2VcH95PXi_zzGmr5YGsw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Peter Maydell Subject: [PULL 23/30] migration: Perform vmsd structure check during tests Date: Tue, 7 Feb 2023 01:56:43 +0100 Message-Id: <20230207005650.1810-24-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731591362100003 Content-Type: text/plain; charset="utf-8" From: "Dr. David Alan Gilbert" Perform a check on vmsd structures during test runs in the hope of catching any missing terminators and other simple screwups. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Peter Maydell Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/savevm.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index 5c3e5b1bb5..e9cf4999ad 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -66,6 +66,7 @@ #include "net/announce.h" #include "qemu/yank.h" #include "yank_functions.h" +#include "sysemu/qtest.h" =20 const unsigned int postcopy_ram_discard_version; =20 @@ -804,6 +805,42 @@ void unregister_savevm(VMStateIf *obj, const char *ids= tr, void *opaque) } } =20 +/* + * Perform some basic checks on vmsd's at registration + * time. + */ +static void vmstate_check(const VMStateDescription *vmsd) +{ + const VMStateField *field =3D vmsd->fields; + const VMStateDescription **subsection =3D vmsd->subsections; + + if (field) { + while (field->name) { + if (field->flags & (VMS_STRUCT | VMS_VSTRUCT)) { + /* Recurse to sub structures */ + vmstate_check(field->vmsd); + } + /* Carry on */ + field++; + } + /* Check for the end of field list canary */ + if (field->flags !=3D VMS_END) { + error_report("VMSTATE not ending with VMS_END: %s", vmsd->name= ); + g_assert_not_reached(); + } + } + + while (subsection && *subsection) { + /* + * The name of a subsection should start with the name of the + * current object. + */ + assert(!strncmp(vmsd->name, (*subsection)->name, strlen(vmsd->name= ))); + vmstate_check(*subsection); + subsection++; + } +} + int vmstate_register_with_alias_id(VMStateIf *obj, uint32_t instance_id, const VMStateDescription *vmsd, void *opaque, int alias_id, @@ -849,6 +886,11 @@ int vmstate_register_with_alias_id(VMStateIf *obj, uin= t32_t instance_id, } else { se->instance_id =3D instance_id; } + + /* Perform a recursive sanity check during the test runs */ + if (qtest_enabled()) { + vmstate_check(vmsd); + } assert(!se->compat || se->instance_id =3D=3D 0); savevm_state_handler_insert(se); return 0; --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731683; cv=none; d=zohomail.com; s=zohoarc; b=jAqfQedOWQ0oKOTUKq1ZRQtBFAa18uxKaZIFT8127sXWNXnfc3F6jJ717xxRKC6nW+t/wRwBJyIwSbru5E3z0qN/rFLmEM2f9Pm88sTeVaNY/5Mgns4QxEYZaLT/eZQCR3puNeyT2F4wihLTdrXV/xQyTZ7+zztj+K1CFzT6C8w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731683; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hudZzsRpagzcLI/XtX7LAUqMBRpe4HeJEe9q7k4gr1A=; b=D25XZaOL0AMkwBFhScs8l3dcBQZtj1VcQBegC5uY1sCRISDkDncJ97vI5r8ik9PDH1gWuQE9P10aD1KHcq8ZKQ5fK4zobYlCvGPFoW+VpS5r5nOEeYdkxft3aU7RP6gPmTUdnHJ8jtFzSuAslRLKEzqlGuJsA8/9L88gVofVzJc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731683436986.7428281947003; Mon, 6 Feb 2023 17:01:23 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCKJ-0000Vr-L8; Mon, 06 Feb 2023 19:59:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKH-0000Rs-HX for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:13 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKF-0003yC-VF for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:13 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-357-x1a2aZd-N7OUrSZJE7rzRw-1; Mon, 06 Feb 2023 19:59:08 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3C25980D0EE; Tue, 7 Feb 2023 00:59:07 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0986F492C3C; Tue, 7 Feb 2023 00:59:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731551; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hudZzsRpagzcLI/XtX7LAUqMBRpe4HeJEe9q7k4gr1A=; b=TP6NHmQta45BY+8R8rh6pu+0rXHO8gFYef0N6bovD9ffwlhFZsOtAwDNu4E00U95MayzRc dyCdASpxtOu/5IqoUcfFxiYSqOLIkMtVyF4tdQ9lGEsTVZaH8SNF948u1b8cu1wEqh93EQ ufiQTo+CgLq/S7BfqhX+ij3ieiF1nDw= X-MC-Unique: x1a2aZd-N7OUrSZJE7rzRw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Zhenzhong Duan , Peter Xu Subject: [PULL 24/30] migration/dirtyrate: Show sample pages only in page-sampling mode Date: Tue, 7 Feb 2023 01:56:44 +0100 Message-Id: <20230207005650.1810-25-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731683836100001 Content-Type: text/plain; charset="utf-8" From: Zhenzhong Duan The value of "Sample Pages" is confusing in mode other than page-sampling. See below: (qemu) calc_dirty_rate -b 10 520 (qemu) info dirty_rate Status: measuring Start Time: 11646834 (ms) Sample Pages: 520 (per GB) Period: 10 (sec) Mode: dirty-bitmap Dirty rate: (not ready) (qemu) info dirty_rate Status: measured Start Time: 11646834 (ms) Sample Pages: 0 (per GB) Period: 10 (sec) Mode: dirty-bitmap Dirty rate: 2 (MB/s) While it's totally useless in dirty-ring and dirty-bitmap mode, fix to show it only in page-sampling mode. Signed-off-by: Zhenzhong Duan Reviewed-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/dirtyrate.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index 4bfb97fc68..575d48c397 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -714,8 +714,8 @@ void qmp_calc_dirty_rate(int64_t calc_time, mode =3D DIRTY_RATE_MEASURE_MODE_PAGE_SAMPLING; } =20 - if (has_sample_pages && mode =3D=3D DIRTY_RATE_MEASURE_MODE_DIRTY_RING= ) { - error_setg(errp, "either sample-pages or dirty-ring can be specifi= ed."); + if (has_sample_pages && mode !=3D DIRTY_RATE_MEASURE_MODE_PAGE_SAMPLIN= G) { + error_setg(errp, "sample-pages is used only in page-sampling mode"= ); return; } =20 @@ -785,8 +785,10 @@ void hmp_info_dirty_rate(Monitor *mon, const QDict *qd= ict) DirtyRateStatus_str(info->status)); monitor_printf(mon, "Start Time: %"PRIi64" (ms)\n", info->start_time); - monitor_printf(mon, "Sample Pages: %"PRIu64" (per GB)\n", - info->sample_pages); + if (info->mode =3D=3D DIRTY_RATE_MEASURE_MODE_PAGE_SAMPLING) { + monitor_printf(mon, "Sample Pages: %"PRIu64" (per GB)\n", + info->sample_pages); + } monitor_printf(mon, "Period: %"PRIi64" (sec)\n", info->calc_time); monitor_printf(mon, "Mode: %s\n", --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731720; cv=none; d=zohomail.com; s=zohoarc; b=RGaj6Uu9orCCRNHCmVw3dLS1uWgWNreUm3OGfEOHqNfyHL41Y3rPTWHecSdIjUsWF8FpnG4VXvbn/Cf/ZDBWLhAqsTuZMxdQsYhyuFw0IIyTGUvhG9edasZTuzb6cES2CWJKnYCFbP6p3S4MErFBKYuGsGdhrwAeOTWR5bU09eU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731720; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fKPuk3A4ncKmTLyWDDc1BjCWFLugOIjTFFZfMtsDaIk=; b=frXxcx47pBJqvCSGflwpSJti3ulWI1300JJ272PtS1wgA+Qg8KxgrkEyg7LwRjBbxsDbtBtMcYf3qjahzcJTx2hBfk1SO27qy7ur1kb5V4Id7O8vbF62P0HH5OOY0/dnxtScWAp7EGchW0ajPf5v14ZSlIaJkk4R+0cWAVKRe0Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731720552445.03900679225615; Mon, 6 Feb 2023 17:02:00 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCKZ-0001Fe-An; Mon, 06 Feb 2023 19:59:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKP-0000x9-Ca for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKN-0003yx-8n for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:21 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-367-ZM73jhy3M1WBrMixcPUzLA-1; Mon, 06 Feb 2023 19:59:14 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9A2292A59560; Tue, 7 Feb 2023 00:59:12 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 88820492C3C; Tue, 7 Feb 2023 00:59:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKPuk3A4ncKmTLyWDDc1BjCWFLugOIjTFFZfMtsDaIk=; b=hqGpsIbuijZkIabSHtN9SwAehSfosVq5VC0CRa6gnWohsrSZyjdSsJuLFg4GzRR3+IfkQg H9z5XRDFMYIA9dh2MCftE0a03FPmgw6KoKvUZL8kvpMOPeDqCB0ByB2+zfg09ezQK7hG0L M96kD3VOQGVY2qrEs1vs6qDZVR/Vgek= X-MC-Unique: ZM73jhy3M1WBrMixcPUzLA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , "manish.mishra" , Peter Xu Subject: [PULL 25/30] io: Add support for MSG_PEEK for socket channel Date: Tue, 7 Feb 2023 01:56:45 +0100 Message-Id: <20230207005650.1810-26-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731722146100011 Content-Type: text/plain; charset="utf-8" From: "manish.mishra" MSG_PEEK peeks at the channel, The data is treated as unread and the next read shall still return this data. This support is currently added only for socket class. Extra parameter 'flags' is added to io_readv calls to pass extra read flags like MSG_PEEK. Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrange Reviewed-by: Juan Quintela Suggested-by: Daniel P. Berrange Signed-off-by: manish.mishra Signed-off-by: Juan Quintela --- include/io/channel.h | 6 ++++++ chardev/char-socket.c | 4 ++-- io/channel-buffer.c | 1 + io/channel-command.c | 1 + io/channel-file.c | 1 + io/channel-null.c | 1 + io/channel-socket.c | 19 ++++++++++++++++++- io/channel-tls.c | 1 + io/channel-websock.c | 1 + io/channel.c | 16 ++++++++++++---- migration/channel-block.c | 1 + migration/rdma.c | 1 + scsi/qemu-pr-helper.c | 2 +- tests/qtest/tpm-emu.c | 2 +- tests/unit/test-io-channel-socket.c | 1 + util/vhost-user-server.c | 2 +- 16 files changed, 50 insertions(+), 10 deletions(-) diff --git a/include/io/channel.h b/include/io/channel.h index 78b15f7870..153fbd2904 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -34,6 +34,8 @@ OBJECT_DECLARE_TYPE(QIOChannel, QIOChannelClass, =20 #define QIO_CHANNEL_WRITE_FLAG_ZERO_COPY 0x1 =20 +#define QIO_CHANNEL_READ_FLAG_MSG_PEEK 0x1 + typedef enum QIOChannelFeature QIOChannelFeature; =20 enum QIOChannelFeature { @@ -41,6 +43,7 @@ enum QIOChannelFeature { QIO_CHANNEL_FEATURE_SHUTDOWN, QIO_CHANNEL_FEATURE_LISTEN, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY, + QIO_CHANNEL_FEATURE_READ_MSG_PEEK, }; =20 =20 @@ -114,6 +117,7 @@ struct QIOChannelClass { size_t niov, int **fds, size_t *nfds, + int flags, Error **errp); int (*io_close)(QIOChannel *ioc, Error **errp); @@ -188,6 +192,7 @@ void qio_channel_set_name(QIOChannel *ioc, * @niov: the length of the @iov array * @fds: pointer to an array that will received file handles * @nfds: pointer filled with number of elements in @fds on return + * @flags: read flags (QIO_CHANNEL_READ_FLAG_*) * @errp: pointer to a NULL-initialized error object * * Read data from the IO channel, storing it in the @@ -224,6 +229,7 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp); =20 =20 diff --git a/chardev/char-socket.c b/chardev/char-socket.c index 29ffe5075e..c2265436ac 100644 --- a/chardev/char-socket.c +++ b/chardev/char-socket.c @@ -283,11 +283,11 @@ static ssize_t tcp_chr_recv(Chardev *chr, char *buf, = size_t len) if (qio_channel_has_feature(s->ioc, QIO_CHANNEL_FEATURE_FD_PASS)) { ret =3D qio_channel_readv_full(s->ioc, &iov, 1, &msgfds, &msgfds_num, - NULL); + 0, NULL); } else { ret =3D qio_channel_readv_full(s->ioc, &iov, 1, NULL, NULL, - NULL); + 0, NULL); } =20 if (msgfds_num) { diff --git a/io/channel-buffer.c b/io/channel-buffer.c index bf52011be2..8096180f85 100644 --- a/io/channel-buffer.c +++ b/io/channel-buffer.c @@ -54,6 +54,7 @@ static ssize_t qio_channel_buffer_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelBuffer *bioc =3D QIO_CHANNEL_BUFFER(ioc); diff --git a/io/channel-command.c b/io/channel-command.c index 74516252ba..e7edd091af 100644 --- a/io/channel-command.c +++ b/io/channel-command.c @@ -203,6 +203,7 @@ static ssize_t qio_channel_command_readv(QIOChannel *io= c, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelCommand *cioc =3D QIO_CHANNEL_COMMAND(ioc); diff --git a/io/channel-file.c b/io/channel-file.c index b67687c2aa..d76663e6ae 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -86,6 +86,7 @@ static ssize_t qio_channel_file_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelFile *fioc =3D QIO_CHANNEL_FILE(ioc); diff --git a/io/channel-null.c b/io/channel-null.c index 75e3781507..4fafdb770d 100644 --- a/io/channel-null.c +++ b/io/channel-null.c @@ -60,6 +60,7 @@ qio_channel_null_readv(QIOChannel *ioc, size_t niov, int **fds G_GNUC_UNUSED, size_t *nfds G_GNUC_UNUSED, + int flags, Error **errp) { QIOChannelNull *nioc =3D QIO_CHANNEL_NULL(ioc); diff --git a/io/channel-socket.c b/io/channel-socket.c index b76dca9cc1..7aca84f61a 100644 --- a/io/channel-socket.c +++ b/io/channel-socket.c @@ -173,6 +173,9 @@ int qio_channel_socket_connect_sync(QIOChannelSocket *i= oc, } #endif =20 + qio_channel_set_feature(QIO_CHANNEL(ioc), + QIO_CHANNEL_FEATURE_READ_MSG_PEEK); + return 0; } =20 @@ -406,6 +409,9 @@ qio_channel_socket_accept(QIOChannelSocket *ioc, } #endif /* WIN32 */ =20 + qio_channel_set_feature(QIO_CHANNEL(cioc), + QIO_CHANNEL_FEATURE_READ_MSG_PEEK); + trace_qio_channel_socket_accept_complete(ioc, cioc, cioc->fd); return cioc; =20 @@ -496,6 +502,7 @@ static ssize_t qio_channel_socket_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelSocket *sioc =3D QIO_CHANNEL_SOCKET(ioc); @@ -517,6 +524,10 @@ static ssize_t qio_channel_socket_readv(QIOChannel *io= c, =20 } =20 + if (flags & QIO_CHANNEL_READ_FLAG_MSG_PEEK) { + sflags |=3D MSG_PEEK; + } + retry: ret =3D recvmsg(sioc->fd, &msg, sflags); if (ret < 0) { @@ -624,11 +635,17 @@ static ssize_t qio_channel_socket_readv(QIOChannel *i= oc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelSocket *sioc =3D QIO_CHANNEL_SOCKET(ioc); ssize_t done =3D 0; ssize_t i; + int sflags =3D 0; + + if (flags & QIO_CHANNEL_READ_FLAG_MSG_PEEK) { + sflags |=3D MSG_PEEK; + } =20 for (i =3D 0; i < niov; i++) { ssize_t ret; @@ -636,7 +653,7 @@ static ssize_t qio_channel_socket_readv(QIOChannel *ioc, ret =3D recv(sioc->fd, iov[i].iov_base, iov[i].iov_len, - 0); + sflags); if (ret < 0) { if (errno =3D=3D EAGAIN) { if (done) { diff --git a/io/channel-tls.c b/io/channel-tls.c index 4ce890a538..c730cb8ec5 100644 --- a/io/channel-tls.c +++ b/io/channel-tls.c @@ -260,6 +260,7 @@ static ssize_t qio_channel_tls_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelTLS *tioc =3D QIO_CHANNEL_TLS(ioc); diff --git a/io/channel-websock.c b/io/channel-websock.c index fb4932ade7..a12acc27cf 100644 --- a/io/channel-websock.c +++ b/io/channel-websock.c @@ -1081,6 +1081,7 @@ static ssize_t qio_channel_websock_readv(QIOChannel *= ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelWebsock *wioc =3D QIO_CHANNEL_WEBSOCK(ioc); diff --git a/io/channel.c b/io/channel.c index 0640941ac5..a8c7f11649 100644 --- a/io/channel.c +++ b/io/channel.c @@ -52,6 +52,7 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelClass *klass =3D QIO_CHANNEL_GET_CLASS(ioc); @@ -63,7 +64,14 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc, return -1; } =20 - return klass->io_readv(ioc, iov, niov, fds, nfds, errp); + if ((flags & QIO_CHANNEL_READ_FLAG_MSG_PEEK) && + !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_READ_MSG_PEEK)) { + error_setg_errno(errp, EINVAL, + "Channel does not support peek read"); + return -1; + } + + return klass->io_readv(ioc, iov, niov, fds, nfds, flags, errp); } =20 =20 @@ -146,7 +154,7 @@ int qio_channel_readv_full_all_eof(QIOChannel *ioc, while ((nlocal_iov > 0) || local_fds) { ssize_t len; len =3D qio_channel_readv_full(ioc, local_iov, nlocal_iov, local_f= ds, - local_nfds, errp); + local_nfds, 0, errp); if (len =3D=3D QIO_CHANNEL_ERR_BLOCK) { if (qemu_in_coroutine()) { qio_channel_yield(ioc, G_IO_IN); @@ -284,7 +292,7 @@ ssize_t qio_channel_readv(QIOChannel *ioc, size_t niov, Error **errp) { - return qio_channel_readv_full(ioc, iov, niov, NULL, NULL, errp); + return qio_channel_readv_full(ioc, iov, niov, NULL, NULL, 0, errp); } =20 =20 @@ -303,7 +311,7 @@ ssize_t qio_channel_read(QIOChannel *ioc, Error **errp) { struct iovec iov =3D { .iov_base =3D buf, .iov_len =3D buflen }; - return qio_channel_readv_full(ioc, &iov, 1, NULL, NULL, errp); + return qio_channel_readv_full(ioc, &iov, 1, NULL, NULL, 0, errp); } =20 =20 diff --git a/migration/channel-block.c b/migration/channel-block.c index f4ab53acdb..b7374363c3 100644 --- a/migration/channel-block.c +++ b/migration/channel-block.c @@ -53,6 +53,7 @@ qio_channel_block_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelBlock *bioc =3D QIO_CHANNEL_BLOCK(ioc); diff --git a/migration/rdma.c b/migration/rdma.c index 0ba1668d70..288eadc2d2 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2857,6 +2857,7 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); diff --git a/scsi/qemu-pr-helper.c b/scsi/qemu-pr-helper.c index 196b78c00d..199227a556 100644 --- a/scsi/qemu-pr-helper.c +++ b/scsi/qemu-pr-helper.c @@ -614,7 +614,7 @@ static int coroutine_fn prh_read(PRHelperClient *client= , void *buf, int sz, iov.iov_base =3D buf; iov.iov_len =3D sz; n_read =3D qio_channel_readv_full(QIO_CHANNEL(client->ioc), &iov, = 1, - &fds, &nfds, errp); + &fds, &nfds, 0, errp); =20 if (n_read =3D=3D QIO_CHANNEL_ERR_BLOCK) { qio_channel_yield(QIO_CHANNEL(client->ioc), G_IO_IN); diff --git a/tests/qtest/tpm-emu.c b/tests/qtest/tpm-emu.c index 73e0000a2c..f05fe12f01 100644 --- a/tests/qtest/tpm-emu.c +++ b/tests/qtest/tpm-emu.c @@ -115,7 +115,7 @@ void *tpm_emu_ctrl_thread(void *data) int *pfd =3D NULL; size_t nfd =3D 0; =20 - qio_channel_readv_full(ioc, &iov, 1, &pfd, &nfd, &error_abort); + qio_channel_readv_full(ioc, &iov, 1, &pfd, &nfd, 0, &error_abort); cmd =3D be32_to_cpu(cmd); g_assert_cmpint(cmd, =3D=3D, CMD_SET_DATAFD); g_assert_cmpint(nfd, =3D=3D, 1); diff --git a/tests/unit/test-io-channel-socket.c b/tests/unit/test-io-chann= el-socket.c index b36a5d972a..b964bb202d 100644 --- a/tests/unit/test-io-channel-socket.c +++ b/tests/unit/test-io-channel-socket.c @@ -460,6 +460,7 @@ static void test_io_channel_unix_fd_pass(void) G_N_ELEMENTS(iorecv), &fdrecv, &nfdrecv, + 0, &error_abort); =20 g_assert(nfdrecv =3D=3D G_N_ELEMENTS(fdsend)); diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c index 232984ace6..145eb17c08 100644 --- a/util/vhost-user-server.c +++ b/util/vhost-user-server.c @@ -116,7 +116,7 @@ vu_message_read(VuDev *vu_dev, int conn_fd, VhostUserMs= g *vmsg) * qio_channel_readv_full may have short reads, keeping calling it * until getting VHOST_USER_HDR_SIZE or 0 bytes in total */ - rc =3D qio_channel_readv_full(ioc, &iov, 1, &fds, &nfds, &local_er= r); + rc =3D qio_channel_readv_full(ioc, &iov, 1, &fds, &nfds, 0, &local= _err); if (rc < 0) { if (rc =3D=3D QIO_CHANNEL_ERR_BLOCK) { assert(local_err =3D=3D NULL); --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731688; cv=none; d=zohomail.com; s=zohoarc; b=DSHhSUYdlaZ1Hd3jlfb3Blg8OD5zlUrnxTUaEN9o+9fnyWKqdRd8DBTLrO/0WA+nf5py+8BG72RD5FLj1Aj9RqjTBQ40PuUXls0eNxqXM8J6V8Ojm+13WMTa2ztDx/tdDm5y5fPdG94WvdIbbYUh32J36kcEYFYRSWQ3rUNnaZs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731688; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=I8OjNtSNWyZgGwfsVEjlXafCsoXgcTJpHlHHk83LMZg=; b=lIHsnYoMhwuF5vWGq6TsWa9+pIEZMswg7q5Y9IxX6jN/jLskD4wh3zzDuazJ1RoCPJxL7W2u9UN7dd/rErVUbr4FpA/9zZcifg/mj5ljvYoIjbKDT4iSOe/Vc1vfRDC5PVaGFlFgDpCvUrD707j1iLT8bgToK+6WlrJgfCZJgaQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731688053139.4745127606585; Mon, 6 Feb 2023 17:01:28 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCKk-0001ur-I5; Mon, 06 Feb 2023 19:59:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKU-0001Mr-HL for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKQ-0003za-O3 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:26 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-256-CKm4hdPcNbyJttxoEQqPtA-1; Mon, 06 Feb 2023 19:59:19 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 57E4A1875041; Tue, 7 Feb 2023 00:59:18 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id E79EC492C3C; Tue, 7 Feb 2023 00:59:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731562; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I8OjNtSNWyZgGwfsVEjlXafCsoXgcTJpHlHHk83LMZg=; b=fpdS8GV9wSYWr6S/pmkFkkDp3BKXFi5NVRXb44ukfATc9hNdUILLoTR2pTWybvdHKmBdV2 EeBuFL37gxISzVfrzkr5KmO3thFD0jq/7AY03K/vLF4f4C7mvc1QfJS7KiVtEWd5gRgXjI ka2yaZF8DvaFFD9QcG9wwLU+Cj7LH+0= X-MC-Unique: CKm4hdPcNbyJttxoEQqPtA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , "manish.mishra" , Peter Xu Subject: [PULL 26/30] migration: check magic value for deciding the mapping of channels Date: Tue, 7 Feb 2023 01:56:46 +0100 Message-Id: <20230207005650.1810-27-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731689947100001 Content-Type: text/plain; charset="utf-8" From: "manish.mishra" Current logic assumes that channel connections on the destination side are always established in the same order as the source and the first one will always be the main channel followed by the multifid or post-copy preemption channel. This may not be always true, as even if a channel has a connection established on the source side it can be in the pending state on the destination side and a newer connection can be established first. Basically causing out of order mapping of channels on the destination side. Currently, all channels except post-copy preempt send a magic number, this patch uses that magic number to decide the type of channel. This logic is applicable only for precopy(multifd) live migration, as mentioned, the post-copy preempt channel does not send any magic number. Also, tls live migrations already does tls handshake before creating other channels, so this issue is not possible with tls, hence this logic is avoided for tls live migrations. This patch uses read peek to check the magic number of channels so that current data/control stream management remains un-effected. Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrange Reviewed-by: Juan Quintela Suggested-by: Daniel P. Berrange Signed-off-by: manish.mishra Signed-off-by: Juan Quintela --- migration/channel.h | 5 ++++ migration/multifd.h | 2 +- migration/postcopy-ram.h | 2 +- migration/channel.c | 45 +++++++++++++++++++++++++++++++++ migration/migration.c | 54 ++++++++++++++++++++++++++++------------ migration/multifd.c | 19 +++++++------- migration/postcopy-ram.c | 5 +--- 7 files changed, 101 insertions(+), 31 deletions(-) diff --git a/migration/channel.h b/migration/channel.h index 67a461c28a..5bdb8208a7 100644 --- a/migration/channel.h +++ b/migration/channel.h @@ -24,4 +24,9 @@ void migration_channel_connect(MigrationState *s, QIOChannel *ioc, const char *hostname, Error *error_in); + +int migration_channel_read_peek(QIOChannel *ioc, + const char *buf, + const size_t buflen, + Error **errp); #endif diff --git a/migration/multifd.h b/migration/multifd.h index e2802a9ce2..ff3aa2e2e9 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -18,7 +18,7 @@ void multifd_save_cleanup(void); int multifd_load_setup(Error **errp); int multifd_load_cleanup(Error **errp); bool multifd_recv_all_channels_created(void); -bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp); +void multifd_recv_new_channel(QIOChannel *ioc, Error **errp); void multifd_recv_sync_main(void); int multifd_send_sync_main(QEMUFile *f); int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset); diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 6147bf7d1d..25881c4127 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -190,7 +190,7 @@ enum PostcopyChannels { RAM_CHANNEL_MAX, }; =20 -bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); +void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); int postcopy_preempt_setup(MigrationState *s, Error **errp); int postcopy_preempt_wait_channel(MigrationState *s); =20 diff --git a/migration/channel.c b/migration/channel.c index 1b0815039f..ca3319a309 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -92,3 +92,48 @@ void migration_channel_connect(MigrationState *s, migrate_fd_connect(s, error); error_free(error); } + + +/** + * @migration_channel_read_peek - Peek at migration channel, without + * actually removing it from channel buffer. + * + * @ioc: the channel object + * @buf: the memory region to read data into + * @buflen: the number of bytes to read in @buf + * @errp: pointer to a NULL-initialized error object + * + * Returns 0 if successful, returns -1 and sets @errp if fails. + */ +int migration_channel_read_peek(QIOChannel *ioc, + const char *buf, + const size_t buflen, + Error **errp) +{ + ssize_t len =3D 0; + struct iovec iov =3D { .iov_base =3D (char *)buf, .iov_len =3D buflen = }; + + while (true) { + len =3D qio_channel_readv_full(ioc, &iov, 1, NULL, NULL, + QIO_CHANNEL_READ_FLAG_MSG_PEEK, errp); + + if (len <=3D 0 && len !=3D QIO_CHANNEL_ERR_BLOCK) { + error_setg(errp, + "Failed to peek at channel"); + return -1; + } + + if (len =3D=3D buflen) { + break; + } + + /* 1ms sleep. */ + if (qemu_in_coroutine()) { + qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 1000000); + } else { + g_usleep(1000); + } + } + + return 0; +} diff --git a/migration/migration.c b/migration/migration.c index 4f4d798d3e..66c74f8e17 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -31,6 +31,7 @@ #include "migration.h" #include "savevm.h" #include "qemu-file.h" +#include "channel.h" #include "migration/vmstate.h" #include "block/block.h" #include "qapi/error.h" @@ -664,10 +665,6 @@ static bool migration_incoming_setup(QEMUFile *f, Erro= r **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 - if (multifd_load_setup(errp) !=3D 0) { - return false; - } - if (!mis->from_src_file) { mis->from_src_file =3D f; } @@ -734,31 +731,56 @@ void migration_ioc_process_incoming(QIOChannel *ioc, = Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); Error *local_err =3D NULL; - bool start_migration; QEMUFile *f; + bool default_channel =3D true; + uint32_t channel_magic =3D 0; + int ret =3D 0; =20 - if (!mis->from_src_file) { - /* The first connection (multifd may have multiple) */ + if (migrate_use_multifd() && !migrate_postcopy_ram() && + qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_READ_MSG_PEEK)) { + /* + * With multiple channels, it is possible that we receive channels + * out of order on destination side, causing incorrect mapping of + * source channels on destination side. Check channel MAGIC to + * decide type of channel. Please note this is best effort, postco= py + * preempt channel does not send any magic number so avoid it for + * postcopy live migration. Also tls live migration already does + * tls handshake while initializing main channel so with tls this + * issue is not possible. + */ + ret =3D migration_channel_read_peek(ioc, (void *)&channel_magic, + sizeof(channel_magic), &local_er= r); + + if (ret !=3D 0) { + error_propagate(errp, local_err); + return; + } + + default_channel =3D (channel_magic =3D=3D cpu_to_be32(QEMU_VM_FILE= _MAGIC)); + } else { + default_channel =3D !mis->from_src_file; + } + + if (multifd_load_setup(errp) !=3D 0) { + error_setg(errp, "Failed to setup multifd channels"); + return; + } + + if (default_channel) { f =3D qemu_file_new_input(ioc); =20 if (!migration_incoming_setup(f, errp)) { return; } - - /* - * Common migration only needs one channel, so we can start - * right now. Some features need more than one channel, we wait. - */ - start_migration =3D !migration_needs_multiple_sockets(); } else { /* Multiple connections */ assert(migration_needs_multiple_sockets()); if (migrate_use_multifd()) { - start_migration =3D multifd_recv_new_channel(ioc, &local_err); + multifd_recv_new_channel(ioc, &local_err); } else { assert(migrate_postcopy_preempt()); f =3D qemu_file_new_input(ioc); - start_migration =3D postcopy_preempt_new_channel(mis, f); + postcopy_preempt_new_channel(mis, f); } if (local_err) { error_propagate(errp, local_err); @@ -766,7 +788,7 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Er= ror **errp) } } =20 - if (start_migration) { + if (migration_has_all_channels()) { /* If it's a recovery, we're done */ if (postcopy_try_recover()) { return; diff --git a/migration/multifd.c b/migration/multifd.c index 000ca4d4ec..eeb4fb87ee 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -1164,9 +1164,14 @@ int multifd_load_setup(Error **errp) uint32_t page_count =3D MULTIFD_PACKET_SIZE / qemu_target_page_size(); uint8_t i; =20 - if (!migrate_use_multifd()) { + /* + * Return successfully if multiFD recv state is already initialised + * or multiFD is not enabled. + */ + if (multifd_recv_state || !migrate_use_multifd()) { return 0; } + if (!migrate_multi_channels_is_allowed()) { error_setg(errp, "multifd is not supported by current protocol"); return -1; @@ -1227,11 +1232,9 @@ bool multifd_recv_all_channels_created(void) =20 /* * Try to receive all multifd channels to get ready for the migration. - * - Return true and do not set @errp when correctly receiving all channel= s; - * - Return false and do not set @errp when correctly receiving the curren= t one; - * - Return false and set @errp when failing to receive the current channe= l. + * Sets @errp when failing to receive the current channel. */ -bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp) +void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) { MultiFDRecvParams *p; Error *local_err =3D NULL; @@ -1244,7 +1247,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error = **errp) "failed to receive packet" " via multifd channel %d: ", qatomic_read(&multifd_recv_state->count)); - return false; + return; } trace_multifd_recv_new_channel(id); =20 @@ -1254,7 +1257,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error = **errp) id); multifd_recv_terminate_threads(local_err); error_propagate(errp, local_err); - return false; + return; } p->c =3D ioc; object_ref(OBJECT(ioc)); @@ -1265,6 +1268,4 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error = **errp) qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, QEMU_THREAD_JOINABLE); qatomic_inc(&multifd_recv_state->count); - return qatomic_read(&multifd_recv_state->count) =3D=3D - migrate_multifd_channels(); } diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 0c55df0e52..b98e95dab0 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1538,7 +1538,7 @@ void postcopy_unregister_shared_ufd(struct PostCopyFD= *pcfd) } } =20 -bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile) +void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile) { /* * The new loading channel has its own threads, so it needs to be @@ -1547,9 +1547,6 @@ bool postcopy_preempt_new_channel(MigrationIncomingSt= ate *mis, QEMUFile *file) qemu_file_set_blocking(file, true); mis->postcopy_qemufile_dst =3D file; trace_postcopy_preempt_new_channel(); - - /* Start the migration immediately */ - return true; } =20 /* --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731863; cv=none; d=zohomail.com; s=zohoarc; b=DtgM2XsfHXMfecP6MupLQpHfi1iJbl8o8aKD9Ari9YNLoFS2lbFpZg/RRq65lFWt43wMMV7+F6Uk0SU1IGFmKZC/kB+F7jQ0vgDvwZHh0L4eqe4c6k/SJ0lsDZAc1dnSeMgChTGJOPohoFzRpoYIjvUyI92Bgb/skKRqL8fbkj0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731863; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/vi3kUI9VRzfEdyEafXxBkT0bxSOiapenyEEaiH00lY=; b=INk9JwvKJ28jakZGzHqykKfbNUliH7HPvpGXxGiuV0WT0X9OaXC7vzi9qBX7aB6dT2gSPr22DWXpISvUSoI5kumfbryY/OzBPKwYN4CbaPOlBWKAZ7CT6pyW1CtZQrfxygOR86Q8+apqSY0jNbB2rNXhgIReZLtz8EFdieNbvFw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731863950649.283532291459; Mon, 6 Feb 2023 17:04:23 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCKl-0002BH-6k; Mon, 06 Feb 2023 19:59:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKZ-0001UC-CM for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:34 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKW-00040I-Tq for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:31 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-582-KPEAZe1yNMqDu1WeIVDh4Q-1; Mon, 06 Feb 2023 19:59:24 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 69AA6811E6E; Tue, 7 Feb 2023 00:59:23 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9E951492C3C; Tue, 7 Feb 2023 00:59:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731568; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/vi3kUI9VRzfEdyEafXxBkT0bxSOiapenyEEaiH00lY=; b=SMavTxTgGZ92fqEpOT2jQ8VR9vTLT42s142eDn3QwmWJKwulPAUWxqX4BhIWtHkUpWFUSd ZfUO7Sis+48euuYQXvPUcYaKEcK8W7Ds8v3iuf2tzeZz4BQJ/GMrlaEV25owzDXPutBql7 L67EKY/iC1xcMNA82ZGcxo15s+Qlayg= X-MC-Unique: KPEAZe1yNMqDu1WeIVDh4Q-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Zhenzhong Duan Subject: [PULL 27/30] multifd: Fix a race on reading MultiFDPages_t.block Date: Tue, 7 Feb 2023 01:56:47 +0100 Message-Id: <20230207005650.1810-28-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731865029100001 Content-Type: text/plain; charset="utf-8" From: Zhenzhong Duan In multifd_queue_page() MultiFDPages_t.block is checked twice. Between the two checks, MultiFDPages_t.block may be reset to NULL by multifd thread. This lead to the 2nd check always true then a redundant page submitted to multifd thread again. Signed-off-by: Zhenzhong Duan Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/multifd.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index eeb4fb87ee..ad89293b4e 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -442,6 +442,7 @@ static int multifd_send_pages(QEMUFile *f) int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset) { MultiFDPages_t *pages =3D multifd_send_state->pages; + bool changed =3D false; =20 if (!pages->block) { pages->block =3D block; @@ -454,14 +455,16 @@ int multifd_queue_page(QEMUFile *f, RAMBlock *block, = ram_addr_t offset) if (pages->num < pages->allocated) { return 1; } + } else { + changed =3D true; } =20 if (multifd_send_pages(f) < 0) { return -1; } =20 - if (pages->block !=3D block) { - return multifd_queue_page(f, block, offset); + if (changed) { + return multifd_queue_page(f, block, offset); } =20 return 1; --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731857; cv=none; d=zohomail.com; s=zohoarc; b=VHpA+suXjoQWs8qkTasIeMHFL5kkMxOwBaSqPdi89n6O844hJaf3rlsptYboOyWchInGBL1NihHBvMOJdEkft86TRWROGZH6dRLkB8wXW1s5S2TZ0EdMYPbsijWeLfl5EHZSiNF85iRxyI3FaiLK5/Op6+etjIReT7F+AIeebBg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731857; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CFfCGNpJ2ofu7yh6AXsFdYvKkqDeKIq/PJ0QkgXtDLY=; b=H/tEjmWlQNUm8IZyaOGcqhlGwcT8eJD6oEU84Ts2EyUKCsrUwKXeeWGmnMahHLImeu97T8s/YmCofxGXcXrar8nKMjHhIvhdZ1eP0b+EkcpJCWqb/rVVL+qKx9yyPV//eXALZsKmstMakSSjoctrb2FQjW8AlQMScYduPohVgdk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731857745639.8972068966513; Mon, 6 Feb 2023 17:04:17 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCKo-0002er-Dn; Mon, 06 Feb 2023 19:59:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKh-0001rA-18 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:40 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKc-000413-AM for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:38 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-625-Qt8smNQgOr23hVwO4GUoWw-1; Mon, 06 Feb 2023 19:59:29 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9B8EB3C025B0; Tue, 7 Feb 2023 00:59:28 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id AF260492C3C; Tue, 7 Feb 2023 00:59:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731572; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CFfCGNpJ2ofu7yh6AXsFdYvKkqDeKIq/PJ0QkgXtDLY=; b=JONUXBuMz603em1p/cy5g3FpH0fojenoYcYaAT9csW1NmNScxexaatvNPjS/U3Ksv6zM52 f1/HAjw8WgtqTFHKRZMxPpUQgBS8Ku0MWDA2lIK0y6pRj4vQw0TTuNRUQ2uRikdxV5lLBz jEB3D7vyrHSDvispDt52RZjm6ogmLG8= X-MC-Unique: Qt8smNQgOr23hVwO4GUoWw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Zhenzhong Duan Subject: [PULL 28/30] multifd: Fix flush of zero copy page send request Date: Tue, 7 Feb 2023 01:56:48 +0100 Message-Id: <20230207005650.1810-29-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731859124100007 Content-Type: text/plain; charset="utf-8" From: Zhenzhong Duan Make IO channel flush call after the inflight request has been drained in multifd thread, or else we may missed to flush the inflight request. Signed-off-by: Zhenzhong Duan Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- .../x86_64-quintela-devices.mak | 7 + .../x86_64-quintela2-devices.mak | 6 + migration/multifd.c | 8 +- migration/multifd.c.orig | 1274 +++++++++++++++++ 4 files changed, 1291 insertions(+), 4 deletions(-) create mode 100644 configs/devices/x86_64-softmmu/x86_64-quintela-devices.= mak create mode 100644 configs/devices/x86_64-softmmu/x86_64-quintela2-devices= .mak create mode 100644 migration/multifd.c.orig diff --git a/configs/devices/x86_64-softmmu/x86_64-quintela-devices.mak b/c= onfigs/devices/x86_64-softmmu/x86_64-quintela-devices.mak new file mode 100644 index 0000000000..ee2bb8c5c9 --- /dev/null +++ b/configs/devices/x86_64-softmmu/x86_64-quintela-devices.mak @@ -0,0 +1,7 @@ +# Boards: +# +CONFIG_ISAPC=3Dn +CONFIG_I440FX=3Dn +CONFIG_Q35=3Dn +CONFIG_MICROVM=3Dy + diff --git a/configs/devices/x86_64-softmmu/x86_64-quintela2-devices.mak b/= configs/devices/x86_64-softmmu/x86_64-quintela2-devices.mak new file mode 100644 index 0000000000..f7e4dae842 --- /dev/null +++ b/configs/devices/x86_64-softmmu/x86_64-quintela2-devices.mak @@ -0,0 +1,6 @@ +# Boards: +# +CONFIG_ISAPC=3Dy +CONFIG_I440FX=3Dy +CONFIG_Q35=3Dy +CONFIG_MICROVM=3Dy diff --git a/migration/multifd.c b/migration/multifd.c index ad89293b4e..437bf6f808 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -630,16 +630,16 @@ int multifd_send_sync_main(QEMUFile *f) stat64_add(&ram_atomic_counters.transferred, p->packet_len); qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); - - if (flush_zero_copy && p->c && (multifd_zero_copy_flush(p->c) < 0)= ) { - return -1; - } } for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 trace_multifd_send_sync_main_wait(p->id); qemu_sem_wait(&p->sem_sync); + + if (flush_zero_copy && p->c && (multifd_zero_copy_flush(p->c) < 0)= ) { + return -1; + } } trace_multifd_send_sync_main(multifd_send_state->packet_num); =20 diff --git a/migration/multifd.c.orig b/migration/multifd.c.orig new file mode 100644 index 0000000000..ad89293b4e --- /dev/null +++ b/migration/multifd.c.orig @@ -0,0 +1,1274 @@ +/* + * Multifd common code + * + * Copyright (c) 2019-2020 Red Hat Inc + * + * Authors: + * Juan Quintela + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/rcu.h" +#include "exec/target_page.h" +#include "sysemu/sysemu.h" +#include "exec/ramblock.h" +#include "qemu/error-report.h" +#include "qapi/error.h" +#include "ram.h" +#include "migration.h" +#include "socket.h" +#include "tls.h" +#include "qemu-file.h" +#include "trace.h" +#include "multifd.h" + +#include "qemu/yank.h" +#include "io/channel-socket.h" +#include "yank_functions.h" + +/* Multiple fd's */ + +#define MULTIFD_MAGIC 0x11223344U +#define MULTIFD_VERSION 1 + +typedef struct { + uint32_t magic; + uint32_t version; + unsigned char uuid[16]; /* QemuUUID */ + uint8_t id; + uint8_t unused1[7]; /* Reserved for future use */ + uint64_t unused2[4]; /* Reserved for future use */ +} __attribute__((packed)) MultiFDInit_t; + +/* Multifd without compression */ + +/** + * nocomp_send_setup: setup send side + * + * For no compression this function does nothing. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int nocomp_send_setup(MultiFDSendParams *p, Error **errp) +{ + return 0; +} + +/** + * nocomp_send_cleanup: cleanup send side + * + * For no compression this function does nothing. + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static void nocomp_send_cleanup(MultiFDSendParams *p, Error **errp) +{ + return; +} + +/** + * nocomp_send_prepare: prepare date to be able to send + * + * For no compression we just have to calculate the size of the + * packet. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int nocomp_send_prepare(MultiFDSendParams *p, Error **errp) +{ + MultiFDPages_t *pages =3D p->pages; + + for (int i =3D 0; i < p->normal_num; i++) { + p->iov[p->iovs_num].iov_base =3D pages->block->host + p->normal[i]; + p->iov[p->iovs_num].iov_len =3D p->page_size; + p->iovs_num++; + } + + p->next_packet_size =3D p->normal_num * p->page_size; + p->flags |=3D MULTIFD_FLAG_NOCOMP; + return 0; +} + +/** + * nocomp_recv_setup: setup receive side + * + * For no compression this function does nothing. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int nocomp_recv_setup(MultiFDRecvParams *p, Error **errp) +{ + return 0; +} + +/** + * nocomp_recv_cleanup: setup receive side + * + * For no compression this function does nothing. + * + * @p: Params for the channel that we are using + */ +static void nocomp_recv_cleanup(MultiFDRecvParams *p) +{ +} + +/** + * nocomp_recv_pages: read the data from the channel into actual pages + * + * For no compression we just need to read things into the correct place. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int nocomp_recv_pages(MultiFDRecvParams *p, Error **errp) +{ + uint32_t flags =3D p->flags & MULTIFD_FLAG_COMPRESSION_MASK; + + if (flags !=3D MULTIFD_FLAG_NOCOMP) { + error_setg(errp, "multifd %u: flags received %x flags expected %x", + p->id, flags, MULTIFD_FLAG_NOCOMP); + return -1; + } + for (int i =3D 0; i < p->normal_num; i++) { + p->iov[i].iov_base =3D p->host + p->normal[i]; + p->iov[i].iov_len =3D p->page_size; + } + return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); +} + +static MultiFDMethods multifd_nocomp_ops =3D { + .send_setup =3D nocomp_send_setup, + .send_cleanup =3D nocomp_send_cleanup, + .send_prepare =3D nocomp_send_prepare, + .recv_setup =3D nocomp_recv_setup, + .recv_cleanup =3D nocomp_recv_cleanup, + .recv_pages =3D nocomp_recv_pages +}; + +static MultiFDMethods *multifd_ops[MULTIFD_COMPRESSION__MAX] =3D { + [MULTIFD_COMPRESSION_NONE] =3D &multifd_nocomp_ops, +}; + +void multifd_register_ops(int method, MultiFDMethods *ops) +{ + assert(0 < method && method < MULTIFD_COMPRESSION__MAX); + multifd_ops[method] =3D ops; +} + +static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) +{ + MultiFDInit_t msg =3D {}; + int ret; + + msg.magic =3D cpu_to_be32(MULTIFD_MAGIC); + msg.version =3D cpu_to_be32(MULTIFD_VERSION); + msg.id =3D p->id; + memcpy(msg.uuid, &qemu_uuid.data, sizeof(msg.uuid)); + + ret =3D qio_channel_write_all(p->c, (char *)&msg, sizeof(msg), errp); + if (ret !=3D 0) { + return -1; + } + return 0; +} + +static int multifd_recv_initial_packet(QIOChannel *c, Error **errp) +{ + MultiFDInit_t msg; + int ret; + + ret =3D qio_channel_read_all(c, (char *)&msg, sizeof(msg), errp); + if (ret !=3D 0) { + return -1; + } + + msg.magic =3D be32_to_cpu(msg.magic); + msg.version =3D be32_to_cpu(msg.version); + + if (msg.magic !=3D MULTIFD_MAGIC) { + error_setg(errp, "multifd: received packet magic %x " + "expected %x", msg.magic, MULTIFD_MAGIC); + return -1; + } + + if (msg.version !=3D MULTIFD_VERSION) { + error_setg(errp, "multifd: received packet version %u " + "expected %u", msg.version, MULTIFD_VERSION); + return -1; + } + + if (memcmp(msg.uuid, &qemu_uuid, sizeof(qemu_uuid))) { + char *uuid =3D qemu_uuid_unparse_strdup(&qemu_uuid); + char *msg_uuid =3D qemu_uuid_unparse_strdup((const QemuUUID *)msg.= uuid); + + error_setg(errp, "multifd: received uuid '%s' and expected " + "uuid '%s' for channel %hhd", msg_uuid, uuid, msg.id); + g_free(uuid); + g_free(msg_uuid); + return -1; + } + + if (msg.id > migrate_multifd_channels()) { + error_setg(errp, "multifd: received channel version %u " + "expected %u", msg.version, MULTIFD_VERSION); + return -1; + } + + return msg.id; +} + +static MultiFDPages_t *multifd_pages_init(size_t size) +{ + MultiFDPages_t *pages =3D g_new0(MultiFDPages_t, 1); + + pages->allocated =3D size; + pages->offset =3D g_new0(ram_addr_t, size); + + return pages; +} + +static void multifd_pages_clear(MultiFDPages_t *pages) +{ + pages->num =3D 0; + pages->allocated =3D 0; + pages->packet_num =3D 0; + pages->block =3D NULL; + g_free(pages->offset); + pages->offset =3D NULL; + g_free(pages); +} + +static void multifd_send_fill_packet(MultiFDSendParams *p) +{ + MultiFDPacket_t *packet =3D p->packet; + int i; + + packet->flags =3D cpu_to_be32(p->flags); + packet->pages_alloc =3D cpu_to_be32(p->pages->allocated); + packet->normal_pages =3D cpu_to_be32(p->normal_num); + packet->next_packet_size =3D cpu_to_be32(p->next_packet_size); + packet->packet_num =3D cpu_to_be64(p->packet_num); + + if (p->pages->block) { + strncpy(packet->ramblock, p->pages->block->idstr, 256); + } + + for (i =3D 0; i < p->normal_num; i++) { + /* there are architectures where ram_addr_t is 32 bit */ + uint64_t temp =3D p->normal[i]; + + packet->offset[i] =3D cpu_to_be64(temp); + } +} + +static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) +{ + MultiFDPacket_t *packet =3D p->packet; + RAMBlock *block; + int i; + + packet->magic =3D be32_to_cpu(packet->magic); + if (packet->magic !=3D MULTIFD_MAGIC) { + error_setg(errp, "multifd: received packet " + "magic %x and expected magic %x", + packet->magic, MULTIFD_MAGIC); + return -1; + } + + packet->version =3D be32_to_cpu(packet->version); + if (packet->version !=3D MULTIFD_VERSION) { + error_setg(errp, "multifd: received packet " + "version %u and expected version %u", + packet->version, MULTIFD_VERSION); + return -1; + } + + p->flags =3D be32_to_cpu(packet->flags); + + packet->pages_alloc =3D be32_to_cpu(packet->pages_alloc); + /* + * If we received a packet that is 100 times bigger than expected + * just stop migration. It is a magic number. + */ + if (packet->pages_alloc > p->page_count) { + error_setg(errp, "multifd: received packet " + "with size %u and expected a size of %u", + packet->pages_alloc, p->page_count) ; + return -1; + } + + p->normal_num =3D be32_to_cpu(packet->normal_pages); + if (p->normal_num > packet->pages_alloc) { + error_setg(errp, "multifd: received packet " + "with %u pages and expected maximum pages are %u", + p->normal_num, packet->pages_alloc) ; + return -1; + } + + p->next_packet_size =3D be32_to_cpu(packet->next_packet_size); + p->packet_num =3D be64_to_cpu(packet->packet_num); + + if (p->normal_num =3D=3D 0) { + return 0; + } + + /* make sure that ramblock is 0 terminated */ + packet->ramblock[255] =3D 0; + block =3D qemu_ram_block_by_name(packet->ramblock); + if (!block) { + error_setg(errp, "multifd: unknown ram block %s", + packet->ramblock); + return -1; + } + + p->host =3D block->host; + for (i =3D 0; i < p->normal_num; i++) { + uint64_t offset =3D be64_to_cpu(packet->offset[i]); + + if (offset > (block->used_length - p->page_size)) { + error_setg(errp, "multifd: offset too long %" PRIu64 + " (max " RAM_ADDR_FMT ")", + offset, block->used_length); + return -1; + } + p->normal[i] =3D offset; + } + + return 0; +} + +struct { + MultiFDSendParams *params; + /* array of pages to sent */ + MultiFDPages_t *pages; + /* global number of generated multifd packets */ + uint64_t packet_num; + /* send channels ready */ + QemuSemaphore channels_ready; + /* + * Have we already run terminate threads. There is a race when it + * happens that we got one error while we are exiting. + * We will use atomic operations. Only valid values are 0 and 1. + */ + int exiting; + /* multifd ops */ + MultiFDMethods *ops; +} *multifd_send_state; + +/* + * How we use multifd_send_state->pages and channel->pages? + * + * We create a pages for each channel, and a main one. Each time that + * we need to send a batch of pages we interchange the ones between + * multifd_send_state and the channel that is sending it. There are + * two reasons for that: + * - to not have to do so many mallocs during migration + * - to make easier to know what to free at the end of migration + * + * This way we always know who is the owner of each "pages" struct, + * and we don't need any locking. It belongs to the migration thread + * or to the channel thread. Switching is safe because the migration + * thread is using the channel mutex when changing it, and the channel + * have to had finish with its own, otherwise pending_job can't be + * false. + */ + +static int multifd_send_pages(QEMUFile *f) +{ + int i; + static int next_channel; + MultiFDSendParams *p =3D NULL; /* make happy gcc */ + MultiFDPages_t *pages =3D multifd_send_state->pages; + uint64_t transferred; + + if (qatomic_read(&multifd_send_state->exiting)) { + return -1; + } + + qemu_sem_wait(&multifd_send_state->channels_ready); + /* + * next_channel can remain from a previous migration that was + * using more channels, so ensure it doesn't overflow if the + * limit is lower now. + */ + next_channel %=3D migrate_multifd_channels(); + for (i =3D next_channel;; i =3D (i + 1) % migrate_multifd_channels()) { + p =3D &multifd_send_state->params[i]; + + qemu_mutex_lock(&p->mutex); + if (p->quit) { + error_report("%s: channel %d has already quit!", __func__, i); + qemu_mutex_unlock(&p->mutex); + return -1; + } + if (!p->pending_job) { + p->pending_job++; + next_channel =3D (i + 1) % migrate_multifd_channels(); + break; + } + qemu_mutex_unlock(&p->mutex); + } + assert(!p->pages->num); + assert(!p->pages->block); + + p->packet_num =3D multifd_send_state->packet_num++; + multifd_send_state->pages =3D p->pages; + p->pages =3D pages; + transferred =3D ((uint64_t) pages->num) * p->page_size + p->packet_len; + qemu_file_acct_rate_limit(f, transferred); + ram_counters.multifd_bytes +=3D transferred; + stat64_add(&ram_atomic_counters.transferred, transferred); + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->sem); + + return 1; +} + +int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset) +{ + MultiFDPages_t *pages =3D multifd_send_state->pages; + bool changed =3D false; + + if (!pages->block) { + pages->block =3D block; + } + + if (pages->block =3D=3D block) { + pages->offset[pages->num] =3D offset; + pages->num++; + + if (pages->num < pages->allocated) { + return 1; + } + } else { + changed =3D true; + } + + if (multifd_send_pages(f) < 0) { + return -1; + } + + if (changed) { + return multifd_queue_page(f, block, offset); + } + + return 1; +} + +static void multifd_send_terminate_threads(Error *err) +{ + int i; + + trace_multifd_send_terminate_threads(err !=3D NULL); + + if (err) { + MigrationState *s =3D migrate_get_current(); + migrate_set_error(s, err); + if (s->state =3D=3D MIGRATION_STATUS_SETUP || + s->state =3D=3D MIGRATION_STATUS_PRE_SWITCHOVER || + s->state =3D=3D MIGRATION_STATUS_DEVICE || + s->state =3D=3D MIGRATION_STATUS_ACTIVE) { + migrate_set_state(&s->state, s->state, + MIGRATION_STATUS_FAILED); + } + } + + /* + * We don't want to exit each threads twice. Depending on where + * we get the error, or if there are two independent errors in two + * threads at the same time, we can end calling this function + * twice. + */ + if (qatomic_xchg(&multifd_send_state->exiting, 1)) { + return; + } + + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + qemu_mutex_lock(&p->mutex); + p->quit =3D true; + qemu_sem_post(&p->sem); + if (p->c) { + qio_channel_shutdown(p->c, QIO_CHANNEL_SHUTDOWN_BOTH, NULL); + } + qemu_mutex_unlock(&p->mutex); + } +} + +void multifd_save_cleanup(void) +{ + int i; + + if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { + return; + } + multifd_send_terminate_threads(NULL); + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + if (p->running) { + qemu_thread_join(&p->thread); + } + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + Error *local_err =3D NULL; + + if (p->registered_yank) { + migration_ioc_unregister_yank(p->c); + } + socket_send_channel_destroy(p->c); + p->c =3D NULL; + qemu_mutex_destroy(&p->mutex); + qemu_sem_destroy(&p->sem); + qemu_sem_destroy(&p->sem_sync); + g_free(p->name); + p->name =3D NULL; + multifd_pages_clear(p->pages); + p->pages =3D NULL; + p->packet_len =3D 0; + g_free(p->packet); + p->packet =3D NULL; + g_free(p->iov); + p->iov =3D NULL; + g_free(p->normal); + p->normal =3D NULL; + multifd_send_state->ops->send_cleanup(p, &local_err); + if (local_err) { + migrate_set_error(migrate_get_current(), local_err); + error_free(local_err); + } + } + qemu_sem_destroy(&multifd_send_state->channels_ready); + g_free(multifd_send_state->params); + multifd_send_state->params =3D NULL; + multifd_pages_clear(multifd_send_state->pages); + multifd_send_state->pages =3D NULL; + g_free(multifd_send_state); + multifd_send_state =3D NULL; +} + +static int multifd_zero_copy_flush(QIOChannel *c) +{ + int ret; + Error *err =3D NULL; + + ret =3D qio_channel_flush(c, &err); + if (ret < 0) { + error_report_err(err); + return -1; + } + if (ret =3D=3D 1) { + dirty_sync_missed_zero_copy(); + } + + return ret; +} + +int multifd_send_sync_main(QEMUFile *f) +{ + int i; + bool flush_zero_copy; + + if (!migrate_use_multifd()) { + return 0; + } + if (multifd_send_state->pages->num) { + if (multifd_send_pages(f) < 0) { + error_report("%s: multifd_send_pages fail", __func__); + return -1; + } + } + + /* + * When using zero-copy, it's necessary to flush the pages before any = of + * the pages can be sent again, so we'll make sure the new version of = the + * pages will always arrive _later_ than the old pages. + * + * Currently we achieve this by flushing the zero-page requested writes + * per ram iteration, but in the future we could potentially optimize = it + * to be less frequent, e.g. only after we finished one whole scanning= of + * all the dirty bitmaps. + */ + + flush_zero_copy =3D migrate_use_zero_copy_send(); + + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + trace_multifd_send_sync_main_signal(p->id); + + qemu_mutex_lock(&p->mutex); + + if (p->quit) { + error_report("%s: channel %d has already quit", __func__, i); + qemu_mutex_unlock(&p->mutex); + return -1; + } + + p->packet_num =3D multifd_send_state->packet_num++; + p->flags |=3D MULTIFD_FLAG_SYNC; + p->pending_job++; + qemu_file_acct_rate_limit(f, p->packet_len); + ram_counters.multifd_bytes +=3D p->packet_len; + stat64_add(&ram_atomic_counters.transferred, p->packet_len); + qemu_mutex_unlock(&p->mutex); + qemu_sem_post(&p->sem); + + if (flush_zero_copy && p->c && (multifd_zero_copy_flush(p->c) < 0)= ) { + return -1; + } + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + trace_multifd_send_sync_main_wait(p->id); + qemu_sem_wait(&p->sem_sync); + } + trace_multifd_send_sync_main(multifd_send_state->packet_num); + + return 0; +} + +static void *multifd_send_thread(void *opaque) +{ + MultiFDSendParams *p =3D opaque; + Error *local_err =3D NULL; + int ret =3D 0; + bool use_zero_copy_send =3D migrate_use_zero_copy_send(); + + trace_multifd_send_thread_start(p->id); + rcu_register_thread(); + + if (multifd_send_initial_packet(p, &local_err) < 0) { + ret =3D -1; + goto out; + } + /* initial packet */ + p->num_packets =3D 1; + + while (true) { + qemu_sem_wait(&p->sem); + + if (qatomic_read(&multifd_send_state->exiting)) { + break; + } + qemu_mutex_lock(&p->mutex); + + if (p->pending_job) { + uint64_t packet_num =3D p->packet_num; + uint32_t flags =3D p->flags; + p->normal_num =3D 0; + + if (use_zero_copy_send) { + p->iovs_num =3D 0; + } else { + p->iovs_num =3D 1; + } + + for (int i =3D 0; i < p->pages->num; i++) { + p->normal[p->normal_num] =3D p->pages->offset[i]; + p->normal_num++; + } + + if (p->normal_num) { + ret =3D multifd_send_state->ops->send_prepare(p, &local_er= r); + if (ret !=3D 0) { + qemu_mutex_unlock(&p->mutex); + break; + } + } + multifd_send_fill_packet(p); + p->flags =3D 0; + p->num_packets++; + p->total_normal_pages +=3D p->normal_num; + p->pages->num =3D 0; + p->pages->block =3D NULL; + qemu_mutex_unlock(&p->mutex); + + trace_multifd_send(p->id, packet_num, p->normal_num, flags, + p->next_packet_size); + + if (use_zero_copy_send) { + /* Send header first, without zerocopy */ + ret =3D qio_channel_write_all(p->c, (void *)p->packet, + p->packet_len, &local_err); + if (ret !=3D 0) { + break; + } + } else { + /* Send header using the same writev call */ + p->iov[0].iov_len =3D p->packet_len; + p->iov[0].iov_base =3D p->packet; + } + + ret =3D qio_channel_writev_full_all(p->c, p->iov, p->iovs_num,= NULL, + 0, p->write_flags, &local_er= r); + if (ret !=3D 0) { + break; + } + + qemu_mutex_lock(&p->mutex); + p->pending_job--; + qemu_mutex_unlock(&p->mutex); + + if (flags & MULTIFD_FLAG_SYNC) { + qemu_sem_post(&p->sem_sync); + } + qemu_sem_post(&multifd_send_state->channels_ready); + } else if (p->quit) { + qemu_mutex_unlock(&p->mutex); + break; + } else { + qemu_mutex_unlock(&p->mutex); + /* sometimes there are spurious wakeups */ + } + } + +out: + if (local_err) { + trace_multifd_send_error(p->id); + multifd_send_terminate_threads(local_err); + error_free(local_err); + } + + /* + * Error happen, I will exit, but I can't just leave, tell + * who pay attention to me. + */ + if (ret !=3D 0) { + qemu_sem_post(&p->sem_sync); + qemu_sem_post(&multifd_send_state->channels_ready); + } + + qemu_mutex_lock(&p->mutex); + p->running =3D false; + qemu_mutex_unlock(&p->mutex); + + rcu_unregister_thread(); + trace_multifd_send_thread_end(p->id, p->num_packets, p->total_normal_p= ages); + + return NULL; +} + +static bool multifd_channel_connect(MultiFDSendParams *p, + QIOChannel *ioc, + Error *error); + +static void multifd_tls_outgoing_handshake(QIOTask *task, + gpointer opaque) +{ + MultiFDSendParams *p =3D opaque; + QIOChannel *ioc =3D QIO_CHANNEL(qio_task_get_source(task)); + Error *err =3D NULL; + + if (qio_task_propagate_error(task, &err)) { + trace_multifd_tls_outgoing_handshake_error(ioc, error_get_pretty(e= rr)); + } else { + trace_multifd_tls_outgoing_handshake_complete(ioc); + } + + if (!multifd_channel_connect(p, ioc, err)) { + /* + * Error happen, mark multifd_send_thread status as 'quit' althoug= h it + * is not created, and then tell who pay attention to me. + */ + p->quit =3D true; + qemu_sem_post(&multifd_send_state->channels_ready); + qemu_sem_post(&p->sem_sync); + } +} + +static void *multifd_tls_handshake_thread(void *opaque) +{ + MultiFDSendParams *p =3D opaque; + QIOChannelTLS *tioc =3D QIO_CHANNEL_TLS(p->c); + + qio_channel_tls_handshake(tioc, + multifd_tls_outgoing_handshake, + p, + NULL, + NULL); + return NULL; +} + +static void multifd_tls_channel_connect(MultiFDSendParams *p, + QIOChannel *ioc, + Error **errp) +{ + MigrationState *s =3D migrate_get_current(); + const char *hostname =3D s->hostname; + QIOChannelTLS *tioc; + + tioc =3D migration_tls_client_create(s, ioc, hostname, errp); + if (!tioc) { + return; + } + + object_unref(OBJECT(ioc)); + trace_multifd_tls_outgoing_handshake_start(ioc, tioc, hostname); + qio_channel_set_name(QIO_CHANNEL(tioc), "multifd-tls-outgoing"); + p->c =3D QIO_CHANNEL(tioc); + qemu_thread_create(&p->thread, "multifd-tls-handshake-worker", + multifd_tls_handshake_thread, p, + QEMU_THREAD_JOINABLE); +} + +static bool multifd_channel_connect(MultiFDSendParams *p, + QIOChannel *ioc, + Error *error) +{ + trace_multifd_set_outgoing_channel( + ioc, object_get_typename(OBJECT(ioc)), + migrate_get_current()->hostname, error); + + if (!error) { + if (migrate_channel_requires_tls_upgrade(ioc)) { + multifd_tls_channel_connect(p, ioc, &error); + if (!error) { + /* + * tls_channel_connect will call back to this + * function after the TLS handshake, + * so we mustn't call multifd_send_thread until then + */ + return true; + } else { + return false; + } + } else { + migration_ioc_register_yank(ioc); + p->registered_yank =3D true; + p->c =3D ioc; + qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, + QEMU_THREAD_JOINABLE); + } + return true; + } + + return false; +} + +static void multifd_new_send_channel_cleanup(MultiFDSendParams *p, + QIOChannel *ioc, Error *err) +{ + migrate_set_error(migrate_get_current(), err); + /* Error happen, we need to tell who pay attention to me */ + qemu_sem_post(&multifd_send_state->channels_ready); + qemu_sem_post(&p->sem_sync); + /* + * Although multifd_send_thread is not created, but main migration + * thread neet to judge whether it is running, so we need to mark + * its status. + */ + p->quit =3D true; + object_unref(OBJECT(ioc)); + error_free(err); +} + +static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) +{ + MultiFDSendParams *p =3D opaque; + QIOChannel *sioc =3D QIO_CHANNEL(qio_task_get_source(task)); + Error *local_err =3D NULL; + + trace_multifd_new_send_channel_async(p->id); + if (qio_task_propagate_error(task, &local_err)) { + goto cleanup; + } else { + p->c =3D QIO_CHANNEL(sioc); + qio_channel_set_delay(p->c, false); + p->running =3D true; + if (!multifd_channel_connect(p, sioc, local_err)) { + goto cleanup; + } + return; + } + +cleanup: + multifd_new_send_channel_cleanup(p, sioc, local_err); +} + +int multifd_save_setup(Error **errp) +{ + int thread_count; + uint32_t page_count =3D MULTIFD_PACKET_SIZE / qemu_target_page_size(); + uint8_t i; + + if (!migrate_use_multifd()) { + return 0; + } + if (!migrate_multi_channels_is_allowed()) { + error_setg(errp, "multifd is not supported by current protocol"); + return -1; + } + + thread_count =3D migrate_multifd_channels(); + multifd_send_state =3D g_malloc0(sizeof(*multifd_send_state)); + multifd_send_state->params =3D g_new0(MultiFDSendParams, thread_count); + multifd_send_state->pages =3D multifd_pages_init(page_count); + qemu_sem_init(&multifd_send_state->channels_ready, 0); + qatomic_set(&multifd_send_state->exiting, 0); + multifd_send_state->ops =3D multifd_ops[migrate_multifd_compression()]; + + for (i =3D 0; i < thread_count; i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + + qemu_mutex_init(&p->mutex); + qemu_sem_init(&p->sem, 0); + qemu_sem_init(&p->sem_sync, 0); + p->quit =3D false; + p->pending_job =3D 0; + p->id =3D i; + p->pages =3D multifd_pages_init(page_count); + p->packet_len =3D sizeof(MultiFDPacket_t) + + sizeof(uint64_t) * page_count; + p->packet =3D g_malloc0(p->packet_len); + p->packet->magic =3D cpu_to_be32(MULTIFD_MAGIC); + p->packet->version =3D cpu_to_be32(MULTIFD_VERSION); + p->name =3D g_strdup_printf("multifdsend_%d", i); + /* We need one extra place for the packet header */ + p->iov =3D g_new0(struct iovec, page_count + 1); + p->normal =3D g_new0(ram_addr_t, page_count); + p->page_size =3D qemu_target_page_size(); + p->page_count =3D page_count; + + if (migrate_use_zero_copy_send()) { + p->write_flags =3D QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; + } else { + p->write_flags =3D 0; + } + + socket_send_channel_create(multifd_new_send_channel_async, p); + } + + for (i =3D 0; i < thread_count; i++) { + MultiFDSendParams *p =3D &multifd_send_state->params[i]; + Error *local_err =3D NULL; + int ret; + + ret =3D multifd_send_state->ops->send_setup(p, &local_err); + if (ret) { + error_propagate(errp, local_err); + return ret; + } + } + return 0; +} + +struct { + MultiFDRecvParams *params; + /* number of created threads */ + int count; + /* syncs main thread and channels */ + QemuSemaphore sem_sync; + /* global number of generated multifd packets */ + uint64_t packet_num; + /* multifd ops */ + MultiFDMethods *ops; +} *multifd_recv_state; + +static void multifd_recv_terminate_threads(Error *err) +{ + int i; + + trace_multifd_recv_terminate_threads(err !=3D NULL); + + if (err) { + MigrationState *s =3D migrate_get_current(); + migrate_set_error(s, err); + if (s->state =3D=3D MIGRATION_STATUS_SETUP || + s->state =3D=3D MIGRATION_STATUS_ACTIVE) { + migrate_set_state(&s->state, s->state, + MIGRATION_STATUS_FAILED); + } + } + + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + qemu_mutex_lock(&p->mutex); + p->quit =3D true; + /* + * We could arrive here for two reasons: + * - normal quit, i.e. everything went fine, just finished + * - error quit: We close the channels so the channel threads + * finish the qio_channel_read_all_eof() + */ + if (p->c) { + qio_channel_shutdown(p->c, QIO_CHANNEL_SHUTDOWN_BOTH, NULL); + } + qemu_mutex_unlock(&p->mutex); + } +} + +int multifd_load_cleanup(Error **errp) +{ + int i; + + if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { + return 0; + } + multifd_recv_terminate_threads(NULL); + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + if (p->running) { + p->quit =3D true; + /* + * multifd_recv_thread may hung at MULTIFD_FLAG_SYNC handle co= de, + * however try to wakeup it without harm in cleanup phase. + */ + qemu_sem_post(&p->sem_sync); + qemu_thread_join(&p->thread); + } + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + migration_ioc_unregister_yank(p->c); + object_unref(OBJECT(p->c)); + p->c =3D NULL; + qemu_mutex_destroy(&p->mutex); + qemu_sem_destroy(&p->sem_sync); + g_free(p->name); + p->name =3D NULL; + p->packet_len =3D 0; + g_free(p->packet); + p->packet =3D NULL; + g_free(p->iov); + p->iov =3D NULL; + g_free(p->normal); + p->normal =3D NULL; + multifd_recv_state->ops->recv_cleanup(p); + } + qemu_sem_destroy(&multifd_recv_state->sem_sync); + g_free(multifd_recv_state->params); + multifd_recv_state->params =3D NULL; + g_free(multifd_recv_state); + multifd_recv_state =3D NULL; + + return 0; +} + +void multifd_recv_sync_main(void) +{ + int i; + + if (!migrate_use_multifd()) { + return; + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + trace_multifd_recv_sync_main_wait(p->id); + qemu_sem_wait(&multifd_recv_state->sem_sync); + } + for (i =3D 0; i < migrate_multifd_channels(); i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + WITH_QEMU_LOCK_GUARD(&p->mutex) { + if (multifd_recv_state->packet_num < p->packet_num) { + multifd_recv_state->packet_num =3D p->packet_num; + } + } + trace_multifd_recv_sync_main_signal(p->id); + qemu_sem_post(&p->sem_sync); + } + trace_multifd_recv_sync_main(multifd_recv_state->packet_num); +} + +static void *multifd_recv_thread(void *opaque) +{ + MultiFDRecvParams *p =3D opaque; + Error *local_err =3D NULL; + int ret; + + trace_multifd_recv_thread_start(p->id); + rcu_register_thread(); + + while (true) { + uint32_t flags; + + if (p->quit) { + break; + } + + ret =3D qio_channel_read_all_eof(p->c, (void *)p->packet, + p->packet_len, &local_err); + if (ret =3D=3D 0) { /* EOF */ + break; + } + if (ret =3D=3D -1) { /* Error */ + break; + } + + qemu_mutex_lock(&p->mutex); + ret =3D multifd_recv_unfill_packet(p, &local_err); + if (ret) { + qemu_mutex_unlock(&p->mutex); + break; + } + + flags =3D p->flags; + /* recv methods don't know how to handle the SYNC flag */ + p->flags &=3D ~MULTIFD_FLAG_SYNC; + trace_multifd_recv(p->id, p->packet_num, p->normal_num, flags, + p->next_packet_size); + p->num_packets++; + p->total_normal_pages +=3D p->normal_num; + qemu_mutex_unlock(&p->mutex); + + if (p->normal_num) { + ret =3D multifd_recv_state->ops->recv_pages(p, &local_err); + if (ret !=3D 0) { + break; + } + } + + if (flags & MULTIFD_FLAG_SYNC) { + qemu_sem_post(&multifd_recv_state->sem_sync); + qemu_sem_wait(&p->sem_sync); + } + } + + if (local_err) { + multifd_recv_terminate_threads(local_err); + error_free(local_err); + } + qemu_mutex_lock(&p->mutex); + p->running =3D false; + qemu_mutex_unlock(&p->mutex); + + rcu_unregister_thread(); + trace_multifd_recv_thread_end(p->id, p->num_packets, p->total_normal_p= ages); + + return NULL; +} + +int multifd_load_setup(Error **errp) +{ + int thread_count; + uint32_t page_count =3D MULTIFD_PACKET_SIZE / qemu_target_page_size(); + uint8_t i; + + /* + * Return successfully if multiFD recv state is already initialised + * or multiFD is not enabled. + */ + if (multifd_recv_state || !migrate_use_multifd()) { + return 0; + } + + if (!migrate_multi_channels_is_allowed()) { + error_setg(errp, "multifd is not supported by current protocol"); + return -1; + } + thread_count =3D migrate_multifd_channels(); + multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); + multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); + qatomic_set(&multifd_recv_state->count, 0); + qemu_sem_init(&multifd_recv_state->sem_sync, 0); + multifd_recv_state->ops =3D multifd_ops[migrate_multifd_compression()]; + + for (i =3D 0; i < thread_count; i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + + qemu_mutex_init(&p->mutex); + qemu_sem_init(&p->sem_sync, 0); + p->quit =3D false; + p->id =3D i; + p->packet_len =3D sizeof(MultiFDPacket_t) + + sizeof(uint64_t) * page_count; + p->packet =3D g_malloc0(p->packet_len); + p->name =3D g_strdup_printf("multifdrecv_%d", i); + p->iov =3D g_new0(struct iovec, page_count); + p->normal =3D g_new0(ram_addr_t, page_count); + p->page_count =3D page_count; + p->page_size =3D qemu_target_page_size(); + } + + for (i =3D 0; i < thread_count; i++) { + MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; + Error *local_err =3D NULL; + int ret; + + ret =3D multifd_recv_state->ops->recv_setup(p, &local_err); + if (ret) { + error_propagate(errp, local_err); + return ret; + } + } + return 0; +} + +bool multifd_recv_all_channels_created(void) +{ + int thread_count =3D migrate_multifd_channels(); + + if (!migrate_use_multifd()) { + return true; + } + + if (!multifd_recv_state) { + /* Called before any connections created */ + return false; + } + + return thread_count =3D=3D qatomic_read(&multifd_recv_state->count); +} + +/* + * Try to receive all multifd channels to get ready for the migration. + * Sets @errp when failing to receive the current channel. + */ +void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) +{ + MultiFDRecvParams *p; + Error *local_err =3D NULL; + int id; + + id =3D multifd_recv_initial_packet(ioc, &local_err); + if (id < 0) { + multifd_recv_terminate_threads(local_err); + error_propagate_prepend(errp, local_err, + "failed to receive packet" + " via multifd channel %d: ", + qatomic_read(&multifd_recv_state->count)); + return; + } + trace_multifd_recv_new_channel(id); + + p =3D &multifd_recv_state->params[id]; + if (p->c !=3D NULL) { + error_setg(&local_err, "multifd: received id '%d' already setup'", + id); + multifd_recv_terminate_threads(local_err); + error_propagate(errp, local_err); + return; + } + p->c =3D ioc; + object_ref(OBJECT(ioc)); + /* initial packet */ + p->num_packets =3D 1; + + p->running =3D true; + qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, + QEMU_THREAD_JOINABLE); + qatomic_inc(&multifd_recv_state->count); +} --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731856; cv=none; d=zohomail.com; s=zohoarc; b=cs1N9m9e2r/yw2fmNLtzAvdubxhFBt1etkWvqwtvxOfR4QLHYeEjkrMZvNQ8rFaqIAQkWDuOd7MpkxIJZh+s+OXuL4hwIxL/8sNKVKwEEMsJEei2AQVF8orccUyeTtDRCNYrniN9jl0td63rufRC57ZuKmVQEgv73GZluTk4E1U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731856; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=w3fY2GTLs2aZFehMLZMK3yX/B0Z2HIIVrnQhAJjxhrs=; b=NOGcNsrxUTs+YoFXnM6Qk41UuT2JvWcBezh4r8PmB05S5GNkpU96PHpYW8fexGoqOSTLm6Kpx1WBufr4EFxWWcPKXkNd4gov1+PayXGgEHSNK52pG/u7cZMGFGMXigCbXOWhGrugWgqFWz+yxyUrE0nQWkl9pKr9Kfn6BpxMsW4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731856367758.9749818976542; Mon, 6 Feb 2023 17:04:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCL0-0003Oz-PZ; Mon, 06 Feb 2023 20:00:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKi-0001sf-Ro for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:40 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKh-00041z-33 for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:40 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-306-XoZu7-gEOHe_VM8UtJfNiw-1; Mon, 06 Feb 2023 19:59:34 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B7A6B1875041; Tue, 7 Feb 2023 00:59:33 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id E8A65492C3C; Tue, 7 Feb 2023 00:59:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731578; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w3fY2GTLs2aZFehMLZMK3yX/B0Z2HIIVrnQhAJjxhrs=; b=ULuyjnBSZ7IhYAAH80jOV6X6ovv+LRbu3x7ZYz/k5UBTCbndMwbyEEH2pd9eIdboXaVWJa WvCpQVr6jQzQ9MIn+xp2RKg4PzKmB17fi+QI0NKPuE/wsOs6M91S8O2PkchLnIWn4vI/D+ V/qiO+WUjJ1R0nTM+IU+2JbN2EwrCrY= X-MC-Unique: XoZu7-gEOHe_VM8UtJfNiw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Jiang Jiacheng Subject: [PULL 29/30] migration: Introduce interface query-migrationthreads Date: Tue, 7 Feb 2023 01:56:49 +0100 Message-Id: <20230207005650.1810-30-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731857009100001 Content-Type: text/plain; charset="utf-8" From: Jiang Jiacheng Introduce interface query-migrationthreads. The interface is used to query information about migration threads and returns with migration thread's name and its id. Introduce threadinfo.c to manage threads with migration. Signed-off-by: Jiang Jiacheng Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- qapi/migration.json | 29 ++++++++++++++++++++++++ migration/threadinfo.h | 28 +++++++++++++++++++++++ migration/threadinfo.c | 51 ++++++++++++++++++++++++++++++++++++++++++ migration/meson.build | 1 + 4 files changed, 109 insertions(+) create mode 100644 migration/threadinfo.h create mode 100644 migration/threadinfo.c diff --git a/qapi/migration.json b/qapi/migration.json index 88ecf86ac8..c84fa10e86 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1958,6 +1958,35 @@ { 'command': 'query-vcpu-dirty-limit', 'returns': [ 'DirtyLimitInfo' ] } =20 +## +# @MigrationThreadInfo: +# +# Information about migrationthreads +# +# @name: the name of migration thread +# +# @thread-id: ID of the underlying host thread +# +# Since: 7.2 +## +{ 'struct': 'MigrationThreadInfo', + 'data': {'name': 'str', + 'thread-id': 'int'} } + +## +# @query-migrationthreads: +# +# Returns information of migration threads +# +# data: migration thread name +# +# returns: information about migration threads +# +# Since: 7.2 +## +{ 'command': 'query-migrationthreads', + 'returns': ['MigrationThreadInfo'] } + ## # @snapshot-save: # diff --git a/migration/threadinfo.h b/migration/threadinfo.h new file mode 100644 index 0000000000..4d69423c0a --- /dev/null +++ b/migration/threadinfo.h @@ -0,0 +1,28 @@ +/* + * Migration Threads info + * + * Copyright (c) 2022 HUAWEI TECHNOLOGIES CO., LTD. + * + * Authors: + * Jiang Jiacheng + * + * This work is licensed under the terms of the GNU GPL, version 2 or lat= er. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/queue.h" +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qapi/qapi-commands-migration.h" + +typedef struct MigrationThread MigrationThread; + +struct MigrationThread { + const char *name; /* the name of migration thread */ + int thread_id; /* ID of the underlying host thread */ + QLIST_ENTRY(MigrationThread) node; +}; + +MigrationThread *MigrationThreadAdd(const char *name, int thread_id); + +void MigrationThreadDel(MigrationThread *info); diff --git a/migration/threadinfo.c b/migration/threadinfo.c new file mode 100644 index 0000000000..1de8b31855 --- /dev/null +++ b/migration/threadinfo.c @@ -0,0 +1,51 @@ +/* + * Migration Threads info + * + * Copyright (c) 2022 HUAWEI TECHNOLOGIES CO., LTD. + * + * Authors: + * Jiang Jiacheng + * + * This work is licensed under the terms of the GNU GPL, version 2 or lat= er. + * See the COPYING file in the top-level directory. + */ + +#include "threadinfo.h" + +static QLIST_HEAD(, MigrationThread) migration_threads; + +MigrationThread *MigrationThreadAdd(const char *name, int thread_id) +{ + MigrationThread *thread =3D g_new0(MigrationThread, 1); + thread->name =3D name; + thread->thread_id =3D thread_id; + + QLIST_INSERT_HEAD(&migration_threads, thread, node); + + return thread; +} + +void MigrationThreadDel(MigrationThread *thread) +{ + if (thread) { + QLIST_REMOVE(thread, node); + g_free(thread); + } +} + +MigrationThreadInfoList *qmp_query_migrationthreads(Error **errp) +{ + MigrationThreadInfoList *head =3D NULL; + MigrationThreadInfoList **tail =3D &head; + MigrationThread *thread =3D NULL; + + QLIST_FOREACH(thread, &migration_threads, node) { + MigrationThreadInfo *info =3D g_new0(MigrationThreadInfo, 1); + info->name =3D g_strdup(thread->name); + info->thread_id =3D thread->thread_id; + + QAPI_LIST_APPEND(tail, info); + } + + return head; +} diff --git a/migration/meson.build b/migration/meson.build index a9e7e18793..0d1bb9f96e 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -26,6 +26,7 @@ softmmu_ss.add(files( 'savevm.c', 'socket.c', 'tls.c', + 'threadinfo.c', ), gnutls) =20 softmmu_ss.add(when: rdma, if_true: files('rdma.c')) --=20 2.39.1 From nobody Fri May 17 01:44:04 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675731718; cv=none; d=zohomail.com; s=zohoarc; b=D50TapEjXI14uB5RppKD1mPF0xPhoNgSUmUN7tgmtVMCyUaGKOQct2CaRHSB6op1IeItP+Mrdk5RCc2xVAKks76GidlDic676Z/UzEEpRcAqMbggx+7Qq7DG1A9coV2qBWfc/UHBFsJD3pPtN7P+MhLQPRxEjiU0Nc965S21u8Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675731718; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AEZrsqZsmcl/2XNXsBwuDppqhKVGTnKg5/u2lNXA0Nk=; b=ULb+/tWDa12T37rHYxl0D1+qBuOSzsHDWfZa3RhZlqJGwchrdv2k4Kne4XVEPs4cnaixh0Xp6wzx/yUCV0SgQrZbF+jIR/mrLpx8GEwpBqM6WOrPMT+kWeXzRokSn8H4pg0zlym5O1H6YLDBhNXZs++B7MiwrlFtX5VrvO8hQL0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675731718565432.3154134827579; Mon, 6 Feb 2023 17:01:58 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pPCLU-0003mz-2D; Mon, 06 Feb 2023 20:00:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKy-0003QH-LB for qemu-devel@nongnu.org; Mon, 06 Feb 2023 20:00:00 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pPCKn-00044M-KA for qemu-devel@nongnu.org; Mon, 06 Feb 2023 19:59:56 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-445-IiZIEu_7NP-yOeYctboWbw-1; Mon, 06 Feb 2023 19:59:40 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C70AB185A794; Tue, 7 Feb 2023 00:59:38 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0EBAA492C3C; Tue, 7 Feb 2023 00:59:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675731585; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AEZrsqZsmcl/2XNXsBwuDppqhKVGTnKg5/u2lNXA0Nk=; b=XhLA2dsXaob2GqHSaHkEg0XIhEcSl8GsHy97teU0UZWYHkON8rF+jJhG/lvSgGPt0hmP6K GXB/ca3ar18ER7bbcl98kq8gJwU+gv0g0GA5FfD7VPdanCzoVMpjpHlS/bJmOALO/7z0bG PlDEDWzEeDen6DSoAC5KaBUThqaUC2s= X-MC-Unique: IiZIEu_7NP-yOeYctboWbw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Stefan Berger , Stefan Hajnoczi , Halil Pasic , John Snow , David Hildenbrand , Fam Zheng , Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laurent Vivier , Vladimir Sementsov-Ogievskiy , qemu-s390x@nongnu.org, Christian Borntraeger , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Coiby Xu , Ilya Leoshkevich , Eduardo Habkost , Yanan Wang , Richard Henderson , Markus Armbruster , Paolo Bonzini , Alex Williamson , Eric Blake , Eric Farman , Jiang Jiacheng Subject: [PULL 30/30] migration: save/delete migration thread info Date: Tue, 7 Feb 2023 01:56:50 +0100 Message-Id: <20230207005650.1810-31-quintela@redhat.com> In-Reply-To: <20230207005650.1810-1-quintela@redhat.com> References: <20230207005650.1810-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675731720027100004 Content-Type: text/plain; charset="utf-8" From: Jiang Jiacheng To support query migration thread infomation, save and delete thread(live_migration and multifdsend) information at thread creation and finish. Signed-off-by: Jiang Jiacheng Signed-off-by: Juan Quintela --- migration/migration.c | 5 +++++ migration/multifd.c | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 66c74f8e17..7a14aa98d8 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -58,6 +58,7 @@ #include "net/announce.h" #include "qemu/queue.h" #include "multifd.h" +#include "threadinfo.h" #include "qemu/yank.h" #include "sysemu/cpus.h" #include "yank_functions.h" @@ -4028,10 +4029,13 @@ static void qemu_savevm_wait_unplug(MigrationState = *s, int old_state, static void *migration_thread(void *opaque) { MigrationState *s =3D opaque; + MigrationThread *thread =3D NULL; int64_t setup_start =3D qemu_clock_get_ms(QEMU_CLOCK_HOST); MigThrError thr_error; bool urgent =3D false; =20 + thread =3D MigrationThreadAdd("live_migration", qemu_get_thread_id()); + rcu_register_thread(); =20 object_ref(OBJECT(s)); @@ -4108,6 +4112,7 @@ static void *migration_thread(void *opaque) migration_iteration_finish(s); object_unref(OBJECT(s)); rcu_unregister_thread(); + MigrationThreadDel(thread); return NULL; } =20 diff --git a/migration/multifd.c b/migration/multifd.c index 437bf6f808..b7ad7002e0 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -24,6 +24,7 @@ #include "qemu-file.h" #include "trace.h" #include "multifd.h" +#include "threadinfo.h" =20 #include "qemu/yank.h" #include "io/channel-socket.h" @@ -649,10 +650,13 @@ int multifd_send_sync_main(QEMUFile *f) static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; + MigrationThread *thread =3D NULL; Error *local_err =3D NULL; int ret =3D 0; bool use_zero_copy_send =3D migrate_use_zero_copy_send(); =20 + thread =3D MigrationThreadAdd(p->name, qemu_get_thread_id()); + trace_multifd_send_thread_start(p->id); rcu_register_thread(); =20 @@ -762,6 +766,7 @@ out: qemu_mutex_unlock(&p->mutex); =20 rcu_unregister_thread(); + MigrationThreadDel(thread); trace_multifd_send_thread_end(p->id, p->num_packets, p->total_normal_p= ages); =20 return NULL; --=20 2.39.1