From nobody Fri Nov 14 16:56:48 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1762402931; cv=none; d=zohomail.com; s=zohoarc; b=djNeu7xYdAGptqbt0jg5ihOuebcD5ka+HGk+1GcCa1DkuGtz6OKMIRJkCSEFNlgc0rQTZRN0p5z75nTiywx4vVSnwnMXOeO8vxvzPmpTxb9Bh4tU5etMCZzPdZsnmYl20UHrLX/yFzhIBNpW0YiSps+5kSq+sVfHvG3guSWz8+8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1762402931; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=iMoS62TTWtfNbXchXBkVWsgElMw7jkRAfmWhsZa6vm4=; b=XyjoUQ/ftq4bOXk7tWmZVgBay/jp6ji4xomVSCTDkDwnmjZmtL//OaqcUj8mavBcucZtfN+CC6EDxN6qmWA2vbq5P8VPECy305D9NE9I5V3Bww9vTA2rZX9BopYQvPKLPLIJf8dQHZtBvZ0+yHxmoEsQ/QgtPIRx7K8Zmr4ASD0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 176240293123772.33632990637079; Wed, 5 Nov 2025 20:22:11 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vGrUx-0006ku-2K; Wed, 05 Nov 2025 23:21:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vGrUs-0006k0-HO for qemu-devel@nongnu.org; Wed, 05 Nov 2025 23:21:18 -0500 Received: from mgamail.intel.com ([198.175.65.19]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vGrUo-0000q1-6k for qemu-devel@nongnu.org; Wed, 05 Nov 2025 23:21:15 -0500 Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2025 20:21:13 -0800 Received: from unknown (HELO gnr-sp-2s-612.sh.intel.com) ([10.112.230.229]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Nov 2025 20:21:10 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1762402874; x=1793938874; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=A7i/AJqHEJyiDSM7PjscbOebyDU3VW9QraGVFc93DXs=; b=Kbl2zzeQEppP4WNA6tPcJ091AB9gI+Zhyd9a/nwMf85QhP/Xbd+awYKX uv5cwLeQD7o8CMHVlUKU9Pl+cjEpgbOd4pXXKvmMj1KNA4xPcm5/pzbzu RUqU2YT4ynaSboM/wR1mS1KyZoz6YOjG/h6p7vwi6+ZCnk/HWYEhNiD8n sM0fKwoRNtl8LGgKDZlAiDbL4aBZlwPcgt4MVIU5jXH9uZ0qLkhsYRuFh zPmopQGQOF+3E61ikW/YT74ZChltR4DB3qapvmYEj6fq95TiN77YpUw0k 2m7X7VBX6F/x2LOUdqnpUhzAEZsqGWHu3wwDpzJT3qRcpGgIuuWUVI5aa g==; X-CSE-ConnectionGUID: qcoxqqDuTz60fNj99H1EfQ== X-CSE-MsgGUID: hZ/Q7SlKR4irvENKVIxELQ== X-IronPort-AV: E=McAfee;i="6800,10657,11604"; a="64412871" X-IronPort-AV: E=Sophos;i="6.19,283,1754982000"; d="scan'208";a="64412871" X-CSE-ConnectionGUID: 0+Xa7k7pRNGrtwvF8I8a7A== X-CSE-MsgGUID: XPhaYn1MQOqPbXd05qUJwA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.19,283,1754982000"; d="scan'208";a="186901300" From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex@shazbot.org, clg@redhat.com, mst@redhat.com, jasowang@redhat.com, yi.l.liu@intel.com, clement.mathieu--drif@eviden.com, eric.auger@redhat.com, joao.m.martins@oracle.com, avihaih@nvidia.com, xudong.hao@intel.com, giovanni.cabiddu@intel.com, rohith.s.r@intel.com, mark.gross@intel.com, arjan.van.de.ven@intel.com, Zhenzhong Duan Subject: [PATCH v5 8/9] vfio/migration: Add migration blocker if VM memory is too large to cause unmap_bitmap failure Date: Wed, 5 Nov 2025 23:20:25 -0500 Message-ID: <20251106042027.856594-9-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20251106042027.856594-1-zhenzhong.duan@intel.com> References: <20251106042027.856594-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=198.175.65.19; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -48 X-Spam_score: -4.9 X-Spam_bar: ---- X-Spam_report: (-4.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.517, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1762402933271158500 Content-Type: text/plain; charset="utf-8" With default config, kernel VFIO IOMMU type1 driver limits dirty bitmap to 256MB for unmap_bitmap ioctl so the maximum guest memory region is no more than 8TB size for the ioctl to succeed. Be conservative here to limit total guest memory to max value supported by unmap_bitmap ioctl or else add a migration blocker. IOMMUFD backend doesn't have such limit, one can use it if there is a need to migrate such large VM. Suggested-by: Yi Liu Signed-off-by: Zhenzhong Duan --- hw/vfio/migration.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 4c06e3db93..86e5b7ab55 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -16,6 +16,7 @@ #include =20 #include "system/runstate.h" +#include "hw/boards.h" #include "hw/vfio/vfio-device.h" #include "hw/vfio/vfio-migration.h" #include "migration/misc.h" @@ -1152,6 +1153,32 @@ static bool vfio_viommu_preset(VFIODevice *vbasedev) return vbasedev->bcontainer->space->as !=3D &address_space_memory; } =20 +static bool vfio_dirty_tracking_exceed_limit(VFIODevice *vbasedev) +{ + VFIOContainer *bcontainer =3D vbasedev->bcontainer; + uint64_t max_size, page_size; + + if (!bcontainer->dirty_pages_supported) { + return false; + } + + /* + * VFIO IOMMU type1 driver has limitation of bitmap size on unmap_bitm= ap + * ioctl(), calculate the limit and compare with guest memory size to + * catch dirty tracking failure early. + * + * This limit is 8TB with default kernel and QEMU config, we are a bit + * conservative here as VM memory layout may be nonconsecutive or VM + * can run with vIOMMU enabled so the limitation could be relaxed. One + * can also switch to use IOMMUFD backend if there is a need to migrate + * large VM. + */ + page_size =3D 1 << ctz64(bcontainer->dirty_pgsizes); + max_size =3D bcontainer->max_dirty_bitmap_size * BITS_PER_BYTE * page_= size; + + return current_machine->ram_size > max_size; +} + /* * Return true when either migration initialized or blocker registered. * Currently only return false when adding blocker fails which will @@ -1193,6 +1220,13 @@ bool vfio_migration_realize(VFIODevice *vbasedev, Er= ror **errp) goto add_blocker; } =20 + if (vfio_dirty_tracking_exceed_limit(vbasedev)) { + error_setg(&err, "%s: Migration is currently not supported wit= h " + "large memory VM due to dirty tracking limitation i= n " + "backend", vbasedev->name); + goto add_blocker; + } + warn_report("%s: VFIO device doesn't support device and " "IOMMU dirty tracking", vbasedev->name); } --=20 2.47.1