From nobody Wed Feb 11 04:20:00 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1685469732; cv=none; d=zohomail.com; s=zohoarc; b=CpxnKB4UWR0kccjJoaWH+8oo68Q424GZCXyPL6Djrh6J31ug/qLaJuncEyLicuRpsPQrmU8lllgkoZYDuEe4Hbt5iThoxzxN95tORyPxNwkFCGwaVqOvh+yjVQSF+f2C0VuLRgo/lrOWDwP5qE5voKnYQ9pqcQTQkWdgo6seR0c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1685469732; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hHvviXGKXz7/JvAkgQz3l7QwnqzLJhzckRGwXC/3Pmg=; b=AFjoj5jyLLJtV6aMbOKWsbarWT+/aBik7QecC8eF364UyfVc9JzK6zuoNOmWEbP+nK1R90klJF4wjATetXAEM1BXo6M8egzr4/NN/50stKEOWnmftkHlePij1j35lrtowi9k7ojKfvneDTYtU9rQS3XEzscTQgaMrAADwttXCvc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1685469732577187.02540471177088; Tue, 30 May 2023 11:02:12 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43eW-00026e-8y; Tue, 30 May 2023 14:01:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eU-0001vs-JG for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:58 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eS-0002jh-4K for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:58 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE54jf020902; Tue, 30 May 2023 18:00:51 GMT Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhb93gfv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:51 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHfg5K014618; Tue, 30 May 2023 18:00:50 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48v34-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:50 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspZ011459; Tue, 30 May 2023 18:00:50 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-13; Tue, 30 May 2023 18:00:49 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=hHvviXGKXz7/JvAkgQz3l7QwnqzLJhzckRGwXC/3Pmg=; b=YzOkA8/I0hn/bNMv1kb/UMi7g1Jf/+Q/FsQEgqXgGI2qupz9AKeg3WyApkg3dSrv6O/x 9NrWPhnRAEQb65LovxLXYm+Ie6xGnm4Yb/44urH13+x6mSJ1rNj0fCp65av5laWTcIX+ 5W83vpepuPB4J3/HnP9LygoGJVCq0yVUqoIalCDHWsTWym84ckAmClFCbpUt1KjetPjX 8NzPYzoM2dQ2DmlZr7LirvjJ2QSwS3jdqP3tz8PYPs4pit/JklXuxJmyBdup4ZAWVwYX VcvsmcIZIYrC+WEA6eXFGc4aqT1E4MXc85ln3UPEd1sDd4u6B0MZsNBS9wqu4IBiG5rL dw== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 12/15] vfio/common: Support device dirty page tracking with vIOMMU Date: Tue, 30 May 2023 18:59:34 +0100 Message-Id: <20230530175937.24202-13-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-GUID: gvG2qt7Ghnse6XfhqvrkWZ4taSyibWv2 X-Proofpoint-ORIG-GUID: gvG2qt7Ghnse6XfhqvrkWZ4taSyibWv2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1685469734655100003 Content-Type: text/plain; charset="utf-8" Currently, device dirty page tracking with vIOMMU is not supported, and a blocker is added and the migration is prevented. When vIOMMU is used, IOVA ranges are DMA mapped/unmapped on the fly as requesting by the vIOMMU. These IOVA ranges can potentially be mapped anywhere in the vIOMMU IOVA space as advertised by the VMM. To support device dirty tracking when vIOMMU enabled instead create the dirty ranges based on the vIOMMU provided limits, which leads to the tracking of the whole IOVA space regardless of what devices use. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 64 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 54 insertions(+), 10 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index b8d97577f856..d2897aceedae 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -447,6 +447,31 @@ static bool vfio_viommu_preset(void) return false; } =20 +static int vfio_viommu_get_max_iova(hwaddr *max_iova) +{ + VFIOAddressSpace *space; + int ret =3D -EINVAL; + + QLIST_FOREACH(space, &vfio_address_spaces, list) { + if (space->as =3D=3D &address_space_memory) { + continue; + } + + if (!space->iommu_mr) { + break; + } + + ret =3D memory_region_iommu_get_attr(space->iommu_mr, + IOMMU_ATTR_MAX_IOVA, + max_iova); + if (ret) { + break; + } + } + + return ret; +} + int vfio_block_giommu_migration(Error **errp) { int ret; @@ -1483,10 +1508,11 @@ static const MemoryListener vfio_dirty_tracking_lis= tener =3D { .region_add =3D vfio_listener_dirty_tracking_update, }; =20 -static void vfio_dirty_tracking_init(VFIOContainer *container, +static int vfio_dirty_tracking_init(VFIOContainer *container, VFIODirtyRanges *ranges) { VFIODirtyRangesListener dirty; + int ret; =20 memset(&dirty, 0, sizeof(dirty)); dirty.ranges.min32 =3D UINT32_MAX; @@ -1494,17 +1520,29 @@ static void vfio_dirty_tracking_init(VFIOContainer = *container, dirty.listener =3D vfio_dirty_tracking_listener; dirty.container =3D container; =20 - memory_listener_register(&dirty.listener, - container->space->as); + if (vfio_viommu_preset()) { + hwaddr iommu_max_iova; + + ret =3D vfio_viommu_get_max_iova(&iommu_max_iova); + if (ret) { + return ret; + } + + vfio_dirty_tracking_update(0, iommu_max_iova, &dirty.ranges); + } else { + memory_listener_register(&dirty.listener, + container->space->as); + /* + * The memory listener is synchronous, and used to calculate the r= ange + * to dirty tracking. Unregister it after we are done as we are not + * interested in any follow-up updates. + */ + memory_listener_unregister(&dirty.listener); + } =20 *ranges =3D dirty.ranges; =20 - /* - * The memory listener is synchronous, and used to calculate the range - * to dirty tracking. Unregister it after we are done as we are not - * interested in any follow-up updates. - */ - memory_listener_unregister(&dirty.listener); + return 0; } =20 static void vfio_devices_dma_logging_stop(VFIOContainer *container) @@ -1609,7 +1647,13 @@ static int vfio_devices_dma_logging_start(VFIOContai= ner *container) VFIOGroup *group; int ret =3D 0; =20 - vfio_dirty_tracking_init(container, &ranges); + ret =3D vfio_dirty_tracking_init(container, &ranges); + if (ret) { + error_report("Failed to init DMA logging ranges, err %d", + ret); + return -EOPNOTSUPP; + } + feature =3D vfio_device_feature_dma_logging_start_create(container, &ranges); if (!feature) { --=20 2.39.3