From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894309; cv=none; d=zohomail.com; s=zohoarc; b=Gz8dFpKjsTAFjkMaiDwnuabTeeiqXu1rcwN1OvaB7ZCeLYmbFY1E+GhrLNYERFNOCxGf0IW3xfes1S45nbiEg3hUNqWXfqSojmx86Cd5mnk/H+Z06ztSSBny9PJMkKpzEGMD1ISNniDyPB8i+JBAifdGisVsdSDOp5u//vucRZM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894309; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ph7NhPNJqQFVQZvkebi126vtqIT7j0fmOZoCxa+ciPc=; b=JUASWMY2qF6CithLcA2duV6wPly+HcoR7bNv2rZRVlrNnhZkMegoQOUQ9dVR3KApTDQ3lxf79x84lezc0hUPWyp8S9Mj46wbRdY864J8XMDMwqzZca1TQz0B+nywksbsw/W2L0ButIf5kJNLDazP+JDCeS7ZhQ0iv9Jes2LvArY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894309077806.6505719682192; Fri, 3 Mar 2023 17:45:09 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGwt-0006xh-Lz; Fri, 03 Mar 2023 20:44:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGws-0006xQ-Rn for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:34 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGwq-0005eu-S0 for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:34 -0500 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 323Ndqic004251; Sat, 4 Mar 2023 01:44:31 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3nybb2rqwj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:31 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241XvUC015714; Sat, 4 Mar 2023 01:44:30 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-2; Sat, 04 Mar 2023 01:44:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=ph7NhPNJqQFVQZvkebi126vtqIT7j0fmOZoCxa+ciPc=; b=jNR2qGjF7KXnduYy0uV6PdPsaJ4Lj6S9qxfY7z0ogMoFtnNJQm1CBX4R99zqFDFVpVr+ 9yH8qo9OhkYKjPd1YfjEa3T9752jLL186ppnar/EIp3/MvPe/8hdOxWNWzsao2t3fGql 8ys36wnDjwaxPQ+bMWzLU45lo+v/sOtGNK6WsFr+zroYJJoTcY+nA1vPViZNeLvjZ601 sCEbn8ml0GrUUW5s4soHyPTh7lqtjo7TU6/kVRKXiPrplbzeVTiUQtuG1p/lyeOWpL0p 0f7bBj6sQowG3zB5ZcQZ+zH9tTcQKeGgEVCMeKoeMmm037/X2C72gdhbC7L+TQDQ9CIQ gg== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v3 01/13] vfio/common: Fix error reporting in vfio_get_dirty_bitmap() Date: Sat, 4 Mar 2023 01:43:31 +0000 Message-Id: <20230304014343.33646-2-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=770 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: aYhyEpiBN1uf39joTya1NZdk5DzhdjCM X-Proofpoint-ORIG-GUID: aYhyEpiBN1uf39joTya1NZdk5DzhdjCM Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894310807100001 From: Avihai Horon Return -errno instead of -1 if VFIO_IOMMU_DIRTY_PAGES ioctl fails in vfio_get_dirty_bitmap(). Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index bab83c0e55cb..9fc305448fa2 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1335,6 +1335,7 @@ static int vfio_get_dirty_bitmap(VFIOContainer *conta= iner, uint64_t iova, =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { + ret =3D -errno; error_report("Failed to get dirty bitmap for iova: 0x%"PRIx64 " size: 0x%"PRIx64" err: %d", (uint64_t)range->iova, (uint64_t)range->size, errno); --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894400; cv=none; d=zohomail.com; s=zohoarc; b=lR8wN2NC1gvNjYaB+Oori9yQfplo90uGegvUYz3BxmSCgZIPhVjpqwPtzx02FkuzCiLF30+QBO78aCAAibNxQrtbpTtfIRZcaVhIVVwoAteYXaOLN9IwGOD1+CBVtSWflBwLQBkTgwbzHMcjWnt/FvaXLIdiPuxOyQ2GFt6q4sw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894400; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xk8wvTPv6dJhnVr411G6qxujVLuqM/QjT1VPuR/+yew=; b=IPqwF5gH65yDF6OT3GZbAowmjdWpkcaXejpq3bYnYEWb2kjzJ7YWlDnppfjSCJgzGfpLUQ8DQoiufY7sxIFfoZ+nIQ5pfTAo0D7a2OoLlN6UpEX6xbXUjTPUYNnb23FrHn7iqzInfiwyOWGa55RrnRfN5ctSyveEdmoJGyw9bsI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894400541703.9757130228919; Fri, 3 Mar 2023 17:46:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGwx-0006yS-B4; Fri, 03 Mar 2023 20:44:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGwv-0006yE-W7 for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:38 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGwu-0005fV-8I for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:37 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3241iYH8017574; Sat, 4 Mar 2023 01:44:34 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p3vk1801e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:34 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241XvUD015714; Sat, 4 Mar 2023 01:44:33 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-3; Sat, 04 Mar 2023 01:44:32 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=xk8wvTPv6dJhnVr411G6qxujVLuqM/QjT1VPuR/+yew=; b=H8q2xyYhh1JdLVDG4KeL5bY9/RS59e5o0ymKVL6pob37ccVA0H1I91oDllRbW3QJ39FO hrOOnFC00PR0ryCzrPKugsFLB1l0otb8RJowD6SCcfaEYE4mjaIFdrLWEDLzU1vmTYo0 UXdT0BZNBaZwclRoG29PKptCVhvd/jkt9227R6ASrAt9EoZ33bURjfvULOs4ejbACDLG GVgL51388llZ6bSQMwj4VeHRYCDThtdutGUSwrra2H19WEbkihZnM/IiDQn7dsKfej3L HIefHgS46fEjmue0cedLtuIfBAFqZuO4Yg7wLmnKi25MC5sTJiskNdqBwJoLabpBFHm9 tA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v3 02/13] vfio/common: Fix wrong %m usages Date: Sat, 4 Mar 2023 01:43:32 +0000 Message-Id: <20230304014343.33646-3-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=813 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: HlJGIx6ft1MhsLj0_xvTM9uhGx2YrJpe X-Proofpoint-ORIG-GUID: HlJGIx6ft1MhsLj0_xvTM9uhGx2YrJpe Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894402376100003 From: Avihai Horon There are several places where the %m conversion is used if one of vfio_dma_map(), vfio_dma_unmap() or vfio_get_dirty_bitmap() fail. The %m usage in these places is wrong since %m relies on errno value while the above functions don't report errors via errno. Fix it by using strerror() with the returned value instead. Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 9fc305448fa2..4d26e9cccf91 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -703,17 +703,17 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, I= OMMUTLBEntry *iotlb) read_only); if (ret) { error_report("vfio_dma_map(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx", %p) =3D %d (%m)", + "0x%"HWADDR_PRIx", %p) =3D %d (%s)", container, iova, - iotlb->addr_mask + 1, vaddr, ret); + iotlb->addr_mask + 1, vaddr, ret, strerror(-ret)); } } else { ret =3D vfio_dma_unmap(container, iova, iotlb->addr_mask + 1, iotl= b); if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") =3D %d (%m)", + "0x%"HWADDR_PRIx") =3D %d (%s)", container, iova, - iotlb->addr_mask + 1, ret); + iotlb->addr_mask + 1, ret, strerror(-ret)); } } out: @@ -1095,8 +1095,9 @@ static void vfio_listener_region_add(MemoryListener *= listener, vaddr, section->readonly); if (ret) { error_setg(&err, "vfio_dma_map(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx", %p) =3D %d (%m)", - container, iova, int128_get64(llsize), vaddr, ret); + "0x%"HWADDR_PRIx", %p) =3D %d (%s)", + container, iova, int128_get64(llsize), vaddr, ret, + strerror(-ret)); if (memory_region_is_ram_device(section->mr)) { /* Allow unexpected mappings not to be fatal for RAM devices */ error_report_err(err); @@ -1228,16 +1229,18 @@ static void vfio_listener_region_del(MemoryListener= *listener, ret =3D vfio_dma_unmap(container, iova, int128_get64(llsize), = NULL); if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") =3D %d (%m)", - container, iova, int128_get64(llsize), ret); + "0x%"HWADDR_PRIx") =3D %d (%s)", + container, iova, int128_get64(llsize), ret, + strerror(-ret)); } iova +=3D int128_get64(llsize); } ret =3D vfio_dma_unmap(container, iova, int128_get64(llsize), NULL= ); if (ret) { error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") =3D %d (%m)", - container, iova, int128_get64(llsize), ret); + "0x%"HWADDR_PRIx") =3D %d (%s)", + container, iova, int128_get64(llsize), ret, + strerror(-ret)); } } =20 @@ -1384,9 +1387,9 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifier= *n, IOMMUTLBEntry *iotlb) translated_addr); if (ret) { error_report("vfio_iommu_map_dirty_notify(%p, 0x%"HWADDR_PRIx"= , " - "0x%"HWADDR_PRIx") =3D %d (%m)", - container, iova, - iotlb->addr_mask + 1, ret); + "0x%"HWADDR_PRIx") =3D %d (%s)", + container, iova, iotlb->addr_mask + 1, ret, + strerror(-ret)); } } rcu_read_unlock(); --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894309; cv=none; d=zohomail.com; s=zohoarc; b=QwrZZbt94wif5IMFHLUVyjjFYhXYGsEw03TiO2KxfupnBr0pTqIaHw+aMM9qqmko6MDmafTK3+4McbiBUmLTfZnWc/HvyZBkC9y6bDoZB8M0lVmChEGRQTUFu8HZwgYONSbp/mVchhyGgGC0XIj44CQYGrEALPpu2BeyxgSjQ7o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894309; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GiBCAPHsWsCaxFjIYhTKruwe4Yu7rgg+z8yv2dJvMcY=; b=nMImU6dSnJH8QeXbsFtbquPHHU3BGxuE04ehW3+c0Jsh1qhsgK+1RSVaNLgS+hUm04W4lMs8gT6MjAy6fqc4WLSapbM6unjZrUgmGmpK0fAcr9aE15EFH5psGGviC38Ehz82TZm642aTnD12nE4gar1hHWk4u6BAShLibKu0bUA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894309031287.8034874181802; Fri, 3 Mar 2023 17:45:09 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGx0-0006z9-V4; Fri, 03 Mar 2023 20:44:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGwy-0006yj-Fy for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:40 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGww-0005fn-Lh for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:40 -0500 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 323NU0sU023874; Sat, 4 Mar 2023 01:44:37 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3nyba7rhbm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:37 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241XvUE015714; Sat, 4 Mar 2023 01:44:35 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-4; Sat, 04 Mar 2023 01:44:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2022-7-12; bh=GiBCAPHsWsCaxFjIYhTKruwe4Yu7rgg+z8yv2dJvMcY=; b=zNMgiy1KfC3ib9iFL/jFg7I4/IY6zKMV/WKy3Vv3myAdPba7HiaGZkTcpBKlQct4QPUG pDVGcH4rcG1Haop+HzW82bS589sHZejWc0+2sJIbo5pO5/z+MWp4uCbQodkXShRQKy62 BdrzoW6yi2w8I7ZPb6LALMGGM1oNMz0CvghQZ8MW/TnN/9icz2yN+WkS45YuAZM8Vi5l qnHM+nMkCgO5m1AurxufVv1nxcXWrgTCYqUy51H5ZhlWs346qlI/DZ/H0czIGB5FgYU8 hyaOHt+9p4mI1YFfFz4G8Lhdx1jwJ7r6fHkL39nDNOpYp1DF2usloSGel6Fhe8KsUIo+ OQ== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon Subject: [PATCH v3 03/13] vfio/common: Abort migration if dirty log start/stop/sync fails Date: Sat, 4 Mar 2023 01:43:33 +0000 Message-Id: <20230304014343.33646-4-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-ORIG-GUID: HJTWvlV2Oe5jlCdtggy85It7sUnNFjAP X-Proofpoint-GUID: HJTWvlV2Oe5jlCdtggy85It7sUnNFjAP Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894310836100003 From: Avihai Horon If VFIO dirty pages log start/stop/sync fails during migration, migration should be aborted as pages dirtied by VFIO devices might not be reported properly. This is not the case today, where in such scenario only an error is printed. Fix it by aborting migration in the above scenario. Fixes: 758b96b61d5c ("vfio/migrate: Move switch of dirty tracking into vfio= _memory_listener") Fixes: b6dd6504e303 ("vfio: Add vfio_listener_log_sync to mark dirty pages") Fixes: 9e7b0442f23a ("vfio: Add ioctl to get dirty pages bitmap during dma = unmap") Signed-off-by: Avihai Horon Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 53 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 45 insertions(+), 8 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 4d26e9cccf91..4c801513136a 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -42,6 +42,7 @@ #include "migration/migration.h" #include "migration/misc.h" #include "migration/blocker.h" +#include "migration/qemu-file.h" #include "sysemu/tpm.h" =20 VFIOGroupList vfio_group_list =3D @@ -390,6 +391,19 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker =3D NULL; } =20 +static void vfio_set_migration_error(int err) +{ + MigrationState *ms =3D migrate_get_current(); + + if (migration_is_setup_or_active(ms->state)) { + WITH_QEMU_LOCK_GUARD(&ms->qemu_file_lock) { + if (ms->to_dst_file) { + qemu_file_set_error(ms->to_dst_file, err); + } + } + } +} + static bool vfio_devices_all_dirty_tracking(VFIOContainer *container) { VFIOGroup *group; @@ -680,6 +694,7 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOM= MUTLBEntry *iotlb) if (iotlb->target_as !=3D &address_space_memory) { error_report("Wrong target AS \"%s\", only system memory is allowe= d", iotlb->target_as->name ? iotlb->target_as->name : "no= ne"); + vfio_set_migration_error(-EINVAL); return; } =20 @@ -714,6 +729,7 @@ static void vfio_iommu_map_notify(IOMMUNotifier *n, IOM= MUTLBEntry *iotlb) "0x%"HWADDR_PRIx") =3D %d (%s)", container, iova, iotlb->addr_mask + 1, ret, strerror(-ret)); + vfio_set_migration_error(ret); } } out: @@ -1259,7 +1275,7 @@ static void vfio_listener_region_del(MemoryListener *= listener, } } =20 -static void vfio_set_dirty_page_tracking(VFIOContainer *container, bool st= art) +static int vfio_set_dirty_page_tracking(VFIOContainer *container, bool sta= rt) { int ret; struct vfio_iommu_type1_dirty_bitmap dirty =3D { @@ -1267,7 +1283,7 @@ static void vfio_set_dirty_page_tracking(VFIOContaine= r *container, bool start) }; =20 if (!container->dirty_pages_supported) { - return; + return 0; } =20 if (start) { @@ -1278,23 +1294,34 @@ static void vfio_set_dirty_page_tracking(VFIOContai= ner *container, bool start) =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, &dirty); if (ret) { + ret =3D -errno; error_report("Failed to set dirty tracking flag 0x%x errno: %d", dirty.flags, errno); } + + return ret; } =20 static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + int ret; =20 - vfio_set_dirty_page_tracking(container, true); + ret =3D vfio_set_dirty_page_tracking(container, true); + if (ret) { + vfio_set_migration_error(ret); + } } =20 static void vfio_listener_log_global_stop(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + int ret; =20 - vfio_set_dirty_page_tracking(container, false); + ret =3D vfio_set_dirty_page_tracking(container, false); + if (ret) { + vfio_set_migration_error(ret); + } } =20 static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, @@ -1370,19 +1397,18 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifi= er *n, IOMMUTLBEntry *iotlb) VFIOContainer *container =3D giommu->container; hwaddr iova =3D iotlb->iova + giommu->iommu_offset; ram_addr_t translated_addr; + int ret =3D -EINVAL; =20 trace_vfio_iommu_map_dirty_notify(iova, iova + iotlb->addr_mask); =20 if (iotlb->target_as !=3D &address_space_memory) { error_report("Wrong target AS \"%s\", only system memory is allowe= d", iotlb->target_as->name ? iotlb->target_as->name : "no= ne"); - return; + goto out; } =20 rcu_read_lock(); if (vfio_get_xlat_addr(iotlb, NULL, &translated_addr, NULL)) { - int ret; - ret =3D vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + = 1, translated_addr); if (ret) { @@ -1393,6 +1419,11 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifie= r *n, IOMMUTLBEntry *iotlb) } } rcu_read_unlock(); + +out: + if (ret) { + vfio_set_migration_error(ret); + } } =20 static int vfio_ram_discard_get_dirty_bitmap(MemoryRegionSection *section, @@ -1485,13 +1516,19 @@ static void vfio_listener_log_sync(MemoryListener *= listener, MemoryRegionSection *section) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + int ret; =20 if (vfio_listener_skipped_section(section)) { return; } =20 if (vfio_devices_all_dirty_tracking(container)) { - vfio_sync_dirty_bitmap(container, section); + ret =3D vfio_sync_dirty_bitmap(container, section); + if (ret) { + error_report("vfio: Failed to sync dirty bitmap, err: %d (%s)"= , ret, + strerror(-ret)); + vfio_set_migration_error(ret); + } } } =20 --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894422; cv=none; d=zohomail.com; s=zohoarc; b=ib/jga8pcThSvMiyeroPBYInq/H0yjS4l9xPjhghFRmtwIl8zRuZf1pdenlnB8JN8nGDU4XbqYWzxKyM5SPrWfQgsm1GnBlmdfS92yjDUChVCi8a9ylKdVspBZCFb1RvYyy5XtDWkkRdtnGjawEBIiSej9/GXlczrlr4I4MzU4U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894422; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=v3pY00oNfyn9nEoxJHcnF/C+T2Maa7YaItHfdgTxWhk=; b=f/y28lj1cD6REQahgj9P3f+/wxMewtF4p+/QBVCyvHRzj/04oCRWYFlbv7WDDTw6DTI3KJGJfgWdmXp0YZkg3QRQ8isimjMgxZNavv99fkZz6FN4voUwdNWuuD7GRYggw15yV2NaLthEcA4OtNmCMxmlrgmf8WyrUOIcCU4WfD8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 167789442211375.71618504937601; Fri, 3 Mar 2023 17:47:02 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGx3-0006zc-El; Fri, 03 Mar 2023 20:44:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGx2-0006zE-5K for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:44 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGx0-0005gK-8P for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:43 -0500 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 323MvSUd004323; Sat, 4 Mar 2023 01:44:41 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3nybb2rqwx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:40 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241XqOb015662; Sat, 4 Mar 2023 01:44:39 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g79t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:39 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0f016769; Sat, 4 Mar 2023 01:44:38 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-5; Sat, 04 Mar 2023 01:44:38 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=v3pY00oNfyn9nEoxJHcnF/C+T2Maa7YaItHfdgTxWhk=; b=PiHJwQ54kfP+v2NigsrVn8e07gW2sFwC/t2teRKoVcOMWFAs1vpgx7LO77rTCzwfPceU u0pKD83h3kOaXz6/+mgRCHokVCoH0gmLLSDn3EF6lePhwRsIeIIlLUXQm5LYqckTHjtL ubjkKHGFnfHvhELSXdBT3+ZEcz5IVcp+gWzIeB7OtWMjogiOMHYemaeY0F92RPz7bO9F VbUQzepy/Gi3/Ys35L6MOH2yd0xb+3i2bNnDYIpuM8lH1pOHs1OBG/cupDlruKVe0v9Q YKpjdqi9b5SEfhKTxfbr2ht/z4XZsrk2tHFMMilfJFbDMmc+mK8xlh7eZYI4cdIxxFmu CA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v3 04/13] vfio/common: Add VFIOBitmap and alloc function Date: Sat, 4 Mar 2023 01:43:34 +0000 Message-Id: <20230304014343.33646-5-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: VNbyLRA_siwXiuTaQql6kjyJTu-lqmWx X-Proofpoint-ORIG-GUID: VNbyLRA_siwXiuTaQql6kjyJTu-lqmWx Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894422476100001 Content-Type: text/plain; charset="utf-8" From: Avihai Horon There are already two places where dirty page bitmap allocation and calculations are done in open code. With device dirty page tracking being added in next patches, there are going to be even more places. To avoid code duplication, introduce VFIOBitmap struct and corresponding alloc function and use them where applicable. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 75 +++++++++++++++++++++++++++++------------------- 1 file changed, 46 insertions(+), 29 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 4c801513136a..151e7f40b73d 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -320,6 +320,27 @@ const MemoryRegionOps vfio_region_ops =3D { * Device state interfaces */ =20 +typedef struct { + unsigned long *bitmap; + hwaddr size; + hwaddr pages; +} VFIOBitmap; + +static int vfio_bitmap_alloc(VFIOBitmap *vbmap, hwaddr size) +{ + vbmap->pages =3D REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_size= (); + vbmap->size =3D ROUND_UP(vbmap->pages, sizeof(__u64) * BITS_PER_BYTE) / + BITS_PER_BYTE; + vbmap->bitmap =3D g_try_malloc0(vbmap->size); + if (!vbmap->bitmap) { + errno =3D ENOMEM; + + return -errno; + } + + return 0; +} + bool vfio_mig_active(void) { VFIOGroup *group; @@ -468,9 +489,14 @@ static int vfio_dma_unmap_bitmap(VFIOContainer *contai= ner, { struct vfio_iommu_type1_dma_unmap *unmap; struct vfio_bitmap *bitmap; - uint64_t pages =3D REAL_HOST_PAGE_ALIGN(size) / qemu_real_host_page_si= ze(); + VFIOBitmap vbmap; int ret; =20 + ret =3D vfio_bitmap_alloc(&vbmap, size); + if (ret) { + return -errno; + } + unmap =3D g_malloc0(sizeof(*unmap) + sizeof(*bitmap)); =20 unmap->argsz =3D sizeof(*unmap) + sizeof(*bitmap); @@ -484,35 +510,28 @@ static int vfio_dma_unmap_bitmap(VFIOContainer *conta= iner, * qemu_real_host_page_size to mark those dirty. Hence set bitmap_pgsi= ze * to qemu_real_host_page_size. */ - bitmap->pgsize =3D qemu_real_host_page_size(); - bitmap->size =3D ROUND_UP(pages, sizeof(__u64) * BITS_PER_BYTE) / - BITS_PER_BYTE; + bitmap->size =3D vbmap.size; + bitmap->data =3D (__u64 *)vbmap.bitmap; =20 - if (bitmap->size > container->max_dirty_bitmap_size) { - error_report("UNMAP: Size of bitmap too big 0x%"PRIx64, - (uint64_t)bitmap->size); + if (vbmap.size > container->max_dirty_bitmap_size) { + error_report("UNMAP: Size of bitmap too big 0x%"PRIx64, vbmap.size= ); ret =3D -E2BIG; goto unmap_exit; } =20 - bitmap->data =3D g_try_malloc0(bitmap->size); - if (!bitmap->data) { - ret =3D -ENOMEM; - goto unmap_exit; - } - ret =3D ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, unmap); if (!ret) { - cpu_physical_memory_set_dirty_lebitmap((unsigned long *)bitmap->da= ta, - iotlb->translated_addr, pages); + cpu_physical_memory_set_dirty_lebitmap(vbmap.bitmap, + iotlb->translated_addr, vbmap.pages); } else { error_report("VFIO_UNMAP_DMA with DIRTY_BITMAP : %m"); } =20 - g_free(bitmap->data); unmap_exit: g_free(unmap); + g_free(vbmap.bitmap); + return ret; } =20 @@ -1329,7 +1348,7 @@ static int vfio_get_dirty_bitmap(VFIOContainer *conta= iner, uint64_t iova, { struct vfio_iommu_type1_dirty_bitmap *dbitmap; struct vfio_iommu_type1_dirty_bitmap_get *range; - uint64_t pages; + VFIOBitmap vbmap; int ret; =20 if (!container->dirty_pages_supported) { @@ -1339,6 +1358,11 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, return 0; } =20 + ret =3D vfio_bitmap_alloc(&vbmap, size); + if (ret) { + return -errno; + } + dbitmap =3D g_malloc0(sizeof(*dbitmap) + sizeof(*range)); =20 dbitmap->argsz =3D sizeof(*dbitmap) + sizeof(*range); @@ -1353,15 +1377,8 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, * to qemu_real_host_page_size. */ range->bitmap.pgsize =3D qemu_real_host_page_size(); - - pages =3D REAL_HOST_PAGE_ALIGN(range->size) / qemu_real_host_page_size= (); - range->bitmap.size =3D ROUND_UP(pages, sizeof(__u64) * BITS_PER_BYTE) / - BITS_PER_BYTE; - range->bitmap.data =3D g_try_malloc0(range->bitmap.size); - if (!range->bitmap.data) { - ret =3D -ENOMEM; - goto err_out; - } + range->bitmap.size =3D vbmap.size; + range->bitmap.data =3D (__u64 *)vbmap.bitmap; =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { @@ -1372,14 +1389,14 @@ static int vfio_get_dirty_bitmap(VFIOContainer *con= tainer, uint64_t iova, goto err_out; } =20 - cpu_physical_memory_set_dirty_lebitmap((unsigned long *)range->bitmap.= data, - ram_addr, pages); + cpu_physical_memory_set_dirty_lebitmap(vbmap.bitmap, ram_addr, + vbmap.pages); =20 trace_vfio_get_dirty_bitmap(container->fd, range->iova, range->size, range->bitmap.size, ram_addr); err_out: - g_free(range->bitmap.data); g_free(dbitmap); + g_free(vbmap.bitmap); =20 return ret; } --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894436; cv=none; d=zohomail.com; s=zohoarc; b=J6RzJU0nzTqKwuPj+AmyiYK+s8is4KMHCMRiQqOJud82atimumKg9m7hvcxIXBrvD7Jn1ovOl3Swf5uBlEpN8Lu7AzZxeads9dHVU0wYbIFgiw26tIFrkeNcACAkntTXUJIAXP9Kbi7PyaWMqYtmlamizuzu2QtNB+s7aOOfTcQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894436; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XmxQiPZk6Jr6FvaNtthcRDWhdAD4oAVDL+leTtdsWB4=; b=AAQDOX3VcG0xen5tXXIIcqzQAXHzv9iXDQCTeVFYRfYe6uaSpmOwVYxKo5bUBZeZJf3m+0Gx4YuA8ihrfSXU3hTPdzqryPzylVovAJNyXKGq+7jGJptD28x+m37DPO/jb5I4JL45uJlSlXsdsb2CzKfDZqiA/RvNvLyGtkPPjaQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894436461388.08138320435285; Fri, 3 Mar 2023 17:47:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGx6-0006zn-4J; Fri, 03 Mar 2023 20:44:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGx4-0006zf-IA for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:46 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGx2-0005gi-Js for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:46 -0500 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3241XjH8028344; Sat, 4 Mar 2023 01:44:43 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3nyb7x0r8u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:43 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241XqOd015662; Sat, 4 Mar 2023 01:44:41 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g7aw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:41 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0h016769; Sat, 4 Mar 2023 01:44:41 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-6; Sat, 04 Mar 2023 01:44:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=XmxQiPZk6Jr6FvaNtthcRDWhdAD4oAVDL+leTtdsWB4=; b=kmoH1IOBOeVnkQtYtcyDtghb5M7mMEHDLDyphZS8vPMrXOh9urx3Akb/rDOdSlIkqXQa F5dkzy0rRmG2ZfZT+FlC8KMwm+7Cvqsra/Hbwco3W3i+YnEmBBkrs9fblQrBhE6SnQDu cbG+ShyYXZzxQ76/jMEIA8iywrUE2qiOba+jOamAVpeCjC1JDnZylX+zzfhWZ7u0SpvI pwbfK/ULnui+GDAAsQLYoylEu0anJswph/1d32kWV8Xk/oMcw8YFnermuqu2K1Lfr3nr K0rMRA7HSvSH4WJlneMxRQdlTX+a9ug9rhu9uZJaQA19suz8pocSrrgOC0PXEscqkfEH 7g== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 05/13] vfio/common: Add helper to validate iova/end against hostwin Date: Sat, 4 Mar 2023 01:43:35 +0000 Message-Id: <20230304014343.33646-6-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=753 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: Ytd9aiyMv3lO-XCW8HePImXjXyLRkEXq X-Proofpoint-ORIG-GUID: Ytd9aiyMv3lO-XCW8HePImXjXyLRkEXq Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894438506100007 Content-Type: text/plain; charset="utf-8" In preparation to be used in device dirty tracking, move the code that finds the container host DMA window against a iova range. This avoids duplication on the common checks across listener callbacks. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 151e7f40b73d..80f3a1c44a01 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -903,6 +903,22 @@ static void vfio_unregister_ram_discard_listener(VFIOC= ontainer *container, g_free(vrdl); } =20 +static VFIOHostDMAWindow *vfio_find_hostwin(VFIOContainer *container, + hwaddr iova, hwaddr end) +{ + VFIOHostDMAWindow *hostwin; + bool hostwin_found =3D false; + + QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { + if (hostwin->min_iova <=3D iova && end <=3D hostwin->max_iova) { + hostwin_found =3D true; + break; + } + } + + return hostwin_found ? hostwin : NULL; +} + static bool vfio_known_safe_misalignment(MemoryRegionSection *section) { MemoryRegion *mr =3D section->mr; @@ -928,7 +944,6 @@ static void vfio_listener_region_add(MemoryListener *li= stener, void *vaddr; int ret; VFIOHostDMAWindow *hostwin; - bool hostwin_found; Error *err =3D NULL; =20 if (vfio_listener_skipped_section(section)) { @@ -1029,15 +1044,8 @@ static void vfio_listener_region_add(MemoryListener = *listener, #endif } =20 - hostwin_found =3D false; - QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { - if (hostwin->min_iova <=3D iova && end <=3D hostwin->max_iova) { - hostwin_found =3D true; - break; - } - } - - if (!hostwin_found) { + hostwin =3D vfio_find_hostwin(container, iova, end); + if (!hostwin) { error_setg(&err, "Container %p can't map guest IOVA region" " 0x%"HWADDR_PRIx"..0x%"HWADDR_PRIx, container, iova, e= nd); goto fail; @@ -1239,15 +1247,9 @@ static void vfio_listener_region_del(MemoryListener = *listener, if (memory_region_is_ram_device(section->mr)) { hwaddr pgmask; VFIOHostDMAWindow *hostwin; - bool hostwin_found =3D false; =20 - QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { - if (hostwin->min_iova <=3D iova && end <=3D hostwin->max_iova)= { - hostwin_found =3D true; - break; - } - } - assert(hostwin_found); /* or region_add() would have failed */ + hostwin =3D vfio_find_hostwin(container, iova, end); + assert(hostwin); /* or region_add() would have failed */ =20 pgmask =3D (1ULL << ctz64(hostwin->iova_pgsizes)) - 1; try_unmap =3D !((iova & pgmask) || (int128_get64(llsize) & pgmask)= ); --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894390; cv=none; d=zohomail.com; s=zohoarc; b=Z8V58g2Yda31GOhBykQ5STqFOLi8wSw8mi5THSrndAN8I2FF0K4K9aDP4ufKwl4X0Ihk1paxbh3IrAIKWwCRNgLvkYZYbcXaflOkHEGyiQYsaRSuRQxtPnz0dnIiWftAVFDHpBL/WmMDtSEWo0Cw6egTnD7fzV3t7KpPO361Gi4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894390; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0saykVJKcvcjXtn/rbhkDq7mKeHhVUAcMOlUxmoB8lY=; b=NV4sh5DqU+2oES0uM+aD7Hvi7yINCT5vP1fu9KxUwhbLjyiOFslgdeGYdWap/VOh/7TwSnjPfFZhH1K3ntGJPvSsrPCDD5ZLr4iQlhX0bA9QjepfVI1w/Nb8gaxsSmtVU6ZcRzNrEOs4V1760C5hPIDGQmhvfqPZCIi9ZTAAsao= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894390261975.4134817580062; Fri, 3 Mar 2023 17:46:30 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGx8-000711-RF; Fri, 03 Mar 2023 20:44:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGx6-000702-Pt for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:48 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGx4-0005in-Ua for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:48 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3241hlCD015658; Sat, 4 Mar 2023 01:44:45 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p3vk1801x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:45 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241XfrK015499; Sat, 4 Mar 2023 01:44:44 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g7bu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:44 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0j016769; Sat, 4 Mar 2023 01:44:43 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-7; Sat, 04 Mar 2023 01:44:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=0saykVJKcvcjXtn/rbhkDq7mKeHhVUAcMOlUxmoB8lY=; b=UPmR3ccUng3ufYnjf3RRWVwvWu8f7WsQNiI5UFRJWl85Kg97QNKjGnSZyMKfbUqmf8A+ HizjV9WQILxqIeV4vawc4W9IA9DR9OLfT0SARitd7acPWN0fYvuLenK/gTeZv/gOLq2t w5Xpsr1e1D+wIpliAZQ2FtP1gHwII9e6RSWjMPdZnrGiGxIS2SV9GaungH/YnO4tH2rz tPV9CPXUL+R459jNUfvqEtRAIyYVP/Zz+lAjXWQjjTAyP+raCWZTu2fLiLHVinQave8H doG3e0Me8s+vzpp4xLV1qB0rKg/i3otSDyOyO+zCy6Eo802U0IW9Qf2a5CWCQ9A2VNrm 8g== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 06/13] vfio/common: Consolidate skip/invalid section into helper Date: Sat, 4 Mar 2023 01:43:36 +0000 Message-Id: <20230304014343.33646-7-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=588 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: 7Qs5HDSzwqD9B8pC7KpQirE0cX1NkNke X-Proofpoint-ORIG-GUID: 7Qs5HDSzwqD9B8pC7KpQirE0cX1NkNke Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894392327100003 Content-Type: text/plain; charset="utf-8" The checks are replicated against region_add and region_del and will be soon added in another memory listener dedicated for dirty tracking. Move these into a new helper for avoid duplication. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 52 +++++++++++++++++++----------------------------- 1 file changed, 21 insertions(+), 31 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 80f3a1c44a01..ed908e303dbb 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -935,23 +935,14 @@ static bool vfio_known_safe_misalignment(MemoryRegion= Section *section) return true; } =20 -static void vfio_listener_region_add(MemoryListener *listener, - MemoryRegionSection *section) +static bool vfio_listener_valid_section(MemoryRegionSection *section) { - VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); - hwaddr iova, end; - Int128 llend, llsize; - void *vaddr; - int ret; - VFIOHostDMAWindow *hostwin; - Error *err =3D NULL; - if (vfio_listener_skipped_section(section)) { trace_vfio_listener_region_add_skip( section->offset_within_address_space, section->offset_within_address_space + int128_get64(int128_sub(section->size, int128_one()))); - return; + return false; } =20 if (unlikely((section->offset_within_address_space & @@ -966,6 +957,24 @@ static void vfio_listener_region_add(MemoryListener *l= istener, section->offset_within_region, qemu_real_host_page_size()); } + return false; + } + + return true; +} + +static void vfio_listener_region_add(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); + hwaddr iova, end; + Int128 llend, llsize; + void *vaddr; + int ret; + VFIOHostDMAWindow *hostwin; + Error *err =3D NULL; + + if (!vfio_listener_valid_section(section)) { return; } =20 @@ -1184,26 +1193,7 @@ static void vfio_listener_region_del(MemoryListener = *listener, int ret; bool try_unmap =3D true; =20 - if (vfio_listener_skipped_section(section)) { - trace_vfio_listener_region_del_skip( - section->offset_within_address_space, - section->offset_within_address_space + - int128_get64(int128_sub(section->size, int128_one()))); - return; - } - - if (unlikely((section->offset_within_address_space & - ~qemu_real_host_page_mask()) !=3D - (section->offset_within_region & ~qemu_real_host_page_mas= k()))) { - if (!vfio_known_safe_misalignment(section)) { - error_report("%s received unaligned region %s iova=3D0x%"PRIx64 - " offset_within_region=3D0x%"PRIx64 - " qemu_real_host_page_size=3D0x%"PRIxPTR, - __func__, memory_region_name(section->mr), - section->offset_within_address_space, - section->offset_within_region, - qemu_real_host_page_size()); - } + if (!vfio_listener_valid_section(section)) { return; } =20 --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894331; cv=none; d=zohomail.com; s=zohoarc; b=N8eeZiEwKPnv/kynDcZIbYh+GD4/HM9VYN9sq8SL0J+OYOY2QchFYRnRGQlG64/Rm6IsBc1nzu52m0ABKpQHE16WYPoDgl230R6uL+rxMod7xkB35/uw1zOJRCknXWJYNazmzTa7gX6Fdul1cQ6LDvTQTXzN8icQH7R8EzYhtFw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894331; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PYvffhJeqghnbX1YeU8rdpMLDEz1j85B0a2RRv4ozZ8=; b=BEbKmNPL458yYpZpHtMbHGmiwwTUmpIzEq8VORoIqUz55KuKcOnWllZQ0zPR7414pQP7SUSMtC5BSCIllDKuRTMWmGiMRNHEiI0OR+8M2XpiKznuy+DKkTmQw1YWVDbYAEQn7LllcFN/tZRS5JXSOzQAdlsgGgNfNBZ1ObZyYFk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894331163733.3227041516343; Fri, 3 Mar 2023 17:45:31 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGxA-00071O-FX; Fri, 03 Mar 2023 20:44:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGx9-00071B-LU for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:51 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGx7-0005mQ-Ny for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:51 -0500 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 323LZZTB023780; Sat, 4 Mar 2023 01:44:48 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3nyba7rhby-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:48 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241Xd8W015487; Sat, 4 Mar 2023 01:44:46 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g7cv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:46 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0l016769; Sat, 4 Mar 2023 01:44:46 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-8; Sat, 04 Mar 2023 01:44:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=PYvffhJeqghnbX1YeU8rdpMLDEz1j85B0a2RRv4ozZ8=; b=xaZnDcizhckqn1dhwcpeMBcQTJRUR0r0dBtrwnSbufkJ6WD0P/ujQ79tUFTXw8eeip89 vORzQlW6tLeD5C4rLpqozBtB39DNda1997mnxBSpD0RA6BUQMOglDtgZacWqNCpSgvXW 3yAjqXle6mBBv/luEbHr2+mHbofjl0oK5kS67sw+REsczZEIBh/gn80jBnK+CmHkdp2F U2vOD4Pqb0QeUxod4YAvcozd/i8UTlOTI60QgqhnT6NSfX7+4g6A5RU/3B0w1AaKeVc4 0A+8SWjJHL/8egr4Nh259K0dk7jalfHb+NorF1T2xecUBD32H2jKMBj1tvwio29I7ixr wg== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Joao Martins , Avihai Horon Subject: [PATCH v3 07/13] vfio/common: Record DMA mapped IOVA ranges Date: Sat, 4 Mar 2023 01:43:37 +0000 Message-Id: <20230304014343.33646-8-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-ORIG-GUID: ylx_mCpb41Gv3mOisAH75_OP5ZVpHTVz X-Proofpoint-GUID: ylx_mCpb41Gv3mOisAH75_OP5ZVpHTVz Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894332723100003 Content-Type: text/plain; charset="utf-8" According to the device DMA logging uAPI, IOVA ranges to be logged by the device must be provided all at once upon DMA logging start. As preparation for the following patches which will add device dirty page tracking, keep a record of all DMA mapped IOVA ranges so later they can be used for DMA logging start. Note that when vIOMMU is enabled DMA mapped IOVA ranges are not tracked. This is due to the dynamic nature of vIOMMU DMA mapping/unmapping. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 84 +++++++++++++++++++++++++++++++++++ hw/vfio/trace-events | 1 + include/hw/vfio/vfio-common.h | 11 +++++ 3 files changed, 96 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index ed908e303dbb..d84e5fd86bb4 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -44,6 +44,7 @@ #include "migration/blocker.h" #include "migration/qemu-file.h" #include "sysemu/tpm.h" +#include "qemu/iova-tree.h" =20 VFIOGroupList vfio_group_list =3D QLIST_HEAD_INITIALIZER(vfio_group_list); @@ -1313,11 +1314,94 @@ static int vfio_set_dirty_page_tracking(VFIOContain= er *container, bool start) return ret; } =20 +/* + * Called for the dirty tracking memory listener to calculate the iova/end + * for a given memory region section. The checks here, replicate the logic + * in vfio_listener_region_{add,del}() used for the same purpose. And thus + * both listener should be kept in sync. + */ +static bool vfio_get_section_iova_range(VFIOContainer *container, + MemoryRegionSection *section, + hwaddr *out_iova, hwaddr *out_end) +{ + Int128 llend; + hwaddr iova; + + iova =3D REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); + llend =3D int128_make64(section->offset_within_address_space); + llend =3D int128_add(llend, section->size); + llend =3D int128_and(llend, int128_exts64(qemu_real_host_page_mask())); + + if (int128_ge(int128_make64(iova), llend)) { + return false; + } + + *out_iova =3D iova; + *out_end =3D int128_get64(llend) - 1; + return true; +} + +static void vfio_dirty_tracking_update(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIOContainer *container =3D container_of(listener, VFIOContainer, + tracking_listener); + VFIODirtyTrackingRange *range =3D &container->tracking_range; + hwaddr max32 =3D (1ULL << 32) - 1ULL; + hwaddr iova, end; + + if (!vfio_listener_valid_section(section) || + !vfio_get_section_iova_range(container, section, &iova, &end)) { + return; + } + + WITH_QEMU_LOCK_GUARD(&container->tracking_mutex) { + if (iova < max32 && end <=3D max32) { + if (range->min32 > iova) { + range->min32 =3D iova; + } + if (range->max32 < end) { + range->max32 =3D end; + } + trace_vfio_device_dirty_tracking_update(iova, end, + range->min32, range->max32); + } else { + if (!range->min64 || range->min64 > iova) { + range->min64 =3D iova; + } + if (range->max64 < end) { + range->max64 =3D end; + } + trace_vfio_device_dirty_tracking_update(iova, end, + range->min64, range->max64); + } + } + return; +} + +static const MemoryListener vfio_dirty_tracking_listener =3D { + .name =3D "vfio-tracking", + .region_add =3D vfio_dirty_tracking_update, +}; + +static void vfio_dirty_tracking_init(VFIOContainer *container) +{ + memset(&container->tracking_range, 0, sizeof(container->tracking_range= )); + qemu_mutex_init(&container->tracking_mutex); + container->tracking_listener =3D vfio_dirty_tracking_listener; + memory_listener_register(&container->tracking_listener, + container->space->as); + memory_listener_unregister(&container->tracking_listener); + qemu_mutex_destroy(&container->tracking_mutex); +} + static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); int ret; =20 + vfio_dirty_tracking_init(container); + ret =3D vfio_set_dirty_page_tracking(container, true); if (ret) { vfio_set_migration_error(ret); diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 669d9fe07cd9..d97a6de17921 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -104,6 +104,7 @@ vfio_known_safe_misalignment(const char *name, uint64_t= iova, uint64_t offset_wi vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint6= 4_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=3D0x%"PRIx64"= is not aligned to 0x%"PRIx64" and cannot be mapped for DMA" vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING regi= on_del 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRI= x64" - 0x%"PRIx64 +vfio_device_dirty_tracking_update(uint64_t start, uint64_t end, uint64_t m= in, uint64_t max) "section 0x%"PRIx64" - 0x%"PRIx64" -> update [0x%"PRIx64"= - 0x%"PRIx64"]" vfio_disconnect_container(int fd) "close container->fd=3D%d" vfio_put_group(int fd) "close group->fd=3D%d" vfio_get_device(const char * name, unsigned int flags, unsigned int num_re= gions, unsigned int num_irqs) "Device %s flags: %u, regions: %u, irqs: %u" diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 87524c64a443..96791add2719 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -23,6 +23,7 @@ =20 #include "exec/memory.h" #include "qemu/queue.h" +#include "qemu/iova-tree.h" #include "qemu/notify.h" #include "ui/console.h" #include "hw/display/ramfb.h" @@ -68,6 +69,13 @@ typedef struct VFIOMigration { size_t data_buffer_size; } VFIOMigration; =20 +typedef struct VFIODirtyTrackingRange { + hwaddr min32; + hwaddr max32; + hwaddr min64; + hwaddr max64; +} VFIODirtyTrackingRange; + typedef struct VFIOAddressSpace { AddressSpace *as; QLIST_HEAD(, VFIOContainer) containers; @@ -89,6 +97,9 @@ typedef struct VFIOContainer { uint64_t max_dirty_bitmap_size; unsigned long pgsizes; unsigned int dma_max_mappings; + VFIODirtyTrackingRange tracking_range; + QemuMutex tracking_mutex; + MemoryListener tracking_listener; QLIST_HEAD(, VFIOGuestIOMMU) giommu_list; QLIST_HEAD(, VFIOHostDMAWindow) hostwin_list; QLIST_HEAD(, VFIOGroup) group_list; --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894309; cv=none; d=zohomail.com; s=zohoarc; b=gXlSJqeOosnhXOrScbS9CrRNEztPL+/ZqPsZ6SFzLagpitsc98MOfVl9flUTVwV/4sbyvwb3FoDSyTOHYbRKGH36TVngLsAsIJf08QHZZd4lqtx0MiGT7u0EZLWwjhsfubUM9yukEIGhTHJuuUkIvq/vjjuC8vJGWeDEUZshWTE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894309; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KOegUf8ho1EKTnum2BepF5VWFtkK3WmKYKSCk6PtILc=; b=ZgN8PmPYYa7P/4IMPPq0a997KOBq/47loz4nlh+zTcGveoZ5sYh+PGG7pOsgsS2sFYUZcZEs61THcA8XDf91hb+AiAC17SAedi3kn5jsfFacRQnrofPev38GBXD2ntFURwnCojU0cNjUDjpJrGwmCCbK772ss9KVTjMg8gVQ14A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894309094587.0759414450506; Fri, 3 Mar 2023 17:45:09 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGxF-000722-2R; Fri, 03 Mar 2023 20:44:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxC-00071n-LJ for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:54 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxA-0005nZ-OC for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:54 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3241hwEo015725; Sat, 4 Mar 2023 01:44:51 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p3vk18021-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:50 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241XTp1014975; Sat, 4 Mar 2023 01:44:49 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g7du-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:49 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0n016769; Sat, 4 Mar 2023 01:44:48 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-9; Sat, 04 Mar 2023 01:44:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=KOegUf8ho1EKTnum2BepF5VWFtkK3WmKYKSCk6PtILc=; b=NGzEuHG0FJpTm9mf8+23oba2trVoL1c1m6CYcNxYxvS8ODruIVG8N4DCjONw/vyyMacp uOgcoq6wrKpJ4VtxSDOvM3E0Yd/TdaIRc5PYfdUjnJqKupckDU5TUAnkxfUf+2IRgugz 9lxfzYzLtpEF1ijoVtmJ29e1/c9GkskD/zJEAr0bBMW+Nc3v8MNiBcqs7CfXBLpdZ26K P6FLR3TdJr3kL1ThwRLMPlNNMr8wC2wx5dFjlOnQ5ITzRhzi1Qv94KeAfFbr+KQ6/pJG QwLAidoUQcORYBLy3Sgq1XORB1BbJsL5rbNDfjvaZI9aEBAgYPyXwbxsAfhTeE9xF/WC jQ== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Joao Martins , Avihai Horon Subject: [PATCH v3 08/13] vfio/common: Add device dirty page tracking start/stop Date: Sat, 4 Mar 2023 01:43:38 +0000 Message-Id: <20230304014343.33646-9-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=889 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: cJG5Ty6waptXB2UhNLqWVjP7tiVWy7gb X-Proofpoint-ORIG-GUID: cJG5Ty6waptXB2UhNLqWVjP7tiVWy7gb Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894310851100005 Content-Type: text/plain; charset="utf-8" Add device dirty page tracking start/stop functionality. This uses the device DMA logging uAPI to start and stop dirty page tracking by device. Device dirty page tracking is used only if all devices within a container support device dirty page tracking. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 166 +++++++++++++++++++++++++++++++++- hw/vfio/trace-events | 1 + include/hw/vfio/vfio-common.h | 2 + 3 files changed, 166 insertions(+), 3 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index d84e5fd86bb4..aa0df0604704 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -453,6 +453,22 @@ static bool vfio_devices_all_dirty_tracking(VFIOContai= ner *container) return true; } =20 +static bool vfio_devices_all_device_dirty_tracking(VFIOContainer *containe= r) +{ + VFIOGroup *group; + VFIODevice *vbasedev; + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (!vbasedev->dirty_pages_supported) { + return false; + } + } + } + + return true; +} + /* * Check if all VFIO devices are running and migration is active, which is * essentially equivalent to the migration being in pre-copy phase. @@ -1395,15 +1411,152 @@ static void vfio_dirty_tracking_init(VFIOContainer= *container) qemu_mutex_destroy(&container->tracking_mutex); } =20 +static int vfio_devices_dma_logging_set(VFIOContainer *container, + struct vfio_device_feature *featur= e) +{ + bool status =3D (feature->flags & VFIO_DEVICE_FEATURE_MASK) =3D=3D + VFIO_DEVICE_FEATURE_DMA_LOGGING_START; + VFIODevice *vbasedev; + VFIOGroup *group; + int ret =3D 0; + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (vbasedev->dirty_tracking =3D=3D status) { + continue; + } + + ret =3D ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); + if (ret) { + ret =3D -errno; + error_report("%s: Failed to set DMA logging %s, err %d (%s= )", + vbasedev->name, status ? "start" : "stop", re= t, + strerror(errno)); + goto out; + } + vbasedev->dirty_tracking =3D status; + } + } + +out: + return ret; +} + +static int vfio_devices_dma_logging_stop(VFIOContainer *container) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + + feature->argsz =3D sizeof(buf); + feature->flags =3D VFIO_DEVICE_FEATURE_SET; + feature->flags |=3D VFIO_DEVICE_FEATURE_DMA_LOGGING_STOP; + + return vfio_devices_dma_logging_set(container, feature); +} + +static struct vfio_device_feature * +vfio_device_feature_dma_logging_start_create(VFIOContainer *container) +{ + struct vfio_device_feature *feature; + size_t feature_size; + struct vfio_device_feature_dma_logging_control *control; + struct vfio_device_feature_dma_logging_range *ranges; + VFIODirtyTrackingRange *tracking =3D &container->tracking_range; + + feature_size =3D sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_dma_logging_control); + feature =3D g_try_malloc0(feature_size); + if (!feature) { + errno =3D ENOMEM; + return NULL; + } + feature->argsz =3D feature_size; + feature->flags =3D VFIO_DEVICE_FEATURE_SET; + feature->flags |=3D VFIO_DEVICE_FEATURE_DMA_LOGGING_START; + + control =3D (struct vfio_device_feature_dma_logging_control *)feature-= >data; + control->page_size =3D qemu_real_host_page_size(); + + /* + * DMA logging uAPI guarantees to support at least a number of ranges = that + * fits into a single host kernel base page. + */ + control->num_ranges =3D !!tracking->max32 + !!tracking->max64; + ranges =3D g_try_new0(struct vfio_device_feature_dma_logging_range, + control->num_ranges); + if (!ranges) { + g_free(feature); + errno =3D ENOMEM; + + return NULL; + } + + control->ranges =3D (__aligned_u64)ranges; + if (tracking->max32) { + ranges->iova =3D tracking->min32; + ranges->length =3D (tracking->max32 - tracking->min32) + 1; + ranges++; + } + if (tracking->max64) { + ranges->iova =3D tracking->min64; + ranges->length =3D (tracking->max64 - tracking->min64) + 1; + } + + trace_vfio_device_dirty_tracking_start(control->num_ranges, + tracking->min32, tracking->max3= 2, + tracking->min64, tracking->max6= 4); + + return feature; +} + +static void vfio_device_feature_dma_logging_start_destroy( + struct vfio_device_feature *feature) +{ + struct vfio_device_feature_dma_logging_control *control =3D + (struct vfio_device_feature_dma_logging_control *)feature->data; + struct vfio_device_feature_dma_logging_range *ranges =3D + (struct vfio_device_feature_dma_logging_range *)control->ranges; + + g_free(ranges); + g_free(feature); +} + +static int vfio_devices_dma_logging_start(VFIOContainer *container) +{ + struct vfio_device_feature *feature; + int ret =3D 0; + + vfio_dirty_tracking_init(container); + feature =3D vfio_device_feature_dma_logging_start_create(container); + if (!feature) { + return -errno; + } + + ret =3D vfio_devices_dma_logging_set(container, feature); + if (ret) { + vfio_devices_dma_logging_stop(container); + } + + vfio_device_feature_dma_logging_start_destroy(feature); + + return ret; +} + static void vfio_listener_log_global_start(MemoryListener *listener) { VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); int ret; =20 - vfio_dirty_tracking_init(container); + if (vfio_devices_all_device_dirty_tracking(container)) { + ret =3D vfio_devices_dma_logging_start(container); + } else { + ret =3D vfio_set_dirty_page_tracking(container, true); + } =20 - ret =3D vfio_set_dirty_page_tracking(container, true); if (ret) { + error_report("vfio: Could not start dirty page tracking, err: %d (= %s)", + ret, strerror(-ret)); vfio_set_migration_error(ret); } } @@ -1413,8 +1566,15 @@ static void vfio_listener_log_global_stop(MemoryList= ener *listener) VFIOContainer *container =3D container_of(listener, VFIOContainer, lis= tener); int ret; =20 - ret =3D vfio_set_dirty_page_tracking(container, false); + if (vfio_devices_all_device_dirty_tracking(container)) { + ret =3D vfio_devices_dma_logging_stop(container); + } else { + ret =3D vfio_set_dirty_page_tracking(container, false); + } + if (ret) { + error_report("vfio: Could not stop dirty page tracking, err: %d (%= s)", + ret, strerror(-ret)); vfio_set_migration_error(ret); } } diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index d97a6de17921..7a7e0cfe5b23 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -105,6 +105,7 @@ vfio_listener_region_add_no_dma_map(const char *name, u= int64_t iova, uint64_t si vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING regi= on_del 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRI= x64" - 0x%"PRIx64 vfio_device_dirty_tracking_update(uint64_t start, uint64_t end, uint64_t m= in, uint64_t max) "section 0x%"PRIx64" - 0x%"PRIx64" -> update [0x%"PRIx64"= - 0x%"PRIx64"]" +vfio_device_dirty_tracking_start(int nr_ranges, uint64_t min32, uint64_t m= ax32, uint64_t min64, uint64_t max64) "nr_ranges %d 32:[0x%"PRIx64" - 0x%"P= RIx64"], 64:[0x%"PRIx64" - 0x%"PRIx64"]" vfio_disconnect_container(int fd) "close container->fd=3D%d" vfio_put_group(int fd) "close group->fd=3D%d" vfio_get_device(const char * name, unsigned int flags, unsigned int num_re= gions, unsigned int num_irqs) "Device %s flags: %u, regions: %u, irqs: %u" diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 96791add2719..1cbbccd91e11 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -154,6 +154,8 @@ typedef struct VFIODevice { VFIOMigration *migration; Error *migration_blocker; OnOffAuto pre_copy_dirty_page_tracking; + bool dirty_pages_supported; + bool dirty_tracking; } VFIODevice; =20 struct VFIODeviceOps { --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894316; cv=none; d=zohomail.com; s=zohoarc; b=T5lvDHsdmOcwBRJj5NS98tHMxFUYGj1BR8iQ3ZTnJVdS+t6QEott35/fSBcW82e7HSjEJJfheixQUmLoZooxZzkngHcaxB6cALxfjA1Nfd/KgmKvfpaOuI4shurMg1UW9irkMKbrG5q0ZFVfAv+HcNGt9dgCzAJOYq7cEbO/qFI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894316; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=+2rYVbTAzQosWIDqUcFnJ3JXbhY6DqzkleL46W9BMCs=; b=JSsSDaAvHWnetMSIb13/IW9Ial5P1KfLtN+GWO3LVhB7lSbefHs3QHfwgzaCYyqENajAq50LOM7HlXIk/2WzTetO30v+/Ma5MNKcS78jcIOhmCBcLc1oCtDGGFXjHsPUrFBY19C5d8dsufAhvGJbKLziP3KRywMJxFUy/Q3+vtY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894316275947.1442721649752; Fri, 3 Mar 2023 17:45:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGxH-00072Z-25; Fri, 03 Mar 2023 20:44:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxF-00072D-6i for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:57 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxD-0005no-9C for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:56 -0500 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 323LZZnA023779; Sat, 4 Mar 2023 01:44:54 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3nyba7rhc1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:53 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241XfrM015499; Sat, 4 Mar 2023 01:44:52 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g7es-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:52 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0p016769; Sat, 4 Mar 2023 01:44:51 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-10; Sat, 04 Mar 2023 01:44:51 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=+2rYVbTAzQosWIDqUcFnJ3JXbhY6DqzkleL46W9BMCs=; b=iRmTSbeZy+lKO37eE8JtWAki2cAXMtZFQb0rvWagHQEf2h+5Jx3fuSfi1bKCtVdz6rNT r+ycaBTdC4fRUGu1sTtHw1MTY7wKYg0Uh76u6K7AKEmqLBma1gmtMoDRX08al4YURt0h uVWun5M6fnHsiQiGqdCWyhY+lVS9GX2z85nDVikMnJbMHAgReyHb7jCEoh9otzwzmHuX fTX/+asnzaWR2s21xy1xwGAUo9vXNWbk2pSxev4lOB0prUfp97fOfYUtvHRvWxtZbTT1 M8fecfDB6Dh2I735o/FH/KBTv1MesCkKGXHsOIpQU0SCAsyT34M2w1PzRFyI3JOKynUz DA== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v3 09/13] vfio/common: Extract code from vfio_get_dirty_bitmap() to new function Date: Sat, 4 Mar 2023 01:43:39 +0000 Message-Id: <20230304014343.33646-10-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=651 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-ORIG-GUID: dzvGIwJ7yXeqS8l2j0o85UrxSXboKJA- X-Proofpoint-GUID: dzvGIwJ7yXeqS8l2j0o85UrxSXboKJA- Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894316740100001 Content-Type: text/plain; charset="utf-8" From: Avihai Horon Extract the VFIO_IOMMU_DIRTY_PAGES ioctl code in vfio_get_dirty_bitmap() to its own function. This will help the code to be more readable after next patch will add device dirty page bitmap sync functionality. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 57 +++++++++++++++++++++++++++++------------------- 1 file changed, 35 insertions(+), 22 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index aa0df0604704..b0c7d03279ab 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1579,26 +1579,13 @@ static void vfio_listener_log_global_stop(MemoryLis= tener *listener) } } =20 -static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, - uint64_t size, ram_addr_t ram_addr) +static int vfio_query_dirty_bitmap(VFIOContainer *container, VFIOBitmap *v= bmap, + hwaddr iova, hwaddr size) { struct vfio_iommu_type1_dirty_bitmap *dbitmap; struct vfio_iommu_type1_dirty_bitmap_get *range; - VFIOBitmap vbmap; int ret; =20 - if (!container->dirty_pages_supported) { - cpu_physical_memory_set_dirty_range(ram_addr, size, - tcg_enabled() ? DIRTY_CLIENTS_= ALL : - DIRTY_CLIENTS_NOCODE); - return 0; - } - - ret =3D vfio_bitmap_alloc(&vbmap, size); - if (ret) { - return -errno; - } - dbitmap =3D g_malloc0(sizeof(*dbitmap) + sizeof(*range)); =20 dbitmap->argsz =3D sizeof(*dbitmap) + sizeof(*range); @@ -1613,8 +1600,8 @@ static int vfio_get_dirty_bitmap(VFIOContainer *conta= iner, uint64_t iova, * to qemu_real_host_page_size. */ range->bitmap.pgsize =3D qemu_real_host_page_size(); - range->bitmap.size =3D vbmap.size; - range->bitmap.data =3D (__u64 *)vbmap.bitmap; + range->bitmap.size =3D vbmap->size; + range->bitmap.data =3D (__u64 *)vbmap->bitmap; =20 ret =3D ioctl(container->fd, VFIO_IOMMU_DIRTY_PAGES, dbitmap); if (ret) { @@ -1622,16 +1609,42 @@ static int vfio_get_dirty_bitmap(VFIOContainer *con= tainer, uint64_t iova, error_report("Failed to get dirty bitmap for iova: 0x%"PRIx64 " size: 0x%"PRIx64" err: %d", (uint64_t)range->iova, (uint64_t)range->size, errno); - goto err_out; + } + + g_free(dbitmap); + + return ret; +} + +static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, + uint64_t size, ram_addr_t ram_addr) +{ + VFIOBitmap vbmap; + int ret; + + if (!container->dirty_pages_supported) { + cpu_physical_memory_set_dirty_range(ram_addr, size, + tcg_enabled() ? DIRTY_CLIENTS_= ALL : + DIRTY_CLIENTS_NOCODE); + return 0; + } + + ret =3D vfio_bitmap_alloc(&vbmap, size); + if (ret) { + return -errno; + } + + ret =3D vfio_query_dirty_bitmap(container, &vbmap, iova, size); + if (ret) { + goto out; } =20 cpu_physical_memory_set_dirty_lebitmap(vbmap.bitmap, ram_addr, vbmap.pages); =20 - trace_vfio_get_dirty_bitmap(container->fd, range->iova, range->size, - range->bitmap.size, ram_addr); -err_out: - g_free(dbitmap); + trace_vfio_get_dirty_bitmap(container->fd, iova, size, vbmap.size, + ram_addr); +out: g_free(vbmap.bitmap); =20 return ret; --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894379; cv=none; d=zohomail.com; s=zohoarc; b=ESkX8r/jT1IQZuPDoC+ZuSL3tYfNwzxDFIRi4WzrOnFWu0LF78622OWNkriey+bEgeEPbmbtDFN35D5NheDKHOqjzR8j0eiJrtriY42tgaF2SYE4XQKXdWLdxJf7rz75Ur9tRUkuaKrHQdgB0A0lqPiyv/sWswgezbi3mdACrXM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894379; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cRt27q5U/3xAdBQBpzWlZ5ZYHLvyqiaFBO0MPtv1dmM=; b=HVO8oordaeLy+wsAxQBW9xQ+gBKP9A/fCHLeV/MpVZ4YXiPz8zPXDqXL2I5x5IOzxb1nYy+Vq9Cg4xoC6gOfzaG+EgBEqy38QchORin4J0MZD+ic83A2A+0lK9F5OgLSIqwVFjXZ9RH7rRUnDnzlrk5SZTyY0CWR/gj15N6EQCk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894379772192.05873231898568; Fri, 3 Mar 2023 17:46:19 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGxI-00072r-LP; Fri, 03 Mar 2023 20:45:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxG-00072P-TW for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:58 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxF-0005o3-7R for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:44:58 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 323MsRr8006782; Sat, 4 Mar 2023 01:44:55 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3nyba2gwu2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:55 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241YGml015889; Sat, 4 Mar 2023 01:44:55 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g7ff-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:54 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0r016769; Sat, 4 Mar 2023 01:44:54 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-11; Sat, 04 Mar 2023 01:44:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=cRt27q5U/3xAdBQBpzWlZ5ZYHLvyqiaFBO0MPtv1dmM=; b=RJ//5w+QMahMWG01Hk0Hqwv1vNRtmsx9O9HU78yGJFyJjErQj/k1rCYuJyQXNSwDdO8n +7gw3cCrDjH+STjf+O6emoEbZK2cPH8Rvm8B7qFakAVBKr3qEKo69VfZlSyaEVCFiqjb EjQ9NEmEszRDQdgGv07n1DrCboJ4v5Qnz1pyCfgB89dd101wIFl2oXBhKMn4a835+RkB tuHxRkP5I1ftL/AldneJhyJJRfOmeRjbRBnJVfinISymU9Hde99XQhGgdSarwFxPd72g odWol1XyWTzAjQYxs1d66xjlSAmFJsW/c8BGfbfLY14YrSjebwsJ+iKIfgwueWJeRLzP Dw== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Joao Martins , Avihai Horon Subject: [PATCH v3 10/13] vfio/common: Add device dirty page bitmap sync Date: Sat, 4 Mar 2023 01:43:40 +0000 Message-Id: <20230304014343.33646-11-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=924 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: VBWDncvRrLHeTHUYLj8hsSmwGmQ2qW1g X-Proofpoint-ORIG-GUID: VBWDncvRrLHeTHUYLj8hsSmwGmQ2qW1g Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894380304100001 Content-Type: text/plain; charset="utf-8" Add device dirty page bitmap sync functionality. This uses the device DMA logging uAPI to sync dirty page bitmap from the device. Device dirty page bitmap sync is used only if all devices within a container support device dirty page tracking. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 88 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 79 insertions(+), 9 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index b0c7d03279ab..5b8456975e97 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -342,6 +342,9 @@ static int vfio_bitmap_alloc(VFIOBitmap *vbmap, hwaddr = size) return 0; } =20 +static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, + uint64_t size, ram_addr_t ram_addr); + bool vfio_mig_active(void) { VFIOGroup *group; @@ -565,10 +568,16 @@ static int vfio_dma_unmap(VFIOContainer *container, .iova =3D iova, .size =3D size, }; + bool need_dirty_sync =3D false; + int ret; + + if (iotlb && vfio_devices_all_running_and_mig_active(container)) { + if (!vfio_devices_all_device_dirty_tracking(container) && + container->dirty_pages_supported) { + return vfio_dma_unmap_bitmap(container, iova, size, iotlb); + } =20 - if (iotlb && container->dirty_pages_supported && - vfio_devices_all_running_and_mig_active(container)) { - return vfio_dma_unmap_bitmap(container, iova, size, iotlb); + need_dirty_sync =3D true; } =20 while (ioctl(container->fd, VFIO_IOMMU_UNMAP_DMA, &unmap)) { @@ -594,10 +603,12 @@ static int vfio_dma_unmap(VFIOContainer *container, return -errno; } =20 - if (iotlb && vfio_devices_all_running_and_mig_active(container)) { - cpu_physical_memory_set_dirty_range(iotlb->translated_addr, size, - tcg_enabled() ? DIRTY_CLIENTS_= ALL : - DIRTY_CLIENTS_NOCODE); + if (need_dirty_sync) { + ret =3D vfio_get_dirty_bitmap(container, iova, size, + iotlb->translated_addr); + if (ret) { + return ret; + } } =20 return 0; @@ -1579,6 +1590,58 @@ static void vfio_listener_log_global_stop(MemoryList= ener *listener) } } =20 +static int vfio_device_dma_logging_report(VFIODevice *vbasedev, hwaddr iov= a, + hwaddr size, void *bitmap) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_dma_logging_repo= rt), + sizeof(__aligned_u64))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + struct vfio_device_feature_dma_logging_report *report =3D + (struct vfio_device_feature_dma_logging_report *)feature->data; + + report->iova =3D iova; + report->length =3D size; + report->page_size =3D qemu_real_host_page_size(); + report->bitmap =3D (__aligned_u64)bitmap; + + feature->argsz =3D sizeof(buf); + feature->flags =3D + VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_DMA_LOGGING_REPORT; + + if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + return -errno; + } + + return 0; +} + +static int vfio_devices_query_dirty_bitmap(VFIOContainer *container, + VFIOBitmap *vbmap, hwaddr iova, + hwaddr size) +{ + VFIODevice *vbasedev; + VFIOGroup *group; + int ret; + + QLIST_FOREACH(group, &container->group_list, container_next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + ret =3D vfio_device_dma_logging_report(vbasedev, iova, size, + vbmap->bitmap); + if (ret) { + error_report("%s: Failed to get DMA logging report, iova: " + "0x%" HWADDR_PRIx ", size: 0x%" HWADDR_PRIx + ", err: %d (%s)", + vbasedev->name, iova, size, ret, strerror(-re= t)); + + return ret; + } + } + } + + return 0; +} + static int vfio_query_dirty_bitmap(VFIOContainer *container, VFIOBitmap *v= bmap, hwaddr iova, hwaddr size) { @@ -1619,10 +1682,12 @@ static int vfio_query_dirty_bitmap(VFIOContainer *c= ontainer, VFIOBitmap *vbmap, static int vfio_get_dirty_bitmap(VFIOContainer *container, uint64_t iova, uint64_t size, ram_addr_t ram_addr) { + bool all_device_dirty_tracking =3D + vfio_devices_all_device_dirty_tracking(container); VFIOBitmap vbmap; int ret; =20 - if (!container->dirty_pages_supported) { + if (!container->dirty_pages_supported && !all_device_dirty_tracking) { cpu_physical_memory_set_dirty_range(ram_addr, size, tcg_enabled() ? DIRTY_CLIENTS_= ALL : DIRTY_CLIENTS_NOCODE); @@ -1634,7 +1699,12 @@ static int vfio_get_dirty_bitmap(VFIOContainer *cont= ainer, uint64_t iova, return -errno; } =20 - ret =3D vfio_query_dirty_bitmap(container, &vbmap, iova, size); + if (all_device_dirty_tracking) { + ret =3D vfio_devices_query_dirty_bitmap(container, &vbmap, iova, s= ize); + } else { + ret =3D vfio_query_dirty_bitmap(container, &vbmap, iova, size); + } + if (ret) { goto out; } --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894394; cv=none; d=zohomail.com; s=zohoarc; b=Rkmb4mNm1u6dX3AYnqFDMt/akLkdhvtl0zwNu5tEaEzcWFj5cV2PlyPrn8iif/4X7Lm+DW1IAo5MrKhGCPiTktsa18dBxt5/13+ND4PcEY5nDJZIbxhchpOeRGAm8vc6PouwGlVURfTr+x+kQXmInMtTtjxbnR6ejaPintmdp10= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894394; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2kCOGSsmi6n+f+mtgQt/sYjswS65xZxOKhkRqpF7wL8=; b=deO5tAenGUAZqdDCfuTvpl5qa6ZS/1B+O9y/siQ0Bp3P03M1BykyjxL/jVzSW+uQ42/sK146dbgR4nh8KPBPyXzppykoaQnG4uloQOLD+GQPOqbYXTmlX9l5+CA7otwCKzt4fH/0+Y2e4LwngjSGnE8skSBW5CzUcC/Q9ZsSj3g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894394955341.3129430575625; Fri, 3 Mar 2023 17:46:34 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGxO-00073l-Ca; Fri, 03 Mar 2023 20:45:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxK-00073S-00 for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:45:02 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxI-0005ok-5F for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:45:01 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3241hZTM015303; Sat, 4 Mar 2023 01:44:59 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p3vk18025-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:58 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241Y8pj015769; Sat, 4 Mar 2023 01:44:57 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g7gg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:57 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0t016769; Sat, 4 Mar 2023 01:44:56 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-12; Sat, 04 Mar 2023 01:44:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=2kCOGSsmi6n+f+mtgQt/sYjswS65xZxOKhkRqpF7wL8=; b=ZeqciY0pKVCGUr1eoo8fG4KsFgrFggFu3CfDfHuClfXpp7Jvor38FupaI/yj1ITyXitY 6O9hEUNTvJcEiyX2/FErsR11ViVe9s/DDcqfaZnPc+SgNmVh6MSAFgG/hC6YB4CO0o26 +5xCWVwdg5YAtKhbRstPapYfR9LJyd9pCOu9WTI9Tam6v6jbyrv6YxNqrjpEt3kJSnzo FKKLd8+/TGwcRzhdKjeWPAWGiShFnozBcw3IxwAtHdj32ncb+E1NUSKHvFKVi8VxjhgL oywskgHHain+MIBSQjw+BA5LQglRd5whJe/2gn6GQeHSq59AX/qJXY7Fgml8CVqPhxmo Dg== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Joao Martins Subject: [PATCH v3 11/13] vfio/migration: Block migration with vIOMMU Date: Sat, 4 Mar 2023 01:43:41 +0000 Message-Id: <20230304014343.33646-12-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=632 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: VbGDZxCXmTHJSzfDKVJPtm4nJdggh5kT X-Proofpoint-ORIG-GUID: VbGDZxCXmTHJSzfDKVJPtm4nJdggh5kT Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894396353100003 Content-Type: text/plain; charset="utf-8" Migrating with vIOMMU will require either tracking maximum IOMMU supported address space (e.g. 39/48 address width on Intel) or range-track current mappings and dirty track the new ones post starting dirty tracking. This will be done as a separate series, so add a live migration blocker until that is fixed. Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/common.c | 51 +++++++++++++++++++++++++++++++++++ hw/vfio/migration.c | 6 +++++ include/hw/vfio/vfio-common.h | 2 ++ 3 files changed, 59 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 5b8456975e97..9b909f856722 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -365,6 +365,7 @@ bool vfio_mig_active(void) } =20 static Error *multiple_devices_migration_blocker; +static Error *giommu_migration_blocker; =20 static unsigned int vfio_migratable_device_num(void) { @@ -416,6 +417,56 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker =3D NULL; } =20 +static unsigned int vfio_use_iommu_device_num(void) +{ + VFIOGroup *group; + VFIODevice *vbasedev; + unsigned int device_num =3D 0; + + QLIST_FOREACH(group, &vfio_group_list, next) { + QLIST_FOREACH(vbasedev, &group->device_list, next) { + if (vbasedev->group->container->space->as !=3D + &address_space_memory) { + device_num++; + } + } + } + + return device_num; +} + +int vfio_block_giommu_migration(Error **errp) +{ + int ret; + + if (giommu_migration_blocker || + !vfio_use_iommu_device_num()) { + return 0; + } + + error_setg(&giommu_migration_blocker, + "Migration is currently not supported with vIOMMU enabled"); + ret =3D migrate_add_blocker(giommu_migration_blocker, errp); + if (ret < 0) { + error_free(giommu_migration_blocker); + giommu_migration_blocker =3D NULL; + } + + return ret; +} + +void vfio_unblock_giommu_migration(void) +{ + if (!giommu_migration_blocker || + vfio_use_iommu_device_num()) { + return; + } + + migrate_del_blocker(giommu_migration_blocker); + error_free(giommu_migration_blocker); + giommu_migration_blocker =3D NULL; +} + static void vfio_set_migration_error(int err) { MigrationState *ms =3D migrate_get_current(); diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index a2c3d9bade7f..3e75868ae7a9 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -634,6 +634,11 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error *= *errp) return ret; } =20 + ret =3D vfio_block_giommu_migration(errp); + if (ret) { + return ret; + } + trace_vfio_migration_probe(vbasedev->name); return 0; =20 @@ -659,6 +664,7 @@ void vfio_migration_finalize(VFIODevice *vbasedev) unregister_savevm(VMSTATE_IF(vbasedev->dev), "vfio", vbasedev); vfio_migration_exit(vbasedev); vfio_unblock_multiple_devices_migration(); + vfio_unblock_giommu_migration(); } =20 if (vbasedev->migration_blocker) { diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 1cbbccd91e11..38e44258925b 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -233,6 +233,8 @@ extern VFIOGroupList vfio_group_list; bool vfio_mig_active(void); int vfio_block_multiple_devices_migration(Error **errp); void vfio_unblock_multiple_devices_migration(void); +int vfio_block_giommu_migration(Error **errp); +void vfio_unblock_giommu_migration(void); int64_t vfio_mig_bytes_transferred(void); =20 #ifdef CONFIG_LINUX --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894405; cv=none; d=zohomail.com; s=zohoarc; b=BufSdh+8DU9kMsQGZBhTGoTZYr2QQbbw9ek/E+EFefIKwK3thme4x1MXImVOqLbSIXreurN8NwDNIfTy06PtlvOO7sTRgqUD0alN1O9kdI3lUJ8AW+ODHd3ByuPbisJQZFjFxSWzto0LXl64RJUwiveaI5447nRHHC/Kd2PJJps= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894405; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=erm6jcUnq0dCHUR6vJxy8tc4+tJla+sUFwT5HcfOMyU=; b=ZKiaa6cJxYEaqHhxFnN2qURFa7M8mBuGUkGq6sbQWLMRrcM9YKdOw9UEawlEUeyEtmkuIB6WZZ9QQGoTPFayUOVshT6oA9drSCOS52wKxX8/giYYvTO2bZ9nRiF42sy0BG/wofqXPB2b1vsIc2/jt2It5JUnnkS/qglY5KvThAU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 167789440527225.79476366915901; Fri, 3 Mar 2023 17:46:45 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGxZ-0007EN-1p; Fri, 03 Mar 2023 20:45:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxM-00073n-Q6 for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:45:06 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxL-0005pM-0m for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:45:04 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3241hlCF015658; Sat, 4 Mar 2023 01:45:01 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p3vk18027-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:45:01 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241XhKD015522; Sat, 4 Mar 2023 01:45:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g7ha-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:44:59 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0v016769; Sat, 4 Mar 2023 01:44:59 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-13; Sat, 04 Mar 2023 01:44:59 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=erm6jcUnq0dCHUR6vJxy8tc4+tJla+sUFwT5HcfOMyU=; b=wql6u6yX2464A1RcSBqeti4jXCMDFNzDoX2H7D3/iXpZbOvP346xJSQ58nMdOJYeOFr1 Y3fwV9kl74HquMteL0gwxM4/4zGtoXCMVk8FL02lbNLIH4wC1mODClSENbyd/FUT7qJy 58mwKD3wwTa7anykgNBeTGFGk9pq+/EIv0PxQGO8q8fZJtn9sFeGEfj9Gqu1mUCaJZL3 uR8enX/xCGPShBaFbpcMIR3itouOKR7vBbLLwRxAehfK4ne8454pYtrKmvZuO6gdkc28 fwQoLInKuGiEQAG7KQOl12tN40pIYAjMXq6IamMka3cx0V+55nq0QBDJMVCrR4VOCfnH 0Q== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Joao Martins , Avihai Horon Subject: [PATCH v3 12/13] vfio/migration: Query device dirty page tracking support Date: Sat, 4 Mar 2023 01:43:42 +0000 Message-Id: <20230304014343.33646-13-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: kmWKTFjRqgiltGZSMFV9ml8avh1Tl_xB X-Proofpoint-ORIG-GUID: kmWKTFjRqgiltGZSMFV9ml8avh1Tl_xB Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894406385100001 Content-Type: text/plain; charset="utf-8" Now that everything has been set up for device dirty page tracking, query the device for device dirty page tracking support. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- hw/vfio/migration.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 3e75868ae7a9..da3aa596b3ec 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -555,6 +555,19 @@ static int vfio_migration_query_flags(VFIODevice *vbas= edev, uint64_t *mig_flags) return 0; } =20 +static bool vfio_dma_logging_supported(VFIODevice *vbasedev) +{ + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + + feature->argsz =3D sizeof(buf); + feature->flags =3D + VFIO_DEVICE_FEATURE_PROBE | VFIO_DEVICE_FEATURE_DMA_LOGGING_START; + + return !ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature); +} + static int vfio_migration_init(VFIODevice *vbasedev) { int ret; @@ -589,6 +602,8 @@ static int vfio_migration_init(VFIODevice *vbasedev) migration->device_state =3D VFIO_DEVICE_STATE_RUNNING; migration->data_fd =3D -1; =20 + vbasedev->dirty_pages_supported =3D vfio_dma_logging_supported(vbasede= v); + oid =3D vmstate_if_get_id(VMSTATE_IF(DEVICE(obj))); if (oid) { path =3D g_strdup_printf("%s/vfio", oid); --=20 2.17.2 From nobody Tue May 7 16:40:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1677894434; cv=none; d=zohomail.com; s=zohoarc; b=lMKqbz4ET3vt+DelgY1FqqhquQ9sP4WTtAeIHdlPK95uhdeodJhRN8k2PGNU53skYOBb7DTqHuWy6aLyO2qorXodP4xhPUgVfzrmambMktmRxGA0XAsUN2FoKBQrTH0miZITRPzro8p+viu/FbfyM7QH7llN9i7+aSY0UvVp0i0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677894434; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=S8N/340cX9iXMQcJJeOcV59jvlpnhDzM55cnZWaHBpM=; b=OdlLWnZyTbu7DhfyRKAz3ksbHPvQTj0f2RYOetW9oFco9+Z9EMhxhyCRSDNZQ1jmNkMW1gnqDN2PDhpLZftyLnfi31JSmYf590l8+rje3/HYwUnzZaz295oOP4o2SFdUFhSe+uGiVMQlVYl/iYUA7aa5H0CfEusDBJuu39VEn4A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677894434221328.2389941826474; Fri, 3 Mar 2023 17:47:14 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYGxa-0007Md-77; Fri, 03 Mar 2023 20:45:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxQ-0007Au-3P for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:45:13 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYGxN-000600-GY for qemu-devel@nongnu.org; Fri, 03 Mar 2023 20:45:07 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3241Xe0W031636; Sat, 4 Mar 2023 01:45:04 GMT Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3p3vddg0be-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:45:03 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 3241Xdlo015475; Sat, 4 Mar 2023 01:45:03 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3p3ve8g7kx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 04 Mar 2023 01:45:03 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3241YZ0x016769; Sat, 4 Mar 2023 01:45:02 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-185-117.vpn.oracle.com [10.175.185.117]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3p3ve8g73b-14; Sat, 04 Mar 2023 01:45:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=S8N/340cX9iXMQcJJeOcV59jvlpnhDzM55cnZWaHBpM=; b=tNhr+qspgz6/N850XpNL4JpOv1qYb5+1wdJ6eI+IyKEtgUhArWYLQ8fbr8apOA6g3VMt LzBVnNgN5zdq6EqguUCSsQuPUkdvotDYKyeY3F535nskZK2ttfYCIVBobH9AMvAOla7t q/QbXk/2ycGpruc/ZcN7YQAV/Zl1C0bEKL7dHV88Ar43YBvCjLL64yS+vP6KiJfhO25g z0f8Fx60ywJ4eXB0RIhv0C9gsBAEsGtRhVhB9DVt2Vnuidk2SR2HvnGVkNnmz/0iTxtT 6WGy2g+52MRMtwqEOkNqPWA3SJJlmqqfCcfxRcV+KnN0EmPCegv/ctYJOOpK6oEcgU7c eg== From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Yishai Hadas , Jason Gunthorpe , Maor Gottlieb , Kirti Wankhede , Tarun Gupta , Avihai Horon , Joao Martins Subject: [PATCH v3 13/13] docs/devel: Document VFIO device dirty page tracking Date: Sat, 4 Mar 2023 01:43:43 +0000 Message-Id: <20230304014343.33646-14-joao.m.martins@oracle.com> In-Reply-To: <20230304014343.33646-1-joao.m.martins@oracle.com> References: <20230304014343.33646-1-joao.m.martins@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-03_07,2023-03-03_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 mlxlogscore=952 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303040011 X-Proofpoint-GUID: pjVccxvJqdyiwI6VoVVNjmMbXNCNQCQk X-Proofpoint-ORIG-GUID: pjVccxvJqdyiwI6VoVVNjmMbXNCNQCQk Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZM-MESSAGEID: 1677894434513100001 Content-Type: text/plain; charset="utf-8" From: Avihai Horon Adjust the VFIO dirty page tracking documentation and add a section to describe device dirty page tracking. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Reviewed-by: C=C3=A9dric Le Goater --- docs/devel/vfio-migration.rst | 46 +++++++++++++++++++++++------------ 1 file changed, 31 insertions(+), 15 deletions(-) diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst index c214c73e2818..1b68ccf11529 100644 --- a/docs/devel/vfio-migration.rst +++ b/docs/devel/vfio-migration.rst @@ -59,22 +59,37 @@ System memory dirty pages tracking ---------------------------------- =20 A ``log_global_start`` and ``log_global_stop`` memory listener callback in= forms -the VFIO IOMMU module to start and stop dirty page tracking. A ``log_sync`` -memory listener callback marks those system memory pages as dirty which are -used for DMA by the VFIO device. The dirty pages bitmap is queried per -container. All pages pinned by the vendor driver through external APIs hav= e to -be marked as dirty during migration. When there are CPU writes, CPU dirty = page -tracking can identify dirtied pages, but any page pinned by the vendor dri= ver -can also be written by the device. There is currently no device or IOMMU -support for dirty page tracking in hardware. +the VFIO dirty tracking module to start and stop dirty page tracking. A +``log_sync`` memory listener callback queries the dirty page bitmap from t= he +dirty tracking module and marks system memory pages which were DMA-ed by t= he +VFIO device as dirty. The dirty page bitmap is queried per container. + +Currently there are two ways dirty page tracking can be done: +(1) Device dirty tracking: +In this method the device is responsible to log and report its DMAs. This +method can be used only if the device is capable of tracking its DMAs. +Discovering device capability, starting and stopping dirty tracking, and +syncing the dirty bitmaps from the device are done using the DMA logging u= API. +More info about the uAPI can be found in the comments of the +``vfio_device_feature_dma_logging_control`` and +``vfio_device_feature_dma_logging_report`` structures in the header file +linux-headers/linux/vfio.h. + +(2) VFIO IOMMU module: +In this method dirty tracking is done by IOMMU. However, there is currentl= y no +IOMMU support for dirty page tracking. For this reason, all pages are +perpetually marked dirty, unless the device driver pins pages through exte= rnal +APIs in which case only those pinned pages are perpetually marked dirty. + +If the above two methods are not supported, all pages are perpetually mark= ed +dirty by QEMU. =20 By default, dirty pages are tracked during pre-copy as well as stop-and-co= py -phase. So, a page pinned by the vendor driver will be copied to the destin= ation -in both phases. Copying dirty pages in pre-copy phase helps QEMU to predic= t if -it can achieve its downtime tolerances. If QEMU during pre-copy phase keeps -finding dirty pages continuously, then it understands that even in stop-an= d-copy -phase, it is likely to find dirty pages and can predict the downtime -accordingly. +phase. So, a page marked as dirty will be copied to the destination in both +phases. Copying dirty pages in pre-copy phase helps QEMU to predict if it = can +achieve its downtime tolerances. If QEMU during pre-copy phase keeps findi= ng +dirty pages continuously, then it understands that even in stop-and-copy p= hase, +it is likely to find dirty pages and can predict the downtime accordingly. =20 QEMU also provides a per device opt-out option ``pre-copy-dirty-page-track= ing`` which disables querying the dirty bitmap during pre-copy phase. If it is s= et to @@ -89,7 +104,8 @@ phase of migration. In that case, the unmap ioctl return= s any dirty pages in that range and QEMU reports corresponding guest physical pages dirty. Duri= ng stop-and-copy phase, an IOMMU notifier is used to get a callback for mapped pages and then dirty pages bitmap is fetched from VFIO IOMMU modules for t= hose -mapped ranges. +mapped ranges. If device dirty tracking is enabled with vIOMMU, live migra= tion +will be blocked. =20 Flow of state changes during Live migration =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --=20 2.17.2