From nobody Fri Dec 19 13:47:15 2025 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1229A2D949C for ; Mon, 13 Oct 2025 05:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760333566; cv=none; b=g5wlV/9M4/eAzdLwufxjjk0iUkHCGA09wQ5BXBhnelpZucEv7ZOiYRbouj5vifXV5nXf3m4CWWUf9UorPlxWb8Tw/vIQHnMtG6HGIruxnM7PC94GKViNOHnJy2MkPSpTYnF/uItKskFOGdW/nJU5zkidJrGiN0thy+Hm9Y20UH0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760333566; c=relaxed/simple; bh=6XcZ9MVYEQvbzX0ab/p/XqkK7vVjmvZOH+HTZVqz2Bw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-ID:References: In-Reply-To:To:CC; b=kBCR0xrHQCPObUfNm0DfNm2NlVj0cHEAKBNmnedhrF+VOOs//VzvMLadFw7mbvljsPUqyZ+uJHSn7Dn9Q2fWPBg9Bnxm8ftKobTAsnhCAHw0h/gsdXF2svnaSFxu7mzBBKlC3yiA8FbbY2hFH0+3Eap6wiyZE9GW7JtYlkkVW5I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=fb.com header.i=@fb.com header.b=fKBli/z2; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fb.com header.i=@fb.com header.b="fKBli/z2" Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 59CMgNrG1450873 for ; Sun, 12 Oct 2025 22:32:44 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2025-q2; bh=R8mY5ErWJ0OEynwU1tO7R/gc5Tv7LC/H9b+NJx7lPuQ=; b=fKBli/z2Ufxm BbRF8srZ8PJHqPga6Yg7pPTmdQAtvDuPgNZtIhOVHRbQHdvMxu5WYfjFOUaUCotS 849etk12tRJfSaMplwlm6XiYN8WETQDs58NvUE1c8EmLzslmQ2qlavAdZDp966qB vjsQrWTHQZTw4foIp3FiAFYKaUNyRB4bFPqSbXrXs7v63/UsO2EL6sxNDbvCMVOR EV9+YC0aOF7lulUK0fywsmG1qiiR/36/IHu+MitZYCwfCEcZM/S0xMKp+G+9/ETv PZq1uOjf9KisEYDg/gr1MUjU1gH3DMC3/38Jd/eHQa95uu/3gIqIgoD4ltFPcV1v kLFOl03KfQ== Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 49rnbm172b-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Sun, 12 Oct 2025 22:32:44 -0700 (PDT) Received: from twshared18070.28.prn2.facebook.com (2620:10d:c085:208::7cb7) by mail.thefacebook.com (2620:10d:c08b:78::2ac9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.20; Mon, 13 Oct 2025 05:32:42 +0000 Received: by devgpu015.cco6.facebook.com (Postfix, from userid 199522) id 9E257102FE47; Sun, 12 Oct 2025 22:32:30 -0700 (PDT) From: Alex Mastro Date: Sun, 12 Oct 2025 22:32:24 -0700 Subject: [PATCH v4 1/3] vfio/type1: sanitize for overflow using check_*_overflow Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-ID: <20251012-fix-unmap-v4-1-9eefc90ed14c@fb.com> References: <20251012-fix-unmap-v4-0-9eefc90ed14c@fb.com> In-Reply-To: <20251012-fix-unmap-v4-0-9eefc90ed14c@fb.com> To: Alex Williamson CC: Jason Gunthorpe , Alejandro Jimenez , , , Alex Mastro X-Mailer: b4 0.13.0 X-FB-Internal: Safe X-Proofpoint-GUID: 8FuvjVKYvSuWD-WCgttlJj_BNarlD53B X-Authority-Analysis: v=2.4 cv=NfjrFmD4 c=1 sm=1 tr=0 ts=68ec8efc cx=c_pps a=CB4LiSf2rd0gKozIdrpkBw==:117 a=CB4LiSf2rd0gKozIdrpkBw==:17 a=IkcTkHD0fZMA:10 a=x6icFKpwvdMA:10 a=FOH2dFAWAAAA:8 a=kr0RIOuGBwfI36ecCNoA:9 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: 8FuvjVKYvSuWD-WCgttlJj_BNarlD53B X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMDEzMDAyNiBTYWx0ZWRfX4U/3rI9nOSOP laT5MuXGpcA1qgZrFQNqbadsiF2orBujosuuBdUcv3i4vQdck4j9iw+0hyrlXKQbVDEvG5DPjNu CW9jyNcoxz6kPs/w9dealQ3DzTXDEfPMESL6bIxQWizVdRlcjp4Vq4DQwtStiJkDmaz7SixE9Ca 9vfsx20tt1Zyx7MEooUjNiFK6ALz6G3j0zMjg3R9fHUshkcYXfMh51QuVrN4PAsU9o9n5GcXqff 02ssBiFaxqnW6Fa3OdAtPijfXJj3D1tk8DJCZ08W900dxTcdwsF68Cn87jwPYlFK+/eQ1+tLLwF DGDP3QZZeJrpZY+8U0mvSCzOx+X1tw2dSgvaG7jvjMfPZ8lYTJya1Ze8dr9byc1MzbKPQd5kMgv U1tRtkZtdyr8I2u4k8rS6lk+aowBzw== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-10-13_02,2025-10-06_01,2025-03-28_01 Adopt check_*_overflow functions to clearly express overflow check intent. Signed-off-by: Alex Mastro Reviewed-by: Alejandro Jimenez Reviewed-by: Jason Gunthorpe Tested-by: Alejandro Jimenez --- drivers/vfio/vfio_iommu_type1.c | 86 ++++++++++++++++++++++++++++++-------= ---- 1 file changed, 63 insertions(+), 23 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type= 1.c index f8d68fe77b41..1ac056b27f27 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -37,6 +37,7 @@ #include #include #include +#include #include "vfio.h" =20 #define DRIVER_VERSION "0.2" @@ -180,7 +181,7 @@ static struct vfio_dma *vfio_find_dma(struct vfio_iommu= *iommu, } =20 static struct rb_node *vfio_find_dma_first_node(struct vfio_iommu *iommu, - dma_addr_t start, u64 size) + dma_addr_t start, size_t size) { struct rb_node *res =3D NULL; struct rb_node *node =3D iommu->dma_list.rb_node; @@ -825,14 +826,20 @@ static int vfio_iommu_type1_pin_pages(void *iommu_dat= a, unsigned long remote_vaddr; struct vfio_dma *dma; bool do_accounting; + dma_addr_t iova_end; + size_t iova_size; =20 - if (!iommu || !pages) + if (!iommu || !pages || npage <=3D 0) return -EINVAL; =20 /* Supported for v2 version only */ if (!iommu->v2) return -EACCES; =20 + if (check_mul_overflow(npage, PAGE_SIZE, &iova_size) || + check_add_overflow(user_iova, iova_size - 1, &iova_end)) + return -EOVERFLOW; + mutex_lock(&iommu->lock); =20 if (WARN_ONCE(iommu->vaddr_invalid_count, @@ -938,12 +945,21 @@ static void vfio_iommu_type1_unpin_pages(void *iommu_= data, { struct vfio_iommu *iommu =3D iommu_data; bool do_accounting; + dma_addr_t iova_end; + size_t iova_size; int i; =20 /* Supported for v2 version only */ if (WARN_ON(!iommu->v2)) return; =20 + if (WARN_ON(npage <=3D 0)) + return; + + if (WARN_ON(check_mul_overflow(npage, PAGE_SIZE, &iova_size) || + check_add_overflow(user_iova, iova_size - 1, &iova_end))) + return; + mutex_lock(&iommu->lock); =20 do_accounting =3D list_empty(&iommu->domain_list); @@ -1304,7 +1320,8 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iommu, int ret =3D -EINVAL, retries =3D 0; unsigned long pgshift; dma_addr_t iova =3D unmap->iova; - u64 size =3D unmap->size; + dma_addr_t iova_end; + size_t size =3D unmap->size; bool unmap_all =3D unmap->flags & VFIO_DMA_UNMAP_FLAG_ALL; bool invalidate_vaddr =3D unmap->flags & VFIO_DMA_UNMAP_FLAG_VADDR; struct rb_node *n, *first_n; @@ -1317,6 +1334,11 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iomm= u, goto unlock; } =20 + if (iova !=3D unmap->iova || size !=3D unmap->size) { + ret =3D -EOVERFLOW; + goto unlock; + } + pgshift =3D __ffs(iommu->pgsize_bitmap); pgsize =3D (size_t)1 << pgshift; =20 @@ -1326,10 +1348,15 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iom= mu, if (unmap_all) { if (iova || size) goto unlock; - size =3D U64_MAX; - } else if (!size || size & (pgsize - 1) || - iova + size - 1 < iova || size > SIZE_MAX) { - goto unlock; + size =3D SIZE_MAX; + } else { + if (!size || size & (pgsize - 1)) + goto unlock; + + if (check_add_overflow(iova, size - 1, &iova_end)) { + ret =3D -EOVERFLOW; + goto unlock; + } } =20 /* When dirty tracking is enabled, allow only min supported pgsize */ @@ -1376,7 +1403,7 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iommu, if (dma && dma->iova !=3D iova) goto unlock; =20 - dma =3D vfio_find_dma(iommu, iova + size - 1, 0); + dma =3D vfio_find_dma(iommu, iova_end, 0); if (dma && dma->iova + dma->size !=3D iova + size) goto unlock; } @@ -1578,7 +1605,9 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu, { bool set_vaddr =3D map->flags & VFIO_DMA_MAP_FLAG_VADDR; dma_addr_t iova =3D map->iova; + dma_addr_t iova_end; unsigned long vaddr =3D map->vaddr; + unsigned long vaddr_end; size_t size =3D map->size; int ret =3D 0, prot =3D 0; size_t pgsize; @@ -1586,8 +1615,15 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu, =20 /* Verify that none of our __u64 fields overflow */ if (map->size !=3D size || map->vaddr !=3D vaddr || map->iova !=3D iova) + return -EOVERFLOW; + + if (!size) return -EINVAL; =20 + if (check_add_overflow(iova, size - 1, &iova_end) || + check_add_overflow(vaddr, size - 1, &vaddr_end)) + return -EOVERFLOW; + /* READ/WRITE from device perspective */ if (map->flags & VFIO_DMA_MAP_FLAG_WRITE) prot |=3D IOMMU_WRITE; @@ -1603,13 +1639,7 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu, =20 WARN_ON((pgsize - 1) & PAGE_MASK); =20 - if (!size || (size | iova | vaddr) & (pgsize - 1)) { - ret =3D -EINVAL; - goto out_unlock; - } - - /* Don't allow IOVA or virtual address wrap */ - if (iova + size - 1 < iova || vaddr + size - 1 < vaddr) { + if ((size | iova | vaddr) & (pgsize - 1)) { ret =3D -EINVAL; goto out_unlock; } @@ -1640,7 +1670,7 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu, goto out_unlock; } =20 - if (!vfio_iommu_iova_dma_valid(iommu, iova, iova + size - 1)) { + if (!vfio_iommu_iova_dma_valid(iommu, iova, iova_end)) { ret =3D -EINVAL; goto out_unlock; } @@ -2907,7 +2937,8 @@ static int vfio_iommu_type1_dirty_pages(struct vfio_i= ommu *iommu, struct vfio_iommu_type1_dirty_bitmap_get range; unsigned long pgshift; size_t data_size =3D dirty.argsz - minsz; - size_t iommu_pgsize; + size_t size, iommu_pgsize; + dma_addr_t iova, iova_end; =20 if (!data_size || data_size < sizeof(range)) return -EINVAL; @@ -2916,14 +2947,24 @@ static int vfio_iommu_type1_dirty_pages(struct vfio= _iommu *iommu, sizeof(range))) return -EFAULT; =20 - if (range.iova + range.size < range.iova) + iova =3D range.iova; + size =3D range.size; + + if (iova !=3D range.iova || size !=3D range.size) + return -EOVERFLOW; + + if (!size) return -EINVAL; + + if (check_add_overflow(iova, size - 1, &iova_end)) + return -EOVERFLOW; + if (!access_ok((void __user *)range.bitmap.data, range.bitmap.size)) return -EINVAL; =20 pgshift =3D __ffs(range.bitmap.pgsize); - ret =3D verify_bitmap_size(range.size >> pgshift, + ret =3D verify_bitmap_size(size >> pgshift, range.bitmap.size); if (ret) return ret; @@ -2937,19 +2978,18 @@ static int vfio_iommu_type1_dirty_pages(struct vfio= _iommu *iommu, ret =3D -EINVAL; goto out_unlock; } - if (range.iova & (iommu_pgsize - 1)) { + if (iova & (iommu_pgsize - 1)) { ret =3D -EINVAL; goto out_unlock; } - if (!range.size || range.size & (iommu_pgsize - 1)) { + if (size & (iommu_pgsize - 1)) { ret =3D -EINVAL; goto out_unlock; } =20 if (iommu->dirty_page_tracking) ret =3D vfio_iova_dirty_bitmap(range.bitmap.data, - iommu, range.iova, - range.size, + iommu, iova, size, range.bitmap.pgsize); else ret =3D -EINVAL; --=20 2.47.3 From nobody Fri Dec 19 13:47:15 2025 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C89C2DCF44 for ; Mon, 13 Oct 2025 05:32:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760333571; cv=none; b=UC2o5M91ZTfOlFWLfzupAt4FssTgw46gPAJHcCb5CrXVvryzJVepucJYBBG+uSHX9vZTNy/F6RQ9KFICBO0SsJdPEjO3A8j1bhLZaW9eehQ5OIsd8G+S5TPXmstAZadBYZg5sG4RndwQCq7cvkcnD/CbOhlfMl9VEtzLz41Xr3w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760333571; c=relaxed/simple; bh=bL2Lm2n4e9930GjnhR9kYIAZMluDFF1rbD3fEq8ntKM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-ID:References: In-Reply-To:To:CC; b=PXb40g3ltyoYYmR0nwee2xtyu9Vx0wodJ2iVsYHcE8cUlm90rF2IA8xfJJ0TfS0lwRen+jZA0yFj9VeZq0bWAQFvZX4BtoGLnHO07EmhX1JonQeKIyYk8kuy3Nbk071ErSJ1A+AiaJk18a+oe7G3TdQP3gNtXQIZgVtEkF+fses= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=fb.com header.i=@fb.com header.b=sgaXgrnB; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fb.com header.i=@fb.com header.b="sgaXgrnB" Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 59D3DWht536898 for ; Sun, 12 Oct 2025 22:32:48 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2025-q2; bh=9P0Pku55fBiFAorYlOSjBsBGriWgJG02HANwpST4hjY=; b=sgaXgrnBRCJI yNXHOkA9WIDcYkpxODIHQAAEyZsxODY65tUKsGSpE3SAJF3QqAA7skBhzdL6FX+E e2awIN1Hu3iBOe7y8zNjyPMHWOoysCh9B2AX+JXqI5YVSuJ7veg/8xbgMwb1QuGf xUi5PC5DQO1taLPDY4XZZzsDgCo0Vf2dxP6442FO0BwiEQKTc6uw3M/4hSukE+bU c2FonPCywb9nh6VkSxLrSUSlkm0XYmcOj0r981vU3es/dw4KUO2ZGNNgxzG/1lR4 /FfkXhDnyf2rEMeXiujYZ5XDsO4eNXHMSrIcNpkOkEwY52+yrHBDAZLfQQ/bSW6n D3+oN5S4iQ== Received: from maileast.thefacebook.com ([163.114.135.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 49rsb6gdyw-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Sun, 12 Oct 2025 22:32:48 -0700 (PDT) Received: from twshared38445.28.prn2.facebook.com (2620:10d:c0a8:1b::8e35) by mail.thefacebook.com (2620:10d:c0a9:6f::8fd4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.20; Mon, 13 Oct 2025 05:32:44 +0000 Received: by devgpu015.cco6.facebook.com (Postfix, from userid 199522) id 9E4A4102FE48; Sun, 12 Oct 2025 22:32:30 -0700 (PDT) From: Alex Mastro Date: Sun, 12 Oct 2025 22:32:25 -0700 Subject: [PATCH v4 2/3] vfio/type1: move iova increment to unmap_unpin_* caller Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-ID: <20251012-fix-unmap-v4-2-9eefc90ed14c@fb.com> References: <20251012-fix-unmap-v4-0-9eefc90ed14c@fb.com> In-Reply-To: <20251012-fix-unmap-v4-0-9eefc90ed14c@fb.com> To: Alex Williamson CC: Jason Gunthorpe , Alejandro Jimenez , , , Alex Mastro X-Mailer: b4 0.13.0 X-FB-Internal: Safe X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMDEzMDAyNiBTYWx0ZWRfX/68TFfxS9wKo U5D7OAo4dnBkryzl4LKxpYYsnfuIitO6tQsmq6bx7sQI6soiD3nuxJ3Yq0nSaIkKi4rQpqhAhvC fKKcTdULfQIEr8sxoJQ6XtSyMJcmVIfERjk+Is77Vnt7deS5Kdj4CVC+ypCFYWAmrYm++8fnNlY gfqNllojb6H8HL1zTyQUvDy3tI1Ju1Bm95jCq5vfvgMSAts0McW85XA/QNTxlTTmMuw/ODsI0VS meJM6aFhdoSbXi7DPha2kSgtCwxuY8I6zsLVPtYV+AROE5tItOqvHLx6SPlHZyKJmC7wQJDM+P5 VixXoycbtETXIxXOElpTt4Kb4hQFihI5EFeWB0flqQ9+pBqmmhjuyc9Jj9wkLDHSdPuX4XhgnDW DJLurvWsyiROqbQUkbzUeDiODPm4qg== X-Proofpoint-ORIG-GUID: ANX-ujIdqNxi6tuZf2PZSJxnihRRVEuv X-Authority-Analysis: v=2.4 cv=BarVE7t2 c=1 sm=1 tr=0 ts=68ec8f00 cx=c_pps a=MfjaFnPeirRr97d5FC5oHw==:117 a=MfjaFnPeirRr97d5FC5oHw==:17 a=IkcTkHD0fZMA:10 a=x6icFKpwvdMA:10 a=FOH2dFAWAAAA:8 a=Bkl4-CcWoNymJLHaMFkA:9 a=QEXdDO2ut3YA:10 X-Proofpoint-GUID: ANX-ujIdqNxi6tuZf2PZSJxnihRRVEuv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-10-13_02,2025-10-06_01,2025-03-28_01 Move incrementing iova to the caller of these functions as part of preparing to handle end of address space map/unmap. Signed-off-by: Alex Mastro Reviewed-by: Alejandro Jimenez Reviewed-by: Jason Gunthorpe Tested-by: Alejandro Jimenez --- drivers/vfio/vfio_iommu_type1.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type= 1.c index 1ac056b27f27..48b84a7af2e1 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -1013,7 +1013,7 @@ static long vfio_sync_unpin(struct vfio_dma *dma, str= uct vfio_domain *domain, #define VFIO_IOMMU_TLB_SYNC_MAX 512 =20 static size_t unmap_unpin_fast(struct vfio_domain *domain, - struct vfio_dma *dma, dma_addr_t *iova, + struct vfio_dma *dma, dma_addr_t iova, size_t len, phys_addr_t phys, long *unlocked, struct list_head *unmapped_list, int *unmapped_cnt, @@ -1023,18 +1023,17 @@ static size_t unmap_unpin_fast(struct vfio_domain *= domain, struct vfio_regions *entry =3D kzalloc(sizeof(*entry), GFP_KERNEL); =20 if (entry) { - unmapped =3D iommu_unmap_fast(domain->domain, *iova, len, + unmapped =3D iommu_unmap_fast(domain->domain, iova, len, iotlb_gather); =20 if (!unmapped) { kfree(entry); } else { - entry->iova =3D *iova; + entry->iova =3D iova; entry->phys =3D phys; entry->len =3D unmapped; list_add_tail(&entry->list, unmapped_list); =20 - *iova +=3D unmapped; (*unmapped_cnt)++; } } @@ -1053,18 +1052,17 @@ static size_t unmap_unpin_fast(struct vfio_domain *= domain, } =20 static size_t unmap_unpin_slow(struct vfio_domain *domain, - struct vfio_dma *dma, dma_addr_t *iova, + struct vfio_dma *dma, dma_addr_t iova, size_t len, phys_addr_t phys, long *unlocked) { - size_t unmapped =3D iommu_unmap(domain->domain, *iova, len); + size_t unmapped =3D iommu_unmap(domain->domain, iova, len); =20 if (unmapped) { - *unlocked +=3D vfio_unpin_pages_remote(dma, *iova, + *unlocked +=3D vfio_unpin_pages_remote(dma, iova, phys >> PAGE_SHIFT, unmapped >> PAGE_SHIFT, false); - *iova +=3D unmapped; cond_resched(); } return unmapped; @@ -1127,16 +1125,18 @@ static long vfio_unmap_unpin(struct vfio_iommu *iom= mu, struct vfio_dma *dma, * First, try to use fast unmap/unpin. In case of failure, * switch to slow unmap/unpin path. */ - unmapped =3D unmap_unpin_fast(domain, dma, &iova, len, phys, + unmapped =3D unmap_unpin_fast(domain, dma, iova, len, phys, &unlocked, &unmapped_region_list, &unmapped_region_cnt, &iotlb_gather); if (!unmapped) { - unmapped =3D unmap_unpin_slow(domain, dma, &iova, len, + unmapped =3D unmap_unpin_slow(domain, dma, iova, len, phys, &unlocked); if (WARN_ON(!unmapped)) break; } + + iova +=3D unmapped; } =20 dma->iommu_mapped =3D false; --=20 2.47.3 From nobody Fri Dec 19 13:47:15 2025 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D5D442DA753 for ; Mon, 13 Oct 2025 05:32:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760333567; cv=none; b=euikDJ0T+pThw/B+nph750eNM8j0xcIO/Gv7tpmqVCs5dtYR6J7EmEqkYJEnDQFD591Hxp9Aw/kwqNaWMRVwYeFxbO7uep05lUb5Ix8gEJEkCO2v1EwBEMfzYUCDfzGZraLRSUE29rtaOAdjcUpYxk3J10c50kEGJisuxkpgW9Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760333567; c=relaxed/simple; bh=4hYhb+EPmlaOc3bN+TTQRMFugJwG6oxY36pVB85jVgQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-ID:References: In-Reply-To:To:CC; b=bR8xVOfDy19MDv4+tceBGim7x24Li0w5OcSzdkrGTf8b9mHPyUd130CutaEPOX0T6icH+ijM7rz1mOsTZABbhb6A1a8lkEEK3e+n0/j3bnMhyyzPpUUK779ss6tEATBSsZJeNt1oGgeLnzM/F6P5BA6gV/54raWHnh29kl/OWv0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=fb.com header.i=@fb.com header.b=GgqCHrje; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fb.com header.i=@fb.com header.b="GgqCHrje" Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 59CMgNrI1450873 for ; Sun, 12 Oct 2025 22:32:45 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2025-q2; bh=5W/oWlGSgrgcpquGT5WzQZk6fkYCHamp032I6BK5EHA=; b=GgqCHrjeo6Dd Cxuia4OULEpPoPjM39fFVIHZaA/PW8whz1mKEqQhh7awmMa0tGvsIhuKPruYJYia i1EwkH3FFSUijxwIYgL6/txAJxcdeetzbO0BoI6cJQblQvGlRgOMHUD8YCXkKLS3 /4oYCtIpoI4cYLuVwt/pl8D4mf53WHTbBdxasTVIx//R9XL9af1+mcmO3rWddRVb RkWePLcePeDHGduBLPDqtqN0nRD66MsOGq0E9HFxOOm6oB+yd3NOrp5vmzbS08uq IX2TH4aFXP93mDDJPTm+6Lz1J0LiaXwNzfjRGMonOvG84A922+mJLdhQ8hVIcgMG faFolgT3Mg== Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 49rnbm172b-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Sun, 12 Oct 2025 22:32:44 -0700 (PDT) Received: from twshared28243.32.prn2.facebook.com (2620:10d:c085:208::7cb7) by mail.thefacebook.com (2620:10d:c08b:78::2ac9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.20; Mon, 13 Oct 2025 05:32:44 +0000 Received: by devgpu015.cco6.facebook.com (Postfix, from userid 199522) id 9E654102FE49; Sun, 12 Oct 2025 22:32:30 -0700 (PDT) From: Alex Mastro Date: Sun, 12 Oct 2025 22:32:26 -0700 Subject: [PATCH v4 3/3] vfio/type1: handle DMA map/unmap up to the addressable limit Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-ID: <20251012-fix-unmap-v4-3-9eefc90ed14c@fb.com> References: <20251012-fix-unmap-v4-0-9eefc90ed14c@fb.com> In-Reply-To: <20251012-fix-unmap-v4-0-9eefc90ed14c@fb.com> To: Alex Williamson CC: Jason Gunthorpe , Alejandro Jimenez , , , Alex Mastro X-Mailer: b4 0.13.0 X-FB-Internal: Safe X-Proofpoint-GUID: oIAy4rlBuigx-9in3aAUaUiiGqGth9ff X-Authority-Analysis: v=2.4 cv=NfjrFmD4 c=1 sm=1 tr=0 ts=68ec8efd cx=c_pps a=CB4LiSf2rd0gKozIdrpkBw==:117 a=CB4LiSf2rd0gKozIdrpkBw==:17 a=IkcTkHD0fZMA:10 a=x6icFKpwvdMA:10 a=FOH2dFAWAAAA:8 a=P34euavMcYKjluB8mAYA:9 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: oIAy4rlBuigx-9in3aAUaUiiGqGth9ff X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMDEzMDAyNiBTYWx0ZWRfX5ggvlzXn+AtQ ZoE7Na2oCyB14AWIuDcWZ6lPPf6vqkvp3Rx7X+MC9p2JMq8SYT92+DMkjzKbDcFqJrWQGi2H+Lm swKiA5XlXuLYfWHWjnNy01ugiVsea8wjV76CCQLf1YE5Dj98T8dlK2D3xlfwmU2FGyhAKMJ3Ms2 2A/o7Ap6YB0DUTJ1zi8Rf0U/hxI+ar0BpJX7b24SPgJWMga1ncvtnxjF+IXygM6XOpUtTXNbNOg HCDWfU5TNDpC+VseBAQI5ZO33b6mURhlcoY7mtxVsS7cHMZLx+dJg0/oEqvXEeHrKmWKWdUE0YT rMl8Kx7Cy9QeAt6coP7PSQvV6wgzWikMxZx+q3gWyzQnX0R1nBxt39Z3gco2cvqwsFFI0lQi+II hXqKoT8TgvdxbuWmn+5ULPCpLVrIcQ== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-10-13_02,2025-10-06_01,2025-03-28_01 Handle DMA map/unmap operations up to the addressable limit by comparing against inclusive end-of-range limits, and changing iteration to perform relative traversals across range sizes, rather than absolute traversals across addresses. vfio_link_dma inserts a zero-sized vfio_dma into the rb-tree, and is only used for that purpose, so discard the size from consideration for the insertion point. Signed-off-by: Alex Mastro Reviewed-by: Alejandro Jimenez Reviewed-by: Jason Gunthorpe Tested-by: Alejandro Jimenez --- drivers/vfio/vfio_iommu_type1.c | 77 ++++++++++++++++++++++---------------= ---- 1 file changed, 42 insertions(+), 35 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type= 1.c index 48b84a7af2e1..a65625dcf708 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -166,12 +166,14 @@ static struct vfio_dma *vfio_find_dma(struct vfio_iom= mu *iommu, { struct rb_node *node =3D iommu->dma_list.rb_node; =20 + WARN_ON(!size); + while (node) { struct vfio_dma *dma =3D rb_entry(node, struct vfio_dma, node); =20 - if (start + size <=3D dma->iova) + if (start + size - 1 < dma->iova) node =3D node->rb_left; - else if (start >=3D dma->iova + dma->size) + else if (start > dma->iova + dma->size - 1) node =3D node->rb_right; else return dma; @@ -181,16 +183,19 @@ static struct vfio_dma *vfio_find_dma(struct vfio_iom= mu *iommu, } =20 static struct rb_node *vfio_find_dma_first_node(struct vfio_iommu *iommu, - dma_addr_t start, size_t size) + dma_addr_t start, + dma_addr_t end) { struct rb_node *res =3D NULL; struct rb_node *node =3D iommu->dma_list.rb_node; struct vfio_dma *dma_res =3D NULL; =20 + WARN_ON(end < start); + while (node) { struct vfio_dma *dma =3D rb_entry(node, struct vfio_dma, node); =20 - if (start < dma->iova + dma->size) { + if (start <=3D dma->iova + dma->size - 1) { res =3D node; dma_res =3D dma; if (start >=3D dma->iova) @@ -200,7 +205,7 @@ static struct rb_node *vfio_find_dma_first_node(struct = vfio_iommu *iommu, node =3D node->rb_right; } } - if (res && size && dma_res->iova >=3D start + size) + if (res && dma_res->iova > end) res =3D NULL; return res; } @@ -210,11 +215,13 @@ static void vfio_link_dma(struct vfio_iommu *iommu, s= truct vfio_dma *new) struct rb_node **link =3D &iommu->dma_list.rb_node, *parent =3D NULL; struct vfio_dma *dma; =20 + WARN_ON(new->size !=3D 0); + while (*link) { parent =3D *link; dma =3D rb_entry(parent, struct vfio_dma, node); =20 - if (new->iova + new->size <=3D dma->iova) + if (new->iova <=3D dma->iova) link =3D &(*link)->rb_left; else link =3D &(*link)->rb_right; @@ -1071,12 +1078,12 @@ static size_t unmap_unpin_slow(struct vfio_domain *= domain, static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dm= a, bool do_accounting) { - dma_addr_t iova =3D dma->iova, end =3D dma->iova + dma->size; struct vfio_domain *domain, *d; LIST_HEAD(unmapped_region_list); struct iommu_iotlb_gather iotlb_gather; int unmapped_region_cnt =3D 0; long unlocked =3D 0; + size_t pos =3D 0; =20 if (!dma->size) return 0; @@ -1100,13 +1107,14 @@ static long vfio_unmap_unpin(struct vfio_iommu *iom= mu, struct vfio_dma *dma, } =20 iommu_iotlb_gather_init(&iotlb_gather); - while (iova < end) { + while (pos < dma->size) { size_t unmapped, len; phys_addr_t phys, next; + dma_addr_t iova =3D dma->iova + pos; =20 phys =3D iommu_iova_to_phys(domain->domain, iova); if (WARN_ON(!phys)) { - iova +=3D PAGE_SIZE; + pos +=3D PAGE_SIZE; continue; } =20 @@ -1115,7 +1123,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu= , struct vfio_dma *dma, * may require hardware cache flushing, try to find the * largest contiguous physical memory chunk to unmap. */ - for (len =3D PAGE_SIZE; iova + len < end; len +=3D PAGE_SIZE) { + for (len =3D PAGE_SIZE; pos + len < dma->size; len +=3D PAGE_SIZE) { next =3D iommu_iova_to_phys(domain->domain, iova + len); if (next !=3D phys + len) break; @@ -1136,7 +1144,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu= , struct vfio_dma *dma, break; } =20 - iova +=3D unmapped; + pos +=3D unmapped; } =20 dma->iommu_mapped =3D false; @@ -1228,7 +1236,7 @@ static int update_user_bitmap(u64 __user *bitmap, str= uct vfio_iommu *iommu, } =20 static int vfio_iova_dirty_bitmap(u64 __user *bitmap, struct vfio_iommu *i= ommu, - dma_addr_t iova, size_t size, size_t pgsize) + dma_addr_t iova, dma_addr_t iova_end, size_t pgsize) { struct vfio_dma *dma; struct rb_node *n; @@ -1245,8 +1253,8 @@ static int vfio_iova_dirty_bitmap(u64 __user *bitmap,= struct vfio_iommu *iommu, if (dma && dma->iova !=3D iova) return -EINVAL; =20 - dma =3D vfio_find_dma(iommu, iova + size - 1, 0); - if (dma && dma->iova + dma->size !=3D iova + size) + dma =3D vfio_find_dma(iommu, iova_end, 1); + if (dma && dma->iova + dma->size - 1 !=3D iova_end) return -EINVAL; =20 for (n =3D rb_first(&iommu->dma_list); n; n =3D rb_next(n)) { @@ -1255,7 +1263,7 @@ static int vfio_iova_dirty_bitmap(u64 __user *bitmap,= struct vfio_iommu *iommu, if (dma->iova < iova) continue; =20 - if (dma->iova > iova + size - 1) + if (dma->iova > iova_end) break; =20 ret =3D update_user_bitmap(bitmap, iommu, dma, iova, pgsize); @@ -1348,7 +1356,7 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iommu, if (unmap_all) { if (iova || size) goto unlock; - size =3D SIZE_MAX; + iova_end =3D ~(dma_addr_t)0; } else { if (!size || size & (pgsize - 1)) goto unlock; @@ -1403,17 +1411,17 @@ static int vfio_dma_do_unmap(struct vfio_iommu *iom= mu, if (dma && dma->iova !=3D iova) goto unlock; =20 - dma =3D vfio_find_dma(iommu, iova_end, 0); - if (dma && dma->iova + dma->size !=3D iova + size) + dma =3D vfio_find_dma(iommu, iova_end, 1); + if (dma && dma->iova + dma->size - 1 !=3D iova_end) goto unlock; } =20 ret =3D 0; - n =3D first_n =3D vfio_find_dma_first_node(iommu, iova, size); + n =3D first_n =3D vfio_find_dma_first_node(iommu, iova, iova_end); =20 while (n) { dma =3D rb_entry(n, struct vfio_dma, node); - if (dma->iova >=3D iova + size) + if (dma->iova > iova_end) break; =20 if (!iommu->v2 && iova > dma->iova) @@ -1743,12 +1751,12 @@ static int vfio_iommu_replay(struct vfio_iommu *iom= mu, =20 for (; n; n =3D rb_next(n)) { struct vfio_dma *dma; - dma_addr_t iova; + size_t pos =3D 0; =20 dma =3D rb_entry(n, struct vfio_dma, node); - iova =3D dma->iova; =20 - while (iova < dma->iova + dma->size) { + while (pos < dma->size) { + dma_addr_t iova =3D dma->iova + pos; phys_addr_t phys; size_t size; =20 @@ -1764,14 +1772,14 @@ static int vfio_iommu_replay(struct vfio_iommu *iom= mu, phys =3D iommu_iova_to_phys(d->domain, iova); =20 if (WARN_ON(!phys)) { - iova +=3D PAGE_SIZE; + pos +=3D PAGE_SIZE; continue; } =20 size =3D PAGE_SIZE; p =3D phys + size; i =3D iova + size; - while (i < dma->iova + dma->size && + while (pos + size < dma->size && p =3D=3D iommu_iova_to_phys(d->domain, i)) { size +=3D PAGE_SIZE; p +=3D PAGE_SIZE; @@ -1779,9 +1787,8 @@ static int vfio_iommu_replay(struct vfio_iommu *iommu, } } else { unsigned long pfn; - unsigned long vaddr =3D dma->vaddr + - (iova - dma->iova); - size_t n =3D dma->iova + dma->size - iova; + unsigned long vaddr =3D dma->vaddr + pos; + size_t n =3D dma->size - pos; long npage; =20 npage =3D vfio_pin_pages_remote(dma, vaddr, @@ -1812,7 +1819,7 @@ static int vfio_iommu_replay(struct vfio_iommu *iommu, goto unwind; } =20 - iova +=3D size; + pos +=3D size; } } =20 @@ -1829,29 +1836,29 @@ static int vfio_iommu_replay(struct vfio_iommu *iom= mu, unwind: for (; n; n =3D rb_prev(n)) { struct vfio_dma *dma =3D rb_entry(n, struct vfio_dma, node); - dma_addr_t iova; + size_t pos =3D 0; =20 if (dma->iommu_mapped) { iommu_unmap(domain->domain, dma->iova, dma->size); continue; } =20 - iova =3D dma->iova; - while (iova < dma->iova + dma->size) { + while (pos < dma->size) { + dma_addr_t iova =3D dma->iova + pos; phys_addr_t phys, p; size_t size; dma_addr_t i; =20 phys =3D iommu_iova_to_phys(domain->domain, iova); if (!phys) { - iova +=3D PAGE_SIZE; + pos +=3D PAGE_SIZE; continue; } =20 size =3D PAGE_SIZE; p =3D phys + size; i =3D iova + size; - while (i < dma->iova + dma->size && + while (pos + size < dma->size && p =3D=3D iommu_iova_to_phys(domain->domain, i)) { size +=3D PAGE_SIZE; p +=3D PAGE_SIZE; @@ -2989,7 +2996,7 @@ static int vfio_iommu_type1_dirty_pages(struct vfio_i= ommu *iommu, =20 if (iommu->dirty_page_tracking) ret =3D vfio_iova_dirty_bitmap(range.bitmap.data, - iommu, iova, size, + iommu, iova, iova_end, range.bitmap.pgsize); else ret =3D -EINVAL; --=20 2.47.3