From nobody Thu Oct 9 04:52:43 2025 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD9CD1DED5C for ; Fri, 20 Jun 2025 03:24:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750389853; cv=none; b=VaRj72QzmUXWeKq3Jr9w55dxbG9jIOW//GknQCvjLybGErUHLryRLwbMo577UCdtjpsDXiIW6X3qqa3MbBNOwAopjrYrh8Tg+QSkBeZXIzcW/xfY0tlbuN5YC1timbD15Ik8ZdNscTJTntdAL/DKc/7nOJGQhnXEWLBSjncLzRU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750389853; c=relaxed/simple; bh=uV3cHhXrkTvBqdB8VYyLYO8kapwhTK0/BJjfokGJUfw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rR9QBplp+AJmE18019htkIBVIW7caqoGSxzT/WEOE9layh1hUhf3uWb0je+XYsjYj2rjlO2gIfbGqbMKg6cnpyi1JTihucjtm0AdnPtNS/UfSmnbQBSSQpOnueipLmoxCuY0EHuU5nfBlvR3fpD6tsAjWyr/614ttjwnW2Io/Q8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=Ej4mijS4; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="Ej4mijS4" Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-74264d1832eso1725181b3a.0 for ; Thu, 19 Jun 2025 20:24:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1750389850; x=1750994650; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DSlkXQNydyRdWXir99epy9/y/pcz1kH+JURzeIlDKkA=; b=Ej4mijS4RguUTdUbbaLOvuxEsljl/l/1iHRMlpTICXszE6evvhFc5rsBHh5A0GDPIy rm30X/nF6cpOgmAVbZOfUy2xoCWIiiggtWnXEeh8PdTPABjM76pYy43n1M04Xw8Cc3NV VwfYxOyODA2MN58ml6vts3HfRjJUN14dyO23MrdXi0KaLtCArihoIbu89ZtqphCUPa5l Ujto4oLw3XUKrgO9qS68Ml8VCkpO/JyJSAb+D/PREvvZXWMvcfRBWyc4L4Gb/x4lmx+7 oYFNBV/zWQ8dRYVIQ7AxXh7im/dMQw8rgt4e1ghPrOzZFFwZ2LT5U5yGLA6pHzuze2y7 6MbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750389850; x=1750994650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DSlkXQNydyRdWXir99epy9/y/pcz1kH+JURzeIlDKkA=; b=WsHdl4ySoHg8eVa7coDsl1tC04Z8Ghw9bUCWqW2WETPET9VfkU/4CFnpFMPxEQxMKr KQF/sidbGN2xbXdyd7QLET1qA3LC6xtLG29HKnchiziufdpPZY9Xmvl5mdLbpfBOy4ru 32vNKZ7Q6SMi8XxSNG6AzGMdNZUlWrJrAbYr6093PWCq41mf00/hshq0yQ7WTNfn0WQR eVkR5oHJHXtGp9a5WJa1Qkhf2gv2ES1dtLFFBwFWSKjYHSw+6blvizqNFkr4J8bxwayP ZiDCVvBLc5rFUdbaA/Szc4SgWZhoPB946ftBr9Tj9sy60dZI7NfaUfF0L1C8qBHZOWhM IZQw== X-Forwarded-Encrypted: i=1; AJvYcCVtQWkBLRjRjD4AR5Jtc57Z0UOQ5voLkqSRbBRzGznbGesnMxYiok5A+FhukFJJNzdGirWaVsIS+QZKzsA=@vger.kernel.org X-Gm-Message-State: AOJu0Yy2zSXmnzCO/BE/XCFtaKBO13qORUBUH6LBDUkqeho854S6Jx4z 3TOCmU6e7QrsKKgv/UGujkkTAhS1hxdy7gieCUX5F3r2x8tcdHEp6331BBMgO022zl8= X-Gm-Gg: ASbGnctl644HTbbO/cbSxY/qj/SdWTgkxxYVIwiDmH6Q/MStTV//cowFX9IVjIkYzrz lXM3xrM20da56b9+QvUc7bmt4zFpFKiZJzMA3UZ2mrpDmQRIU8LkQVPdJ9+kW9EXpFfS9LpAWVn EWgIyDJiIUDBmZ9jVnb/zCUdjcdZh6Nh7+odklNVudrwYN5u/Mua9VtW4t3IfBXu7F7ypoR1XVQ TshV/viXGvt2k3aXrAkhQVRsDoTG/g+rtWx4i/wNxcUw58PpimoW/ZEYbkfNZk179Qb03uUndct JTjszXGomAAxn91AcuwsLfBux1RONYP+XTbaVQEvt/lKWehj72hB8DABxEp+yVmtFiMkDZde/DY kG4Jz612vkFLr5YCW3jfAiIw= X-Google-Smtp-Source: AGHT+IEsXcfjbayevRNoihfpHC+h/sIDZOjdQav72EKNFmITtaUo3YN4YHLpYj329boEKbDYcfD4jg== X-Received: by 2002:a05:6a20:a10c:b0:204:4573:d854 with SMTP id adf61e73a8af0-22026d58631mr2053781637.9.1750389850193; Thu, 19 Jun 2025 20:24:10 -0700 (PDT) Received: from localhost.localdomain ([203.208.189.7]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-b31f12423e3sm490565a12.47.2025.06.19.20.24.07 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 19 Jun 2025 20:24:09 -0700 (PDT) From: lizhe.67@bytedance.com To: alex.williamson@redhat.com, jgg@ziepe.ca, david@redhat.com Cc: peterx@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, lizhe.67@bytedance.com Subject: [PATCH v5 1/3] vfio/type1: batch vfio_find_vpfn() in function vfio_unpin_pages_remote() Date: Fri, 20 Jun 2025 11:23:42 +0800 Message-ID: <20250620032344.13382-2-lizhe.67@bytedance.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250620032344.13382-1-lizhe.67@bytedance.com> References: <20250620032344.13382-1-lizhe.67@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Li Zhe This patch is based on patch 'vfio/type1: optimize vfio_pin_pages_remote() for large folios'[1]. The function vpfn_pages() can help us determine the number of vpfn nodes on the vpfn rb tree within a specified range. This allows us to avoid searching for each vpfn individually in the function vfio_unpin_pages_remote(). This patch batches the vfio_find_vpfn() calls in function vfio_unpin_pages_remote(). [1]: https://lore.kernel.org/all/20250529064947.38433-1-lizhe.67@bytedance.= com/ Signed-off-by: Li Zhe --- drivers/vfio/vfio_iommu_type1.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type= 1.c index 28ee4b8d39ae..e952bf8bdfab 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -805,16 +805,12 @@ static long vfio_unpin_pages_remote(struct vfio_dma *= dma, dma_addr_t iova, unsigned long pfn, unsigned long npage, bool do_accounting) { - long unlocked =3D 0, locked =3D 0; + long unlocked =3D 0, locked =3D vpfn_pages(dma, iova, npage); long i; =20 - for (i =3D 0; i < npage; i++, iova +=3D PAGE_SIZE) { - if (put_pfn(pfn++, dma->prot)) { + for (i =3D 0; i < npage; i++) + if (put_pfn(pfn++, dma->prot)) unlocked++; - if (vfio_find_vpfn(dma, iova)) - locked++; - } - } =20 if (do_accounting) vfio_lock_acct(dma, locked - unlocked, true); --=20 2.20.1 From nobody Thu Oct 9 04:52:43 2025 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 759F21DD0EF for ; Fri, 20 Jun 2025 03:24:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750389859; cv=none; b=KnflWHx77+qrG5rlaiXfjqqUOM5x9TCyWNzXqapdBx0vJZ89ZfUHfb0Kmv0KHZlD4Ga8ciJXuS0iFojpmL0tm6FZLE/dE51fKZRSItxHoSDbP4VEgi5U1yHoNdDQp1t0ZgI6AfOQtQ8LzD5ZBwgbC/yZFoMuXy1RPm+IP4AQc1c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750389859; c=relaxed/simple; bh=aVi+q7O/crMw2zBHRVPGvYU26xciOHQulJaxqwWbTeo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u/ZWbJji7d8pthUW6/V+y5mPlaA9b8wvAReHIOClNAGRuno1sUV5tYp4KcH5KdPI244cfDbgVubJYfK8TqkgFNW4+5IoO5FN0wSTUlW+YB3XTQo9a6aRKdO7uZ1lpuip3Gx4my8GZJRuXZCa2RHUzFo4yvhrJf1j0TNP6nLSsIM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=GEv4zMUu; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="GEv4zMUu" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-747fba9f962so993723b3a.0 for ; Thu, 19 Jun 2025 20:24:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1750389855; x=1750994655; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nA8lu1ILmrG85n9fJ0bjb2wIucT5x1eh7C3/xgb3MvM=; b=GEv4zMUuvMrSyBBMOaakEMNj5WP580hzoZihSL0BZoi9eMFc7Ier6Nj2VfgAEono02 POHM5fBuRIzL8bh3EzlHuhS9gui+ZWicusAJbACLRPE7hlFICKmcJDuEjNDC4nFry9SC 7pLQj/WStR41RBkCEyipImiq060HAAWAOjlIAQRDDEmkLW4Fw9s1TN2xCFONa7zBsMgi Q8iz68q33oocaxSRYPdJhHb4ucdc5w9B56JvTpO2VABGpOpvdiwrNCX4bQrt6FKiIr3Y kTvB/fEvus0n6L6Usu6VwgJOAjIcrs9has8mz89KR7NuJZ6BK0UkaKN5OcFHAF9MHhwm /j5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750389855; x=1750994655; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nA8lu1ILmrG85n9fJ0bjb2wIucT5x1eh7C3/xgb3MvM=; b=lX5oll9RFSVsxG67hbQysbPPg+KRGm21YpC0AMOxStdeuGUmRMiiM2xl25fTKabjD/ GpFax7JFngUuw7v1kYx70CS70VffZxQzPaeu1NEZBc8LTJP7V1Etb6Se2YjbllVImDLJ jt/Y4Ujo/RoO4l/5yaMSRAUC4uDstEP5ACj6ZK9qcUw0JfhhNOOKbTxN7XSfL1xmKMUG 9vCQ6ros2uQhnF7FM95HhYQQggLKM506QwMxpj0L28lQ+Iwle3nR8Kw9xtJ7QPrujWBd 6VGBorXXNbI4SZRtKP444ZLQHlhViBIxWOMorTdfo3WKkbS3+gO0S94GC4cIVYDgPihh ljiA== X-Forwarded-Encrypted: i=1; AJvYcCXnpsSaNisuZScFkPqvap7OV6LyhLFqpNJ/c/WJsII3Z3HSbt+vo4NiQMwggtLNFKIfL4oKJhF1uUoeV5A=@vger.kernel.org X-Gm-Message-State: AOJu0Ywq7Wnt+1x9lvErN6HR0dbiV2XM03uvmotxHxikZyhNAbxNu6nX 4XdrcRHK2RBFsGQupKN6Bmxu2sU0gCiJT0Bsw5XJyfqvkrhEGSBG+Qw0Vrj8TULBhhItJTiAma5 9tK02 X-Gm-Gg: ASbGncuZAyZmvhuNlifvdUCPdXRM6ZuL1HyOdLFnwMrD+g9H16eTtqZWbRjOQ6z16U3 5HQ4hP2i1MyF6NhrrrXPklAhaYdzCORFBY+khHi9utntiu+Fs2ZAS80IbMy0tSP4Jv3+TB+4Giy iAbQvR3CqmFr3Neo0A0XPhQzWsfpSBBDebYEx29UfawTOxgzv1oEw99blkajJ4CLU7Mlr65c+p6 mMRniIT1+yVBitzWFhztbra3GgcIkhBkfyZvL/DHSl1ovBX67uzKkVakWV+eD0e/FCb7ZHF44pw iOUFRs5L49euf4Ai2Upbw6REl8VKNOK7J3hF1CB6sv2ivy6oIrfTTWZyHiPGmP3Q8RPNbj+g67N R+RgaPMNP75dvC/SfLYLVw1s= X-Google-Smtp-Source: AGHT+IErDrjxvnczjGX7Ziw3nenkBC6Zy/yUAkwXq28ayHDSFAw6u+ds5wFBRcu3M1Dusg9exASQ7g== X-Received: by 2002:a05:6a00:9496:b0:746:195b:bf1c with SMTP id d2e1a72fcca58-7490f5a435emr1097281b3a.10.1750389854555; Thu, 19 Jun 2025 20:24:14 -0700 (PDT) Received: from localhost.localdomain ([203.208.189.7]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-b31f12423e3sm490565a12.47.2025.06.19.20.24.11 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 19 Jun 2025 20:24:14 -0700 (PDT) From: lizhe.67@bytedance.com To: alex.williamson@redhat.com, jgg@ziepe.ca, david@redhat.com Cc: peterx@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, lizhe.67@bytedance.com Subject: [PATCH v5 2/3] vfio/type1: introduce a new member has_rsvd for struct vfio_dma Date: Fri, 20 Jun 2025 11:23:43 +0800 Message-ID: <20250620032344.13382-3-lizhe.67@bytedance.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250620032344.13382-1-lizhe.67@bytedance.com> References: <20250620032344.13382-1-lizhe.67@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Li Zhe Introduce a new member has_rsvd for struct vfio_dma. This member is used to indicate whether there are any reserved or invalid pfns in the region represented by this vfio_dma. If it is true, it indicates that there is at least one pfn in this region that is either reserved or invalid. Signed-off-by: Li Zhe --- drivers/vfio/vfio_iommu_type1.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type= 1.c index e952bf8bdfab..8827e315e3d8 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -93,6 +93,10 @@ struct vfio_dma { bool iommu_mapped; bool lock_cap; /* capable(CAP_IPC_LOCK) */ bool vaddr_invalid; + /* + * Any reserved or invalid pfns within this range? + */ + bool has_rsvd; struct task_struct *task; struct rb_root pfn_list; /* Ex-user pinned pfn list */ unsigned long *bitmap; @@ -785,6 +789,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma,= unsigned long vaddr, } =20 out: + dma->has_rsvd |=3D rsvd; ret =3D vfio_lock_acct(dma, lock_acct, false); =20 unpin_out: --=20 2.20.1 From nobody Thu Oct 9 04:52:43 2025 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BAFFA1E835D for ; Fri, 20 Jun 2025 03:24:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750389862; cv=none; b=PbKp3xuMeEmOsgISngmBR2RHQNAiItUIuIhuynPLZQU33pwHNaTQHs+7ZI7wpHRJ/w96esq3TZZuJOsxLiQwsfySKhypEcfoGTA/R0VhnG5XGXfNZqEq6rX20KHYuS0TGiW6kq+Pj1o0EnpEEcB4SmVzf3qGwAVfmNzNj2rKklM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750389862; c=relaxed/simple; bh=SYK+UWbrGNX3UbkkR8jkQm/pIxA9wBivQLpXpvCwK4A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rrX5kUIPbQgLvbAFZWHlwmi7Zi8nawg7XiPLnqaX+K0Owe/R8R6qno4VLKdZo0HkpQ6ujxVDRc6NK0R1Oo6lY/CtebhCMYCgvcBzDo0rgN8KmdpS104l431mOPgOKOqb0elRenneluElxw69nnhuF0O7k9ql3YsYHXfBhGuRtE0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=bo6RUVHE; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="bo6RUVHE" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-747fc7506d4so990674b3a.0 for ; Thu, 19 Jun 2025 20:24:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1750389859; x=1750994659; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=z2YzMDv78WPG5/xYOI3b72eSmmaa1xBCqOTfcIeZ+L0=; b=bo6RUVHEWj/aNYwXo+exHdKMyNT9eaDfbshbnp7uwMLRiI5XUuL7N+OocSQAKHCmEJ 5KtGZk0WCGgOovC3XQucCpXaq9pz6/NIY8+6dEyZj920Wku6UgRflxnj1YDImZ9dSCqV gBj5W8q4LEzod2CAgqQbi/iWh02ImbWvos7FqcqBpEXp2xevp1EitbZXfBx0BtJI24Dx WT9T8zsKHA3bDO9Ebo3fyPMIZW0O8Vzi6Mq6U1eD4IK8WL39rzjfsw3yHlsS6Z29PnqM r67c0z0f91pBiCriqyqPbg4Gs+TOY+M258LLJvMEmWnAF8f1EV4jQGgSY35cFfaVjodr GS0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750389859; x=1750994659; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=z2YzMDv78WPG5/xYOI3b72eSmmaa1xBCqOTfcIeZ+L0=; b=V27F/IDMA93BPsMHbJ9TFq7XqR4KnHs4sWlgp9K0gZQxS9CX7g0qysmmGTv9ydzygc 0cfEECkT2CIjFrDuCHyBp2MIaMuDLaZ/ih7ST1PDcZ2NsYNOwI7iK6VmJjLMAmipt/Qb VnRU7U7GTrK5rQBn+nOWJvL9jDk+SKnei9FHLth5+1yXDpoXkl9dIXKnmOSPbA30v2pa AxXoUg8+rucrPMqr4qy61P38L1I3FG2WJDvKsh5vQHB5ky0Egdi6wOJwJpf8XklX5n5w 9R4utMaWR2CcfE41hAXtIXlSZN0bUUPICY6gk4HNjAmkuE+2IunW1k/NnkGEjVNyp45W BCqQ== X-Forwarded-Encrypted: i=1; AJvYcCVDyxH84jF2jy+G1qDhvJPbPHudhCxVoKyODC55HeAgUJqq5JJaCJKuXgy5HdsOUN3iRB6jOV645oUH+bc=@vger.kernel.org X-Gm-Message-State: AOJu0YxNW9m8+0e7ndvTj6FWz/UlOBm+HhN5HGgB/6XFU9+LgNw8xr5s sxTgkPqqrSgawdwasqpSZzR0gpY5qit9fr7UUGKs80Xg94iyG+aa5/H88Gv/rzyu87E= X-Gm-Gg: ASbGncuv5fUwsJy8ENg8OOER/5W55g67tpus1XUO3/uawhORNxdKKHibywW8D2xZZRa sMG9IBYNbWJMSI+s+ekxrcAAWPmZsP/E8cFiP6WFCktfGztUnyN7Ggza8g9DreDRO0jFTn6vvMb HyVCOzM+oPmdichbJ9sL9fs2FKljmisv2M++VdGMQ3rHMUD0YCjpU20IAyBVvs3rsLXnxlI1gD7 0V3OzLtt0roc9FKzAdzLuUAlhIiOfoa0a7ss/XOuLIvQTmeCendu6oL5HDDQnVx9XuY9prVKKaV 5ZXXqTkeRtBI4L3adRvNxM332CcZnzff3KoTGJtlesrFT/tUW/13Uo5T7eMAPws58oLzSHoDWzV t32QpHtbgtwU3 X-Google-Smtp-Source: AGHT+IGXeKmUfFbuX8aA0TnmPyTEPtqZ2Lj0/H9EkIAI3yrkuvUIjWM6dz6wmIwtNCHRZ1tMl1fTIg== X-Received: by 2002:a05:6a20:6a23:b0:1f5:7007:9eb8 with SMTP id adf61e73a8af0-22026e94585mr1952273637.16.1750389858932; Thu, 19 Jun 2025 20:24:18 -0700 (PDT) Received: from localhost.localdomain ([203.208.189.7]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-b31f12423e3sm490565a12.47.2025.06.19.20.24.15 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 19 Jun 2025 20:24:18 -0700 (PDT) From: lizhe.67@bytedance.com To: alex.williamson@redhat.com, jgg@ziepe.ca, david@redhat.com Cc: peterx@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, lizhe.67@bytedance.com Subject: [PATCH v5 3/3] vfio/type1: optimize vfio_unpin_pages_remote() for large folio Date: Fri, 20 Jun 2025 11:23:44 +0800 Message-ID: <20250620032344.13382-4-lizhe.67@bytedance.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250620032344.13382-1-lizhe.67@bytedance.com> References: <20250620032344.13382-1-lizhe.67@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Li Zhe When vfio_unpin_pages_remote() is called with a range of addresses that includes large folios, the function currently performs individual put_pfn() operations for each page. This can lead to significant performance overheads, especially when dealing with large ranges of pages. It would be very rare for reserved PFNs and non reserved will to be mixed within the same range. So this patch utilizes the has_rsvd variable introduced in the previous patch to determine whether batch put_pfn() operations can be performed. Moreover, compared to put_pfn(), unpin_user_page_range_dirty_lock() is capable of handling large folio scenarios more efficiently. The performance test results, based on v6.15, for completing the 16G VFIO IOMMU DMA unmapping, obtained through unit test[1] with slight modifications[2], are as follows. Base(v6.15): ./vfio-pci-mem-dma-map 0000:03:00.0 16 Suggested-by: Jason Gunthorpe ------- AVERAGE (MADV_HUGEPAGE) -------- VFIO MAP DMA in 0.047 s (338.6 GB/s) VFIO UNMAP DMA in 0.138 s (116.2 GB/s) ------- AVERAGE (MAP_POPULATE) -------- VFIO MAP DMA in 0.280 s (57.2 GB/s) VFIO UNMAP DMA in 0.312 s (51.3 GB/s) ------- AVERAGE (HUGETLBFS) -------- VFIO MAP DMA in 0.052 s (308.3 GB/s) VFIO UNMAP DMA in 0.139 s (115.1 GB/s) Map[3] + This patchset: ------- AVERAGE (MADV_HUGEPAGE) -------- VFIO MAP DMA in 0.028 s (563.9 GB/s) VFIO UNMAP DMA in 0.049 s (325.1 GB/s) ------- AVERAGE (MAP_POPULATE) -------- VFIO MAP DMA in 0.292 s (54.7 GB/s) VFIO UNMAP DMA in 0.292 s (54.9 GB/s) ------- AVERAGE (HUGETLBFS) -------- VFIO MAP DMA in 0.033 s (491.3 GB/s) VFIO UNMAP DMA in 0.049 s (323.9 GB/s) For large folio, we achieve an approximate 64% performance improvement in the VFIO UNMAP DMA item. For small folios, the performance test results appear to show no significant changes. [1]: https://github.com/awilliam/tests/blob/vfio-pci-mem-dma-map/vfio-pci-m= em-dma-map.c [2]: https://lore.kernel.org/all/20250610031013.98556-1-lizhe.67@bytedance.= com/ [3]: https://lore.kernel.org/all/20250529064947.38433-1-lizhe.67@bytedance.= com/ Suggested-by: Jason Gunthorpe Signed-off-by: Li Zhe --- drivers/vfio/vfio_iommu_type1.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type= 1.c index 8827e315e3d8..88a54b44df5b 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -806,17 +806,29 @@ static long vfio_pin_pages_remote(struct vfio_dma *dm= a, unsigned long vaddr, return pinned; } =20 +static inline void put_valid_unreserved_pfns(unsigned long start_pfn, + unsigned long npage, int prot) +{ + unpin_user_page_range_dirty_lock(pfn_to_page(start_pfn), npage, + prot & IOMMU_WRITE); +} + static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova, unsigned long pfn, unsigned long npage, bool do_accounting) { long unlocked =3D 0, locked =3D vpfn_pages(dma, iova, npage); - long i; =20 - for (i =3D 0; i < npage; i++) - if (put_pfn(pfn++, dma->prot)) - unlocked++; + if (dma->has_rsvd) { + long i; =20 + for (i =3D 0; i < npage; i++) + if (put_pfn(pfn++, dma->prot)) + unlocked++; + } else { + put_valid_unreserved_pfns(pfn, npage, dma->prot); + unlocked =3D npage; + } if (do_accounting) vfio_lock_acct(dma, locked - unlocked, true); =20 --=20 2.20.1