From nobody Mon Jun 8 17:38:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8ED23161A2 for ; Wed, 27 May 2026 18:48:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779907683; cv=none; b=s0HFAsGytcFzuG3UK89tbMbm1FEoFDngv+gOHe1ovP87YVn7wKiwsvs9l161kJ2NCwfcwag1oi2K2j7HpM6pVzFSpnkgq87HwrxFKq0aKtkcyDkBojWJfEkNH1Ho/Xh6MfAzkP9Vvz9+axBjnKb3SRm9kbVCxRLTNIv/5iB/ztc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779907683; c=relaxed/simple; bh=mPLMNMKJ9/Ad9AH/Z7iIBLoXtKJIcCJMgPPnz/g380w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XWucqFkTz3cbg0jAHKJ2rLlCVMuPg7LUhlUMfnMgEyI69CuLduxj57ojaISFxM9syREAwngkeYc/HOr95iRz6riib/wTcNPgdtCrlJGQOnNin0o7dvRYw2XYJjlbVh+0O6olf2S43m9nouwKUlv5hZ7CT+LQHP4o74atOmwLb1Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gr2yuIiX; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gr2yuIiX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BA9111F00A3E; Wed, 27 May 2026 18:47:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1779907681; bh=ZxtKBAoMGY0dqUDczpf/VIzNJfYrrM3Xd71tALEzopQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=gr2yuIiX/P/O2PNJpAF1ruWuSkVo/3UsgZmscmZBGHP9V4h9dlm88SAXIjr4OqEWf 9aHpzWbyd/zUZWTX7HTgRyjTo50UkURWD1Rs43Lzw1ION1jS+xHLObpkJeRLH0Et5W dTJ2j91wJHMl01exagRu7JO7KMb/uUZZ5nwRQ+5BgXhXZPW15ru544aQ9z2BCGrxY4 JOz0AMQt3BeT3nP3b3mZCHVKO3YW5bS+Zub3gPYL++4ZGooRKVrwqXHLEK/ttM+4OS dGEKBOvt4T5PZOqhwxBwITFYFEpRHdtDPIDChp43b/HvMtY9zXYtJTT7PqVKoV63kY EfdmDk2aknCCA== From: Mike Rapoport To: Andrew Morton Cc: David Carlier , David Hildenbrand , Heechan Kang , "Liam R. Howlett" , Lorenzo Stoakes , Michael Bommarito , Mike Rapoport , Peter Xu , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/3] userfaultfd: verify VMA state across UFFDIO_COPY retry Date: Wed, 27 May 2026 21:47:49 +0300 Message-ID: <20260527184751.4147364-2-rppt@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260527184751.4147364-1-rppt@kernel.org> References: <20260527184751.4147364-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" mfill_copy_folio_retry() drops the VMA lock for copy_from_user() and reacquires it afterwards. The destination VMA can be replaced during that window. The existing check compares vma_uffd_ops() before and after the retry, but if a shmem VMA with MAP_SHARED is replaced with a shmem VMA with MAP_PRIVATE (or vice versa) the replacement goes undetected. The change from MAP_PRIVATE to MAP_SHARED will treat the folio allocated with shmem_alloc_folio() as anonymous and this will cause BUG() when mfill_atomic_install_pte() will try to folio_add_new_anon_rmap(). The change from MAP_SHARED to MAP_PRIVATE allows injection of folios into the page cache of the original VMA. There is no need to change for hugetlb because it never uses mfill_copy_folio_retry(). Introduce helpers for more comprehensive comparison of VMA state: - mfill_retry_state_save() to save the relevant VMA state into a struct mfill_retry_state (original uffd_ops, relevant VMA flags, vm_file and pgoff) before dropping the lock - mfill_retry_state_changed() to compare the saved state with the state of the VMA acquired after retaking the locks - mfill_retry_state_put() to release vm_file pinning. Use DEFINE_FREE() cleanup to wrap mfill_retry_state_put() to avoid complicating error handling paths in mfill_copy_folio_retry(). Fixes: 292411fda25b ("mm/userfaultfd: detect VMA type change after copy ret= ry in mfill_copy_folio_retry()") Fixes: 6ab703034f14 ("userfaultfd: mfill_atomic(): remove retry logic") Suggested-by: Peter Xu Co-developed-by: David Carlier Signed-off-by: David Carlier Co-developed-by: Michael Bommarito Signed-off-by: Michael Bommarito Signed-off-by: Mike Rapoport (Microsoft) Reviewed-by: Lorenzo Stoakes --- mm/userfaultfd.c | 85 +++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 73 insertions(+), 12 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 180bad42fc79..e5d2fb3ce2c1 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -14,6 +14,8 @@ #include #include #include +#include +#include #include #include #include "internal.h" @@ -443,16 +445,80 @@ static int mfill_copy_folio_locked(struct folio *foli= o, unsigned long src_addr) return ret; } =20 -static int mfill_copy_folio_retry(struct mfill_state *state, +#define MFILL_RETRY_STATE_VMA_FLAGS \ + append_vma_flags(__VMA_UFFD_FLAGS, VMA_SHARED_BIT) + +/* + * VMA state saved before dropping the locks in mfill_copy_folio_retry(). + * Used to detect VMA replacement or incompatible changes after reacquirin= g the + * locks. + */ +struct mfill_retry_state { + const struct vm_uffd_ops *ops; + struct file *file; + vma_flags_t flags; + pgoff_t pgoff; +}; + +static void mfill_retry_state_save(struct mfill_retry_state *s, + struct vm_area_struct *vma) +{ + s->flags =3D vma_flags_and_mask(&vma->flags, MFILL_RETRY_STATE_VMA_FLAGS); + s->ops =3D vma_uffd_ops(vma); + s->pgoff =3D vma->vm_pgoff; + + if (vma->vm_file) + s->file =3D get_file(vma->vm_file); +} + +static bool mfill_retry_state_changed(struct mfill_retry_state *state, + struct vm_area_struct *vma) +{ + vma_flags_t flags =3D vma_flags_and_mask(&vma->flags, + MFILL_RETRY_STATE_VMA_FLAGS); + + /* Have any UFFD flags (missing, WP, minor) changed? */ + if (!vma_flags_same_pair(&state->flags, &flags)) + return true; + + /* VMA type or effective uffd_ops changed while the lock was dropped */ + if (state->ops !=3D vma_uffd_ops(vma)) + return true; + + /* VMA was anonymous before; changed only if it no longer is */ + if (!state->file) + return !vma_is_anonymous(vma); + + /* VMA was file backed, but file, inode or offset has changed */ + if (!vma->vm_file || vma->vm_file->f_inode !=3D state->file->f_inode || + state->file !=3D vma->vm_file || vma->vm_pgoff !=3D state->pgoff) + return true; + + return false; +} + +static void mfill_retry_state_put(struct mfill_retry_state *s) +{ + if (s->file) + fput(s->file); +} + +DEFINE_FREE(retry_put, struct mfill_retry_state *, + if (_T) mfill_retry_state_put(_T)); + +static int mfill_copy_folio_retry(struct mfill_state *mfill_state, struct folio *folio) { - const struct vm_uffd_ops *orig_ops =3D vma_uffd_ops(state->vma); - unsigned long src_addr =3D state->src_addr; + struct mfill_retry_state retry_state =3D { 0 }; + struct mfill_retry_state *for_free __free(retry_put) =3D &retry_state; + unsigned long src_addr =3D mfill_state->src_addr; void *kaddr; int err; =20 + mfill_retry_state_save(&retry_state, mfill_state->vma); + /* retry copying with mm_lock dropped */ - mfill_put_vma(state); + mfill_put_vma(mfill_state); =20 kaddr =3D kmap_local_folio(folio, 0); err =3D copy_from_user(kaddr, (const void __user *) src_addr, PAGE_SIZE); @@ -463,19 +529,14 @@ static int mfill_copy_folio_retry(struct mfill_state = *state, flush_dcache_folio(folio); =20 /* reget VMA and PMD, they could change underneath us */ - err =3D mfill_get_vma(state); + err =3D mfill_get_vma(mfill_state); if (err) return err; =20 - /* - * The VMA type may have changed while the lock was dropped - * (e.g. replaced with a hugetlb mapping), making the caller's - * ops pointer stale. - */ - if (vma_uffd_ops(state->vma) !=3D orig_ops) + if (mfill_retry_state_changed(&retry_state, mfill_state->vma)) return -EAGAIN; =20 - err =3D mfill_establish_pmd(state); + err =3D mfill_establish_pmd(mfill_state); if (err) return err; =20 --=20 2.53.0 From nobody Mon Jun 8 17:38:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEAF62F7F18 for ; Wed, 27 May 2026 18:48:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779907686; cv=none; b=bFLx6aVqsFvPUmI4CbKlZ5uaYHKvxeDHCGCtVf6sUhTtawPFg5DErDdj8BciiB/99JueyMTbdiyXingyxx+xbCEVXz+YIq26+5T2JJLcYJz7nX2atfeJM/Zd35XmW0P7xUq0Eoq02sD5hCRRHVCc3IkfgJ+erO0bgrS0WoRzLGY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779907686; c=relaxed/simple; bh=yAIrwa4E1hakK2iMBSw2aOMZzmR2OGKXwNNjWcIHMIQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=E5+IJz4Gzp1fylBcnud8xINZWsnMZmxSpwpEVKDsZwTT0mrmjwmDV1clSOeNC6pZ70JQJhs2bGG9A7gwsiPmTGgG4O+p8geYwbzbOJEN5eMKuuxALgU2r+jBAZRZPgZmUnlewGyrKt/xjlC8oyYZitDjl2nKo/81IpS3kk61q60= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kRoX7yVU; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kRoX7yVU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E818E1F000E9; Wed, 27 May 2026 18:48:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1779907684; bh=dRSYO9/9bR0VX2Or4HoVo4m/cqxBZNmu+Ypk7OqiXPg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=kRoX7yVUnMQec8ELSRsqOSyrWPmRFWm7oaNBr1RunHRiYkJHwdhXT5uoI65FaoN5Y gzZbvEahTLDAyoPjZtjtRSTQ8zLOi4QfP43lSm2BZNvvIqc3CSmKRyNtuxI4J2cIz5 uMBXfXKV9xWlkR3LnSITJJrCH7OtzVvJe4aWMmP4EhxeIDBf61g/f7OZ/zrcguyrOp XLPylXlGh106yhi8JWllMWWVYSjjprsp6UszlhijGbptAYcQOgBK2M8XMln9r51nKh KVM5zemZT8bUdHY+M9EoKmx/gKtXrgHIrYZAR6PRCqDENbi8lv23dFDSbZ+nYcR6TP KZvgchARk1NYA== From: Mike Rapoport To: Andrew Morton Cc: David Carlier , David Hildenbrand , Heechan Kang , "Liam R. Howlett" , Lorenzo Stoakes , Michael Bommarito , Mike Rapoport , Peter Xu , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/3] userfaultfd: refuse to __mfill_atomic_pte() for unsupported VMAs Date: Wed, 27 May 2026 21:47:50 +0300 Message-ID: <20260527184751.4147364-3-rppt@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260527184751.4147364-1-rppt@kernel.org> References: <20260527184751.4147364-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" __mfill_atomic_pte() unconditionally dereferences ops because there is an assumption that VMAs that can undergo mfill_* operations are vetted on registration and must have valid vm_uffd_ops. Add a guard against potential bugs and make sure __mfill_atomic_pte() bails out if ops is NULL. Suggested-by: Lorenzo Stoakes Fixes: ad9ac3081332 ("userfaultfd: introduce vm_uffd_ops->alloc_folio()") Signed-off-by: Mike Rapoport (Microsoft) Reviewed-by: David CARLIER Reviewed-by: Lorenzo Stoakes --- mm/userfaultfd.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index e5d2fb3ce2c1..2872c71bbf36 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -552,6 +552,11 @@ static int __mfill_atomic_pte(struct mfill_state *stat= e, struct folio *folio; int ret; =20 + if (!ops) { + VM_WARN_ONCE(1, "UFFDIO_COPY for unsupported VMA"); + return -EOPNOTSUPP; + } + folio =3D ops->alloc_folio(state->vma, state->dst_addr); if (!folio) return -ENOMEM; --=20 2.53.0 From nobody Mon Jun 8 17:38:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C4813161AD for ; Wed, 27 May 2026 18:48:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779907689; cv=none; b=gmfBskv6wWRKdn/WX8LAaoxLG7uVQk4XGaS2v3sVL6As3gJ1Zqt10yZkJ3DMboTv6STX5BM8lEgR0++zNDlzkM6xFTI/0sb2wgLIG/2fCH2oTt/p/88VVmtQ1nY6Ja3a1v/8Yb6/tvxFib+MN6yYA+tRvEZOInrEvbN0ddq/7q0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779907689; c=relaxed/simple; bh=4cIsa04PN1OdjLGHfj5sUlYjZlutQyUTFbZvwJNN9vs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BpirW2TPoHieO+kLRbtohSgpnyfYjVKVbZcl6eZCslL4YrONzyMa7CRQIlFyx77qnkarwTdAs8J/PHQjv5sKRHTpefBq96UfdyJS7vG0y60MyBB9WkTpS3jpHWtebV1nEluhfORQd3klWlclSuxjRE7Z3wz/ZywZtu+6iSdB7ck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MydiH42z; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MydiH42z" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 200E11F00A3D; Wed, 27 May 2026 18:48:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1779907687; bh=07ezutEcInVG//yT0ldXpRUrdrAzNSr842HzE2J9pPU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=MydiH42zD2bX1JV7tCa3KUOOt4FutOpXpMqIykH5OrWBjIlX80mFOaFz/iCyq89Dy p6Tp8luXT72Tq+Km8mCwL0b/DxrsAtIknnypO8kh666+qL7Tv5vV6UGlyN9+X9haaw Oy5/1x+wWonkUSdl2+dHuSjyBWDDvc6PrZm456vfAUhXs2Dcr+VNM8J5memLbt94+O 9Wr97k8aBGuueLEyI8SLfiAo79l49ah9ioUQyYmNLBOjAw6Ff+12JSHR9HQbsZdajC fk5PtvmP3vYKzub50UpAcyCQT52xTt4TPJ0R70irEgsu6Vtv7pMrd5oSj22Gc9vFD9 eqLQsHv1g22WQ== From: Mike Rapoport To: Andrew Morton Cc: David Carlier , David Hildenbrand , Heechan Kang , "Liam R. Howlett" , Lorenzo Stoakes , Michael Bommarito , Mike Rapoport , Peter Xu , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/3] userfaultfd: remove redundant check in vm_uffd_ops() Date: Wed, 27 May 2026 21:47:51 +0300 Message-ID: <20260527184751.4147364-4-rppt@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260527184751.4147364-1-rppt@kernel.org> References: <20260527184751.4147364-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" Lorenzo says: static const struct vm_uffd_ops *vma_uffd_ops(struct vm_area_struct *vma) { if (vma_is_anonymous(vma)) return &anon_uffd_ops; return vma->vm_ops ? vma->vm_ops->uffd_ops : NULL; } This is doing a redundant check _and_ making life confusing, as if !vma->vm_ops is a condition that can be reached there, it can't, as vma_is_anonymous() is literally a !vma->vm_ops check :) Remove the redundant check. Suggested-by: Lorenzo Stoakes Fixes: 0f48947c4232 ("userfaultfd: introduce vm_uffd_ops") Signed-off-by: Mike Rapoport (Microsoft) Reviewed-by: Lorenzo Stoakes --- mm/userfaultfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 2872c71bbf36..80cc8be5725f 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -68,7 +68,7 @@ static const struct vm_uffd_ops *vma_uffd_ops(struct vm_a= rea_struct *vma) { if (vma_is_anonymous(vma)) return &anon_uffd_ops; - return vma->vm_ops ? vma->vm_ops->uffd_ops : NULL; + return vma->vm_ops->uffd_ops; } =20 static __always_inline --=20 2.53.0