From nobody Sun Feb 8 03:27:00 2026 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE24D18733E for ; Mon, 12 Aug 2024 16:49:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723481351; cv=none; b=uOLqf4Upq22aXH6polwc8mh/wDjEeswAyKR9anLMVEtP4nPztX4IIZVo5vEBNSHkSrcfv+XPDjOFLCXEBXZCIFDET8w+8U1C6N1RsNA460lss6mwrj5CisPlTeRipky/1C5m32H98k7vN/UMeFLdVYTBSgjp1XFKOUqTTf9NExU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723481351; c=relaxed/simple; bh=cyEqwtJdd73Z0XZVi41RqSUaVwXAJZY8AuIAhW9WDHE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=XO9m8nDwgcv4JZzdez32hs/85oND4hAbUQEBQQ6OPLX0pVYIhlaeP8iaA0lkTFDoMxQgGdeZM+gou+9LQ6qccKyQ6RlNrNoYuwS4eHOnOVY+gA1CBSoRZjBS0uGC2QVUDm8e5k5aiOTS7TaSzhv4hSZp95il8XZTsnAas4233ME= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jrr5QlJk; arc=none smtp.client-ip=209.85.128.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jrr5QlJk" Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-42807cb6afdso325e9.1 for ; Mon, 12 Aug 2024 09:49:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723481348; x=1724086148; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=DJlU1qRayR5F9JUx5MXjL9SPDXmWr10IpUL5fPzVi5M=; b=jrr5QlJkrAeiL6YYo8aqEvpIYe5Rs5W4q34Nkle1saIIvWO/zaKlCtPd1c2yw5kvMg RW84C2XurUApx7ZVzBxlI8/gevTZ2TnDr+3zsia5rzfkMfLyF6/jgOAu0GlsPYPJcD12 TREldQH5fOVes+htr7+AiVthxK9LAoZjsC9kmwtuCitN4ESAu42M3UYaDUr5kP/lTQe0 oyYmxevmqmzvvfBL3NlaJ0G28SDI8k1CQs+lPocmibQmikDsZUsl8Jo9GlZUmubiVi3K a9G1bHnO3oWCcM35HD5mV/oO2cq6JhO615TAgyKMlB+JIrBhN6EzcIVkIqUenzVMMc52 WWaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723481348; x=1724086148; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DJlU1qRayR5F9JUx5MXjL9SPDXmWr10IpUL5fPzVi5M=; b=JB924uiahV37hUB+m7SVFOfWSnVNGmgNFiMrflFvi8J8wdut6NeD9sLthsaBkr3G1F iIFkNAUG0cbbEYQAzejdq7e4enypMQnQAYfXV28qFyrBrRlMSaNLbs6UYkoMWUhbi76V 7IVBHBhGZSj4X/FJIDWtXFk+/Vz/y1yebPV5cgV0G6WbkAqQnPRtlDU2tx2EwkKdlZV3 8eMKgM5apl5rxqb8LpnfVy7xkc+5dq2ooZpj3K0JvDENjy3LA6qv3IL1GPIqv0iaa3H4 wpXWpQDMu2QRU5Yzm1RSCoeQfAQFVXoFv08eNdB3wfA/cm01ErjGeWdfx25rE9f/t/Mf OmMQ== X-Forwarded-Encrypted: i=1; AJvYcCXMVGwlpxaiuFoPRLfZIKe7SO6NXcxO2GrNmTEY9J5SPRW/owks10bCMgJzPf9FGa+eJC6fBD2cKWJ5Qrb8n3wghPXYp67jPzzLe8PJ X-Gm-Message-State: AOJu0Yzciu6NoDZkEUwJYi+7pEvEYrbm+ql3v7z+EPqIvfCGpZOPh4up 2dXOLsLG6Iak08ttlhBv+skYevVM/Ded0/CX6wPNMK6ajV7ooKPWj0a4S2nIUw== X-Google-Smtp-Source: AGHT+IF3zlvPwjqUofQiL0ZC5eEQF3fcknxLRKQAkZEkHYcLKefHOkHnk0bRtOc+kUzO6CYwrPRQqA== X-Received: by 2002:a05:600c:1d9d:b0:428:e6eb:1340 with SMTP id 5b1f17b1804b1-429c7ab73f3mr3104265e9.4.1723480961010; Mon, 12 Aug 2024 09:42:41 -0700 (PDT) Received: from localhost ([2a00:79e0:9d:4:731e:4844:d154:4cec]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-36e4ecc7ab6sm7913066f8f.104.2024.08.12.09.42.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Aug 2024 09:42:40 -0700 (PDT) From: Jann Horn Date: Mon, 12 Aug 2024 18:42:16 +0200 Subject: [PATCH 1/2] userfaultfd: Fix pmd_trans_huge() recheck race Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240812-uffd-thp-flip-fix-v1-1-4fc1db7ccdd0@google.com> References: <20240812-uffd-thp-flip-fix-v1-0-4fc1db7ccdd0@google.com> In-Reply-To: <20240812-uffd-thp-flip-fix-v1-0-4fc1db7ccdd0@google.com> To: Andrew Morton , Pavel Emelyanov , Andrea Arcangeli , Hugh Dickins Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jann Horn , stable@vger.kernel.org X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1723480955; l=1791; i=jannh@google.com; s=20240730; h=from:subject:message-id; bh=cyEqwtJdd73Z0XZVi41RqSUaVwXAJZY8AuIAhW9WDHE=; b=aJA7M7sFrH6fD8l5+lxzjeojnHNOZEjoqfdSfrX7+15r0/hEZsfJhoS9jBzIQjwlC/09kjeLt aliIZZ7VCkXCOia7BI5iwegqhOGJUdxnMWHy2QCvS7PsVHzM0h5yHQr X-Developer-Key: i=jannh@google.com; a=ed25519; pk=AljNtGOzXeF6khBXDJVVvwSEkVDGnnZZYqfWhP1V+C8= The following race can occur: mfill_atomic other thread =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D pmdp_get_lockless() [reads none pmd] __pte_alloc [no-op] BUG_ON(pmd_none(*dst_pmd)) I have experimentally verified this in a kernel with extra mdelay() calls; the BUG_ON(pmd_none(*dst_pmd)) triggers. On kernels newer than commit 0d940a9b270b ("mm/pgtable: allow pte_offset_map[_lock]() to fail"), this can't lead to anything worse than a BUG_ON(), since the page table access helpers are actually designed to deal with page tables concurrently disappearing; but on older kernels (<=3D6.4), I think we could probably theoretically race past the two BUG_ON= () checks and end up treating a hugepage as a page table. Cc: stable@vger.kernel.org Fixes: c1a4de99fada ("userfaultfd: mcopy_atomic|mfill_zeropage: UFFDIO_COPY= |UFFDIO_ZEROPAGE preparation") Signed-off-by: Jann Horn Acked-by: David Hildenbrand --- mm/userfaultfd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index e54e5c8907fa..ec3750467aa5 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -801,7 +801,8 @@ static __always_inline ssize_t mfill_atomic(struct user= faultfd_ctx *ctx, break; } /* If an huge pmd materialized from under us fail */ - if (unlikely(pmd_trans_huge(*dst_pmd))) { + dst_pmdval =3D pmdp_get_lockless(dst_pmd); + if (unlikely(pmd_none(dst_pmdval) || pmd_trans_huge(dst_pmdval))) { err =3D -EFAULT; break; } --=20 2.46.0.76.ge559c4bf1a-goog From nobody Sun Feb 8 03:27:00 2026 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2C5018733E for ; Mon, 12 Aug 2024 16:50:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723481439; cv=none; b=HT1na+EWPYSRIc/btnG7L3Th4AgXFhg3Pd3OXI4dNwMRRJ2jGS1kqAyDUlg+NqUiezZHeGqymkicf1sBhfTNr2QmL+Ko0v5Ij8L2FAzVSJbwbB7i2RHLtQm+zp6gPxG3cyUo0qZdOVoXNVvFoj14Ol6IgnIZ8KQzvPhZBpgwH0U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723481439; c=relaxed/simple; bh=DZesoZHTRWCbLwwNstyM7gJ5jwQEg9bMdX+DVGz6a38=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=X52QUjRLGmE3wlWFI4EauYuw/m/CLd7AwuLWcSS9IcgKFTGgYNsTNpwaX8vk4EtWAGzkAhFd2/Vf2s8LaVAR39RRfZVQqLmbpfAdhhWQ99MRZHXNOYCqFjzZi5aFZ9ybzgCEm4GHKUNTrvxcN4CQ6VqCG/BWn3COHwFECcNwwQg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=u7jBI1sN; arc=none smtp.client-ip=209.85.128.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="u7jBI1sN" Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-429d1a9363aso1095e9.1 for ; Mon, 12 Aug 2024 09:50:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723481436; x=1724086236; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=/Pwh5Yck0NJ63ftZanRvFS8ng+FvBnwXPy4oihzU0QE=; b=u7jBI1sNG2GpbW+11ThXucQ7Af/EK0mwEbMae6PEyatOY7XfkVzh9bdadozuzyG2un R0X9SGIEMC6h1zvwkraY4GlhEWaZox9/jTQwn9hwI+9IHdh6mS01GW6rIo42YlklgpSl /RsGPbJg/6qhHvt9awfjyt1YhMXNTi5YKWhIhu/FdIbi4BC8sIMjixPw+DrNbUKlTJKP Fp+/CIbkUALY0gKcTY2pAdiOM0Zpt+BRuMrNj1odstIeWtVTp6P7M4XZnnQ73LFAHeV+ 8JIssFc6Vay0fMVECJkLWOXLYSvSusaapr/jTagDfsG16rcij9JsFkAOlh0f2Aped+4q kY8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723481436; x=1724086236; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/Pwh5Yck0NJ63ftZanRvFS8ng+FvBnwXPy4oihzU0QE=; b=QVJuJd+3ooRMUqAx6b1kqyhVwCIBU0XUs2pluZSxUMd+HAp3KfmLPFrV2gccSnsCOt QQX9TKIHgrfW20f5sXi4B4UdxRpm+zaj2TklqoDHeVvWfSNDowZm7Xn77pF+IJieDdVg sgrIARHFUjRhoOwNybYq607zvinpNdXmddG5LC3aKRfZBhx6Dkp2ehgDpMJRQvRYaeeE mJ1fTDes4NAWQ+Mq1+BPiKCy9zL7A9GQiRxrz2BPws++q7yvklaCo6ZrGwRsqHVuUWLd eCBike8G0pRmjZQcomPEFQ+1gr5AElUyBAy3MJTyo1bY5qbSVTy0APyf9iayrlVAVFi/ bjPA== X-Forwarded-Encrypted: i=1; AJvYcCX1RSivQbPhK22UzrXxi8llflIEHO58DjcEq56uRZDYfPoeMU6rsVgwXVeZEpT7TeGi7WkQ1FdTeJK0I+36VFr3Ik072CjbWr8LgX7X X-Gm-Message-State: AOJu0YygoiMiUgb3IBl4v4qDAqlBBJTgIOUTxjDgeIbTFCpTt3SZwKLQ gBEkuZpC+AVaQ/cNlRdccIXPtBSD4cm95BkhVpKOM3WBlrFgnR/AN0TavwRHnED4s+BqBXl6bPU jeTvF X-Google-Smtp-Source: AGHT+IHfHCfh/WSuaqttUqyZE+PyNP9LtWGsB/jH2gcAVk2r+RVkhkgxxEdBJPnJp7LHUeTtjOy/kw== X-Received: by 2002:a05:600c:3d0f:b0:424:898b:522b with SMTP id 5b1f17b1804b1-429c827a41dmr2993975e9.1.1723480961958; Mon, 12 Aug 2024 09:42:41 -0700 (PDT) Received: from localhost ([2a00:79e0:9d:4:731e:4844:d154:4cec]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-36e4ebd3416sm7942206f8f.100.2024.08.12.09.42.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Aug 2024 09:42:41 -0700 (PDT) From: Jann Horn Date: Mon, 12 Aug 2024 18:42:17 +0200 Subject: [PATCH 2/2] userfaultfd: Don't BUG_ON() if khugepaged yanks our page table Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240812-uffd-thp-flip-fix-v1-2-4fc1db7ccdd0@google.com> References: <20240812-uffd-thp-flip-fix-v1-0-4fc1db7ccdd0@google.com> In-Reply-To: <20240812-uffd-thp-flip-fix-v1-0-4fc1db7ccdd0@google.com> To: Andrew Morton , Pavel Emelyanov , Andrea Arcangeli , Hugh Dickins Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jann Horn , stable@vger.kernel.org X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1723480955; l=1297; i=jannh@google.com; s=20240730; h=from:subject:message-id; bh=DZesoZHTRWCbLwwNstyM7gJ5jwQEg9bMdX+DVGz6a38=; b=ZH3EnaCLnxnBAMGfLvSGP0Eka8HeLuZeSjPlA42GsqIFj8e3TEDej8ARd5JVzffIeA0SjAHpP K2ENAU38N1DDTzCRll9WxUJ6b3Q4wSjqJacAcWC80k9zPshbwINOyyv X-Developer-Key: i=jannh@google.com; a=ed25519; pk=AljNtGOzXeF6khBXDJVVvwSEkVDGnnZZYqfWhP1V+C8= Since khugepaged was changed to allow retracting page tables in file mappings without holding the mmap lock, these BUG_ON()s are wrong - get rid of them. We could also remove the preceding "if (unlikely(...))" block, but then we could reach pte_offset_map_lock() with transhuge pages not just for file mappings but also for anonymous mappings - which would probably be fine but I think is not necessarily expected. Cc: stable@vger.kernel.org Fixes: 1d65b771bc08 ("mm/khugepaged: retract_page_tables() without mmap or = vma lock") Signed-off-by: Jann Horn Acked-by: David Hildenbrand Reviewed-by: Qi Zheng --- mm/userfaultfd.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index ec3750467aa5..0dfa97db6feb 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -806,9 +806,10 @@ static __always_inline ssize_t mfill_atomic(struct use= rfaultfd_ctx *ctx, err =3D -EFAULT; break; } - - BUG_ON(pmd_none(*dst_pmd)); - BUG_ON(pmd_trans_huge(*dst_pmd)); + /* + * For shmem mappings, khugepaged is allowed to remove page + * tables under us; pte_offset_map_lock() will deal with that. + */ =20 err =3D mfill_atomic_pte(dst_pmd, dst_vma, dst_addr, src_addr, flags, &folio); --=20 2.46.0.76.ge559c4bf1a-goog