From nobody Sat Feb 7 18:21:15 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3B8D13C3C0 for ; Tue, 26 Mar 2024 14:32:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711463564; cv=none; b=P8gLgobohkW4rRGI7Qm4PvFnRddUeppDa8S5Hoo4tx5XeQltzpTB5zWBaJIf3FSZ44s2ByOKP7NDH4fNSPSfFd4Iq8J0sPN5Tqs/abUnT9QZ6SY666XJlKb3Ngr/NZZIq/TgAPD6mjn0VdmZg8t1dOuroEt6KUhSLisFYZzN/6g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711463564; c=relaxed/simple; bh=3BfObOBItLmXWASUgTdfkgiwVlyf/Sm1OH87Hu2jEv4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u/Dn4rGQdLUc9NwUaHf1YrVu5GopapS8rcUZX/0XX7aWQglDVVR3Wotf6xxpq2U6nlGxOxa1aqhwpPXavZYkZxEvqZcYb4PiGXzvlQNQMpH0NKXTBD/iLCaMsr+OSEO68Xi2CctE2HLZoXbczcrXaufDe+H6aeJvD+T6gipjUI0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=W+PsGvPt; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="W+PsGvPt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711463561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jCQ18XUYGXxAblag6bvSJIJrwTPD6Hhm3KxukBaiG2A=; b=W+PsGvPto/7dV1XNioWFiWvQemJgIGerwYHPPYBPk0Nlukjx7FW34eU5wclCSi7KLJWmP7 yaEhusXhG0Jy5kB1URnwyPGQDbUOwm+8kBGcT6iZRPYw6Bx81T9ds3qAH27PSkWRjWiBZt xbYlTDSkmSygB1prZ7YGKRg/O20g1as= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-130-SUIk7uSiPqOSHPDYataC4g-1; Tue, 26 Mar 2024 10:32:38 -0400 X-MC-Unique: SUIk7uSiPqOSHPDYataC4g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 95BD23C0C889; Tue, 26 Mar 2024 14:32:37 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6784340C6CBE; Tue, 26 Mar 2024 14:32:34 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Rapoport , Miklos Szeredi , Lorenzo Stoakes , xingwei lee , yue sun , Miklos Szeredi , stable@vger.kernel.org Subject: [PATCH v2 1/3] mm/secretmem: fix GUP-fast succeeding on secretmem folios Date: Tue, 26 Mar 2024 15:32:08 +0100 Message-ID: <20240326143210.291116-2-david@redhat.com> In-Reply-To: <20240326143210.291116-1-david@redhat.com> References: <20240326143210.291116-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 Content-Type: text/plain; charset="utf-8" folio_is_secretmem() currently relies on secretmem folios being LRU folios, to save some cycles. However, folios might reside in a folio batch without the LRU flag set, or temporarily have their LRU flag cleared. Consequently, the LRU flag is unreliable for this purpose. In particular, this is the case when secretmem_fault() allocates a fresh page and calls filemap_add_folio()->folio_add_lru(). The folio might be added to the per-cpu folio batch and won't get the LRU flag set until the batch was drained using e.g., lru_add_drain(). Consequently, folio_is_secretmem() might not detect secretmem folios and GUP-fast can succeed in grabbing a secretmem folio, crashing the kernel when we would later try reading/writing to the folio, because the folio has been unmapped from the directmap. Fix it by removing that unreliable check. Reported-by: xingwei lee Reported-by: yue sun Closes: https://lore.kernel.org/lkml/CABOYnLyevJeravW=3DQrH0JUPYEcDN160aZFb= 7kwndm-J2rmz0HQ@mail.gmail.com/ Debugged-by: Miklos Szeredi Tested-by: Miklos Szeredi Fixes: 1507f51255c9 ("mm: introduce memfd_secret system call to create "sec= ret" memory areas") Cc: Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) --- include/linux/secretmem.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/secretmem.h b/include/linux/secretmem.h index 35f3a4a8ceb1..acf7e1a3f3de 100644 --- a/include/linux/secretmem.h +++ b/include/linux/secretmem.h @@ -13,10 +13,10 @@ static inline bool folio_is_secretmem(struct folio *fol= io) /* * Using folio_mapping() is quite slow because of the actual call * instruction. - * We know that secretmem pages are not compound and LRU so we can + * We know that secretmem pages are not compound, so we can * save a couple of cycles here. */ - if (folio_test_large(folio) || !folio_test_lru(folio)) + if (folio_test_large(folio)) return false; =20 mapping =3D (struct address_space *) --=20 2.43.2 From nobody Sat Feb 7 18:21:15 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB5E513C3E1 for ; Tue, 26 Mar 2024 14:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711463566; cv=none; b=q1BhxYEzci5kiLgKwVmS8ThnpThGdX/tDEUzKQ2BPCDrrGKdNdATx35Wrfbo8kAph5yp7s87PTtCyHX7S9sQyudWxF06c0kNE/PQ0AmKo74Y8VMv2hsS6miqI50NMJEINt8xD9JLqkH10I/vZeMK/4xJKu0atUZkyGU7vQKz7P8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711463566; c=relaxed/simple; bh=+q7oN0RHHkf3xdx5Sx6lpZeZdUjteYjleOK/Z4hnVB4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AciRNZMT9MwPqQ8uUvLLVsjVr6eaYngC1a/X6AJe4lu12rc3iXUF5D0bgBnBD4eaKHkxFNheazQKmHbJJEbP8KFQ6reyX6NHEE8lAwx1A4dg9HmtXStgIIDrMjGSmqkPNvn+NU1OUJWL0n3x0W3gMCLdOoPjIsyEIh72O8mPCWU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=dkDUmpzS; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="dkDUmpzS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711463563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=82tUC6dlxq3xFltomUX9Np5kaF2oYCvvKygUhhN8Yg8=; b=dkDUmpzSBbDYePIxUFf2apar35ClW59JMbswPzAgxpqpqYWIeDe50L6XsYodHUx0WtBN0o RiYMSL36pDxsQ3tFoQqdAIcifoJgzkGy9XWR4rWfFeZo+d/RfhN61jRG0PcY5vezu7QvKY k+dCPhw6ArAT+uKjmE6B8RnDQK7ANUA= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-318-Xgt-kVftPyGYoQ5JQWdHag-1; Tue, 26 Mar 2024 10:32:40 -0400 X-MC-Unique: Xgt-kVftPyGYoQ5JQWdHag-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EE76C382C468; Tue, 26 Mar 2024 14:32:39 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 01750400D698; Tue, 26 Mar 2024 14:32:37 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Rapoport , Miklos Szeredi , Lorenzo Stoakes , xingwei lee , yue sun Subject: [PATCH v2 2/3] selftests/memfd_secret: add vmsplice() test Date: Tue, 26 Mar 2024 15:32:09 +0100 Message-ID: <20240326143210.291116-3-david@redhat.com> In-Reply-To: <20240326143210.291116-1-david@redhat.com> References: <20240326143210.291116-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 Content-Type: text/plain; charset="utf-8" Let's add a simple reproducer for a scenario where GUP-fast could succeed on secretmem folios, making vmsplice() succeed instead of failing. The reproducer is based on a reproducer [1] by Miklos Szeredi. We want to perform two tests: vmsplice() when a fresh page was just faulted in, and vmsplice() on an existing page after munmap() that would drain certain LRU caches/batches in the kernel. In an ideal world, we could use fallocate(FALLOC_FL_PUNCH_HOLE) / MADV_REMOVE to remove any existing page. As that is currently not possible, run the test before any other tests that would allocate memory in the secretmem fd. Perform the ftruncate() only once, and check the return value. [1] https://lkml.kernel.org/r/CAJfpegt3UCsMmxd0taOY11Uaw5U=3DeS1fE5dn0wZX3H= F0oy8-oQ@mail.gmail.com Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) --- tools/testing/selftests/mm/memfd_secret.c | 51 ++++++++++++++++++++++- 1 file changed, 49 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/mm/memfd_secret.c b/tools/testing/self= tests/mm/memfd_secret.c index 9b298f6a04b3..9a0597310a76 100644 --- a/tools/testing/selftests/mm/memfd_secret.c +++ b/tools/testing/selftests/mm/memfd_secret.c @@ -20,6 +20,7 @@ #include #include #include +#include =20 #include "../kselftest.h" =20 @@ -83,6 +84,45 @@ static void test_mlock_limit(int fd) pass("mlock limit is respected\n"); } =20 +static void test_vmsplice(int fd, const char *desc) +{ + ssize_t transferred; + struct iovec iov; + int pipefd[2]; + char *mem; + + if (pipe(pipefd)) { + fail("pipe failed: %s\n", strerror(errno)); + return; + } + + mem =3D mmap(NULL, page_size, prot, mode, fd, 0); + if (mem =3D=3D MAP_FAILED) { + fail("Unable to mmap secret memory\n"); + goto close_pipe; + } + + /* + * vmsplice() may use GUP-fast, which must also fail. Prefault the + * page table, so GUP-fast could find it. + */ + memset(mem, PATTERN, page_size); + + iov.iov_base =3D mem; + iov.iov_len =3D page_size; + transferred =3D vmsplice(pipefd[1], &iov, 1, 0); + + if (transferred < 0 && errno =3D=3D EFAULT) + pass("vmsplice is blocked as expected with %s\n", desc); + else + fail("vmsplice: unexpected memory access with %s\n", desc); + + munmap(mem, page_size); +close_pipe: + close(pipefd[0]); + close(pipefd[1]); +} + static void try_process_vm_read(int fd, int pipefd[2]) { struct iovec liov, riov; @@ -187,7 +227,6 @@ static void test_remote_access(int fd, const char *name, return; } =20 - ftruncate(fd, page_size); memset(mem, PATTERN, page_size); =20 if (write(pipefd[1], &mem, sizeof(mem)) < 0) { @@ -258,7 +297,7 @@ static void prepare(void) strerror(errno)); } =20 -#define NUM_TESTS 4 +#define NUM_TESTS 6 =20 int main(int argc, char *argv[]) { @@ -277,9 +316,17 @@ int main(int argc, char *argv[]) ksft_exit_fail_msg("memfd_secret failed: %s\n", strerror(errno)); } + if (ftruncate(fd, page_size)) + ksft_exit_fail_msg("ftruncate failed: %s\n", strerror(errno)); =20 test_mlock_limit(fd); test_file_apis(fd); + /* + * We have to run the first vmsplice test before any secretmem page was + * allocated for this fd. + */ + test_vmsplice(fd, "fresh page"); + test_vmsplice(fd, "existing page"); test_process_vm_read(fd); test_ptrace(fd); =20 --=20 2.43.2 From nobody Sat Feb 7 18:21:15 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D357413C3C6 for ; Tue, 26 Mar 2024 14:32:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711463569; cv=none; b=OeL79u4Jtoz+yBSryXWjTUAq9NhRMNNIkbmatvYtNIbXhOPjiL5wAB40BJIT+WTqbuOUO/fiUc1o0N2ZdOKFnoqP2lmr3ljf6iwZPfdQ3USNYwvQOhConftyrF61YoCBf8a0aN76OUo8q8C6Vb7+DfeHBKl3KvzJPeNmfpPADaA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711463569; c=relaxed/simple; bh=Ft/fl115PWaojyHnfcG8GXvmIPoq3i6y0/vZnrjpc5E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uhBBm6pbjL1VSN7rCe9jswvQsUaK4Q+jfQWMO/Z12pZLWgaMJN6l6BcmZ/fkS9HZIDk2zf5hRduFX3gRU9IT5gW3Pm3S5w0SkbPJ8FsX4PmjzC/LhWF10o74Sxo+kOT4q0/RevNjleMEOJq/XCt0CgpIlgRB4Z0Y0q8OwrM9f70= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=I8XCaH3M; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="I8XCaH3M" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711463566; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cZdlw1b/kcLUrflUxwTe3odXoIWBotMqjI0wjKr5kQg=; b=I8XCaH3MriK0krX9t8Bk33sAUo91Ul5JveN3GaRcSzM8ZGaxIBq1+8sUmftkJOoU0KQmFc agxhvV0sEHzHHJdzzIh79ORVUSMnvbzlORpopzUYevhhvpDCr8H5VJlbklKzHFXEec8TvV sWZvQJh+OuBOF9NAh/4zO5YYUaFw1Rk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-468-AuwEDpdsMcGdxkD-3r8M6g-1; Tue, 26 Mar 2024 10:32:43 -0400 X-MC-Unique: AuwEDpdsMcGdxkD-3r8M6g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B404D800266; Tue, 26 Mar 2024 14:32:42 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43F7540D2982; Tue, 26 Mar 2024 14:32:40 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Rapoport , Miklos Szeredi , Lorenzo Stoakes , xingwei lee , yue sun Subject: [PATCH v2 3/3] mm: merge folio_is_secretmem() and folio_fast_pin_allowed() into gup_fast_folio_allowed() Date: Tue, 26 Mar 2024 15:32:10 +0100 Message-ID: <20240326143210.291116-4-david@redhat.com> In-Reply-To: <20240326143210.291116-1-david@redhat.com> References: <20240326143210.291116-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 Content-Type: text/plain; charset="utf-8" folio_is_secretmem() is currently only used during GUP-fast. Nowadays, folio_fast_pin_allowed() performs similar checks during GUP-fast and contains a lot of careful handling -- READ_ONCE() -- , sanity checks -- lockdep_assert_irqs_disabled() -- and helpful comments on how this handling is safe and correct. So let's merge folio_is_secretmem() into folio_fast_pin_allowed(). Rename folio_fast_pin_allowed() to gup_fast_folio_allowed(), to better match the new semantics. Reviewed-by: Mike Rapoport (IBM) Signed-off-by: David Hildenbrand --- include/linux/secretmem.h | 21 ++--------------- mm/gup.c | 48 +++++++++++++++++++++++---------------- 2 files changed, 30 insertions(+), 39 deletions(-) diff --git a/include/linux/secretmem.h b/include/linux/secretmem.h index acf7e1a3f3de..e918f96881f5 100644 --- a/include/linux/secretmem.h +++ b/include/linux/secretmem.h @@ -6,25 +6,8 @@ =20 extern const struct address_space_operations secretmem_aops; =20 -static inline bool folio_is_secretmem(struct folio *folio) +static inline bool secretmem_mapping(struct address_space *mapping) { - struct address_space *mapping; - - /* - * Using folio_mapping() is quite slow because of the actual call - * instruction. - * We know that secretmem pages are not compound, so we can - * save a couple of cycles here. - */ - if (folio_test_large(folio)) - return false; - - mapping =3D (struct address_space *) - ((unsigned long)folio->mapping & ~PAGE_MAPPING_FLAGS); - - if (!mapping || mapping !=3D folio->mapping) - return false; - return mapping->a_ops =3D=3D &secretmem_aops; } =20 @@ -38,7 +21,7 @@ static inline bool vma_is_secretmem(struct vm_area_struct= *vma) return false; } =20 -static inline bool folio_is_secretmem(struct folio *folio) +static inline bool secretmem_mapping(struct address_space *mapping) { return false; } diff --git a/mm/gup.c b/mm/gup.c index e7510b6ce765..03b74b148e30 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2466,12 +2466,14 @@ EXPORT_SYMBOL(get_user_pages_unlocked); #ifdef CONFIG_HAVE_FAST_GUP =20 /* - * Used in the GUP-fast path to determine whether a pin is permitted for a - * specific folio. + * Used in the GUP-fast path to determine whether GUP is permitted to work= on + * a specific folio. * * This call assumes the caller has pinned the folio, that the lowest page= table * level still points to this folio, and that interrupts have been disable= d. * + * GUP-fast must reject all secretmem folios. + * * Writing to pinned file-backed dirty tracked folios is inherently proble= matic * (see comment describing the writable_file_mapping_allowed() function). = We * therefore try to avoid the most egregious case of a long-term mapping d= oing @@ -2481,25 +2483,34 @@ EXPORT_SYMBOL(get_user_pages_unlocked); * in the fast path, so instead we whitelist known good cases and if in do= ubt, * fall back to the slow path. */ -static bool folio_fast_pin_allowed(struct folio *folio, unsigned int flags) +static bool gup_fast_folio_allowed(struct folio *folio, unsigned int flags) { + bool reject_file_backed =3D false; struct address_space *mapping; + bool check_secretmem =3D false; unsigned long mapping_flags; =20 /* * If we aren't pinning then no problematic write can occur. A long term * pin is the most egregious case so this is the one we disallow. */ - if ((flags & (FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE)) !=3D + if ((flags & (FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE)) =3D=3D (FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE)) - return true; + reject_file_backed =3D true; + + /* We hold a folio reference, so we can safely access folio fields. */ =20 - /* The folio is pinned, so we can safely access folio fields. */ + /* secretmem folios are always order-0 folios. */ + if (IS_ENABLED(CONFIG_SECRETMEM) && !folio_test_large(folio)) + check_secretmem =3D true; + + if (!reject_file_backed && !check_secretmem) + return true; =20 if (WARN_ON_ONCE(folio_test_slab(folio))) return false; =20 - /* hugetlb mappings do not require dirty-tracking. */ + /* hugetlb neither requires dirty-tracking nor can be secretmem. */ if (folio_test_hugetlb(folio)) return true; =20 @@ -2535,10 +2546,12 @@ static bool folio_fast_pin_allowed(struct folio *fo= lio, unsigned int flags) =20 /* * At this point, we know the mapping is non-null and points to an - * address_space object. The only remaining whitelisted file system is - * shmem. + * address_space object. */ - return shmem_mapping(mapping); + if (check_secretmem && secretmem_mapping(mapping)) + return false; + /* The only remaining allowed file system is shmem. */ + return !reject_file_backed || shmem_mapping(mapping); } =20 static void __maybe_unused undo_dev_pagemap(int *nr, int nr_start, @@ -2624,18 +2637,13 @@ static int gup_pte_range(pmd_t pmd, pmd_t *pmdp, un= signed long addr, if (!folio) goto pte_unmap; =20 - if (unlikely(folio_is_secretmem(folio))) { - gup_put_folio(folio, 1, flags); - goto pte_unmap; - } - if (unlikely(pmd_val(pmd) !=3D pmd_val(*pmdp)) || unlikely(pte_val(pte) !=3D pte_val(ptep_get(ptep)))) { gup_put_folio(folio, 1, flags); goto pte_unmap; } =20 - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, 1, flags); goto pte_unmap; } @@ -2832,7 +2840,7 @@ static int gup_hugepte(pte_t *ptep, unsigned long sz,= unsigned long addr, return 0; } =20 - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, refs, flags); return 0; } @@ -2903,7 +2911,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsi= gned long addr, return 0; } =20 - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, refs, flags); return 0; } @@ -2947,7 +2955,7 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsi= gned long addr, return 0; } =20 - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, refs, flags); return 0; } @@ -2992,7 +3000,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsi= gned long addr, return 0; } =20 - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, refs, flags); return 0; } --=20 2.43.2