From nobody Sun Feb 8 11:44:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A72B6EB64D9 for ; Thu, 29 Jun 2023 15:56:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232646AbjF2P4B (ORCPT ); Thu, 29 Jun 2023 11:56:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59694 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232587AbjF2Pzy (ORCPT ); Thu, 29 Jun 2023 11:55:54 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BAF0359E for ; Thu, 29 Jun 2023 08:55:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1688054104; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2eueBEW3aL+xqYTSeuQ3xb53C/MZpWPSC4TMM2avShg=; b=G9XzZqYOU/JTDCv8bVABRfWkJn3eN8FJFRcvnYtMrU+jHNmMBpkgziQJsXMBxEq3PA2zv+ ZvRd4PPVkpcyh6CjPbUCzwNDpUp1IgKMp+bDCVAMxl+7ol0I6Bhq5kvlz7qyazWCnGzImv UaH8DaClqOqj1ymP+LiwwVrbC4wllkk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-381-pxvax7y_Nu26XJJKh4xM3A-1; Thu, 29 Jun 2023 11:54:59 -0400 X-MC-Unique: pxvax7y_Nu26XJJKh4xM3A-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 66B8A280AA42; Thu, 29 Jun 2023 15:54:39 +0000 (UTC) Received: from warthog.procyon.org.uk.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0EF7D4CD0C3; Thu, 29 Jun 2023 15:54:37 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Matthew Wilcox , Dave Chinner , Matt Whitlock , Linus Torvalds , Jens Axboe , linux-fsdevel@kvack.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig , linux-fsdevel@vger.kernel.org Subject: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns Date: Thu, 29 Jun 2023 16:54:30 +0100 Message-ID: <20230629155433.4170837-2-dhowells@redhat.com> In-Reply-To: <20230629155433.4170837-1-dhowells@redhat.com> References: <20230629155433.4170837-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Splicing data from, say, a file into a pipe currently leaves the source pages in the pipe after splice() returns - but this means that those pages can be subsequently modified by shared-writable mmap(), write(), fallocate(), etc. before they're consumed. Fix this by stealing the pages in splice() before they're added to the pipe if no one else is using them or has them mapped and copying them otherwise. Reported-by: Matt Whitlock Link: https://lore.kernel.org/r/ec804f26-fa76-4fbe-9b1c-8fbbd829b735@mattwh= itlock.name/ Signed-off-by: David Howells cc: Matthew Wilcox cc: Dave Chinner cc: Christoph Hellwig cc: Jens Axboe cc: linux-fsdevel@vger.kernel.org --- mm/filemap.c | 92 ++++++++++++++++++++++++++++++++++++++++++++++++--- mm/internal.h | 4 +-- mm/shmem.c | 8 +++-- 3 files changed, 95 insertions(+), 9 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 9e44a49bbd74..a002df515966 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2838,15 +2838,87 @@ generic_file_read_iter(struct kiocb *iocb, struct i= ov_iter *iter) } EXPORT_SYMBOL(generic_file_read_iter); =20 +static inline void copy_folio_to_folio(struct folio *src, size_t src_offse= t, + struct folio *dst, size_t dst_offset, + size_t size) +{ + void *p, *q; + + while (size > 0) { + size_t part =3D min3(PAGE_SIZE - src_offset % PAGE_SIZE, + PAGE_SIZE - dst_offset % PAGE_SIZE, + size); + + p =3D kmap_local_folio(src, src_offset); + q =3D kmap_local_folio(dst, dst_offset); + memcpy(q, p, part); + kunmap_local(p); + kunmap_local(q); + src_offset +=3D part; + dst_offset +=3D part; + size -=3D part; + } +} + /* - * Splice subpages from a folio into a pipe. + * Splice data from a folio into a pipe. The folio is stolen if no one el= se is + * using it and copied otherwise. We can't put the folio into the pipe st= ill + * attached to the pagecache as that allows someone to modify it after the + * splice. */ -size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, - struct folio *folio, loff_t fpos, size_t size) +ssize_t splice_folio_into_pipe(struct pipe_inode_info *pipe, + struct folio *folio, loff_t fpos, size_t size) { + struct address_space *mapping; + struct folio *copy =3D NULL; struct page *page; + unsigned int flags =3D 0; + ssize_t ret; size_t spliced =3D 0, offset =3D offset_in_folio(folio, fpos); =20 + folio_lock(folio); + + mapping =3D folio_mapping(folio); + ret =3D -ENODATA; + if (!folio->mapping) + goto err_unlock; /* Truncated */ + ret =3D -EIO; + if (!folio_test_uptodate(folio)) + goto err_unlock; + + /* + * At least for ext2 with nobh option, we need to wait on writeback + * completing on this folio, since we'll remove it from the pagecache. + * Otherwise truncate wont wait on the folio, allowing the disk blocks + * to be reused by someone else before we actually wrote our data to + * them. fs corruption ensues. + */ + folio_wait_writeback(folio); + + if (folio_has_private(folio) && + !filemap_release_folio(folio, GFP_KERNEL)) + goto need_copy; + + /* If we succeed in removing the mapping, set LRU flag and add it. */ + if (remove_mapping(mapping, folio)) { + folio_unlock(folio); + flags =3D PIPE_BUF_FLAG_LRU; + goto add_to_pipe; + } + +need_copy: + folio_unlock(folio); + + copy =3D folio_alloc(GFP_KERNEL, 0); + if (!copy) + return -ENOMEM; + + size =3D min(size, PAGE_SIZE - offset % PAGE_SIZE); + copy_folio_to_folio(folio, offset, copy, 0, size); + folio =3D copy; + offset =3D 0; + +add_to_pipe: page =3D folio_page(folio, offset / PAGE_SIZE); size =3D min(size, folio_size(folio) - offset); offset %=3D PAGE_SIZE; @@ -2861,6 +2933,7 @@ size_t splice_folio_into_pipe(struct pipe_inode_info = *pipe, .page =3D page, .offset =3D offset, .len =3D part, + .flags =3D flags, }; folio_get(folio); pipe->head++; @@ -2869,7 +2942,13 @@ size_t splice_folio_into_pipe(struct pipe_inode_info= *pipe, offset =3D 0; } =20 + if (copy) + folio_put(copy); return spliced; + +err_unlock: + folio_unlock(folio); + return ret; } =20 /** @@ -2947,7 +3026,7 @@ ssize_t filemap_splice_read(struct file *in, loff_t *= ppos, =20 for (i =3D 0; i < folio_batch_count(&fbatch); i++) { struct folio *folio =3D fbatch.folios[i]; - size_t n; + ssize_t n; =20 if (folio_pos(folio) >=3D end_offset) goto out; @@ -2963,8 +3042,11 @@ ssize_t filemap_splice_read(struct file *in, loff_t = *ppos, =20 n =3D min_t(loff_t, len, isize - *ppos); n =3D splice_folio_into_pipe(pipe, folio, *ppos, n); - if (!n) + if (n <=3D 0) { + if (n < 0) + error =3D n; goto out; + } len -=3D n; total_spliced +=3D n; *ppos +=3D n; diff --git a/mm/internal.h b/mm/internal.h index a7d9e980429a..ae395e0f31d5 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -881,8 +881,8 @@ struct migration_target_control { /* * mm/filemap.c */ -size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, - struct folio *folio, loff_t fpos, size_t size); +ssize_t splice_folio_into_pipe(struct pipe_inode_info *pipe, + struct folio *folio, loff_t fpos, size_t size); =20 /* * mm/vmalloc.c diff --git a/mm/shmem.c b/mm/shmem.c index 2f2e0e618072..969931b0f00e 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2783,7 +2783,8 @@ static ssize_t shmem_file_splice_read(struct file *in= , loff_t *ppos, struct inode *inode =3D file_inode(in); struct address_space *mapping =3D inode->i_mapping; struct folio *folio =3D NULL; - size_t total_spliced =3D 0, used, npages, n, part; + ssize_t n; + size_t total_spliced =3D 0, used, npages, part; loff_t isize; int error =3D 0; =20 @@ -2844,8 +2845,11 @@ static ssize_t shmem_file_splice_read(struct file *i= n, loff_t *ppos, n =3D splice_zeropage_into_pipe(pipe, *ppos, len); } =20 - if (!n) + if (n <=3D 0) { + if (n < 0) + error =3D n; break; + } len -=3D n; total_spliced +=3D n; *ppos +=3D n;