From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18985C7EE2D for ; Fri, 19 May 2023 07:42:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230413AbjESHmL (ORCPT ); Fri, 19 May 2023 03:42:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230387AbjESHmD (ORCPT ); Fri, 19 May 2023 03:42:03 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EAF34E6E for ; Fri, 19 May 2023 00:41:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482072; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BN8k90GM05xGcXUwqmiE5oyub34c+VlFBbq2fGi5jWo=; b=WV4vqEtwM2u95piKVcv32VaYxD2vcjyM03G/gmvCkyvXDkeZY4EGvyG8fWkGQXSUFdv9kR +y+mcCwjPDHb/YwxqQzftKWdM3rHj/zpFsqfzdKHGPzMfKiJSDF0gEH1iRp2eXMWsxeB7O 9Sa01w6D4Ibqb+/VLZSJZX7Q1yUo3MA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-541-i2ERw50uOo69R7scCkPSwg-1; Fri, 19 May 2023 03:41:08 -0400 X-MC-Unique: i2ERw50uOo69R7scCkPSwg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DCDB029AB3E6; Fri, 19 May 2023 07:41:06 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id B6BAC1121314; Fri, 19 May 2023 07:41:04 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Steve French , Christoph Hellwig , John Hubbard Subject: [PATCH v20 01/32] splice: Fix filemap of a blockdev Date: Fri, 19 May 2023 08:40:16 +0100 Message-Id: <20230519074047.1739879-2-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Fix filemap_splice_read() to use file->f_mapping->host, not file->f_inode, as the source of the file size because in the case of a block device, file->f_inode points to the block-special file (which is typically 0 length) and not the backing store. Fixes: 07073eb01c5f ("splice: Add a func to do a splice from a buffered fil= e without ITER_PIPE") Signed-off-by: David Howells cc: Steve French cc: Jens Axboe cc: Christoph Hellwig cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org Reviewed-by: Christoph Hellwig --- mm/filemap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index b4c9bd368b7e..a2006936a6ae 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2900,7 +2900,7 @@ ssize_t filemap_splice_read(struct file *in, loff_t *= ppos, do { cond_resched(); =20 - if (*ppos >=3D i_size_read(file_inode(in))) + if (*ppos >=3D i_size_read(in->f_mapping->host)) break; =20 iocb.ki_pos =3D *ppos; @@ -2916,7 +2916,7 @@ ssize_t filemap_splice_read(struct file *in, loff_t *= ppos, * part of the page is not copied back to userspace (unless * another truncate extends the file - this is desired though). */ - isize =3D i_size_read(file_inode(in)); + isize =3D i_size_read(in->f_mapping->host); if (unlikely(*ppos >=3D isize)) break; end_offset =3D min_t(loff_t, isize, *ppos + len); From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F080C7EE2C for ; Fri, 19 May 2023 07:42:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230522AbjESHm0 (ORCPT ); Fri, 19 May 2023 03:42:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230437AbjESHmM (ORCPT ); Fri, 19 May 2023 03:42:12 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F9DF10F0 for ; Fri, 19 May 2023 00:41:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482081; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sXlwBEApq0PjRnmqqIdGIYb9yjH9GX5fVKdD8XUANt8=; b=D1FExhMVD+Rv7j1guKI4G2RIyie9b5ZuDqUkg8muKxqrnorzACqpj50phjFsvZ4vcvMZar 2ePpLz+zKay7pEvts0iRj2KrXG9iRdYXhwrBAWI8ltYYkCos9urZPk8YUVUEt2UcieuEUL i/DU5ttHuNToYK5AC6S4X9A1+jfk2WA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-1-IyiQq-xAM9m13EaOaEApEQ-1; Fri, 19 May 2023 03:41:10 -0400 X-MC-Unique: IyiQq-xAM9m13EaOaEApEQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E69E5380450F; Fri, 19 May 2023 07:41:09 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id B1B6F2166B25; Fri, 19 May 2023 07:41:07 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v20 02/32] splice: Clean up direct_splice_read() a bit Date: Fri, 19 May 2023 08:40:17 +0100 Message-Id: <20230519074047.1739879-3-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Do a couple of cleanups to direct_splice_read(): (1) Cast to struct page **, not void *. (2) Simplify the calculation of the number of pages to keep/reclaim in direct_splice_read(). Suggested-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org --- fs/splice.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 3e06611d19ae..4db3eee49423 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -312,7 +312,7 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppo= s, struct kiocb kiocb; struct page **pages; ssize_t ret; - size_t used, npages, chunk, remain, reclaim; + size_t used, npages, chunk, remain, keep =3D 0; int i; =20 /* Work out how much data we can actually add into the pipe */ @@ -326,7 +326,7 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppo= s, if (!bv) return -ENOMEM; =20 - pages =3D (void *)(bv + npages); + pages =3D (struct page **)(bv + npages); npages =3D alloc_pages_bulk_array(GFP_USER, npages, pages); if (!npages) { kfree(bv); @@ -349,11 +349,8 @@ ssize_t direct_splice_read(struct file *in, loff_t *pp= os, kiocb.ki_pos =3D *ppos; ret =3D call_read_iter(in, &kiocb, &to); =20 - reclaim =3D npages * PAGE_SIZE; - remain =3D 0; if (ret > 0) { - reclaim -=3D ret; - remain =3D ret; + keep =3D DIV_ROUND_UP(ret, PAGE_SIZE); *ppos =3D kiocb.ki_pos; file_accessed(in); } else if (ret < 0) { @@ -366,14 +363,12 @@ ssize_t direct_splice_read(struct file *in, loff_t *p= pos, } =20 /* Free any pages that didn't get touched at all. */ - reclaim /=3D PAGE_SIZE; - if (reclaim) { - npages -=3D reclaim; - release_pages(pages + npages, reclaim); - } + if (keep < npages) + release_pages(pages + keep, npages - keep); =20 /* Push the remaining pages into the pipe. */ - for (i =3D 0; i < npages; i++) { + remain =3D ret; + for (i =3D 0; i < keep; i++) { struct pipe_buffer *buf =3D pipe_head_buf(pipe); =20 chunk =3D min_t(size_t, remain, PAGE_SIZE); From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E61FC7EE2E for ; Fri, 19 May 2023 07:42:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230482AbjESHmP (ORCPT ); Fri, 19 May 2023 03:42:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230317AbjESHmI (ORCPT ); Fri, 19 May 2023 03:42:08 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02BA71AC for ; Fri, 19 May 2023 00:41:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482077; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HJSls9joKnzamhDgZoRi/AKfts16br7tyBuA30GIBac=; b=QFYDajIts4baAcveTWbNX5MjBC+5bl+sLftufvPkOoCN3vvQJvhGzuJ8SFC43xwUrwSAUP Ii83EIhRLxBuwbFoqRRBUtBZA4SFt8MG/r22T+3F9xoBlevmm0zeodyticrH0U8X2n6A4X x3peDFK955G7cdwz6rqh5wfWUslFxHY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-662-unilBiGRMYCJnhOgY90jPw-1; Fri, 19 May 2023 03:41:13 -0400 X-MC-Unique: unilBiGRMYCJnhOgY90jPw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B0BCD29AB3E8; Fri, 19 May 2023 07:41:12 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 894042166B28; Fri, 19 May 2023 07:41:10 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig Subject: [PATCH v20 03/32] splice: Make direct_read_splice() limit to eof where appropriate Date: Fri, 19 May 2023 08:40:18 +0100 Message-Id: <20230519074047.1739879-4-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Make direct_read_splice() limit the read to the end of the file for regular files and block devices, thereby reducing the amount of allocation it will do in such a case. This means that the blockdev code doesn't require any special handling as filemap_read_splice() also limits to i_size. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- fs/splice.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/fs/splice.c b/fs/splice.c index 4db3eee49423..89c8516554d1 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -315,6 +315,19 @@ ssize_t direct_splice_read(struct file *in, loff_t *pp= os, size_t used, npages, chunk, remain, keep =3D 0; int i; =20 + if (!len) + return 0; + + if (S_ISREG(file_inode(in)->i_mode) || + S_ISBLK(file_inode(in)->i_mode)) { + loff_t i_size =3D i_size_read(in->f_mapping->host); + + if (*ppos >=3D i_size) + return 0; + if (len > i_size - *ppos) + len =3D i_size - *ppos; + } + /* Work out how much data we can actually add into the pipe */ used =3D pipe_occupancy(pipe->head, pipe->tail); npages =3D max_t(ssize_t, pipe->max_usage - used, 0); From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3481C7EE2A for ; Fri, 19 May 2023 07:42:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229810AbjESHmT (ORCPT ); Fri, 19 May 2023 03:42:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230338AbjESHmK (ORCPT ); Fri, 19 May 2023 03:42:10 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 220A8198 for ; Fri, 19 May 2023 00:41:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482082; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ajb+DQf4ARiWjgPxDf7oBmcuXX7xbhqUpLXrqXXlXzo=; b=SvKMN+lc8tIWVUMsgGOOLAyymGEWoHKpASP5t6NNinLLsGTZYay3RGaDxDtG27OU4nyFCY TEG9mC03ssc8PIintCku8FjZYFSzbKreKIwz6dYxvB1xVd+6DPXGDJJbjPh3Tk8i1UiHrs 70N+0G0cAd7tB4glVOZ1BAgVCzxJPpk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-650-m-DjV4UkMcasoW9pDJqpqQ-1; Fri, 19 May 2023 03:41:17 -0400 X-MC-Unique: m-DjV4UkMcasoW9pDJqpqQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4A7D7185A791; Fri, 19 May 2023 07:41:16 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7EEE77C2A; Fri, 19 May 2023 07:41:13 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Miklos Szeredi , John Hubbard , linux-unionfs@vger.kernel.org Subject: [PATCH v20 04/32] splice: Make do_splice_to() generic and export it Date: Fri, 19 May 2023 08:40:19 +0100 Message-Id: <20230519074047.1739879-5-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Rename do_splice_to() to vfs_splice_read() and export it so that it can be used as a helper when calling down to a lower layer filesystem as it performs all the necessary checks[1]. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Miklos Szeredi cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/CAJfpeguGksS3sCigmRi9hJdUec8qtM9f+_9jC1rJhs= XT+dV01w@mail.gmail.com/ [1] --- fs/splice.c | 27 ++++++++++++++++++++------- include/linux/splice.h | 3 +++ 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 89c8516554d1..1e0b7c7038b5 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -881,12 +881,24 @@ static long do_splice_from(struct pipe_inode_info *pi= pe, struct file *out, return out->f_op->splice_write(pipe, out, ppos, len, flags); } =20 -/* - * Attempt to initiate a splice from a file to a pipe. +/** + * vfs_splice_read - Read data from a file and splice it into a pipe + * @in: File to splice from + * @ppos: Input file offset + * @pipe: Pipe to splice to + * @len: Number of bytes to splice + * @flags: Splice modifier flags (SPLICE_F_*) + * + * Splice the requested amount of data from the input file to the pipe. T= his + * is synchronous as the caller must hold the pipe lock across the entire + * operation. + * + * If successful, it returns the amount of data spliced, 0 if it hit the E= OF or + * a hole and a negative error code otherwise. */ -static long do_splice_to(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, size_t len, - unsigned int flags) +long vfs_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, size_t len, + unsigned int flags) { unsigned int p_space; int ret; @@ -909,6 +921,7 @@ static long do_splice_to(struct file *in, loff_t *ppos, return warn_unsupported(in, "read"); return in->f_op->splice_read(in, ppos, pipe, len, flags); } +EXPORT_SYMBOL_GPL(vfs_splice_read); =20 /** * splice_direct_to_actor - splices data directly between two non-pipes @@ -978,7 +991,7 @@ ssize_t splice_direct_to_actor(struct file *in, struct = splice_desc *sd, size_t read_len; loff_t pos =3D sd->pos, prev_pos =3D pos; =20 - ret =3D do_splice_to(in, &pos, pipe, len, flags); + ret =3D vfs_splice_read(in, &pos, pipe, len, flags); if (unlikely(ret <=3D 0)) goto out_release; =20 @@ -1126,7 +1139,7 @@ long splice_file_to_pipe(struct file *in, pipe_lock(opipe); ret =3D wait_for_space(opipe, flags); if (!ret) - ret =3D do_splice_to(in, offset, opipe, len, flags); + ret =3D vfs_splice_read(in, offset, opipe, len, flags); pipe_unlock(opipe); if (ret > 0) wakeup_pipe_readers(opipe); diff --git a/include/linux/splice.h b/include/linux/splice.h index a55179fd60fc..8f052c3dae95 100644 --- a/include/linux/splice.h +++ b/include/linux/splice.h @@ -76,6 +76,9 @@ extern ssize_t splice_to_pipe(struct pipe_inode_info *, struct splice_pipe_desc *); extern ssize_t add_to_pipe(struct pipe_inode_info *, struct pipe_buffer *); +long vfs_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, size_t len, + unsigned int flags); extern ssize_t splice_direct_to_actor(struct file *, struct splice_desc *, splice_direct_actor *); extern long do_splice(struct file *in, loff_t *off_in, From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 506DDC77B7A for ; Fri, 19 May 2023 07:43:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231336AbjESHnB (ORCPT ); Fri, 19 May 2023 03:43:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229931AbjESHmo (ORCPT ); Fri, 19 May 2023 03:42:44 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F79610F5 for ; Fri, 19 May 2023 00:41:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482086; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ChgLSgSgCciGGEj2txPdAcXunor0ElTQJtr+fuEaFtA=; b=SXIZYmqWSTazlf7Y5HSs6bXudZmcdg/gg1nnDcKxX7dVIlPkGQsNwdPE+b1RzvsgIXYCPL lCw/yQHbD0s9XzoCT2VIgRDp3x1PORqvgZEQ9P+zeJWm2NCvQmhoq7ACXWYNt7C16g+O3q PV9aoDk3vO+9+m7UOiY0kOFoaL0Fzmw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-673-2UnIjJkHPT6ZoSIApCo6vg-1; Fri, 19 May 2023 03:41:20 -0400 X-MC-Unique: 2UnIjJkHPT6ZoSIApCo6vg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3C96E101A58B; Fri, 19 May 2023 07:41:19 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id D833E2166B27; Fri, 19 May 2023 07:41:16 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , linux-erofs@lists.ozlabs.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org Subject: [PATCH v20 05/32] splice: Make splice from a DAX file use direct_splice_read() Date: Fri, 19 May 2023 08:40:20 +0100 Message-Id: <20230519074047.1739879-6-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Make a read splice from a DAX file go directly to direct_splice_read() to do the reading as filemap_splice_read() is unlikely to find any pagecache to splice. I think this affects only erofs, Ext2, Ext4, fuse and XFS. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: linux-erofs@lists.ozlabs.org cc: linux-ext4@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-xfs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/splice.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/fs/splice.c b/fs/splice.c index 1e0b7c7038b5..7b818b5b18d4 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -421,6 +421,11 @@ ssize_t generic_file_splice_read(struct file *in, loff= _t *ppos, struct kiocb kiocb; int ret; =20 +#ifdef CONFIG_FS_DAX + if (IS_DAX(in->f_mapping->host)) + return direct_splice_read(in, ppos, pipe, len, flags); +#endif + iov_iter_pipe(&to, ITER_DEST, pipe, len); init_sync_kiocb(&kiocb, in); kiocb.ki_pos =3D *ppos; From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 430DBC77B7F for ; Fri, 19 May 2023 07:43:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231200AbjESHnJ (ORCPT ); Fri, 19 May 2023 03:43:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58910 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231207AbjESHms (ORCPT ); Fri, 19 May 2023 03:42:48 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6781F10FF for ; Fri, 19 May 2023 00:41:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482086; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KUqjynheJXCaLvUnn68NO8vUOrcgRITgr951Lzay0lM=; b=c/j2Pl56MKGlL9FpkmZw+GJfSbhBjaDsR9L/8vXwdXhOtdwzA3vc4ZudM3i/r90O1Jksbc 7/7Fw2qGkjqf4MDSJyxRJgkluDOt/tOh/zsR5Ou/s8/p/8aEiIfVVHJ0vgPzGSrpR7AWDF mlHDve4EScYpWKZ7kYAFKWLH4Zcj7ZM= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-81-zAQ1Er1WNnaKexqCxzJVJg-1; Fri, 19 May 2023 03:41:23 -0400 X-MC-Unique: zAQ1Er1WNnaKexqCxzJVJg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 63CC129AB3F4; Fri, 19 May 2023 07:41:22 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id EB0B51121314; Fri, 19 May 2023 07:41:19 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Daniel Golle , Guenter Roeck , Christoph Hellwig , John Hubbard , Hugh Dickins Subject: [PATCH v20 06/32] shmem: Implement splice-read Date: Fri, 19 May 2023 08:40:21 +0100 Message-Id: <20230519074047.1739879-7-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The new filemap_splice_read() has an implicit expectation via filemap_get_pages() that ->read_folio() exists if ->readahead() doesn't fully populate the pagecache of the file it is reading from[1], potentially leading to a jump to NULL if this doesn't exist. shmem, however, (and by extension, tmpfs, ramfs and rootfs), doesn't have ->read_folio(), Work around this by equipping shmem with its own splice-read implementation, based on filemap_splice_read(), but able to paste in zero_page when there's a page missing. Signed-off-by: David Howells cc: Daniel Golle cc: Guenter Roeck cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Hugh Dickins cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/Y+pdHFFTk1TTEBsO@makrotopia.org/ [1] --- Notes: ver #19) - Remove a missed get_page() on the zero page. =20 ver #18) - Don't take/release a ref on the zero page. mm/shmem.c | 134 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 133 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index e40a08c5c6d7..1f504ed982cf 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2731,6 +2731,138 @@ static ssize_t shmem_file_read_iter(struct kiocb *i= ocb, struct iov_iter *to) return retval ? retval : error; } =20 +static bool zero_pipe_buf_get(struct pipe_inode_info *pipe, + struct pipe_buffer *buf) +{ + return true; +} + +static void zero_pipe_buf_release(struct pipe_inode_info *pipe, + struct pipe_buffer *buf) +{ +} + +static bool zero_pipe_buf_try_steal(struct pipe_inode_info *pipe, + struct pipe_buffer *buf) +{ + return false; +} + +static const struct pipe_buf_operations zero_pipe_buf_ops =3D { + .release =3D zero_pipe_buf_release, + .try_steal =3D zero_pipe_buf_try_steal, + .get =3D zero_pipe_buf_get, +}; + +static size_t splice_zeropage_into_pipe(struct pipe_inode_info *pipe, + loff_t fpos, size_t size) +{ + size_t offset =3D fpos & ~PAGE_MASK; + + size =3D min_t(size_t, size, PAGE_SIZE - offset); + + if (!pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { + struct pipe_buffer *buf =3D pipe_head_buf(pipe); + + *buf =3D (struct pipe_buffer) { + .ops =3D &zero_pipe_buf_ops, + .page =3D ZERO_PAGE(0), + .offset =3D offset, + .len =3D size, + }; + pipe->head++; + } + + return size; +} + +static ssize_t shmem_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode =3D file_inode(in); + struct address_space *mapping =3D inode->i_mapping; + struct folio *folio =3D NULL; + size_t total_spliced =3D 0, used, npages, n, part; + loff_t isize; + int error =3D 0; + + /* Work out how much data we can actually add into the pipe */ + used =3D pipe_occupancy(pipe->head, pipe->tail); + npages =3D max_t(ssize_t, pipe->max_usage - used, 0); + len =3D min_t(size_t, len, npages * PAGE_SIZE); + + do { + if (*ppos >=3D i_size_read(inode)) + break; + + error =3D shmem_get_folio(inode, *ppos / PAGE_SIZE, &folio, SGP_READ); + if (error) { + if (error =3D=3D -EINVAL) + error =3D 0; + break; + } + if (folio) { + folio_unlock(folio); + + if (folio_test_hwpoison(folio)) { + error =3D -EIO; + break; + } + } + + /* + * i_size must be checked after we know the pages are Uptodate. + * + * Checking i_size after the check allows us to calculate + * the correct value for "nr", which means the zero-filled + * part of the page is not copied back to userspace (unless + * another truncate extends the file - this is desired though). + */ + isize =3D i_size_read(inode); + if (unlikely(*ppos >=3D isize)) + break; + part =3D min_t(loff_t, isize - *ppos, len); + + if (folio) { + /* + * If users can be writing to this page using arbitrary + * virtual addresses, take care about potential aliasing + * before reading the page on the kernel side. + */ + if (mapping_writably_mapped(mapping)) + flush_dcache_folio(folio); + folio_mark_accessed(folio); + /* + * Ok, we have the page, and it's up-to-date, so we can + * now splice it into the pipe. + */ + n =3D splice_folio_into_pipe(pipe, folio, *ppos, part); + folio_put(folio); + folio =3D NULL; + } else { + n =3D splice_zeropage_into_pipe(pipe, *ppos, len); + } + + if (!n) + break; + len -=3D n; + total_spliced +=3D n; + *ppos +=3D n; + in->f_ra.prev_pos =3D *ppos; + if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) + break; + + cond_resched(); + } while (len); + + if (folio) + folio_put(folio); + + file_accessed(in); + return total_spliced ? total_spliced : error; +} + static loff_t shmem_file_llseek(struct file *file, loff_t offset, int when= ce) { struct address_space *mapping =3D file->f_mapping; @@ -3971,7 +4103,7 @@ static const struct file_operations shmem_file_operat= ions =3D { .read_iter =3D shmem_file_read_iter, .write_iter =3D generic_file_write_iter, .fsync =3D noop_fsync, - .splice_read =3D generic_file_splice_read, + .splice_read =3D shmem_file_splice_read, .splice_write =3D iter_file_splice_write, .fallocate =3D shmem_fallocate, #endif From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDC6EC77B7A for ; Fri, 19 May 2023 07:42:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231279AbjESHmv (ORCPT ); Fri, 19 May 2023 03:42:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229565AbjESHmm (ORCPT ); Fri, 19 May 2023 03:42:42 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8454170B for ; Fri, 19 May 2023 00:41:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482091; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=74aMisHJvcrsbS5g8R7662ZNEA+1A6hMiypJVvVd5yo=; b=gPhoLIB7ysPM5LqXV5pacYaHyund+Pesp4CTtcwCHHn49r27SVH0624kJ47U1Z+2z77Y0Z oRYN93M9sHErrMJ52fYQ4UzFUHDND6XqPJiFkiGedN43+r4Mkx0EMy2TLkuw59RmDN5KV0 HrkB1cGYFgq9fQcTILmPIz5Y0EfUGPU= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-271-zhDNyJpoN6uuRXwKl_ogyg-1; Fri, 19 May 2023 03:41:26 -0400 X-MC-Unique: zhDNyJpoN6uuRXwKl_ogyg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 591D03804098; Fri, 19 May 2023 07:41:25 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1119C2166B25; Fri, 19 May 2023 07:41:22 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard , Miklos Szeredi , linux-unionfs@vger.kernel.org Subject: [PATCH v20 07/32] overlayfs: Implement splice-read Date: Fri, 19 May 2023 08:40:22 +0100 Message-Id: <20230519074047.1739879-8-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Implement splice-read for overlayfs by passing the request down a layer rather than going through generic_file_splice_read() which is going to be changed to assume that ->read_folio() is present on buffered files. Signed-off-by: David Howells cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Miklos Szeredi cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- Notes: ver #17) - Use vfs_splice_read() helper rather than open-coding checks. =20 ver #15) - Remove redundant FMODE_CAN_ODIRECT check on real file. - Do rw_verify_area() on the real file, not the overlay file. - Fix a file leak. fs/overlayfs/file.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c index 7c04f033aadd..86197882ff35 100644 --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -419,6 +419,27 @@ static ssize_t ovl_write_iter(struct kiocb *iocb, stru= ct iov_iter *iter) return ret; } =20 +static ssize_t ovl_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, size_t len, + unsigned int flags) +{ + const struct cred *old_cred; + struct fd real; + ssize_t ret; + + ret =3D ovl_real_fdget(in, &real); + if (ret) + return ret; + + old_cred =3D ovl_override_creds(file_inode(in)->i_sb); + ret =3D vfs_splice_read(real.file, ppos, pipe, len, flags); + revert_creds(old_cred); + ovl_file_accessed(in); + + fdput(real); + return ret; +} + /* * Calling iter_file_splice_write() directly from overlay's f_op may deadl= ock * due to lock order inversion between pipe->mutex in iter_file_splice_wri= te() @@ -695,7 +716,7 @@ const struct file_operations ovl_file_operations =3D { .fallocate =3D ovl_fallocate, .fadvise =3D ovl_fadvise, .flush =3D ovl_flush, - .splice_read =3D generic_file_splice_read, + .splice_read =3D ovl_splice_read, .splice_write =3D ovl_splice_write, =20 .copy_file_range =3D ovl_copy_file_range, From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0544EC77B7A for ; Fri, 19 May 2023 07:43:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231345AbjESHnE (ORCPT ); Fri, 19 May 2023 03:43:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230451AbjESHmr (ORCPT ); Fri, 19 May 2023 03:42:47 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADE63171F for ; Fri, 19 May 2023 00:41:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482094; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Mo6NWBR6hkfydtMnZeO+F5U5GhN+vbQRFCzRyccP/1I=; b=FhNalLiAxiyRSjczVWMIUPXkPc6wMlP8Wdy5ybLjz3uMfyEmzC9jPQOKsLLmYMTke/aiqp 5zks71loNaxuz5NkiHGPqDcHZbMmi5+WPB3iITBBdaKOQCXTV4UbbklCjzehvZq+LkcvwA LJ5gn4csrXPQHSay1rpvc8idmnWSiuM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-587-n4p8TgQ6OGG9Vxma9vRnxA-1; Fri, 19 May 2023 03:41:29 -0400 X-MC-Unique: n4p8TgQ6OGG9Vxma9vRnxA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C01A5800141; Fri, 19 May 2023 07:41:28 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 10DF940CFD45; Fri, 19 May 2023 07:41:25 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Jan Harkes , Christoph Hellwig , John Hubbard , coda@cs.cmu.edu, codalist@coda.cs.cmu.edu, linux-unionfs@vger.kernel.org Subject: [PATCH v20 08/32] coda: Implement splice-read Date: Fri, 19 May 2023 08:40:23 +0100 Message-Id: <20230519074047.1739879-9-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Implement splice-read for coda by passing the request down a layer rather than going through generic_file_splice_read() which is going to be changed to assume that ->read_folio() is present on buffered files. Signed-off-by: David Howells Acked-by: Jan Harkes cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: coda@cs.cmu.edu cc: codalist@coda.cs.cmu.edu cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- Notes: ver #17) - Use vfs_splice_read() helper rather than open-coding checks. fs/coda/file.c | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/fs/coda/file.c b/fs/coda/file.c index 3f3c81e6b1ab..12b26bd13564 100644 --- a/fs/coda/file.c +++ b/fs/coda/file.c @@ -23,6 +23,7 @@ #include #include #include +#include =20 #include #include "coda_psdev.h" @@ -94,6 +95,32 @@ coda_file_write_iter(struct kiocb *iocb, struct iov_iter= *to) return ret; } =20 +static ssize_t +coda_file_splice_read(struct file *coda_file, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *coda_inode =3D file_inode(coda_file); + struct coda_file_info *cfi =3D coda_ftoc(coda_file); + struct file *in =3D cfi->cfi_container; + loff_t ki_pos =3D *ppos; + ssize_t ret; + + ret =3D venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode), + &cfi->cfi_access_intent, + len, ki_pos, CODA_ACCESS_TYPE_READ); + if (ret) + goto finish_read; + + ret =3D vfs_splice_read(in, ppos, pipe, len, flags); + +finish_read: + venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode), + &cfi->cfi_access_intent, + len, ki_pos, CODA_ACCESS_TYPE_READ_FINISH); + return ret; +} + static void coda_vm_open(struct vm_area_struct *vma) { @@ -302,5 +329,5 @@ const struct file_operations coda_file_operations =3D { .open =3D coda_open, .release =3D coda_release, .fsync =3D coda_fsync, - .splice_read =3D generic_file_splice_read, + .splice_read =3D coda_file_splice_read, }; From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04C9AC7EE2C for ; Fri, 19 May 2023 07:43:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231196AbjESHm6 (ORCPT ); Fri, 19 May 2023 03:42:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229570AbjESHmo (ORCPT ); Fri, 19 May 2023 03:42:44 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A94181724 for ; Fri, 19 May 2023 00:41:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482098; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+X2utfWiaKbqYICogIU/HgPsAYww5xpYEgZObAx+E9M=; b=a4x+8of3f1bGfJfA2JYAFxu7hFuV+Pe5468CwiGAlOs+FdduY9D8NAL92m83MELMA5OLjg g8nycs7z1SyiCfejgLgwMGgzw2CBJxRKxzBzdYT6v+HohSPfZZepA1tV2EDoGof0BzAQC7 pY0Ie+piVrtHyBaTI1h1moCJN7oGL+Y= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-647-T0SjJP-GMVq8ZCGHan-Apg-1; Fri, 19 May 2023 03:41:34 -0400 X-MC-Unique: T0SjJP-GMVq8ZCGHan-Apg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F0B92380406F; Fri, 19 May 2023 07:41:32 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 62C2DC0004B; Fri, 19 May 2023 07:41:29 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Greg Kroah-Hartman , Christoph Hellwig , John Hubbard , Miklos Szeredi , Arnd Bergmann Subject: [PATCH v20 09/32] tty, proc, kernfs, random: Use direct_splice_read() Date: Fri, 19 May 2023 08:40:24 +0100 Message-Id: <20230519074047.1739879-10-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use direct_splice_read() for tty, procfs, kernfs and random files rather than going through generic_file_splice_read() as they just copy the file into the output buffer and don't splice pages. This avoids the need for them to have a ->read_folio() to satisfy filemap_splice_read(). Signed-off-by: David Howells Acked-by: Greg Kroah-Hartman cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Miklos Szeredi cc: Arnd Bergmann cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- drivers/char/random.c | 4 ++-- drivers/tty/tty_io.c | 4 ++-- fs/kernfs/file.c | 2 +- fs/proc/inode.c | 4 ++-- fs/proc/proc_sysctl.c | 2 +- fs/proc_namespace.c | 6 +++--- 6 files changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/char/random.c b/drivers/char/random.c index 253f2ddb8913..2b47b500126c 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -1546,7 +1546,7 @@ const struct file_operations random_fops =3D { .compat_ioctl =3D compat_ptr_ioctl, .fasync =3D random_fasync, .llseek =3D noop_llseek, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .splice_write =3D iter_file_splice_write, }; =20 @@ -1557,7 +1557,7 @@ const struct file_operations urandom_fops =3D { .compat_ioctl =3D compat_ptr_ioctl, .fasync =3D random_fasync, .llseek =3D noop_llseek, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .splice_write =3D iter_file_splice_write, }; =20 diff --git a/drivers/tty/tty_io.c b/drivers/tty/tty_io.c index c84be40fb8df..0b910a2af8e4 100644 --- a/drivers/tty/tty_io.c +++ b/drivers/tty/tty_io.c @@ -466,7 +466,7 @@ static const struct file_operations tty_fops =3D { .llseek =3D no_llseek, .read_iter =3D tty_read, .write_iter =3D tty_write, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .splice_write =3D iter_file_splice_write, .poll =3D tty_poll, .unlocked_ioctl =3D tty_ioctl, @@ -481,7 +481,7 @@ static const struct file_operations console_fops =3D { .llseek =3D no_llseek, .read_iter =3D tty_read, .write_iter =3D redirected_tty_write, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .splice_write =3D iter_file_splice_write, .poll =3D tty_poll, .unlocked_ioctl =3D tty_ioctl, diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index 40c4661f15b7..be43afa2138d 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -1011,7 +1011,7 @@ const struct file_operations kernfs_file_fops =3D { .release =3D kernfs_fop_release, .poll =3D kernfs_fop_poll, .fsync =3D noop_fsync, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .splice_write =3D iter_file_splice_write, }; =20 diff --git a/fs/proc/inode.c b/fs/proc/inode.c index f495fdb39151..711f12706469 100644 --- a/fs/proc/inode.c +++ b/fs/proc/inode.c @@ -591,7 +591,7 @@ static const struct file_operations proc_iter_file_ops = =3D { .llseek =3D proc_reg_llseek, .read_iter =3D proc_reg_read_iter, .write =3D proc_reg_write, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .poll =3D proc_reg_poll, .unlocked_ioctl =3D proc_reg_unlocked_ioctl, .mmap =3D proc_reg_mmap, @@ -617,7 +617,7 @@ static const struct file_operations proc_reg_file_ops_c= ompat =3D { static const struct file_operations proc_iter_file_ops_compat =3D { .llseek =3D proc_reg_llseek, .read_iter =3D proc_reg_read_iter, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .write =3D proc_reg_write, .poll =3D proc_reg_poll, .unlocked_ioctl =3D proc_reg_unlocked_ioctl, diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c index 8038833ff5b0..72f1a8ac7802 100644 --- a/fs/proc/proc_sysctl.c +++ b/fs/proc/proc_sysctl.c @@ -868,7 +868,7 @@ static const struct file_operations proc_sys_file_opera= tions =3D { .poll =3D proc_sys_poll, .read_iter =3D proc_sys_read, .write_iter =3D proc_sys_write, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .splice_write =3D iter_file_splice_write, .llseek =3D default_llseek, }; diff --git a/fs/proc_namespace.c b/fs/proc_namespace.c index 846f9455ae22..492abbbeff5e 100644 --- a/fs/proc_namespace.c +++ b/fs/proc_namespace.c @@ -324,7 +324,7 @@ static int mountstats_open(struct inode *inode, struct = file *file) const struct file_operations proc_mounts_operations =3D { .open =3D mounts_open, .read_iter =3D seq_read_iter, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .llseek =3D seq_lseek, .release =3D mounts_release, .poll =3D mounts_poll, @@ -333,7 +333,7 @@ const struct file_operations proc_mounts_operations =3D= { const struct file_operations proc_mountinfo_operations =3D { .open =3D mountinfo_open, .read_iter =3D seq_read_iter, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .llseek =3D seq_lseek, .release =3D mounts_release, .poll =3D mounts_poll, @@ -342,7 +342,7 @@ const struct file_operations proc_mountinfo_operations = =3D { const struct file_operations proc_mountstats_operations =3D { .open =3D mountstats_open, .read_iter =3D seq_read_iter, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .llseek =3D seq_lseek, .release =3D mounts_release, }; From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80538C7EE32 for ; Fri, 19 May 2023 07:42:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231186AbjESHmy (ORCPT ); Fri, 19 May 2023 03:42:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229663AbjESHmo (ORCPT ); Fri, 19 May 2023 03:42:44 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61EE81727 for ; Fri, 19 May 2023 00:41:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482100; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ronE9gr+sRXtILb6fInqA45jNbZvtaGttVkSNNlZgW8=; b=d9bwyEpX11hsNtIbHG1LPT/oQKvurOLUUIw7NOM61s/sJJgUH4WHO4gN6y8SY7WD8sT2x4 nH/XOtXKUEQKrWruRDOKaa3UvTZ7JF2KD3n9W1Ozdnan1VjNzcZbC/uQ6S9VJz9T+VuGgF b8wILmRUW0ALi1BpFRh+bGIUuycekwk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-274-frHc13ceOAyBYDJKrmNfcw-1; Fri, 19 May 2023 03:41:37 -0400 X-MC-Unique: frHc13ceOAyBYDJKrmNfcw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 39B0C80120A; Fri, 19 May 2023 07:41:36 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id A860140CFD45; Fri, 19 May 2023 07:41:33 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Christoph Hellwig , netdev@vger.kernel.org Subject: [PATCH v20 10/32] net: Make sock_splice_read() use direct_splice_read() by default Date: Fri, 19 May 2023 08:40:25 +0100 Message-Id: <20230519074047.1739879-11-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Make sock_splice_read() use direct_splice_read() by default as file_splice_read() will return immediately with 0 as a socket has no pagecache and is a zero-size file. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: netdev@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- net/socket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/socket.c b/net/socket.c index b7e01d0fe082..40b204a47aba 100644 --- a/net/socket.c +++ b/net/socket.c @@ -1093,7 +1093,7 @@ static ssize_t sock_splice_read(struct file *file, lo= ff_t *ppos, struct socket *sock =3D file->private_data; =20 if (unlikely(!sock->ops->splice_read)) - return generic_file_splice_read(file, ppos, pipe, len, flags); + return direct_splice_read(file, ppos, pipe, len, flags); =20 return sock->ops->splice_read(sock, ppos, pipe, len, flags); } From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CAB5C77B7A for ; Fri, 19 May 2023 07:43:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231303AbjESHnc (ORCPT ); Fri, 19 May 2023 03:43:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231266AbjESHmv (ORCPT ); Fri, 19 May 2023 03:42:51 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD46E173C for ; Fri, 19 May 2023 00:41:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482106; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g820KSNTP7OMqJOKF462B/JivphTXg8NZzTw8fVuImA=; b=Y1Y4ogU9sVFKhucR6tVRqjC7fEBt3N7wXjv1pxFWgr0x3HNU2528XdrNM7UttaghDTDSMR TinyBQAE/7rNU+8Gh3ZeGqMYFIQdXFHRrkrgxBUx+9UOulFt2jifSHczl5mMWQ0xxjRtEI Ee14dxk42yx7COKtGjSQ51q0GMTI9cY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-438-CWsHB0HDM6aXruJL4c8kbg-1; Fri, 19 May 2023 03:41:40 -0400 X-MC-Unique: CWsHB0HDM6aXruJL4c8kbg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B8C7E29AB3F4; Fri, 19 May 2023 07:41:39 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id D58D7C0004B; Fri, 19 May 2023 07:41:36 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Dominique Martinet , Eric Van Hensbergen , Latchesar Ionkov , Christian Schoenebeck , v9fs@lists.linux.dev Subject: [PATCH v20 11/32] 9p: Add splice_read stub Date: Fri, 19 May 2023 08:40:26 +0100 Message-Id: <20230519074047.1739879-12-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a splice_read stub for 9p. We should use direct_splice_read() if 9PL_DIRECT is set and filemap_splice_read() otherwise. Note that this doesn't seem to be particularly related to O_DIRECT. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Dominique Martinet cc: Eric Van Hensbergen cc: Latchesar Ionkov cc: Christian Schoenebeck cc: v9fs@lists.linux.dev cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/9p/vfs_file.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/fs/9p/vfs_file.c b/fs/9p/vfs_file.c index 6c31b8c8112d..0f3cb439ab2d 100644 --- a/fs/9p/vfs_file.c +++ b/fs/9p/vfs_file.c @@ -374,6 +374,28 @@ v9fs_file_read_iter(struct kiocb *iocb, struct iov_ite= r *to) return ret; } =20 +/* + * v9fs_file_splice_read - splice-read from a file + * @in: The 9p file to read from + * @ppos: Where to find/update the file position + * @pipe: The pipe to splice into + * @len: The maximum amount of data to splice + * @flags: SPLICE_F_* flags + */ +static ssize_t v9fs_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct p9_fid *fid =3D in->private_data; + + p9_debug(P9_DEBUG_VFS, "fid %d count %zu offset %lld\n", + fid->fid, len, *ppos); + + if (fid->mode & P9L_DIRECT) + return direct_splice_read(in, ppos, pipe, len, flags); + return filemap_splice_read(in, ppos, pipe, len, flags); +} + /** * v9fs_file_write_iter - write to a file * @iocb: The operation parameters @@ -569,7 +591,7 @@ const struct file_operations v9fs_file_operations =3D { .release =3D v9fs_dir_release, .lock =3D v9fs_file_lock, .mmap =3D generic_file_readonly_mmap, - .splice_read =3D generic_file_splice_read, + .splice_read =3D v9fs_file_splice_read, .splice_write =3D iter_file_splice_write, .fsync =3D v9fs_file_fsync, }; @@ -583,7 +605,7 @@ const struct file_operations v9fs_file_operations_dotl = =3D { .lock =3D v9fs_file_lock_dotl, .flock =3D v9fs_file_flock_dotl, .mmap =3D v9fs_file_mmap, - .splice_read =3D generic_file_splice_read, + .splice_read =3D v9fs_file_splice_read, .splice_write =3D iter_file_splice_write, .fsync =3D v9fs_file_fsync_dotl, }; From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2DEEC7EE2A for ; Fri, 19 May 2023 07:43:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231379AbjESHnY (ORCPT ); Fri, 19 May 2023 03:43:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230454AbjESHmu (ORCPT ); Fri, 19 May 2023 03:42:50 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1730EC for ; Fri, 19 May 2023 00:41:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482106; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zBEjCltxDmlnA0oSyuagNhPyHRx1meJicqC3uPUtKyk=; b=NEVtnIljkpKx3z/jm3K1GFMsMivjnkRjWMXnmdC4w2K7RQTf03R9xPfr6x2Xmghcak4zOL IyLOCP8MsZZ5QnvyPnQ83XBeZ2EmZIZtHZDs/0hrASzTmdAIrcTtjv3LJlETwjLmaNc6za XVTp/YobX+18dF0xAacUzbdzOuaDef4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-500-_1BGeCTTMTyFaYpS14WWgQ-1; Fri, 19 May 2023 03:41:43 -0400 X-MC-Unique: _1BGeCTTMTyFaYpS14WWgQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B00A03C02180; Fri, 19 May 2023 07:41:42 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7169740CFD45; Fri, 19 May 2023 07:41:40 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Marc Dionne , linux-afs@lists.infradead.org Subject: [PATCH v20 12/32] afs: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:27 +0100 Message-Id: <20230519074047.1739879-13-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for AFS to call afs_validate() before going into generic_file_splice_read() so that i_size can be brought as up to date as possible. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Marc Dionne cc: linux-afs@lists.infradead.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/afs/file.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/fs/afs/file.c b/fs/afs/file.c index 719b31374879..d8a6b09dadf7 100644 --- a/fs/afs/file.c +++ b/fs/afs/file.c @@ -25,6 +25,9 @@ static void afs_invalidate_folio(struct folio *folio, siz= e_t offset, static bool afs_release_folio(struct folio *folio, gfp_t gfp_flags); =20 static ssize_t afs_file_read_iter(struct kiocb *iocb, struct iov_iter *ite= r); +static ssize_t afs_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags); static void afs_vm_open(struct vm_area_struct *area); static void afs_vm_close(struct vm_area_struct *area); static vm_fault_t afs_vm_map_pages(struct vm_fault *vmf, pgoff_t start_pgo= ff, pgoff_t end_pgoff); @@ -36,7 +39,7 @@ const struct file_operations afs_file_operations =3D { .read_iter =3D afs_file_read_iter, .write_iter =3D afs_file_write, .mmap =3D afs_file_mmap, - .splice_read =3D generic_file_splice_read, + .splice_read =3D afs_file_splice_read, .splice_write =3D iter_file_splice_write, .fsync =3D afs_fsync, .lock =3D afs_lock, @@ -587,3 +590,18 @@ static ssize_t afs_file_read_iter(struct kiocb *iocb, = struct iov_iter *iter) =20 return generic_file_read_iter(iocb, iter); } + +static ssize_t afs_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct afs_vnode *vnode =3D AFS_FS_I(file_inode(in)); + struct afs_file *af =3D in->private_data; + int ret; + + ret =3D afs_validate(vnode, af->key); + if (ret < 0) + return ret; + + return generic_file_splice_read(in, ppos, pipe, len, flags); +} From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AB47C7EE2E for ; Fri, 19 May 2023 07:43:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231173AbjESHn1 (ORCPT ); Fri, 19 May 2023 03:43:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230469AbjESHmu (ORCPT ); Fri, 19 May 2023 03:42:50 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D84821AD for ; Fri, 19 May 2023 00:41:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482108; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ul54+Rfc7Xx/QmGKIvy0/Dk3iAPI5N6ujIbh8BdRPD4=; b=MhAaNh1PNsCutRhq2McwVl4hRwU59wNVEixy+tID9x5CJBpkVesehYeQ37dmjlTjA0GdUb oH5+Ax2PoISzG0YKZFGun7Xrp31C3f/dUE2BMdjf3PtMfQvd7KtDfNcaFZsh04dFN3W3DZ idsTJECzVdIfRbbla0KSguSksvNiftE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-542-yZVFR8FJO82q_4bbCE1uzg-1; Fri, 19 May 2023 03:41:46 -0400 X-MC-Unique: yZVFR8FJO82q_4bbCE1uzg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B75B6185A78E; Fri, 19 May 2023 07:41:45 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5D4D04F2DDE; Fri, 19 May 2023 07:41:43 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Xiubo Li , Ilya Dryomov , ceph-devel@vger.kernel.org Subject: [PATCH v20 13/32] ceph: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:28 +0100 Message-Id: <20230519074047.1739879-14-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for Ceph. This does the inode shutdown check before proceeding and jumps to direct_splice_read() if O_DIRECT is set, the file has inline data or is a synchronous file. We try and get FILE_RD and either FILE_CACHE and/or FILE_LAZYIO caps and hold them across filemap_splice_read(). If we fail to get FILE_CACHE or FILE_LAZYIO capabilities, we use direct_splice_read() instead. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Xiubo Li cc: Ilya Dryomov cc: Jeff Layton cc: ceph-devel@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/ceph/file.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 65 insertions(+), 1 deletion(-) diff --git a/fs/ceph/file.c b/fs/ceph/file.c index f4d8bf7dec88..382dd5901748 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -1745,6 +1745,70 @@ static ssize_t ceph_read_iter(struct kiocb *iocb, st= ruct iov_iter *to) return ret; } =20 +/* + * Wrap filemap_splice_read with checks for cap bits on the inode. + * Atomically grab references, so that those bits are not released + * back to the MDS mid-read. + */ +static ssize_t ceph_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct ceph_file_info *fi =3D in->private_data; + struct inode *inode =3D file_inode(in); + struct ceph_inode_info *ci =3D ceph_inode(inode); + ssize_t ret; + int want =3D 0, got =3D 0; + CEPH_DEFINE_RW_CONTEXT(rw_ctx, 0); + + dout("splice_read %p %llx.%llx %llu~%zu trying to get caps on %p\n", + inode, ceph_vinop(inode), *ppos, len, inode); + + if (ceph_inode_is_shutdown(inode)) + return -ESTALE; + + if ((in->f_flags & O_DIRECT) || + ceph_has_inline_data(ci) || + (fi->flags & CEPH_F_SYNC)) + return direct_splice_read(in, ppos, pipe, len, flags); + + ceph_start_io_read(inode); + + want =3D CEPH_CAP_FILE_CACHE; + if (fi->fmode & CEPH_FILE_MODE_LAZY) + want |=3D CEPH_CAP_FILE_LAZYIO; + + ret =3D ceph_get_caps(in, CEPH_CAP_FILE_RD, want, -1, &got); + if (ret < 0) { + ceph_end_io_read(inode); + return ret; + } + + if ((got & (CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_LAZYIO)) =3D=3D 0) { + dout("splice_read/sync %p %llx.%llx %llu~%zu got cap refs on %s\n", + inode, ceph_vinop(inode), *ppos, len, + ceph_cap_string(got)); + + ceph_end_io_read(inode); + return direct_splice_read(in, ppos, pipe, len, flags); + } + + dout("splice_read %p %llx.%llx %llu~%zu got cap refs on %s\n", + inode, ceph_vinop(inode), *ppos, len, ceph_cap_string(got)); + + rw_ctx.caps =3D got; + ceph_add_rw_context(fi, &rw_ctx); + ret =3D filemap_splice_read(in, ppos, pipe, len, flags); + ceph_del_rw_context(fi, &rw_ctx); + + dout("splice_read %p %llx.%llx dropping cap refs on %s =3D %zd\n", + inode, ceph_vinop(inode), ceph_cap_string(got), ret); + ceph_put_cap_refs(ci, got); + + ceph_end_io_read(inode); + return ret; +} + /* * Take cap references to avoid releasing caps to MDS mid-write. * @@ -2593,7 +2657,7 @@ const struct file_operations ceph_file_fops =3D { .lock =3D ceph_lock, .setlease =3D simple_nosetlease, .flock =3D ceph_flock, - .splice_read =3D generic_file_splice_read, + .splice_read =3D ceph_splice_read, .splice_write =3D iter_file_splice_write, .unlocked_ioctl =3D ceph_ioctl, .compat_ioctl =3D compat_ptr_ioctl, From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA4E8C77B7A for ; Fri, 19 May 2023 07:43:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231442AbjESHnx (ORCPT ); Fri, 19 May 2023 03:43:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231215AbjESHm7 (ORCPT ); Fri, 19 May 2023 03:42:59 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F03F3198C for ; Fri, 19 May 2023 00:41:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482113; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/4iKO0WOSo1UJaMQGu6hpLFM8tphuArX0rBWtTeeSuo=; b=K9o80uaup8/jTx31Q7eRAjBkysMAnUFzY0ZcePKMGhaRs4FQaOKvXkO3gMgWoJuZdWP2E/ lpvpib534GtunAgJ+7438LECHGBxJBtkPwWL8iTIKzOPWPckuyTiUqtNjiQ8lTLFc+fETS Abrzzpz2RjEALmo9PIlR2m7ZH5SK7Qo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-187-6zZLhNJbPDyDHIrP4_P9AQ-1; Fri, 19 May 2023 03:41:50 -0400 X-MC-Unique: 6zZLhNJbPDyDHIrP4_P9AQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1A33D29AB3F0; Fri, 19 May 2023 07:41:49 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 767167C2A; Fri, 19 May 2023 07:41:46 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Tyler Hicks , ecryptfs@vger.kernel.org Subject: [PATCH v20 14/32] ecryptfs: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:29 +0100 Message-Id: <20230519074047.1739879-15-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for ecryptfs to update the access time on the lower file after the operation. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Tyler Hicks cc: ecryptfs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/ecryptfs/file.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/fs/ecryptfs/file.c b/fs/ecryptfs/file.c index 268b74499c28..284395587be0 100644 --- a/fs/ecryptfs/file.c +++ b/fs/ecryptfs/file.c @@ -44,6 +44,31 @@ static ssize_t ecryptfs_read_update_atime(struct kiocb *= iocb, return rc; } =20 +/* + * ecryptfs_splice_read_update_atime + * + * generic_file_splice_read updates the atime of upper layer inode. But, = it + * doesn't give us a chance to update the atime of the lower layer inode. = This + * function is a wrapper to generic_file_read. It updates the atime of the + * lower level inode if generic_file_read returns without any errors. This= is + * to be used only for file reads. The function to be used for directory = reads + * is ecryptfs_read. + */ +static ssize_t ecryptfs_splice_read_update_atime(struct file *in, loff_t *= ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + ssize_t rc; + const struct path *path; + + rc =3D generic_file_splice_read(in, ppos, pipe, len, flags); + if (rc >=3D 0) { + path =3D ecryptfs_dentry_to_lower_path(in->f_path.dentry); + touch_atime(path); + } + return rc; +} + struct ecryptfs_getdents_callback { struct dir_context ctx; struct dir_context *caller; @@ -414,5 +439,5 @@ const struct file_operations ecryptfs_main_fops =3D { .release =3D ecryptfs_release, .fsync =3D ecryptfs_fsync, .fasync =3D ecryptfs_fasync, - .splice_read =3D generic_file_splice_read, + .splice_read =3D ecryptfs_splice_read_update_atime, }; From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FD65C77B7A for ; Fri, 19 May 2023 07:43:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231402AbjESHng (ORCPT ); Fri, 19 May 2023 03:43:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231299AbjESHmy (ORCPT ); Fri, 19 May 2023 03:42:54 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D6DB3195 for ; Fri, 19 May 2023 00:41:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482119; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VhlnyEb6mYomF+UXhy/zVZBJsDbPWQmGyqF6ugPT8o4=; b=g44LcoP0pNvLnM9peU51U9YnDZwpVJPFEcZEM8++6OjKYkVkiAg6OqWyVSUs+ueLPahJiF lqfnHo2rbpcG1vc6E1eI1TB2yoLypWgIwcLMDk1SxrhwTLuYaao55lFxa79uijCxIO1Nyu qy2Ey4WgPpdUBZlgGj1Jzl1akXDoqlA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-290-tq44QnmUO9KmQ-WGEl3PcA-1; Fri, 19 May 2023 03:41:53 -0400 X-MC-Unique: tq44QnmUO9KmQ-WGEl3PcA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7B949800B2A; Fri, 19 May 2023 07:41:52 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id B8AEB4F2DE0; Fri, 19 May 2023 07:41:49 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , "Theodore Ts'o" , Andreas Dilger , linux-ext4@vger.kernel.org Subject: [PATCH v20 15/32] ext4: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:30 +0100 Message-Id: <20230519074047.1739879-16-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for Ext4. This does the inode shutdown check before proceeding. Splicing from DAX files is handled by generic_file_splice_read(). Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: "Theodore Ts'o" cc: Andreas Dilger cc: linux-ext4@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/ext4/file.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index d101b3b0c7da..9f8bbd9d131c 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -147,6 +147,17 @@ static ssize_t ext4_file_read_iter(struct kiocb *iocb,= struct iov_iter *to) return generic_file_read_iter(iocb, to); } =20 +static ssize_t ext4_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode =3D file_inode(in); + + if (unlikely(ext4_forced_shutdown(EXT4_SB(inode->i_sb)))) + return -EIO; + return generic_file_splice_read(in, ppos, pipe, len, flags); +} + /* * Called when an inode is released. Note that this is different * from ext4_file_open: open gets called at every open, but release @@ -957,7 +968,7 @@ const struct file_operations ext4_file_operations =3D { .release =3D ext4_release_file, .fsync =3D ext4_sync_file, .get_unmapped_area =3D thp_get_unmapped_area, - .splice_read =3D generic_file_splice_read, + .splice_read =3D ext4_file_splice_read, .splice_write =3D iter_file_splice_write, .fallocate =3D ext4_fallocate, }; From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4209EC77B7F for ; Fri, 19 May 2023 07:43:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231437AbjESHnu (ORCPT ); Fri, 19 May 2023 03:43:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231329AbjESHnB (ORCPT ); Fri, 19 May 2023 03:43:01 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A680710DF for ; Fri, 19 May 2023 00:42:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482122; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r5qDgkT4BMNnpU06yXa6htF6MTIyeI5iJgctjYsdWto=; b=Y0ycKK5T6mVqGYUc5QLGVd2+jHvV9iK/dLjQ6P/eQEy6BptA3gjSe9YoDu/eeiIKtRMP5J /oCfVw2PqeEg/Ghg636pegW2kEwH9huZifPtyTFmsRjSeh76byiHrNb3nyz+K9P6J+JPhY kYZTG1EsfMXPmVpCBlTzerWxyi95Cm0= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-636-UwmqYBTZNaOSEg7543TOgw-1; Fri, 19 May 2023 03:41:56 -0400 X-MC-Unique: UwmqYBTZNaOSEg7543TOgw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ABE8E1C04B50; Fri, 19 May 2023 07:41:55 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 367671121314; Fri, 19 May 2023 07:41:53 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Jaegeuk Kim , Chao Yu , linux-f2fs-devel@lists.sourceforge.net Subject: [PATCH v20 16/32] f2fs: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:31 +0100 Message-Id: <20230519074047.1739879-17-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for f2fs. This does some checks and tracing before proceeding and will switch from direct-I/O to buffered I/O if forced or if misaligned. It also updates the iostats after doing a buffered I/O. [Note: I wonder if I should only do the tracing if I call filemap_splice_read() as direct_splice_read() will call f2fs_file_read_iter().] Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Jaegeuk Kim cc: Chao Yu cc: linux-f2fs-devel@lists.sourceforge.net cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/f2fs/file.c | 68 ++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 60 insertions(+), 8 deletions(-) diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c index 5ac53d2627d2..3723387f4a87 100644 --- a/fs/f2fs/file.c +++ b/fs/f2fs/file.c @@ -4367,22 +4367,23 @@ static ssize_t f2fs_dio_read_iter(struct kiocb *ioc= b, struct iov_iter *to) return ret; } =20 -static void f2fs_trace_rw_file_path(struct kiocb *iocb, size_t count, int = rw) +static void f2fs_trace_rw_file_path(struct file *file, loff_t pos, size_t = count, + int rw) { - struct inode *inode =3D file_inode(iocb->ki_filp); + struct inode *inode =3D file_inode(file); char *buf, *path; =20 buf =3D f2fs_getname(F2FS_I_SB(inode)); if (!buf) return; - path =3D dentry_path_raw(file_dentry(iocb->ki_filp), buf, PATH_MAX); + path =3D dentry_path_raw(file_dentry(file), buf, PATH_MAX); if (IS_ERR(path)) goto free_buf; if (rw =3D=3D WRITE) - trace_f2fs_datawrite_start(inode, iocb->ki_pos, count, + trace_f2fs_datawrite_start(inode, pos, count, current->pid, path, current->comm); else - trace_f2fs_dataread_start(inode, iocb->ki_pos, count, + trace_f2fs_dataread_start(inode, pos, count, current->pid, path, current->comm); free_buf: f2fs_putname(buf); @@ -4398,7 +4399,8 @@ static ssize_t f2fs_file_read_iter(struct kiocb *iocb= , struct iov_iter *to) return -EOPNOTSUPP; =20 if (trace_f2fs_dataread_start_enabled()) - f2fs_trace_rw_file_path(iocb, iov_iter_count(to), READ); + f2fs_trace_rw_file_path(iocb->ki_filp, iocb->ki_pos, + iov_iter_count(to), READ); =20 if (f2fs_should_use_dio(inode, iocb, to)) { ret =3D f2fs_dio_read_iter(iocb, to); @@ -4413,6 +4415,55 @@ static ssize_t f2fs_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) return ret; } =20 +static ssize_t f2fs_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode =3D file_inode(in); + const loff_t pos =3D *ppos; + ssize_t ret; + + if (!f2fs_is_compress_backend_ready(inode)) + return -EOPNOTSUPP; + + if (trace_f2fs_dataread_start_enabled()) + f2fs_trace_rw_file_path(in, pos, len, READ); + + if (in->f_flags & O_DIRECT) { + if (f2fs_force_buffered_io(inode, READ)) + goto buffered; + + /* + * Direct I/O not aligned to the disk's logical_block_size will + * be attempted, but will fail with -EINVAL. + * + * f2fs additionally requires that direct I/O be aligned to the + * filesystem block size, which is often a stricter + * requirement. However, f2fs traditionally falls back to + * buffered I/O on requests that are logical_block_size-aligned + * but not fs-block aligned. + * + * The below logic implements this behavior. + */ + if (!IS_ALIGNED(pos, i_blocksize(inode)) && + IS_ALIGNED(pos, bdev_logical_block_size(inode->i_sb->s_bdev))) + goto buffered; + ret =3D direct_splice_read(in, ppos, pipe, len, flags); + goto done; + } + +buffered: + ret =3D filemap_splice_read(in, ppos, pipe, len, flags); + if (ret > 0) + f2fs_update_iostat(F2FS_I_SB(inode), inode, + APP_BUFFERED_READ_IO, ret); + +done: + if (trace_f2fs_dataread_end_enabled()) + trace_f2fs_dataread_end(inode, pos, ret); + return ret; +} + static ssize_t f2fs_write_checks(struct kiocb *iocb, struct iov_iter *from) { struct file *file =3D iocb->ki_filp; @@ -4714,7 +4765,8 @@ static ssize_t f2fs_file_write_iter(struct kiocb *ioc= b, struct iov_iter *from) ret =3D preallocated; } else { if (trace_f2fs_datawrite_start_enabled()) - f2fs_trace_rw_file_path(iocb, orig_count, WRITE); + f2fs_trace_rw_file_path(iocb->ki_filp, iocb->ki_pos, + orig_count, WRITE); =20 /* Do the actual write. */ ret =3D dio ? @@ -4919,7 +4971,7 @@ const struct file_operations f2fs_file_operations =3D= { #ifdef CONFIG_COMPAT .compat_ioctl =3D f2fs_compat_ioctl, #endif - .splice_read =3D generic_file_splice_read, + .splice_read =3D f2fs_file_splice_read, .splice_write =3D iter_file_splice_write, .fadvise =3D f2fs_file_fadvise, }; From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D6A6C77B7A for ; Fri, 19 May 2023 07:43:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231448AbjESHn4 (ORCPT ); Fri, 19 May 2023 03:43:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231330AbjESHnB (ORCPT ); Fri, 19 May 2023 03:43:01 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E9A310EF for ; Fri, 19 May 2023 00:42:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482123; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NhBgu+YPWWuhiMuhxeBoG/WfhrSyIDZx4YN21OB8Xyg=; b=KSy9Af7+QCMXPjwGP7ZJWUK0soGYO6KpCNLd07IE3qNAHAArNdl6rySX22Xn1imIcfA5kB T9lgpNEF+cN8ZE/R80ba60f2Epx6/Ka/AY1+u9ZZ0IzcnSLq+zVR89ADZkNRXC8f6B2i3/ NLLTMOG+bvM5OQhenDLooFcMDfF+Cr8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-675-eDOVwGWEMCmaAhO0IYU1iA-1; Fri, 19 May 2023 03:41:59 -0400 X-MC-Unique: eDOVwGWEMCmaAhO0IYU1iA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EABBE3C01D8C; Fri, 19 May 2023 07:41:58 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5AC6F2166B28; Fri, 19 May 2023 07:41:56 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org Subject: [PATCH v20 17/32] nfs: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:32 +0100 Message-Id: <20230519074047.1739879-18-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for NFS. This locks the inode around filemap_splice_read() and revalidates the mapping. It cannot do this around direct_splice_read() as that calls ->read_iter(). Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Trond Myklebust cc: Anna Schumaker cc: linux-nfs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/nfs/file.c | 26 +++++++++++++++++++++++++- fs/nfs/internal.h | 2 ++ fs/nfs/nfs4file.c | 2 +- 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/fs/nfs/file.c b/fs/nfs/file.c index f0edf5a36237..91fe48b47f3c 100644 --- a/fs/nfs/file.c +++ b/fs/nfs/file.c @@ -178,6 +178,30 @@ nfs_file_read(struct kiocb *iocb, struct iov_iter *to) } EXPORT_SYMBOL_GPL(nfs_file_read); =20 +ssize_t +nfs_file_splice_read(struct file *in, loff_t *ppos, struct pipe_inode_info= *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode =3D file_inode(in); + ssize_t result; + + if (in->f_flags & O_DIRECT) + return direct_splice_read(in, ppos, pipe, len, flags); + + dprintk("NFS: splice_read(%pD2, %zu@%lu)\n", in, len, *pos); + + nfs_start_io_read(inode); + result =3D nfs_revalidate_mapping(inode, in->f_mapping); + if (!result) { + result =3D filemap_splice_read(in, ppos, pipe, len, flags); + if (result > 0) + nfs_add_stats(inode, NFSIOS_NORMALREADBYTES, result); + } + nfs_end_io_read(inode); + return result; +} +EXPORT_SYMBOL_GPL(nfs_file_splice_read); + int nfs_file_mmap(struct file * file, struct vm_area_struct * vma) { @@ -879,7 +903,7 @@ const struct file_operations nfs_file_operations =3D { .fsync =3D nfs_file_fsync, .lock =3D nfs_lock, .flock =3D nfs_flock, - .splice_read =3D generic_file_splice_read, + .splice_read =3D nfs_file_splice_read, .splice_write =3D iter_file_splice_write, .check_flags =3D nfs_check_flags, .setlease =3D simple_nosetlease, diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 3cc027d3bd58..b5f21d35d30e 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -416,6 +416,8 @@ static inline __u32 nfs_access_xattr_mask(const struct = nfs_server *server) int nfs_file_fsync(struct file *file, loff_t start, loff_t end, int datasy= nc); loff_t nfs_file_llseek(struct file *, loff_t, int); ssize_t nfs_file_read(struct kiocb *, struct iov_iter *); +ssize_t nfs_file_splice_read(struct file *in, loff_t *ppos, struct pipe_in= ode_info *pipe, + size_t len, unsigned int flags); int nfs_file_mmap(struct file *, struct vm_area_struct *); ssize_t nfs_file_write(struct kiocb *, struct iov_iter *); int nfs_file_release(struct inode *, struct file *); diff --git a/fs/nfs/nfs4file.c b/fs/nfs/nfs4file.c index 2563ed8580f3..4aeadd6e1a6d 100644 --- a/fs/nfs/nfs4file.c +++ b/fs/nfs/nfs4file.c @@ -454,7 +454,7 @@ const struct file_operations nfs4_file_operations =3D { .fsync =3D nfs_file_fsync, .lock =3D nfs_lock, .flock =3D nfs_flock, - .splice_read =3D generic_file_splice_read, + .splice_read =3D nfs_file_splice_read, .splice_write =3D iter_file_splice_write, .check_flags =3D nfs_check_flags, .setlease =3D nfs4_setlease, From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F17FC77B7F for ; Fri, 19 May 2023 07:44:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230417AbjESHoN (ORCPT ); Fri, 19 May 2023 03:44:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231216AbjESHnN (ORCPT ); Fri, 19 May 2023 03:43:13 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B42871997 for ; Fri, 19 May 2023 00:42:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482130; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MIKgygGFPP04jCWudTIx6Exr1BpE2idki489iWC06JA=; b=b5p3EgyaoBdcAW+MZ8niqVl8BOWkVSrN3tj4rrtmwYJgjNIwB8BHoH4XIZ8znDvVZp5qfn hiOdvXS2ezSRQDYUmi6hIzswuzg1/ePfo3zjPHKryT/DKm/GTSegn5ozO1v2s3JHaKe7R7 XViaLEareFGiLerWhC62itqCYgCIttE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-76-BiSEfFWJO5ihxPcX24ACRw-1; Fri, 19 May 2023 03:42:03 -0400 X-MC-Unique: BiSEfFWJO5ihxPcX24ACRw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ADED48027F5; Fri, 19 May 2023 07:42:02 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BEDF40CFD45; Fri, 19 May 2023 07:41:59 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Konstantin Komarov , ntfs3@lists.linux.dev Subject: [PATCH v20 18/32] ntfs3: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:33 +0100 Message-Id: <20230519074047.1739879-19-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for NTFS3 to perform various checks before allowing the operation to proceed. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Konstantin Komarov cc: ntfs3@lists.linux.dev cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/ntfs3/file.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/fs/ntfs3/file.c b/fs/ntfs3/file.c index 9a3d55c367d9..c27814946aa3 100644 --- a/fs/ntfs3/file.c +++ b/fs/ntfs3/file.c @@ -744,6 +744,40 @@ static ssize_t ntfs_file_read_iter(struct kiocb *iocb,= struct iov_iter *iter) return generic_file_read_iter(iocb, iter); } =20 +static ssize_t ntfs_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode =3D in->f_mapping->host; + struct ntfs_inode *ni =3D ntfs_i(inode); + + if (is_encrypted(ni)) { + ntfs_inode_warn(inode, "encrypted i/o not supported"); + return -EOPNOTSUPP; + } + + if (is_compressed(ni) && (in->f_flags & O_DIRECT)) { + ntfs_inode_warn(inode, "direct i/o + compressed not supported"); + return -EOPNOTSUPP; + } + +#ifndef CONFIG_NTFS3_LZX_XPRESS + if (ni->ni_flags & NI_FLAG_COMPRESSED_MASK) { + ntfs_inode_warn( + inode, + "activate CONFIG_NTFS3_LZX_XPRESS to read external compressed files"); + return -EOPNOTSUPP; + } +#endif + + if (is_dedup(ni)) { + ntfs_inode_warn(inode, "read deduplicated not supported"); + return -EOPNOTSUPP; + } + + return generic_file_splice_read(in, ppos, pipe, len, flags); +} + /* * ntfs_get_frame_pages * @@ -1159,7 +1193,7 @@ const struct file_operations ntfs_file_operations =3D= { #ifdef CONFIG_COMPAT .compat_ioctl =3D ntfs_compat_ioctl, #endif - .splice_read =3D generic_file_splice_read, + .splice_read =3D ntfs_file_splice_read, .mmap =3D ntfs_file_mmap, .open =3D ntfs_file_open, .fsync =3D generic_file_fsync, From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AEC7C77B7A for ; Fri, 19 May 2023 07:44:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231501AbjESHow (ORCPT ); Fri, 19 May 2023 03:44:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231409AbjESHni (ORCPT ); Fri, 19 May 2023 03:43:38 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 793A719BA for ; Fri, 19 May 2023 00:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gBG1rbAQ+/pvrFxoUU2GMJeglrqrT3cq++hcE5SQo7w=; b=isULdRdZ5A8Vh41OscA8cV8RG+Qah0NQQm4uY37zr+D81Of0I/IEhSS7okoyt8tQ+xDLnt NQV/85LAC5ILAioT8SPnZxNoSVvqPbi1Smh/g2HyFOmiWiUdBz8BUVeR7n4+g/R/34cpLF pKRspjSKKUbPcTxxqbCDyysJygaKurc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-465-_MfV9O_bPYGXDbkMXz1GRA-1; Fri, 19 May 2023 03:42:06 -0400 X-MC-Unique: _MfV9O_bPYGXDbkMXz1GRA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F0AF585A5BA; Fri, 19 May 2023 07:42:05 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 505E5C0004B; Fri, 19 May 2023 07:42:03 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Mark Fasheh , Joel Becker , Joseph Qi , ocfs2-devel@oss.oracle.com Subject: [PATCH v20 19/32] ocfs2: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:34 +0100 Message-Id: <20230519074047.1739879-20-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for ocfs2. This emits trace lines and does an atime lock/update before calling filemap_splice_read(). It doesn't do this for around direct_splice_read() as that will call ->read_iter(). A couple of new tracepoints are added for this purpose. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Mark Fasheh cc: Joel Becker cc: Joseph Qi cc: ocfs2-devel@oss.oracle.com cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/ocfs2/file.c | 42 +++++++++++++++++++++++++++++++++++++++++- fs/ocfs2/ocfs2_trace.h | 3 +++ 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c index efb09de4343d..27c54a71ec57 100644 --- a/fs/ocfs2/file.c +++ b/fs/ocfs2/file.c @@ -2581,6 +2581,46 @@ static ssize_t ocfs2_file_read_iter(struct kiocb *io= cb, return ret; } =20 +static ssize_t ocfs2_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode =3D file_inode(in); + ssize_t ret =3D 0; + int lock_level =3D 0; + + trace_ocfs2_file_splice_read(inode, in, in->f_path.dentry, + (unsigned long long)OCFS2_I(inode)->ip_blkno, + in->f_path.dentry->d_name.len, + in->f_path.dentry->d_name.name, + 0); + + if (in->f_flags & O_DIRECT) + return direct_splice_read(in, ppos, pipe, len, flags); + + /* + * We're fine letting folks race truncates and extending writes with + * read across the cluster, just like they can locally. Hence no + * rw_lock during read. + * + * Take and drop the meta data lock to update inode fields like i_size. + * This allows the checks down below generic_file_splice_read() a + * chance of actually working. + */ + ret =3D ocfs2_inode_lock_atime(inode, in->f_path.mnt, &lock_level, true); + if (ret < 0) { + if (ret !=3D -EAGAIN) + mlog_errno(ret); + goto bail; + } + ocfs2_inode_unlock(inode, lock_level); + + ret =3D filemap_splice_read(in, ppos, pipe, len, flags); + trace_filemap_splice_read_ret(ret); +bail: + return ret; +} + /* Refer generic_file_llseek_unlocked() */ static loff_t ocfs2_file_llseek(struct file *file, loff_t offset, int when= ce) { @@ -2744,7 +2784,7 @@ const struct file_operations ocfs2_fops =3D { #endif .lock =3D ocfs2_lock, .flock =3D ocfs2_flock, - .splice_read =3D generic_file_splice_read, + .splice_read =3D ocfs2_file_splice_read, .splice_write =3D iter_file_splice_write, .fallocate =3D ocfs2_fallocate, .remap_file_range =3D ocfs2_remap_file_range, diff --git a/fs/ocfs2/ocfs2_trace.h b/fs/ocfs2/ocfs2_trace.h index dc4bce1649c1..b8c3d1702076 100644 --- a/fs/ocfs2/ocfs2_trace.h +++ b/fs/ocfs2/ocfs2_trace.h @@ -1319,6 +1319,8 @@ DEFINE_OCFS2_FILE_OPS(ocfs2_file_splice_write); =20 DEFINE_OCFS2_FILE_OPS(ocfs2_file_read_iter); =20 +DEFINE_OCFS2_FILE_OPS(ocfs2_file_splice_read); + DEFINE_OCFS2_ULL_ULL_ULL_EVENT(ocfs2_truncate_file); =20 DEFINE_OCFS2_ULL_ULL_EVENT(ocfs2_truncate_file_error); @@ -1470,6 +1472,7 @@ TRACE_EVENT(ocfs2_prepare_inode_for_write, ); =20 DEFINE_OCFS2_INT_EVENT(generic_file_read_iter_ret); +DEFINE_OCFS2_INT_EVENT(filemap_splice_read_ret); =20 /* End of trace events for fs/ocfs2/file.c. */ From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EA97C77B7F for ; Fri, 19 May 2023 07:44:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231486AbjESHo1 (ORCPT ); Fri, 19 May 2023 03:44:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230119AbjESHnc (ORCPT ); Fri, 19 May 2023 03:43:32 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D78719F for ; Fri, 19 May 2023 00:42:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482133; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uuCSAVZVpWhK4klEEqx7Ct2l0SGs5ZJdbhyX0+gkImI=; b=RiFdtn8rdi0r6MqnwRMpeS9k8j4SYLjZ278vKPRVJcIkLDB2cQlDnp1kHX9gpE1gItNp3k 9kgaRnq9vAV6IVlcJ5nNRKEk3SBbRyci7a6W5GoXop0zaeIiHXU8/2jUuaye/ofJz6NATs 6xfcwwpAznSFgRT+u+0HdLLFF8qTvog= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-570-9WYmFVhKPfigCOXLKK6NJQ-1; Fri, 19 May 2023 03:42:10 -0400 X-MC-Unique: 9WYmFVhKPfigCOXLKK6NJQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 43746101A53B; Fri, 19 May 2023 07:42:09 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id A998E40D1B60; Fri, 19 May 2023 07:42:06 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Mike Marshall , Martin Brandenburg , devel@lists.orangefs.org Subject: [PATCH v20 20/32] orangefs: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:35 +0100 Message-Id: <20230519074047.1739879-21-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for ocfs2. This increments the read stats and then locks the inode across the call to filemap_splice_read() and a revalidation of the mapping. The lock must not be held across the call to direct_splice_read() as this calls ->read_iter(); this might also mean that the stat increment doesn't need to be done in DIO mode. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Mike Marshall cc: Martin Brandenburg cc: devel@lists.orangefs.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/orangefs/file.c | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/fs/orangefs/file.c b/fs/orangefs/file.c index 1a4301a38aa7..c1eae145e4a8 100644 --- a/fs/orangefs/file.c +++ b/fs/orangefs/file.c @@ -337,6 +337,29 @@ static ssize_t orangefs_file_read_iter(struct kiocb *i= ocb, return ret; } =20 +static ssize_t orangefs_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode =3D file_inode(in); + ssize_t ret; + + orangefs_stats.reads++; + + if (in->f_flags & O_DIRECT) + return direct_splice_read(in, ppos, pipe, len, flags); + + down_read(&inode->i_rwsem); + ret =3D orangefs_revalidate_mapping(inode); + if (ret) + goto out; + + ret =3D filemap_splice_read(in, ppos, pipe, len, flags); +out: + up_read(&inode->i_rwsem); + return ret; +} + static ssize_t orangefs_file_write_iter(struct kiocb *iocb, struct iov_iter *iter) { @@ -556,7 +579,7 @@ const struct file_operations orangefs_file_operations = =3D { .lock =3D orangefs_lock, .mmap =3D orangefs_file_mmap, .open =3D generic_file_open, - .splice_read =3D generic_file_splice_read, + .splice_read =3D orangefs_file_splice_read, .splice_write =3D iter_file_splice_write, .flush =3D orangefs_flush, .release =3D orangefs_file_release, From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A4E0C77B7F for ; Fri, 19 May 2023 07:44:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229649AbjESHoj (ORCPT ); Fri, 19 May 2023 03:44:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231404AbjESHng (ORCPT ); Fri, 19 May 2023 03:43:36 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1EEE41BC8 for ; Fri, 19 May 2023 00:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xmbbe/+Pf//AKyGpf7irbs4+/+24QQIbTBxvLgLUtSY=; b=CNoAd82zd+jKDL04pJ1RMek2uSeoRQ1HCXiNtfkKkvrBk7oVLHJY3I2UqdcPCWCZE/Hu0B XFJ+WXUBFD4RBnWU9+DKQr/0g0n4e5zETScWJyfR4C38cYB0czkqpB4nJwVWKIiceZytGI oCEhWqvBB8Nfsv7r3xhYL3500tbpMhk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-498-MoZDh_T_MgiO891JIsJM_g-1; Fri, 19 May 2023 03:42:13 -0400 X-MC-Unique: MoZDh_T_MgiO891JIsJM_g-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 337F7101A55C; Fri, 19 May 2023 07:42:12 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id D137B492B0A; Fri, 19 May 2023 07:42:09 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , "Darrick J . Wong" , linux-xfs@vger.kernel.org Subject: [PATCH v20 21/32] xfs: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:36 +0100 Message-Id: <20230519074047.1739879-22-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for XFS. This does a stat count and a shutdown check before proceeding. For buffered I/O, it emits a new trace line and locks the inode across the call to filemap_splice_read() and adds to the stats afterwards. Splicing from DAX files either copies the data into a buffer (DIO) or splices from the pagecache. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Darrick J. Wong cc: linux-xfs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/xfs/xfs_file.c | 33 ++++++++++++++++++++++++++++++++- fs/xfs/xfs_trace.h | 2 +- 2 files changed, 33 insertions(+), 2 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index aede746541f8..bb1ec755a709 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -306,6 +306,37 @@ xfs_file_read_iter( return ret; } =20 +STATIC ssize_t +xfs_file_splice_read( + struct file *in, + loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, + unsigned int flags) +{ + struct inode *inode =3D file_inode(in); + struct xfs_inode *ip =3D XFS_I(inode); + struct xfs_mount *mp =3D ip->i_mount; + ssize_t ret =3D 0; + + XFS_STATS_INC(mp, xs_read_calls); + + if (xfs_is_shutdown(mp)) + return -EIO; + + if (in->f_flags & O_DIRECT) + return direct_splice_read(in, ppos, pipe, len, flags); + + trace_xfs_file_splice_read(ip, *ppos, len); + + xfs_ilock(ip, XFS_IOLOCK_SHARED); + ret =3D filemap_splice_read(in, ppos, pipe, len, flags); + xfs_iunlock(ip, XFS_IOLOCK_SHARED); + if (ret > 0) + XFS_STATS_ADD(mp, xs_read_bytes, ret); + return ret; +} + /* * Common pre-write limit and setup checks. * @@ -1423,7 +1454,7 @@ const struct file_operations xfs_file_operations =3D { .llseek =3D xfs_file_llseek, .read_iter =3D xfs_file_read_iter, .write_iter =3D xfs_file_write_iter, - .splice_read =3D generic_file_splice_read, + .splice_read =3D xfs_file_splice_read, .splice_write =3D iter_file_splice_write, .iopoll =3D iocb_bio_iopoll, .unlocked_ioctl =3D xfs_file_ioctl, diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h index cd4ca5b1fcb0..4db669203149 100644 --- a/fs/xfs/xfs_trace.h +++ b/fs/xfs/xfs_trace.h @@ -1445,7 +1445,6 @@ DEFINE_RW_EVENT(xfs_file_direct_write); DEFINE_RW_EVENT(xfs_file_dax_write); DEFINE_RW_EVENT(xfs_reflink_bounce_dio_write); =20 - DECLARE_EVENT_CLASS(xfs_imap_class, TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count, int whichfork, struct xfs_bmbt_irec *irec), @@ -1535,6 +1534,7 @@ DEFINE_SIMPLE_IO_EVENT(xfs_zero_eof); DEFINE_SIMPLE_IO_EVENT(xfs_end_io_direct_write); DEFINE_SIMPLE_IO_EVENT(xfs_end_io_direct_write_unwritten); DEFINE_SIMPLE_IO_EVENT(xfs_end_io_direct_write_append); +DEFINE_SIMPLE_IO_EVENT(xfs_file_splice_read); =20 DECLARE_EVENT_CLASS(xfs_itrunc_class, TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size), From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A11A6C77B7A for ; Fri, 19 May 2023 07:46:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231307AbjESHqg (ORCPT ); Fri, 19 May 2023 03:46:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231136AbjESHqQ (ORCPT ); Fri, 19 May 2023 03:46:16 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8760F1BD5 for ; Fri, 19 May 2023 00:43:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482155; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pyQng7zN0KcMKuuU4UJQFoNAdqIUtbeSzOIbMt9WEc4=; b=HQQrXhhOQB8FLOlUifrSQjkfpWlY3c9ECqS7BXIJhUvcpac57Usvx/V/g3MTcb4C0KGTCS 23VzMU9Pio8qidzI10oGqaz/l55PaF246X5ILV26NOOerSiX+OVC8B8+JavZOo1k26XYlX cGTKv2leaipCT0IbuK83jJEvQp12MHU= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-368-9SkW8NWEPuqdpOcKe74MkQ-1; Fri, 19 May 2023 03:42:16 -0400 X-MC-Unique: 9SkW8NWEPuqdpOcKe74MkQ-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CAF9738035B0; Fri, 19 May 2023 07:42:15 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id E84F4492B0A; Fri, 19 May 2023 07:42:12 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , "Darrick J . Wong" , linux-xfs@vger.kernel.org Subject: [PATCH v20 22/32] zonefs: Provide a splice-read stub Date: Fri, 19 May 2023 08:40:37 +0100 Message-Id: <20230519074047.1739879-23-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide a splice_read stub for zonefs. This does some checks before proceeding. For buffered I/O, it locks the inode across the call to filemap_splice_read() and does a size check in case of truncation. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Darrick J. Wong cc: linux-xfs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org --- fs/zonefs/file.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/fs/zonefs/file.c b/fs/zonefs/file.c index 132f01d3461f..a8d7bae4910d 100644 --- a/fs/zonefs/file.c +++ b/fs/zonefs/file.c @@ -752,6 +752,47 @@ static ssize_t zonefs_file_read_iter(struct kiocb *ioc= b, struct iov_iter *to) return ret; } =20 +static ssize_t zonefs_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode =3D file_inode(in); + struct zonefs_inode_info *zi =3D ZONEFS_I(inode); + struct zonefs_zone *z =3D zonefs_inode_zone(inode); + loff_t isize; + ssize_t ret =3D 0; + + /* Offline zones cannot be read */ + if (unlikely(IS_IMMUTABLE(inode) && !(inode->i_mode & 0777))) + return -EPERM; + + if (*ppos >=3D z->z_capacity) + return 0; + + if (in->f_flags & O_DIRECT) + return direct_splice_read(in, ppos, pipe, len, flags); + + inode_lock_shared(inode); + + /* Limit read operations to written data */ + mutex_lock(&zi->i_truncate_mutex); + isize =3D i_size_read(inode); + if (*ppos >=3D isize) + len =3D 0; + else + len =3D min_t(loff_t, len, isize - *ppos); + mutex_unlock(&zi->i_truncate_mutex); + + if (len > 0) { + ret =3D filemap_splice_read(in, ppos, pipe, len, flags); + if (ret =3D=3D -EIO) + zonefs_io_error(inode, false); + } + + inode_unlock_shared(inode); + return ret; +} + /* * Write open accounting is done only for sequential files. */ @@ -896,7 +937,7 @@ const struct file_operations zonefs_file_operations =3D= { .llseek =3D zonefs_file_llseek, .read_iter =3D zonefs_file_read_iter, .write_iter =3D zonefs_file_write_iter, - .splice_read =3D generic_file_splice_read, + .splice_read =3D zonefs_file_splice_read, .splice_write =3D iter_file_splice_write, .iopoll =3D iocb_bio_iopoll, }; From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32FE3C77B7F for ; Fri, 19 May 2023 07:44:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230451AbjESHor (ORCPT ); Fri, 19 May 2023 03:44:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231342AbjESHoG (ORCPT ); Fri, 19 May 2023 03:44:06 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05E521716 for ; Fri, 19 May 2023 00:42:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mbYjI1mFg3thAEdOWfyRBtPSF9SFp7VqzA4DBX9XOaM=; b=WCrcyP08WfHWAkoqyq8aqaP5sLteQR/xHraMnZxvpd3pherzrYPc2rfTvWryXhRAio9PyP u/k+BCPSoui+qPnsghnWr7hu/g0uI0EY3jQ+UKN3c4k4cccWxhsP7wFKhNEeavvUrCAphf /3Nbs9JmzyAuXBf1DQSilWxP2A9UpWw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-633-8jk5gKwVNnagZwC444Xzmg-1; Fri, 19 May 2023 03:42:19 -0400 X-MC-Unique: 8jk5gKwVNnagZwC444Xzmg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DF47D800C81; Fri, 19 May 2023 07:42:18 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 759107AE4; Fri, 19 May 2023 07:42:16 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Steven Rostedt , Masami Hiramatsu , linux-trace-kernel@vger.kernel.org Subject: [PATCH v20 23/32] splice: Convert trace/seq to use direct_splice_read() Date: Fri, 19 May 2023 08:40:38 +0100 Message-Id: <20230519074047.1739879-24-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" For the splice from the trace seq buffer, just use direct_splice_read(). In the future, something better can probably be done by gifting pages from seq->buf into the pipe, but that would require changing seq->buf into a vmap over an array of pages. Signed-off-by: David Howells cc: Christoph Hellwig cc: Al Viro cc: Jens Axboe cc: Steven Rostedt cc: Masami Hiramatsu cc: linux-kernel@vger.kernel.org cc: linux-trace-kernel@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org Acked-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index ebc59781456a..b664020efcb7 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -5171,7 +5171,7 @@ static const struct file_operations tracing_fops =3D { .open =3D tracing_open, .read =3D seq_read, .read_iter =3D seq_read_iter, - .splice_read =3D generic_file_splice_read, + .splice_read =3D direct_splice_read, .write =3D tracing_write_stub, .llseek =3D tracing_lseek, .release =3D tracing_release, From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 358DCC7EE2C for ; Fri, 19 May 2023 07:45:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231523AbjESHp0 (ORCPT ); Fri, 19 May 2023 03:45:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231392AbjESHoa (ORCPT ); Fri, 19 May 2023 03:44:30 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 397B91FEE for ; Fri, 19 May 2023 00:42:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482145; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KIPsbdoO43iq3tUvemirM6xG5g+su+9PgNJxpKHBcQ0=; b=NeZIYNh4xGDAluvXgKXWCmEjmZqKt74LWOnOB4OWf8BZF7FIXGfkn3ydptCkhwpROYXKSn vosWrMsCZuvH3p2WvsAa7K3Jy+RvF92b/5BGSYuz8jL/9Chg5Cdz3CRP1c59opl3vb7HC1 K7vPYGNR7bVVxmuof3IJ+b0hI0yJhf4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-581-aSvtV5ZQOk2ZrLbIVY_T4A-1; Fri, 19 May 2023 03:42:23 -0400 X-MC-Unique: aSvtV5ZQOk2ZrLbIVY_T4A-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 80C8380013A; Fri, 19 May 2023 07:42:22 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1CE32492B0A; Fri, 19 May 2023 07:42:20 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Steve French , John Hubbard , linux-cifs@vger.kernel.org Subject: [PATCH v20 24/32] splice: Do splice read from a file without using ITER_PIPE Date: Fri, 19 May 2023 08:40:39 +0100 Message-Id: <20230519074047.1739879-25-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Make generic_file_splice_read() use filemap_splice_read() and direct_splice_read() rather than using an ITER_PIPE and call_read_iter(). With this, ITER_PIPE is no longer used. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Steve French cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-cifs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org --- Notes: ver #20) - Use s_maxbytes from the backing store (in->f_mapping), not the front inode (especially for a blockdev). =20 ver #18) - Split out the change to cifs to make it use generic_file_splice_read= (). - Split out the unexport of filemap_splice_read() (still needed by cif= s). fs/splice.c | 31 +++++++------------------------ 1 file changed, 7 insertions(+), 24 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 7b818b5b18d4..56d9802729d0 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -417,34 +417,17 @@ ssize_t generic_file_splice_read(struct file *in, lof= f_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags) { - struct iov_iter to; - struct kiocb kiocb; - int ret; - + if (unlikely(*ppos >=3D in->f_mapping->host->i_sb->s_maxbytes)) + return 0; + if (unlikely(!len)) + return 0; #ifdef CONFIG_FS_DAX if (IS_DAX(in->f_mapping->host)) return direct_splice_read(in, ppos, pipe, len, flags); #endif - - iov_iter_pipe(&to, ITER_DEST, pipe, len); - init_sync_kiocb(&kiocb, in); - kiocb.ki_pos =3D *ppos; - ret =3D call_read_iter(in, &kiocb, &to); - if (ret > 0) { - *ppos =3D kiocb.ki_pos; - file_accessed(in); - } else if (ret < 0) { - /* free what was emitted */ - pipe_discard_from(pipe, to.start_head); - /* - * callers of ->splice_read() expect -EAGAIN on - * "can't put anything in there", rather than -EFAULT. - */ - if (ret =3D=3D -EFAULT) - ret =3D -EAGAIN; - } - - return ret; + if ((in->f_flags & O_DIRECT)) + return direct_splice_read(in, ppos, pipe, len, flags); + return filemap_splice_read(in, ppos, pipe, len, flags); } EXPORT_SYMBOL(generic_file_splice_read); From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08F5AC7EE2C for ; Fri, 19 May 2023 07:45:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231200AbjESHpi (ORCPT ); Fri, 19 May 2023 03:45:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231136AbjESHpF (ORCPT ); Fri, 19 May 2023 03:45:05 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C3714E7F for ; Fri, 19 May 2023 00:42:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g1kSgt8JtA0JiFV4X1i6k/i3VuOpNNHzBh5elSR8Nt8=; b=KTIvCVm8rjMVd5PW0wfLo0iBXcSYQDWNQprLilwuyHP0IlbRUlpUKOUl94HDZyABDmeOMW JiJr41pPQmms954jQ55QaQXOzoGCSQU/wPZ31fR2MvWfo2fyP6CnelMHcH6QbRNJaro7s/ hDOliYJldTmcth1sw9tEs/v06lbjs4s= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-189-_2ymUi_4PoOAUMbTHfhg_A-1; Fri, 19 May 2023 03:42:27 -0400 X-MC-Unique: _2ymUi_4PoOAUMbTHfhg_A-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AAB49101A55C; Fri, 19 May 2023 07:42:26 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1D994C0004B; Fri, 19 May 2023 07:42:23 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Paulo Alcantara , Steve French , John Hubbard , linux-cifs@vger.kernel.org Subject: [PATCH v20 25/32] cifs: Use generic_file_splice_read() Date: Fri, 19 May 2023 08:40:40 +0100 Message-Id: <20230519074047.1739879-26-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Make cifs use generic_file_splice_read() rather than doing it for itself. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: Paulo Alcantara (SUSE) cc: Jens Axboe cc: Steve French cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-cifs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org --- Notes: ver #20) - Don't remove the export of filemap_splice_read(). =20 ver #18) - Split out from change to generic_file_splice_read(). fs/cifs/cifsfs.c | 8 ++++---- fs/cifs/cifsfs.h | 3 --- fs/cifs/file.c | 16 ---------------- 3 files changed, 4 insertions(+), 23 deletions(-) diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c index 43a4d8603db3..257587a6cade 100644 --- a/fs/cifs/cifsfs.c +++ b/fs/cifs/cifsfs.c @@ -1376,7 +1376,7 @@ const struct file_operations cifs_file_ops =3D { .fsync =3D cifs_fsync, .flush =3D cifs_flush, .mmap =3D cifs_file_mmap, - .splice_read =3D cifs_splice_read, + .splice_read =3D generic_file_splice_read, .splice_write =3D iter_file_splice_write, .llseek =3D cifs_llseek, .unlocked_ioctl =3D cifs_ioctl, @@ -1396,7 +1396,7 @@ const struct file_operations cifs_file_strict_ops =3D= { .fsync =3D cifs_strict_fsync, .flush =3D cifs_flush, .mmap =3D cifs_file_strict_mmap, - .splice_read =3D cifs_splice_read, + .splice_read =3D generic_file_splice_read, .splice_write =3D iter_file_splice_write, .llseek =3D cifs_llseek, .unlocked_ioctl =3D cifs_ioctl, @@ -1434,7 +1434,7 @@ const struct file_operations cifs_file_nobrl_ops =3D { .fsync =3D cifs_fsync, .flush =3D cifs_flush, .mmap =3D cifs_file_mmap, - .splice_read =3D cifs_splice_read, + .splice_read =3D generic_file_splice_read, .splice_write =3D iter_file_splice_write, .llseek =3D cifs_llseek, .unlocked_ioctl =3D cifs_ioctl, @@ -1452,7 +1452,7 @@ const struct file_operations cifs_file_strict_nobrl_o= ps =3D { .fsync =3D cifs_strict_fsync, .flush =3D cifs_flush, .mmap =3D cifs_file_strict_mmap, - .splice_read =3D cifs_splice_read, + .splice_read =3D generic_file_splice_read, .splice_write =3D iter_file_splice_write, .llseek =3D cifs_llseek, .unlocked_ioctl =3D cifs_ioctl, diff --git a/fs/cifs/cifsfs.h b/fs/cifs/cifsfs.h index 74cd6fafb33e..d7274eefc666 100644 --- a/fs/cifs/cifsfs.h +++ b/fs/cifs/cifsfs.h @@ -100,9 +100,6 @@ extern ssize_t cifs_strict_readv(struct kiocb *iocb, st= ruct iov_iter *to); extern ssize_t cifs_user_writev(struct kiocb *iocb, struct iov_iter *from); extern ssize_t cifs_direct_writev(struct kiocb *iocb, struct iov_iter *fro= m); extern ssize_t cifs_strict_writev(struct kiocb *iocb, struct iov_iter *fro= m); -extern ssize_t cifs_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, size_t len, - unsigned int flags); extern int cifs_flock(struct file *pfile, int cmd, struct file_lock *plock= ); extern int cifs_lock(struct file *, int, struct file_lock *); extern int cifs_fsync(struct file *, loff_t, loff_t, int); diff --git a/fs/cifs/file.c b/fs/cifs/file.c index c5fcefdfd797..375a8037a3f3 100644 --- a/fs/cifs/file.c +++ b/fs/cifs/file.c @@ -5078,19 +5078,3 @@ const struct address_space_operations cifs_addr_ops_= smallbuf =3D { .launder_folio =3D cifs_launder_folio, .migrate_folio =3D filemap_migrate_folio, }; - -/* - * Splice data from a file into a pipe. - */ -ssize_t cifs_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, size_t len, - unsigned int flags) -{ - if (unlikely(*ppos >=3D file_inode(in)->i_sb->s_maxbytes)) - return 0; - if (unlikely(!len)) - return 0; - if (in->f_flags & O_DIRECT) - return direct_splice_read(in, ppos, pipe, len, flags); - return filemap_splice_read(in, ppos, pipe, len, flags); -} From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DB57C7EE2C for ; Fri, 19 May 2023 07:46:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231499AbjESHqY (ORCPT ); Fri, 19 May 2023 03:46:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58922 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231411AbjESHqL (ORCPT ); Fri, 19 May 2023 03:46:11 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF64E19B7 for ; Fri, 19 May 2023 00:43:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482154; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IZTA75AA4bfNT3x36ufDNmpN8ZwenJYqDvE37aQ1sUs=; b=F4T42uK2kJvWsRNHfr1gcYJVOeKPRXWMIBO35Y7QHGih+L4sYfcirE9yjOvkBt8j63hmAo SW+PYqruLS9prQ4v6qxR9aXGSEPAknagqOUgUAAjPmsfqSTRCcuuBpXUSVvWguMdpWOdzx RCUt15aFdtJjC9ZwcxqM1dkNCT0ppOI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-88-pnfp-Ae-NcKTG-EYqodLzg-1; Fri, 19 May 2023 03:42:30 -0400 X-MC-Unique: pnfp-Ae-NcKTG-EYqodLzg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AB1E638035AD; Fri, 19 May 2023 07:42:29 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6053DC54184; Fri, 19 May 2023 07:42:27 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v20 26/32] iov_iter: Kill ITER_PIPE Date: Fri, 19 May 2023 08:40:41 +0100 Message-Id: <20230519074047.1739879-27-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The ITER_PIPE-type iterator was only used for generic_file_splice_read(), but that has now been switched to either pull pages directly from the pagecache for buffered file splice-reads or to use ITER_BVEC instead for O_DIRECT file splice-reads. This leaves ITER_PIPE unused - so remove it. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org --- Notes: ver #20) - Rebase and remove additional pipe reference. include/linux/uio.h | 14 -- lib/iov_iter.c | 431 +------------------------------------------- mm/filemap.c | 3 +- 3 files changed, 4 insertions(+), 444 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index 044c1d8c230c..60c342bb7ab8 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -11,7 +11,6 @@ #include =20 struct page; -struct pipe_inode_info; =20 typedef unsigned int __bitwise iov_iter_extraction_t; =20 @@ -25,7 +24,6 @@ enum iter_type { ITER_IOVEC, ITER_KVEC, ITER_BVEC, - ITER_PIPE, ITER_XARRAY, ITER_DISCARD, ITER_UBUF, @@ -74,7 +72,6 @@ struct iov_iter { const struct kvec *kvec; const struct bio_vec *bvec; struct xarray *xarray; - struct pipe_inode_info *pipe; void __user *ubuf; }; size_t count; @@ -82,10 +79,6 @@ struct iov_iter { }; union { unsigned long nr_segs; - struct { - unsigned int head; - unsigned int start_head; - }; loff_t xarray_start; }; }; @@ -133,11 +126,6 @@ static inline bool iov_iter_is_bvec(const struct iov_i= ter *i) return iov_iter_type(i) =3D=3D ITER_BVEC; } =20 -static inline bool iov_iter_is_pipe(const struct iov_iter *i) -{ - return iov_iter_type(i) =3D=3D ITER_PIPE; -} - static inline bool iov_iter_is_discard(const struct iov_iter *i) { return iov_iter_type(i) =3D=3D ITER_DISCARD; @@ -286,8 +274,6 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int dir= ection, const struct kvec unsigned long nr_segs, size_t count); void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struc= t bio_vec *bvec, unsigned long nr_segs, size_t count); -void iov_iter_pipe(struct iov_iter *i, unsigned int direction, struct pipe= _inode_info *pipe, - size_t count); void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t c= ount); void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xa= rray *xarray, loff_t start, size_t count); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 960223ed9199..f18138e0292a 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -14,8 +14,6 @@ #include #include =20 -#define PIPE_PARANOIA /* for now */ - /* covers ubuf and kbuf alike */ #define iterate_buf(i, n, base, len, off, __p, STEP) { \ size_t __maybe_unused off =3D 0; \ @@ -198,150 +196,6 @@ static int copyin(void *to, const void __user *from, = size_t n) return res; } =20 -#ifdef PIPE_PARANOIA -static bool sanity(const struct iov_iter *i) -{ - struct pipe_inode_info *pipe =3D i->pipe; - unsigned int p_head =3D pipe->head; - unsigned int p_tail =3D pipe->tail; - unsigned int p_occupancy =3D pipe_occupancy(p_head, p_tail); - unsigned int i_head =3D i->head; - unsigned int idx; - - if (i->last_offset) { - struct pipe_buffer *p; - if (unlikely(p_occupancy =3D=3D 0)) - goto Bad; // pipe must be non-empty - if (unlikely(i_head !=3D p_head - 1)) - goto Bad; // must be at the last buffer... - - p =3D pipe_buf(pipe, i_head); - if (unlikely(p->offset + p->len !=3D abs(i->last_offset))) - goto Bad; // ... at the end of segment - } else { - if (i_head !=3D p_head) - goto Bad; // must be right after the last buffer - } - return true; -Bad: - printk(KERN_ERR "idx =3D %d, offset =3D %d\n", i_head, i->last_offset); - printk(KERN_ERR "head =3D %d, tail =3D %d, buffers =3D %d\n", - p_head, p_tail, pipe->ring_size); - for (idx =3D 0; idx < pipe->ring_size; idx++) - printk(KERN_ERR "[%p %p %d %d]\n", - pipe->bufs[idx].ops, - pipe->bufs[idx].page, - pipe->bufs[idx].offset, - pipe->bufs[idx].len); - WARN_ON(1); - return false; -} -#else -#define sanity(i) true -#endif - -static struct page *push_anon(struct pipe_inode_info *pipe, unsigned size) -{ - struct page *page =3D alloc_page(GFP_USER); - if (page) { - struct pipe_buffer *buf =3D pipe_buf(pipe, pipe->head++); - *buf =3D (struct pipe_buffer) { - .ops =3D &default_pipe_buf_ops, - .page =3D page, - .offset =3D 0, - .len =3D size - }; - } - return page; -} - -static void push_page(struct pipe_inode_info *pipe, struct page *page, - unsigned int offset, unsigned int size) -{ - struct pipe_buffer *buf =3D pipe_buf(pipe, pipe->head++); - *buf =3D (struct pipe_buffer) { - .ops =3D &page_cache_pipe_buf_ops, - .page =3D page, - .offset =3D offset, - .len =3D size - }; - get_page(page); -} - -static inline int last_offset(const struct pipe_buffer *buf) -{ - if (buf->ops =3D=3D &default_pipe_buf_ops) - return buf->len; // buf->offset is 0 for those - else - return -(buf->offset + buf->len); -} - -static struct page *append_pipe(struct iov_iter *i, size_t size, - unsigned int *off) -{ - struct pipe_inode_info *pipe =3D i->pipe; - int offset =3D i->last_offset; - struct pipe_buffer *buf; - struct page *page; - - if (offset > 0 && offset < PAGE_SIZE) { - // some space in the last buffer; add to it - buf =3D pipe_buf(pipe, pipe->head - 1); - size =3D min_t(size_t, size, PAGE_SIZE - offset); - buf->len +=3D size; - i->last_offset +=3D size; - i->count -=3D size; - *off =3D offset; - return buf->page; - } - // OK, we need a new buffer - *off =3D 0; - size =3D min_t(size_t, size, PAGE_SIZE); - if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) - return NULL; - page =3D push_anon(pipe, size); - if (!page) - return NULL; - i->head =3D pipe->head - 1; - i->last_offset =3D size; - i->count -=3D size; - return page; -} - -static size_t copy_page_to_iter_pipe(struct page *page, size_t offset, siz= e_t bytes, - struct iov_iter *i) -{ - struct pipe_inode_info *pipe =3D i->pipe; - unsigned int head =3D pipe->head; - - if (unlikely(bytes > i->count)) - bytes =3D i->count; - - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - if (offset && i->last_offset =3D=3D -offset) { // could we merge it? - struct pipe_buffer *buf =3D pipe_buf(pipe, head - 1); - if (buf->page =3D=3D page) { - buf->len +=3D bytes; - i->last_offset -=3D bytes; - i->count -=3D bytes; - return bytes; - } - } - if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) - return 0; - - push_page(pipe, page, offset, bytes); - i->last_offset =3D -(offset + bytes); - i->head =3D head; - i->count -=3D bytes; - return bytes; -} - /* * fault_in_iov_iter_readable - fault in iov iterator for reading * @i: iterator @@ -446,46 +300,6 @@ void iov_iter_init(struct iov_iter *i, unsigned int di= rection, } EXPORT_SYMBOL(iov_iter_init); =20 -// returns the offset in partial buffer (if any) -static inline unsigned int pipe_npages(const struct iov_iter *i, int *npag= es) -{ - struct pipe_inode_info *pipe =3D i->pipe; - int used =3D pipe->head - pipe->tail; - int off =3D i->last_offset; - - *npages =3D max((int)pipe->max_usage - used, 0); - - if (off > 0 && off < PAGE_SIZE) { // anon and not full - (*npages)++; - return off; - } - return 0; -} - -static size_t copy_pipe_to_iter(const void *addr, size_t bytes, - struct iov_iter *i) -{ - unsigned int off, chunk; - - if (unlikely(bytes > i->count)) - bytes =3D i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - for (size_t n =3D bytes; n; n -=3D chunk) { - struct page *page =3D append_pipe(i, n, &off); - chunk =3D min_t(size_t, n, PAGE_SIZE - off); - if (!page) - return bytes - n; - memcpy_to_page(page, off, addr, chunk); - addr +=3D chunk; - } - return bytes; -} - static __wsum csum_and_memcpy(void *to, const void *from, size_t len, __wsum sum, size_t off) { @@ -493,44 +307,10 @@ static __wsum csum_and_memcpy(void *to, const void *f= rom, size_t len, return csum_block_add(sum, next, off); } =20 -static size_t csum_and_copy_to_pipe_iter(const void *addr, size_t bytes, - struct iov_iter *i, __wsum *sump) -{ - __wsum sum =3D *sump; - size_t off =3D 0; - unsigned int chunk, r; - - if (unlikely(bytes > i->count)) - bytes =3D i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - while (bytes) { - struct page *page =3D append_pipe(i, bytes, &r); - char *p; - - if (!page) - break; - chunk =3D min_t(size_t, bytes, PAGE_SIZE - r); - p =3D kmap_local_page(page); - sum =3D csum_and_memcpy(p + r, addr + off, chunk, sum, off); - kunmap_local(p); - off +=3D chunk; - bytes -=3D chunk; - } - *sump =3D sum; - return off; -} - size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i) { if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_pipe_to_iter(addr, bytes, i); if (user_backed_iter(i)) might_fault(); iterate_and_advance(i, bytes, base, len, off, @@ -552,42 +332,6 @@ static int copyout_mc(void __user *to, const void *fro= m, size_t n) return n; } =20 -static size_t copy_mc_pipe_to_iter(const void *addr, size_t bytes, - struct iov_iter *i) -{ - size_t xfer =3D 0; - unsigned int off, chunk; - - if (unlikely(bytes > i->count)) - bytes =3D i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - while (bytes) { - struct page *page =3D append_pipe(i, bytes, &off); - unsigned long rem; - char *p; - - if (!page) - break; - chunk =3D min_t(size_t, bytes, PAGE_SIZE - off); - p =3D kmap_local_page(page); - rem =3D copy_mc_to_kernel(p + off, addr + xfer, chunk); - chunk -=3D rem; - kunmap_local(p); - xfer +=3D chunk; - bytes -=3D chunk; - if (rem) { - iov_iter_revert(i, rem); - break; - } - } - return xfer; -} - /** * _copy_mc_to_iter - copy to iter with source memory error exception hand= ling * @addr: source kernel address @@ -607,9 +351,8 @@ static size_t copy_mc_pipe_to_iter(const void *addr, si= ze_t bytes, * alignment and poison alignment assumptions to avoid re-triggering * hardware exceptions. * - * * ITER_KVEC, ITER_PIPE, and ITER_BVEC can return short copies. - * Compare to copy_to_iter() where only ITER_IOVEC attempts might return - * a short copy. + * * ITER_KVEC and ITER_BVEC can return short copies. Compare to + * copy_to_iter() where only ITER_IOVEC attempts might return a short co= py. * * Return: number of bytes copied (may be %0) */ @@ -617,8 +360,6 @@ size_t _copy_mc_to_iter(const void *addr, size_t bytes,= struct iov_iter *i) { if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_mc_pipe_to_iter(addr, bytes, i); if (user_backed_iter(i)) might_fault(); __iterate_and_advance(i, bytes, base, len, off, @@ -732,8 +473,6 @@ size_t copy_page_to_iter(struct page *page, size_t offs= et, size_t bytes, return 0; if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_page_to_iter_pipe(page, offset, bytes, i); page +=3D offset / PAGE_SIZE; // first subpage offset %=3D PAGE_SIZE; while (1) { @@ -764,8 +503,6 @@ size_t copy_page_to_iter_nofault(struct page *page, uns= igned offset, size_t byte return 0; if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_page_to_iter_pipe(page, offset, bytes, i); page +=3D offset / PAGE_SIZE; // first subpage offset %=3D PAGE_SIZE; while (1) { @@ -818,36 +555,8 @@ size_t copy_page_from_iter(struct page *page, size_t o= ffset, size_t bytes, } EXPORT_SYMBOL(copy_page_from_iter); =20 -static size_t pipe_zero(size_t bytes, struct iov_iter *i) -{ - unsigned int chunk, off; - - if (unlikely(bytes > i->count)) - bytes =3D i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - for (size_t n =3D bytes; n; n -=3D chunk) { - struct page *page =3D append_pipe(i, n, &off); - char *p; - - if (!page) - return bytes - n; - chunk =3D min_t(size_t, n, PAGE_SIZE - off); - p =3D kmap_local_page(page); - memset(p + off, 0, chunk); - kunmap_local(p); - } - return bytes; -} - size_t iov_iter_zero(size_t bytes, struct iov_iter *i) { - if (unlikely(iov_iter_is_pipe(i))) - return pipe_zero(bytes, i); iterate_and_advance(i, bytes, base, len, count, clear_user(base, len), memset(base, 0, len) @@ -878,32 +587,6 @@ size_t copy_page_from_iter_atomic(struct page *page, u= nsigned offset, size_t byt } EXPORT_SYMBOL(copy_page_from_iter_atomic); =20 -static void pipe_advance(struct iov_iter *i, size_t size) -{ - struct pipe_inode_info *pipe =3D i->pipe; - int off =3D i->last_offset; - - if (!off && !size) { - pipe_discard_from(pipe, i->start_head); // discard everything - return; - } - i->count -=3D size; - while (1) { - struct pipe_buffer *buf =3D pipe_buf(pipe, i->head); - if (off) /* make it relative to the beginning of buffer */ - size +=3D abs(off) - buf->offset; - if (size <=3D buf->len) { - buf->len =3D size; - i->last_offset =3D last_offset(buf); - break; - } - size -=3D buf->len; - i->head++; - off =3D 0; - } - pipe_discard_from(pipe, i->head + 1); // discard everything past this one -} - static void iov_iter_bvec_advance(struct iov_iter *i, size_t size) { const struct bio_vec *bvec, *end; @@ -955,8 +638,6 @@ void iov_iter_advance(struct iov_iter *i, size_t size) iov_iter_iovec_advance(i, size); } else if (iov_iter_is_bvec(i)) { iov_iter_bvec_advance(i, size); - } else if (iov_iter_is_pipe(i)) { - pipe_advance(i, size); } else if (iov_iter_is_discard(i)) { i->count -=3D size; } @@ -970,26 +651,6 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll) if (WARN_ON(unroll > MAX_RW_COUNT)) return; i->count +=3D unroll; - if (unlikely(iov_iter_is_pipe(i))) { - struct pipe_inode_info *pipe =3D i->pipe; - unsigned int head =3D pipe->head; - - while (head > i->start_head) { - struct pipe_buffer *b =3D pipe_buf(pipe, --head); - if (unroll < b->len) { - b->len -=3D unroll; - i->last_offset =3D last_offset(b); - i->head =3D head; - return; - } - unroll -=3D b->len; - pipe_buf_release(pipe, b); - pipe->head--; - } - i->last_offset =3D 0; - i->head =3D head; - return; - } if (unlikely(iov_iter_is_discard(i))) return; if (unroll <=3D i->iov_offset) { @@ -1079,24 +740,6 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int d= irection, } EXPORT_SYMBOL(iov_iter_bvec); =20 -void iov_iter_pipe(struct iov_iter *i, unsigned int direction, - struct pipe_inode_info *pipe, - size_t count) -{ - BUG_ON(direction !=3D READ); - WARN_ON(pipe_full(pipe->head, pipe->tail, pipe->ring_size)); - *i =3D (struct iov_iter){ - .iter_type =3D ITER_PIPE, - .data_source =3D false, - .pipe =3D pipe, - .head =3D pipe->head, - .start_head =3D pipe->head, - .last_offset =3D 0, - .count =3D count - }; -} -EXPORT_SYMBOL(iov_iter_pipe); - /** * iov_iter_xarray - Initialise an I/O iterator to use the pages in an xar= ray * @i: The iterator to initialise. @@ -1224,19 +867,6 @@ bool iov_iter_is_aligned(const struct iov_iter *i, un= signed addr_mask, if (iov_iter_is_bvec(i)) return iov_iter_aligned_bvec(i, addr_mask, len_mask); =20 - if (iov_iter_is_pipe(i)) { - size_t size =3D i->count; - - if (size & len_mask) - return false; - if (size && i->last_offset > 0) { - if (i->last_offset & addr_mask) - return false; - } - - return true; - } - if (iov_iter_is_xarray(i)) { if (i->count & len_mask) return false; @@ -1307,14 +937,6 @@ unsigned long iov_iter_alignment(const struct iov_ite= r *i) if (iov_iter_is_bvec(i)) return iov_iter_alignment_bvec(i); =20 - if (iov_iter_is_pipe(i)) { - size_t size =3D i->count; - - if (size && i->last_offset > 0) - return size | i->last_offset; - return size; - } - if (iov_iter_is_xarray(i)) return (i->xarray_start + i->iov_offset) | i->count; =20 @@ -1367,36 +989,6 @@ static int want_pages_array(struct page ***res, size_= t size, return count; } =20 -static ssize_t pipe_get_pages(struct iov_iter *i, - struct page ***pages, size_t maxsize, unsigned maxpages, - size_t *start) -{ - unsigned int npages, count, off, chunk; - struct page **p; - size_t left; - - if (!sanity(i)) - return -EFAULT; - - *start =3D off =3D pipe_npages(i, &npages); - if (!npages) - return -EFAULT; - count =3D want_pages_array(pages, maxsize, off, min(npages, maxpages)); - if (!count) - return -ENOMEM; - p =3D *pages; - for (npages =3D 0, left =3D maxsize ; npages < count; npages++, left -=3D= chunk) { - struct page *page =3D append_pipe(i, left, &off); - if (!page) - break; - chunk =3D min_t(size_t, left, PAGE_SIZE - off); - get_page(*p++ =3D page); - } - if (!npages) - return -EFAULT; - return maxsize - left; -} - static ssize_t iter_xarray_populate_pages(struct page **pages, struct xarr= ay *xa, pgoff_t index, unsigned int nr_pages) { @@ -1547,8 +1139,6 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_= iter *i, } return maxsize; } - if (iov_iter_is_pipe(i)) - return pipe_get_pages(i, pages, maxsize, maxpages, start); if (iov_iter_is_xarray(i)) return iter_xarray_get_pages(i, pages, maxsize, maxpages, start); return -EFAULT; @@ -1638,9 +1228,7 @@ size_t csum_and_copy_to_iter(const void *addr, size_t= bytes, void *_csstate, } =20 sum =3D csum_shift(csstate->csum, csstate->off); - if (unlikely(iov_iter_is_pipe(i))) - bytes =3D csum_and_copy_to_pipe_iter(addr, bytes, i, &sum); - else iterate_and_advance(i, bytes, base, len, off, ({ + iterate_and_advance(i, bytes, base, len, off, ({ next =3D csum_and_copy_to_user(addr + off, base, len); sum =3D csum_block_add(sum, next, off); next ? 0 : len; @@ -1725,15 +1313,6 @@ int iov_iter_npages(const struct iov_iter *i, int ma= xpages) return iov_npages(i, maxpages); if (iov_iter_is_bvec(i)) return bvec_npages(i, maxpages); - if (iov_iter_is_pipe(i)) { - int npages; - - if (!sanity(i)) - return 0; - - pipe_npages(i, &npages); - return min(npages, maxpages); - } if (iov_iter_is_xarray(i)) { unsigned offset =3D (i->xarray_start + i->iov_offset) % PAGE_SIZE; int npages =3D DIV_ROUND_UP(offset + i->count, PAGE_SIZE); @@ -1746,10 +1325,6 @@ EXPORT_SYMBOL(iov_iter_npages); const void *dup_iter(struct iov_iter *new, struct iov_iter *old, gfp_t fla= gs) { *new =3D *old; - if (unlikely(iov_iter_is_pipe(new))) { - WARN_ON(1); - return NULL; - } if (iov_iter_is_bvec(new)) return new->bvec =3D kmemdup(new->bvec, new->nr_segs * sizeof(struct bio_vec), diff --git a/mm/filemap.c b/mm/filemap.c index a2006936a6ae..394db0c1f197 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2687,8 +2687,7 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_i= ter *iter, if (unlikely(iocb->ki_pos >=3D i_size_read(inode))) break; =20 - error =3D filemap_get_pages(iocb, iter->count, &fbatch, - iov_iter_is_pipe(iter)); + error =3D filemap_get_pages(iocb, iter->count, &fbatch, false); if (error < 0) break; From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFDB6C7EE2C for ; Fri, 19 May 2023 07:46:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231187AbjESHqv (ORCPT ); Fri, 19 May 2023 03:46:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229852AbjESHqZ (ORCPT ); Fri, 19 May 2023 03:46:25 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C3E1726AB for ; Fri, 19 May 2023 00:43:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oZIY9GTPXEPoSMZhe+ryqBhZXmHTyjPJq3hYdlaANCY=; b=SY6kmsjgU1rxdcFdFAhtKgFKrqsYPJXYEai35iQK2mU82uA3aQ03VHVJbp0+9eLUEEXIV9 BwK3z8geN8jbXUoc2linV4kClpjP95V8HCOnOnY13WAFpb/cr5BpF5ylUXAJg3QMbzA/7X f7j97gSwP4zMCGTR0q3nCGBuJjR3yfs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-218-S_Amoo1cMROA4t9BdXJr9g-1; Fri, 19 May 2023 03:42:33 -0400 X-MC-Unique: S_Amoo1cMROA4t9BdXJr9g-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8E12685A5AA; Fri, 19 May 2023 07:42:32 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 503291121314; Fri, 19 May 2023 07:42:30 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, John Hubbard , Dave Chinner , Christoph Hellwig Subject: [PATCH v20 27/32] iomap: Don't get an reference on ZERO_PAGE for direct I/O block zeroing Date: Fri, 19 May 2023 08:40:42 +0100 Message-Id: <20230519074047.1739879-28-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" ZERO_PAGE can't go away, no need to hold an extra reference. Signed-off-by: David Howells Reviewed-by: David Hildenbrand Reviewed-by: John Hubbard Reviewed-by: Dave Chinner Reviewed-by: Christoph Hellwig cc: Al Viro cc: linux-fsdevel@vger.kernel.org --- fs/iomap/direct-io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 019cc87d0fb3..66a9f10e3207 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -203,7 +203,7 @@ static void iomap_dio_zero(const struct iomap_iter *ite= r, struct iomap_dio *dio, bio->bi_private =3D dio; bio->bi_end_io =3D iomap_dio_bio_end_io; =20 - get_page(page); + bio_set_flag(bio, BIO_NO_PAGE_REF); __bio_add_page(bio, page, len, 0); iomap_dio_submit_bio(iter, dio, bio, pos); } From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EFE2C7EE2E for ; Fri, 19 May 2023 07:47:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231584AbjESHrC (ORCPT ); Fri, 19 May 2023 03:47:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231371AbjESHqk (ORCPT ); Fri, 19 May 2023 03:46:40 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79ACC270D for ; Fri, 19 May 2023 00:43:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482161; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=edStQDA/gh3f03KMzWRlpjb2e/gxKf5/7u3ht9bl6/4=; b=ftUtJEE7srthgAdLLhMoy8qotX/Poq+tN51QojqCScQyJZ3ijaUgB7vEQ6iBhFpL7r3Qrh jLBZvVh9gTJi+QUew2kbnCA6ewIzLM3wAp+3LzMa6veqmHQNtiy01OvYiM+/0/JNClzUTI 1J1UBGwPVtDBCZ8R6fS3k0zG2n6322A= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-558-Zn02uDUqNWychxCT98RFHQ-1; Fri, 19 May 2023 03:42:36 -0400 X-MC-Unique: Zn02uDUqNWychxCT98RFHQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 654BA101A531; Fri, 19 May 2023 07:42:35 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 468A240CFD46; Fri, 19 May 2023 07:42:33 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v20 28/32] block: Fix bio_flagged() so that gcc can better optimise it Date: Fri, 19 May 2023 08:40:43 +0100 Message-Id: <20230519074047.1739879-29-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Fix bio_flagged() so that multiple instances of it, such as: if (bio_flagged(bio, BIO_PAGE_REFFED) || bio_flagged(bio, BIO_PAGE_PINNED)) can be combined by the gcc optimiser into a single test in assembly (arguably, this is a compiler optimisation issue[1]). The missed optimisation stems from bio_flagged() comparing the result of the bitwise-AND to zero. This results in an out-of-line bio_release_page() being compiled to something like: <+0>: mov 0x14(%rdi),%eax <+3>: test $0x1,%al <+5>: jne 0xffffffff816dac53 <+7>: test $0x2,%al <+9>: je 0xffffffff816dac5c <+11>: movzbl %sil,%esi <+15>: jmp 0xffffffff816daba1 <__bio_release_pages> <+20>: jmp 0xffffffff81d0b800 <__x86_return_thunk> However, the test is superfluous as the return type is bool. Removing it results in: <+0>: testb $0x3,0x14(%rdi) <+4>: je 0xffffffff816e4af4 <+6>: movzbl %sil,%esi <+10>: jmp 0xffffffff816dab7c <__bio_release_pages> <+15>: jmp 0xffffffff81d0b7c0 <__x86_return_thunk> instead. Also, the MOVZBL instruction looks unnecessary[2] - I think it's just 're-booling' the mark_dirty parameter. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Jens Axboe cc: linux-block@vger.kernel.org Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D108370 [1] Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D108371 [2] Link: https://lore.kernel.org/r/167391056756.2311931.356007731815807265.stg= it@warthog.procyon.org.uk/ # v6 --- include/linux/bio.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index b3e7529ff55e..7f53be035cf0 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -229,7 +229,7 @@ static inline void bio_cnt_set(struct bio *bio, unsigne= d int count) =20 static inline bool bio_flagged(struct bio *bio, unsigned int bit) { - return (bio->bi_flags & (1U << bit)) !=3D 0; + return bio->bi_flags & (1U << bit); } =20 static inline void bio_set_flag(struct bio *bio, unsigned int bit) From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E2BCC77B7F for ; Fri, 19 May 2023 07:47:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231567AbjESHrA (ORCPT ); Fri, 19 May 2023 03:47:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231359AbjESHqi (ORCPT ); Fri, 19 May 2023 03:46:38 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 394052713 for ; Fri, 19 May 2023 00:43:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482162; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TuE+HSvYodh9xYXD2lD0nf71BaZqaWOFZeqfoTjFcWg=; b=L3HFXI2TKmyN3fDBOOsuyxD/lEktqQ6jjNZmrzp0XOcpd+f27a18bCjfE1uCiRsKNZHE45 aAoDLlrObmptWxBdbe5ubUM+3Gm/1PzydMR0UZ16TAEFSRfHr39oAWcaHo8lglop6OPhyr 4dOghZe1PcinwWtzDA7pU2NQG6T0mDo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-536-qF_O3km6NpKFWdiM07-vLQ-1; Fri, 19 May 2023 03:42:39 -0400 X-MC-Unique: qF_O3km6NpKFWdiM07-vLQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 41BD880027F; Fri, 19 May 2023 07:42:38 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0CB432026D49; Fri, 19 May 2023 07:42:35 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v20 29/32] block: Replace BIO_NO_PAGE_REF with BIO_PAGE_REFFED with inverted logic Date: Fri, 19 May 2023 08:40:44 +0100 Message-Id: <20230519074047.1739879-30-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Christoph Hellwig Replace BIO_NO_PAGE_REF with a BIO_PAGE_REFFED flag that has the inverted meaning is only set when a page reference has been acquired that needs to be released by bio_release_pages(). Signed-off-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org Reviewed-by: Kent Overstreet --- Notes: ver #8) - Split out from another patch [hch]. - Don't default to BIO_PAGE_REFFED [hch]. =20 ver #5) - Split from patch that uses iov_iter_extract_pages(). block/bio.c | 2 +- block/blk-map.c | 1 + fs/direct-io.c | 2 ++ fs/iomap/direct-io.c | 1 - include/linux/bio.h | 2 +- include/linux/blk_types.h | 2 +- 6 files changed, 6 insertions(+), 4 deletions(-) diff --git a/block/bio.c b/block/bio.c index 043944fd46eb..8516adeaea26 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1191,7 +1191,6 @@ void bio_iov_bvec_set(struct bio *bio, struct iov_ite= r *iter) bio->bi_io_vec =3D (struct bio_vec *)iter->bvec; bio->bi_iter.bi_bvec_done =3D iter->iov_offset; bio->bi_iter.bi_size =3D size; - bio_set_flag(bio, BIO_NO_PAGE_REF); bio_set_flag(bio, BIO_CLONED); } =20 @@ -1336,6 +1335,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct io= v_iter *iter) return 0; } =20 + bio_set_flag(bio, BIO_PAGE_REFFED); do { ret =3D __bio_iov_iter_get_pages(bio, iter); } while (!ret && iov_iter_count(iter) && !bio_full(bio, 0)); diff --git a/block/blk-map.c b/block/blk-map.c index 04c55f1c492e..33d9f6e89ba6 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -282,6 +282,7 @@ static int bio_map_user_iov(struct request *rq, struct = iov_iter *iter, if (blk_queue_pci_p2pdma(rq->q)) extraction_flags |=3D ITER_ALLOW_P2PDMA; =20 + bio_set_flag(bio, BIO_PAGE_REFFED); while (iov_iter_count(iter)) { struct page **pages, *stack_pages[UIO_FASTIOV]; ssize_t bytes; diff --git a/fs/direct-io.c b/fs/direct-io.c index 0b380bb8a81e..ad20f3428bab 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -402,6 +402,8 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio, bio->bi_end_io =3D dio_bio_end_aio; else bio->bi_end_io =3D dio_bio_end_io; + /* for now require references for all pages */ + bio_set_flag(bio, BIO_PAGE_REFFED); sdio->bio =3D bio; sdio->logical_offset_in_bio =3D sdio->cur_page_fs_offset; } diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 66a9f10e3207..08873f0627dd 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -203,7 +203,6 @@ static void iomap_dio_zero(const struct iomap_iter *ite= r, struct iomap_dio *dio, bio->bi_private =3D dio; bio->bi_end_io =3D iomap_dio_bio_end_io; =20 - bio_set_flag(bio, BIO_NO_PAGE_REF); __bio_add_page(bio, page, len, 0); iomap_dio_submit_bio(iter, dio, bio, pos); } diff --git a/include/linux/bio.h b/include/linux/bio.h index 7f53be035cf0..0922729acd26 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -488,7 +488,7 @@ void zero_fill_bio(struct bio *bio); =20 static inline void bio_release_pages(struct bio *bio, bool mark_dirty) { - if (!bio_flagged(bio, BIO_NO_PAGE_REF)) + if (bio_flagged(bio, BIO_PAGE_REFFED)) __bio_release_pages(bio, mark_dirty); } =20 diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 740afe80f297..dfd2c2cb909d 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -323,7 +323,7 @@ struct bio { * bio flags */ enum { - BIO_NO_PAGE_REF, /* don't put release vec pages */ + BIO_PAGE_REFFED, /* put pages in bio_release_pages() */ BIO_CLONED, /* doesn't own data */ BIO_BOUNCED, /* bio is a bounce bio */ BIO_QUIET, /* Make BIO Quiet */ From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CD60C77B7A for ; Fri, 19 May 2023 07:47:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231605AbjESHra (ORCPT ); Fri, 19 May 2023 03:47:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231546AbjESHqs (ORCPT ); Fri, 19 May 2023 03:46:48 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3FC8A2D5D for ; Fri, 19 May 2023 00:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xt8MS4tNW7WMUMs3pTsS0U26gIFNlNPWhaB+SXg4cNY=; b=cPa+XTE/IXOUgR2pK0zxjg2o0Yk9HEmxGvP2QadAc1ISJHUNnD9CrhuDZ+vbFANkOdoMCc XDHa845VC0Vnj85oUVytSTH380H5IqQQoBOW2oYngHc4WqEcA3b/Fwk1O7FGAV3VvvHwLp o+kPmULl/V9SDvN4uDzjWHrNQkCr8NI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-263-MOrBiR8-MCq-_mwud5C3yw-1; Fri, 19 May 2023 03:42:42 -0400 X-MC-Unique: MOrBiR8-MCq-_mwud5C3yw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3E6753C01DB1; Fri, 19 May 2023 07:42:41 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 01B372026D49; Fri, 19 May 2023 07:42:38 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v20 30/32] block: Add BIO_PAGE_PINNED and associated infrastructure Date: Fri, 19 May 2023 08:40:45 +0100 Message-Id: <20230519074047.1739879-31-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add BIO_PAGE_PINNED to indicate that the pages in a bio are pinned (FOLL_PIN) and that the pin will need removing. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #10) - Drop bio_set_cleanup_mode(), open coding it instead. =20 ver #9) - Only consider pinning in bio_set_cleanup_mode(). Ref'ing pages in struct bio is going away. - page_put_unpin() is removed; call unpin_user_page() and put_page() directly. - Use bio_release_page() in __bio_release_pages(). - BIO_PAGE_PINNED and BIO_PAGE_REFFED can't both be set, so use if-else when testing both of them. =20 ver #8) - Move the infrastructure to clean up pinned pages to this patch [hch]. - Put BIO_PAGE_PINNED before BIO_PAGE_REFFED as the latter should probably be removed at some point. FOLL_PIN can then be renumbered first. block/bio.c | 6 +++--- block/blk.h | 12 ++++++++++++ include/linux/bio.h | 3 ++- include/linux/blk_types.h | 1 + 4 files changed, 18 insertions(+), 4 deletions(-) diff --git a/block/bio.c b/block/bio.c index 8516adeaea26..17bd01ecde36 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1169,7 +1169,7 @@ void __bio_release_pages(struct bio *bio, bool mark_d= irty) bio_for_each_segment_all(bvec, bio, iter_all) { if (mark_dirty && !PageCompound(bvec->bv_page)) set_page_dirty_lock(bvec->bv_page); - put_page(bvec->bv_page); + bio_release_page(bio, bvec->bv_page); } } EXPORT_SYMBOL_GPL(__bio_release_pages); @@ -1489,8 +1489,8 @@ void bio_set_pages_dirty(struct bio *bio) * the BIO and re-dirty the pages in process context. * * It is expected that bio_check_pages_dirty() will wholly own the BIO from - * here on. It will run one put_page() against each page and will run one - * bio_put() against the BIO. + * here on. It will unpin each page and will run one bio_put() against the + * BIO. */ =20 static void bio_dirty_fn(struct work_struct *work); diff --git a/block/blk.h b/block/blk.h index 45547bcf1119..e1ded2ccb3ca 100644 --- a/block/blk.h +++ b/block/blk.h @@ -420,6 +420,18 @@ int bio_add_hw_page(struct request_queue *q, struct bi= o *bio, struct page *page, unsigned int len, unsigned int offset, unsigned int max_sectors, bool *same_page); =20 +/* + * Clean up a page appropriately, where the page may be pinned, may have a + * ref taken on it or neither. + */ +static inline void bio_release_page(struct bio *bio, struct page *page) +{ + if (bio_flagged(bio, BIO_PAGE_PINNED)) + unpin_user_page(page); + else if (bio_flagged(bio, BIO_PAGE_REFFED)) + put_page(page); +} + struct request_queue *blk_alloc_queue(int node_id); =20 int disk_scan_partitions(struct gendisk *disk, fmode_t mode); diff --git a/include/linux/bio.h b/include/linux/bio.h index 0922729acd26..8588bcfbc6ef 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -488,7 +488,8 @@ void zero_fill_bio(struct bio *bio); =20 static inline void bio_release_pages(struct bio *bio, bool mark_dirty) { - if (bio_flagged(bio, BIO_PAGE_REFFED)) + if (bio_flagged(bio, BIO_PAGE_REFFED) || + bio_flagged(bio, BIO_PAGE_PINNED)) __bio_release_pages(bio, mark_dirty); } =20 diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index dfd2c2cb909d..8ef209e3aa96 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -323,6 +323,7 @@ struct bio { * bio flags */ enum { + BIO_PAGE_PINNED, /* Unpin pages in bio_release_pages() */ BIO_PAGE_REFFED, /* put pages in bio_release_pages() */ BIO_CLONED, /* doesn't own data */ BIO_BOUNCED, /* bio is a bounce bio */ From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98925C7EE2A for ; Fri, 19 May 2023 07:47:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231602AbjESHr2 (ORCPT ); Fri, 19 May 2023 03:47:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231424AbjESHqo (ORCPT ); Fri, 19 May 2023 03:46:44 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 952DD18F for ; Fri, 19 May 2023 00:43:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482170; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ehlvfIu1rrXcuy8/PZvsT00k//DN+ilG0RL3yAJWLmk=; b=MetuoGUbuw+WJ7yP28JG1wyTq33lj+a4QBXOPitATTBT4n3n6G5A+LmKIQD51SIjur4H9i A1ICpUp3TLsy16sRWsXP23rlBONbk6/iw0M7cJjXE29bpUZgHAns6cu+08L2erF70dXQGm 3CAZ7obnKVKBVGqUYjhNo/WFGUngQs0= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-373-Z2TszftPOkqunDI3f5UAzQ-1; Fri, 19 May 2023 03:42:45 -0400 X-MC-Unique: Z2TszftPOkqunDI3f5UAzQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F2D9438035AD; Fri, 19 May 2023 07:42:43 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id D7B90C0004B; Fri, 19 May 2023 07:42:41 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v20 31/32] block: Convert bio_iov_iter_get_pages to use iov_iter_extract_pages Date: Fri, 19 May 2023 08:40:46 +0100 Message-Id: <20230519074047.1739879-32-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This will pin pages or leave them unaltered rather than getting a ref on them as appropriate to the iterator. The pages need to be pinned for DIO rather than having refs taken on them to prevent VM copy-on-write from malfunctioning during a concurrent fork() (the result of the I/O could otherwise end up being affected by/visible to the child process). Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #10) - Drop bio_set_cleanup_mode(), open coding it instead. =20 ver #8) - Split the patch up a bit [hch]. - We should only be using pinned/non-pinned pages and not ref'd pages, so adjust the comments appropriately. =20 ver #7) - Don't treat BIO_PAGE_REFFED/PINNED as being the same as FOLL_GET/PIN. =20 ver #5) - Transcribe the FOLL_* flags returned by iov_iter_extract_pages() to BIO_* flags and got rid of bi_cleanup_mode. - Replaced BIO_NO_PAGE_REF to BIO_PAGE_REFFED in the preceding patch. block/bio.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/block/bio.c b/block/bio.c index 17bd01ecde36..798cc4cf3bd2 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1205,7 +1205,7 @@ static int bio_iov_add_page(struct bio *bio, struct p= age *page, } =20 if (same_page) - put_page(page); + bio_release_page(bio, page); return 0; } =20 @@ -1219,7 +1219,7 @@ static int bio_iov_add_zone_append_page(struct bio *b= io, struct page *page, queue_max_zone_append_sectors(q), &same_page) !=3D len) return -EINVAL; if (same_page) - put_page(page); + bio_release_page(bio, page); return 0; } =20 @@ -1230,10 +1230,10 @@ static int bio_iov_add_zone_append_page(struct bio = *bio, struct page *page, * @bio: bio to add pages to * @iter: iov iterator describing the region to be mapped * - * Pins pages from *iter and appends them to @bio's bvec array. The - * pages will have to be released using put_page() when done. - * For multi-segment *iter, this function only adds pages from the - * next non-empty segment of the iov iterator. + * Extracts pages from *iter and appends them to @bio's bvec array. The p= ages + * will have to be cleaned up in the way indicated by the BIO_PAGE_PINNED = flag. + * For a multi-segment *iter, this function only adds pages from the next + * non-empty segment of the iov iterator. */ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) { @@ -1265,9 +1265,9 @@ static int __bio_iov_iter_get_pages(struct bio *bio, = struct iov_iter *iter) * result to ensure the bio's total size is correct. The remainder of * the iov data will be picked up in the next bio iteration. */ - size =3D iov_iter_get_pages(iter, pages, - UINT_MAX - bio->bi_iter.bi_size, - nr_pages, &offset, extraction_flags); + size =3D iov_iter_extract_pages(iter, &pages, + UINT_MAX - bio->bi_iter.bi_size, + nr_pages, extraction_flags, &offset); if (unlikely(size <=3D 0)) return size ? size : -EFAULT; =20 @@ -1300,7 +1300,7 @@ static int __bio_iov_iter_get_pages(struct bio *bio, = struct iov_iter *iter) iov_iter_revert(iter, left); out: while (i < nr_pages) - put_page(pages[i++]); + bio_release_page(bio, pages[i++]); =20 return ret; } @@ -1335,7 +1335,8 @@ int bio_iov_iter_get_pages(struct bio *bio, struct io= v_iter *iter) return 0; } =20 - bio_set_flag(bio, BIO_PAGE_REFFED); + if (iov_iter_extract_will_pin(iter)) + bio_set_flag(bio, BIO_PAGE_PINNED); do { ret =3D __bio_iov_iter_get_pages(bio, iter); } while (!ret && iov_iter_count(iter) && !bio_full(bio, 0)); From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C71EC77B7F for ; Fri, 19 May 2023 07:47:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231543AbjESHrZ (ORCPT ); Fri, 19 May 2023 03:47:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231538AbjESHqr (ORCPT ); Fri, 19 May 2023 03:46:47 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC17E198A for ; Fri, 19 May 2023 00:43:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482171; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0UdxcT7QYPFwRvPO4h85pE794/0OGizygn6UovNdu60=; b=jUVUx3KTwhweTrBjGAPexHrmA0/P6O4LZnJj11WXcf/V6hzdOfJ+Zwh4jwsgczxBs1Xgdu p1I0SkvqlYqClWwlOOG05s8t/8n9z5rtCQkvIsnh1Z90IbRAbGFNIoCFdwoqRoWkD8i6dw siMFv21+6e0BM/UD3NRBWhmQYq4UShQ= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-437-iVklZYPOP7qgZhTLOyp__w-1; Fri, 19 May 2023 03:42:47 -0400 X-MC-Unique: iVklZYPOP7qgZhTLOyp__w-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C287E38035AD; Fri, 19 May 2023 07:42:46 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id A83937C2A; Fri, 19 May 2023 07:42:44 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v20 32/32] block: convert bio_map_user_iov to use iov_iter_extract_pages Date: Fri, 19 May 2023 08:40:47 +0100 Message-Id: <20230519074047.1739879-33-dhowells@redhat.com> In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This will pin pages or leave them unaltered rather than getting a ref on them as appropriate to the iterator. The pages need to be pinned for DIO rather than having refs taken on them to prevent VM copy-on-write from malfunctioning during a concurrent fork() (the result of the I/O could otherwise end up being visible to/affected by the child process). Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #10) - Drop bio_set_cleanup_mode(), open coding it instead. =20 ver #8) - Split the patch up a bit [hch]. - We should only be using pinned/non-pinned pages and not ref'd pages, so adjust the comments appropriately. =20 ver #7) - Don't treat BIO_PAGE_REFFED/PINNED as being the same as FOLL_GET/PIN. =20 ver #5) - Transcribe the FOLL_* flags returned by iov_iter_extract_pages() to BIO_* flags and got rid of bi_cleanup_mode. - Replaced BIO_NO_PAGE_REF to BIO_PAGE_REFFED in the preceding patch. block/blk-map.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/block/blk-map.c b/block/blk-map.c index 33d9f6e89ba6..3551c3ff17cf 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -281,22 +281,21 @@ static int bio_map_user_iov(struct request *rq, struc= t iov_iter *iter, =20 if (blk_queue_pci_p2pdma(rq->q)) extraction_flags |=3D ITER_ALLOW_P2PDMA; + if (iov_iter_extract_will_pin(iter)) + bio_set_flag(bio, BIO_PAGE_PINNED); =20 - bio_set_flag(bio, BIO_PAGE_REFFED); while (iov_iter_count(iter)) { - struct page **pages, *stack_pages[UIO_FASTIOV]; + struct page *stack_pages[UIO_FASTIOV]; + struct page **pages =3D stack_pages; ssize_t bytes; size_t offs; int npages; =20 - if (nr_vecs <=3D ARRAY_SIZE(stack_pages)) { - pages =3D stack_pages; - bytes =3D iov_iter_get_pages(iter, pages, LONG_MAX, - nr_vecs, &offs, extraction_flags); - } else { - bytes =3D iov_iter_get_pages_alloc(iter, &pages, - LONG_MAX, &offs, extraction_flags); - } + if (nr_vecs > ARRAY_SIZE(stack_pages)) + pages =3D NULL; + + bytes =3D iov_iter_extract_pages(iter, &pages, LONG_MAX, + nr_vecs, extraction_flags, &offs); if (unlikely(bytes <=3D 0)) { ret =3D bytes ? bytes : -EFAULT; goto out_unmap; @@ -318,7 +317,7 @@ static int bio_map_user_iov(struct request *rq, struct = iov_iter *iter, if (!bio_add_hw_page(rq->q, bio, page, n, offs, max_sectors, &same_page)) { if (same_page) - put_page(page); + bio_release_page(bio, page); break; } =20 @@ -330,7 +329,7 @@ static int bio_map_user_iov(struct request *rq, struct = iov_iter *iter, * release the pages we didn't map into the bio, if any */ while (j < npages) - put_page(pages[j++]); + bio_release_page(bio, pages[j++]); if (pages !=3D stack_pages) kvfree(pages); /* couldn't stuff something into bio? */ From nobody Wed Sep 10 05:31:03 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDC12C77B7A for ; Fri, 19 May 2023 07:52:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231458AbjESHwW (ORCPT ); Fri, 19 May 2023 03:52:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231403AbjESHwF (ORCPT ); Fri, 19 May 2023 03:52:05 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A15C51BC6 for ; Fri, 19 May 2023 00:50:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684482566; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XeaCD2UlGOxbB6KCWNh7uHI2oUW1y45K4W4A/6wlL3s=; b=efYGnOlta2BWflszXGBpaz3SujAQLCkIXnZn9dkUPFTRRc5xk66ea449S/KFCDrrtW9Aea OsqeV0RGDaB841EC0uLnjgvI1esmIuWR4e31qy/4NFW+VAKBZqwSDl4LjIPANttMk/+cDD lTo58/ZYNFnTymKMJDPxch4YcLTrQIY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-352-oXYVwhe6PISPwOFbVlp0Kg-1; Fri, 19 May 2023 03:49:21 -0400 X-MC-Unique: oXYVwhe6PISPwOFbVlp0Kg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E867B2999B22; Fri, 19 May 2023 07:49:20 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 97D5B4F2DE0; Fri, 19 May 2023 07:49:18 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 From: David Howells In-Reply-To: <20230519074047.1739879-1-dhowells@redhat.com> References: <20230519074047.1739879-1-dhowells@redhat.com> To: Jens Axboe , Al Viro , Christoph Hellwig Cc: dhowells@redhat.com, Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Christian Brauner , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH] iov_iter: Add automatic-alloc for ITER_BVEC and use in direct_splice_read() MIME-Version: 1.0 Content-ID: <1740263.1684482558.1@warthog.procyon.org.uk> Content-Transfer-Encoding: quoted-printable Date: Fri, 19 May 2023 08:49:18 +0100 Message-ID: <1740264.1684482558@warthog.procyon.org.uk> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" If it's a problem that direct_splice_read() always allocates as much memory= as is asked for and that will fit into the pipe when less could be allocated in the case that, say, an O_DIRECT-read will hit a hole and do a short read or= a socket will return less than was asked for, something like the attached modification to ITER_BVEC could be made. David --- iov_iter: Add automatic-alloc for ITER_BVEC and use in direct_splice_read() Add a flag to the iov_iter struct that tells things that write to or allow writing to a BVEC-type iterator that they should allocate pages to fill in any slots in the bio_vec array that have null page pointers. This allows the bufferage in the bvec to be allocated on-demand. Iterators of this type are initialised with iov_iter_bvec_autoalloc() instead of iov_iter_bvec(). Only destination (ie. READ/ITER_DEST) iterators may be used in this fashion. An additional function, iov_iter_auto_alloc() is provided to perform the allocation in the case that the caller wishes to make use of the bio_vec array directly and the block layer is modified to use it. direct_splice_read() is then modified to make use of this. This is less efficient if we know in advance that we want to allocate the full buffer as we can't so easily use bulk allocation, but it does mean in cases where we might not want the full buffer (say we hit a hole in DIO), we don't have to allocate it. Signed-off-by: David Howells cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org --- block/bio.c | 2=20 fs/splice.c | 36 ++++++----------- include/linux/uio.h | 13 ++++-- lib/iov_iter.c | 110 +++++++++++++++++++++++++++++++++++++++++++++++= ++++- 4 files changed, 133 insertions(+), 28 deletions(-) diff --git a/block/bio.c b/block/bio.c index 798cc4cf3bd2..72d5c1125df2 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1330,6 +1330,8 @@ int bio_iov_iter_get_pages(struct bio *bio, struct io= v_iter *iter) int ret =3D 0; =20 if (iov_iter_is_bvec(iter)) { + if (!iov_iter_auto_alloc(iter, iov_iter_count(iter))) + return -ENOMEM; bio_iov_bvec_set(bio, iter); iov_iter_advance(iter, bio->bi_iter.bi_size); return 0; diff --git a/fs/splice.c b/fs/splice.c index 56d9802729d0..30e7a31c5ada 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -310,10 +310,8 @@ ssize_t direct_splice_read(struct file *in, loff_t *pp= os, struct iov_iter to; struct bio_vec *bv; struct kiocb kiocb; - struct page **pages; ssize_t ret; - size_t used, npages, chunk, remain, keep =3D 0; - int i; + size_t used, npages, chunk, remain, keep =3D 0, i; =20 if (!len) return 0; @@ -334,30 +332,14 @@ ssize_t direct_splice_read(struct file *in, loff_t *p= pos, len =3D min_t(size_t, len, npages * PAGE_SIZE); npages =3D DIV_ROUND_UP(len, PAGE_SIZE); =20 - bv =3D kzalloc(array_size(npages, sizeof(bv[0])) + - array_size(npages, sizeof(struct page *)), GFP_KERNEL); + bv =3D kzalloc(array_size(npages, sizeof(bv[0])), GFP_KERNEL); if (!bv) return -ENOMEM; =20 - pages =3D (struct page **)(bv + npages); - npages =3D alloc_pages_bulk_array(GFP_USER, npages, pages); - if (!npages) { - kfree(bv); - return -ENOMEM; - } - remain =3D len =3D min_t(size_t, len, npages * PAGE_SIZE); =20 - for (i =3D 0; i < npages; i++) { - chunk =3D min_t(size_t, PAGE_SIZE, remain); - bv[i].bv_page =3D pages[i]; - bv[i].bv_offset =3D 0; - bv[i].bv_len =3D chunk; - remain -=3D chunk; - } - /* Do the I/O */ - iov_iter_bvec(&to, ITER_DEST, bv, npages, len); + iov_iter_bvec_autoalloc(&to, ITER_DEST, bv, npages, len); init_sync_kiocb(&kiocb, in); kiocb.ki_pos =3D *ppos; ret =3D call_read_iter(in, &kiocb, &to); @@ -376,8 +358,16 @@ ssize_t direct_splice_read(struct file *in, loff_t *pp= os, } =20 /* Free any pages that didn't get touched at all. */ - if (keep < npages) - release_pages(pages + keep, npages - keep); + if (keep < npages) { + struct page **pages =3D (struct page **)&bv[keep]; + size_t j =3D 0; + + for (i =3D keep; i < npages; i++) + if (bv[i].bv_page) + pages[j++] =3D bv[i].bv_page; + if (j) + release_pages(pages, j); + } =20 /* Push the remaining pages into the pipe. */ remain =3D ret; diff --git a/include/linux/uio.h b/include/linux/uio.h index 60c342bb7ab8..6bc2287021d9 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -40,10 +40,11 @@ struct iov_iter_state { =20 struct iov_iter { u8 iter_type; - bool copy_mc; - bool nofault; - bool data_source; - bool user_backed; + bool copy_mc:1; + bool nofault:1; + bool data_source:1; + bool user_backed:1; + bool auto_alloc:1; /* Automatically alloc pages into a bvec */ union { size_t iov_offset; int last_offset; @@ -263,6 +264,7 @@ static inline bool iov_iter_is_copy_mc(const struct iov= _iter *i) } #endif =20 +bool iov_iter_auto_alloc(struct iov_iter *iter, size_t count); size_t iov_iter_zero(size_t bytes, struct iov_iter *); bool iov_iter_is_aligned(const struct iov_iter *i, unsigned addr_mask, unsigned len_mask); @@ -274,6 +276,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int dir= ection, const struct kvec unsigned long nr_segs, size_t count); void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struc= t bio_vec *bvec, unsigned long nr_segs, size_t count); +void iov_iter_bvec_autoalloc(struct iov_iter *i, unsigned int direction, + const struct bio_vec *bvec, unsigned long nr_segs, + size_t count); void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t c= ount); void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xa= rray *xarray, loff_t start, size_t count); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index f18138e0292a..3643f9d80ecc 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -52,7 +52,11 @@ while (n) { \ unsigned offset =3D p->bv_offset + skip; \ unsigned left; \ - void *kaddr =3D kmap_local_page(p->bv_page + \ + void *kaddr; \ + \ + if (!p->bv_page) \ + break; \ + kaddr =3D kmap_local_page(p->bv_page + \ offset / PAGE_SIZE); \ base =3D kaddr + offset % PAGE_SIZE; \ len =3D min(min(n, (size_t)(p->bv_len - skip)), \ @@ -159,6 +163,49 @@ __out: \ #define iterate_and_advance(i, n, base, len, off, I, K) \ __iterate_and_advance(i, n, base, len, off, I, ((void)(K),0)) =20 +/* + * Preallocate pages into the bvec sufficient to store count bytes. + */ +static bool bvec_auto_alloc(struct iov_iter *iter, size_t count) +{ + struct bio_vec *bvec =3D (struct bio_vec *)iter->bvec; + int j; + + if (!count) + return true; + + count +=3D iter->iov_offset; + for (j =3D 0; j < iter->nr_segs; j++) { + if (!bvec[j].bv_page) { + bvec[j].bv_page =3D alloc_page(GFP_KERNEL); + if (!bvec[j].bv_page) + return false; + } + if (bvec[j].bv_len >=3D count) + break; + count -=3D bvec[j].bv_len; + } + + return true; +} + +/** + * iov_iter_auto_alloc - Perform auto-alloc for an iterator + * @iter: The iterator to do the allocation for + * @count: The number of bytes we need to store + * + * Perform auto-allocation on a iterator. This only works with ITER_BVEC-= type + * iterators. It will make sure that pages are allocated sufficient to st= ore + * the specified number of bytes. Returns true if sufficient pages are pr= esent + * in the bvec and false if an allocation failure occurs. + */ +bool iov_iter_auto_alloc(struct iov_iter *iter, size_t count) +{ + return !iov_iter_is_bvec(iter) || !iter->auto_alloc || + bvec_auto_alloc(iter, count); +} +EXPORT_SYMBOL_GPL(iov_iter_auto_alloc); + static int copyout(void __user *to, const void *from, size_t n) { if (should_fail_usercopy()) @@ -313,6 +360,8 @@ size_t _copy_to_iter(const void *addr, size_t bytes, st= ruct iov_iter *i) return 0; if (user_backed_iter(i)) might_fault(); + if (!iov_iter_auto_alloc(i, bytes)) + return 0; iterate_and_advance(i, bytes, base, len, off, copyout(base, addr + off, len), memcpy(base, addr + off, len) @@ -362,6 +411,8 @@ size_t _copy_mc_to_iter(const void *addr, size_t bytes,= struct iov_iter *i) return 0; if (user_backed_iter(i)) might_fault(); + if (!iov_iter_auto_alloc(i, bytes)) + return 0; __iterate_and_advance(i, bytes, base, len, off, copyout_mc(base, addr + off, len), copy_mc_to_kernel(base, addr + off, len) @@ -503,6 +554,8 @@ size_t copy_page_to_iter_nofault(struct page *page, uns= igned offset, size_t byte return 0; if (WARN_ON_ONCE(i->data_source)) return 0; + if (!iov_iter_auto_alloc(i, bytes)) + return 0; page +=3D offset / PAGE_SIZE; // first subpage offset %=3D PAGE_SIZE; while (1) { @@ -557,6 +610,8 @@ EXPORT_SYMBOL(copy_page_from_iter); =20 size_t iov_iter_zero(size_t bytes, struct iov_iter *i) { + if (!iov_iter_auto_alloc(i, bytes)) + return -ENOMEM; iterate_and_advance(i, bytes, base, len, count, clear_user(base, len), memset(base, 0, len) @@ -598,6 +653,7 @@ static void iov_iter_bvec_advance(struct iov_iter *i, s= ize_t size) size +=3D i->iov_offset; =20 for (bvec =3D i->bvec, end =3D bvec + i->nr_segs; bvec < end; bvec++) { + BUG_ON(!bvec->bv_page); if (likely(size < bvec->bv_len)) break; size -=3D bvec->bv_len; @@ -740,6 +796,51 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int di= rection, } EXPORT_SYMBOL(iov_iter_bvec); =20 +/** + * iov_iter_bvec_autoalloc - Initialise a BVEC-type I/O iterator with auto= matic alloc + * @i: The iterator to initialise. + * @direction: The direction of the transfer. + * @bvec: The array of bio_vecs listing the buffer segments + * @nr_segs: The number of segments in @bvec[]. + * @count: The size of the I/O buffer in bytes. + * + * Set up an I/O iterator to insert pages into a bvec as data is written i= nto + * it where NULL pointers exist in the bvec array (if a pointer isn't NULL= , the + * page it points to will just be used). No more than @nr_segs pages will= be + * filled in. Empty slots will have bv_offset set to 0 and bv_len to + * PAGE_SIZE. + * + * If the iterator is reverted, excess pages will be left for the + * caller to clean up. + */ +void iov_iter_bvec_autoalloc(struct iov_iter *i, unsigned int direction, + const struct bio_vec *bvec, unsigned long nr_segs, + size_t count) +{ + struct bio_vec *bv =3D (struct bio_vec *)bvec; + unsigned long j; + + BUG_ON(direction !=3D READ); + *i =3D (struct iov_iter){ + .iter_type =3D ITER_BVEC, + .copy_mc =3D false, + .data_source =3D direction, + .auto_alloc =3D true, + .bvec =3D bvec, + .nr_segs =3D nr_segs, + .iov_offset =3D 0, + .count =3D count + }; + + for (j =3D 0; j < nr_segs; j++) { + if (!bv[j].bv_page) { + bv[j].bv_offset =3D 0; + bv[j].bv_len =3D PAGE_SIZE; + } + } +} +EXPORT_SYMBOL(iov_iter_bvec_autoalloc); + /** * iov_iter_xarray - Initialise an I/O iterator to use the pages in an xar= ray * @i: The iterator to initialise. @@ -1122,6 +1223,8 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_= iter *i, struct page **p; struct page *page; =20 + if (!iov_iter_auto_alloc(i, maxsize)) + return -ENOMEM; page =3D first_bvec_segment(i, &maxsize, start); n =3D want_pages_array(pages, maxsize, *start, maxpages); if (!n) @@ -1226,6 +1329,8 @@ size_t csum_and_copy_to_iter(const void *addr, size_t= bytes, void *_csstate, csstate->off +=3D bytes; return bytes; } + if (!iov_iter_auto_alloc(i, bytes)) + return -ENOMEM; =20 sum =3D csum_shift(csstate->csum, csstate->off); iterate_and_advance(i, bytes, base, len, off, ({ @@ -1664,6 +1769,9 @@ static ssize_t iov_iter_extract_bvec_pages(struct iov= _iter *i, size_t skip =3D i->iov_offset, offset; int k; =20 + if (!iov_iter_auto_alloc(i, maxsize)) + return -ENOMEM; + for (;;) { if (i->nr_segs =3D=3D 0) return 0;