From nobody Fri Jun 19 20:11:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 685BEC433FE for ; Wed, 30 Mar 2022 20:11:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344819AbiC3UNW (ORCPT ); Wed, 30 Mar 2022 16:13:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230131AbiC3UNP (ORCPT ); Wed, 30 Mar 2022 16:13:15 -0400 Received: from box.fidei.email (box.fidei.email [IPv6:2605:2700:0:2:a800:ff:feba:dc44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C919F6542E; Wed, 30 Mar 2022 13:11:28 -0700 (PDT) Received: from authenticated-user (box.fidei.email [71.19.144.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by box.fidei.email (Postfix) with ESMTPSA id 39BEE804FB; Wed, 30 Mar 2022 16:11:28 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=dorminy.me; s=mail; t=1648671088; bh=wtT7OHHCcx98T8VqFZwDN0GGICwV8kFV3QJpvmbxiBo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ruAflQygb4/+91m0+yMOXk3nzbxBAVdLIlMuOgr9ucwz42Br2NAlzYrfHrFvtfpR7 fNEq6CAP1J2+aeXiAm0oO9TsbVIo2Fl1OpHxBKQi6wJIL+tVfIa8BHxZCTSmfHpJwl rOC/n8sjdrbBrEj+rRcTxxuzeg4HAOgb8ZKg97NDQ2R/2+PQr3vz9SQYM3zruxHSVt D6sMA0YeAUYGzZWepPp2cdFcQ0ynzbpDj3fZbeyFLiNw4q+Dre8Be7zBbt7ctSZp4N KihvHA2ZB4yW8sfYnxQAQVB21i75LYY/OMVvbs2ko5BkyIYVHrTNH+YlqRzlqzb4uE vBeFWqGsVmJGQ== From: Sweet Tea Dorminy To: Chris Mason , Josef Bacik , David Sterba , Nick Terrell , linux-kernel@vger.kernel.org, linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Sweet Tea Dorminy , Nikolay Borisov Subject: [PATCH v3 1/2] btrfs: factor out allocating an array of pages Date: Wed, 30 Mar 2022 16:11:22 -0400 Message-Id: <3feb62bb8e605e708a0f839136a7354ec66b7b6b.1648669832.git.sweettea-kernel@dorminy.me> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Several functions currently populate an array of page pointers one allocated page at a time; factor out the common code so as to allow improvements to all of the sites at once. Signed-off-by: Sweet Tea Dorminy Reviewed-by: Nikolay Borisov --- Changes in v3: - expanded 'int r' to 'int ret', reflected renaming throughout - made sure there was space after variable declarations - fixed up kerneldoc to be more btrfs's style Changes in v2:=20 - moved the new btrfs_alloc_page_array() function to extent_io.[ch] --- fs/btrfs/check-integrity.c | 8 ++--- fs/btrfs/compression.c | 39 ++++++++++------------- fs/btrfs/extent_io.c | 65 ++++++++++++++++++++++++++++---------- fs/btrfs/extent_io.h | 2 ++ fs/btrfs/inode.c | 10 +++--- fs/btrfs/raid56.c | 30 +++--------------- 6 files changed, 79 insertions(+), 75 deletions(-) diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c index 7e9f90fa0388..366d5a80f3c5 100644 --- a/fs/btrfs/check-integrity.c +++ b/fs/btrfs/check-integrity.c @@ -1553,11 +1553,9 @@ static int btrfsic_read_block(struct btrfsic_state *= state, return -ENOMEM; block_ctx->datav =3D block_ctx->mem_to_free; block_ctx->pagev =3D (struct page **)(block_ctx->datav + num_pages); - for (i =3D 0; i < num_pages; i++) { - block_ctx->pagev[i] =3D alloc_page(GFP_NOFS); - if (!block_ctx->pagev[i]) - return -1; - } + ret =3D btrfs_alloc_page_array(num_pages, block_ctx->pagev); + if (ret) + return ret; =20 dev_bytenr =3D block_ctx->dev_bytenr; for (i =3D 0; i < num_pages;) { diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index be476f094300..3772195222e9 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -801,8 +801,6 @@ blk_status_t btrfs_submit_compressed_read(struct inode = *inode, struct bio *bio, struct extent_map_tree *em_tree; struct compressed_bio *cb; unsigned int compressed_len; - unsigned int nr_pages; - unsigned int pg_index; struct bio *comp_bio =3D NULL; const u64 disk_bytenr =3D bio->bi_iter.bi_sector << SECTOR_SHIFT; u64 cur_disk_byte =3D disk_bytenr; @@ -812,7 +810,8 @@ blk_status_t btrfs_submit_compressed_read(struct inode = *inode, struct bio *bio, u64 em_start; struct extent_map *em; blk_status_t ret; - int faili =3D 0; + int ret2; + int i; u8 *sums; =20 em_tree =3D &BTRFS_I(inode)->extent_tree; @@ -855,25 +854,20 @@ blk_status_t btrfs_submit_compressed_read(struct inod= e *inode, struct bio *bio, cb->compress_type =3D extent_compress_type(bio_flags); cb->orig_bio =3D bio; =20 - nr_pages =3D DIV_ROUND_UP(compressed_len, PAGE_SIZE); - cb->compressed_pages =3D kcalloc(nr_pages, sizeof(struct page *), - GFP_NOFS); + cb->nr_pages =3D DIV_ROUND_UP(compressed_len, PAGE_SIZE); + cb->compressed_pages =3D kcalloc(cb->nr_pages, sizeof(struct page *), + GFP_NOFS); if (!cb->compressed_pages) { ret =3D BLK_STS_RESOURCE; - goto fail1; + goto fail; } =20 - for (pg_index =3D 0; pg_index < nr_pages; pg_index++) { - cb->compressed_pages[pg_index] =3D alloc_page(GFP_NOFS); - if (!cb->compressed_pages[pg_index]) { - faili =3D pg_index - 1; - ret =3D BLK_STS_RESOURCE; - goto fail2; - } + ret2 =3D btrfs_alloc_page_array(cb->nr_pages, cb->compressed_pages); + if (ret2) { + ret =3D BLK_STS_RESOURCE; + goto fail; } - faili =3D nr_pages - 1; - cb->nr_pages =3D nr_pages; - +=09 add_ra_bio_pages(inode, em_start + em_len, cb); =20 /* include any pages we added in add_ra-bio_pages */ @@ -949,14 +943,15 @@ blk_status_t btrfs_submit_compressed_read(struct inod= e *inode, struct bio *bio, } return BLK_STS_OK; =20 -fail2: - while (faili >=3D 0) { - __free_page(cb->compressed_pages[faili]); - faili--; +fail: + if (cb->compressed_pages) { + for (i =3D 0; i < cb->nr_pages; i++) { + if (cb->compressed_pages[i]) + __free_page(cb->compressed_pages[i]); + } } =20 kfree(cb->compressed_pages); -fail1: kfree(cb); out: free_extent_map(em); diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 53b59944013f..ab4c1c4d1b59 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -3132,6 +3132,33 @@ static void end_bio_extent_readpage(struct bio *bio) bio_put(bio); } =20 +/** + * Populate every slot in a provided array with pages. + * + * @nr_pages: the number of pages to request + * @page_array: the array to fill with pages; any existing non-null entrie= s in + * the array will be skipped + * + * Return: 0 if all pages were able to be allocated; -ENOMEM otherwise, an= d the + * caller is responsible for freeing all non-null page pointers in the arr= ay. + */ +int btrfs_alloc_page_array(unsigned long nr_pages, struct page **page_arra= y) +{ + int i; + + for (i =3D 0; i < nr_pages; i++) { + struct page *page; + + if (page_array[i]) + continue; + page =3D alloc_page(GFP_NOFS); + if (!page) + return -ENOMEM; + page_array[i] =3D page; + } + return 0; +} + /* * Initialize the members up to but not including 'bio'. Use after allocat= ing a * new bio by bio_alloc_bioset as it does not initialize the bytes outside= of @@ -5898,9 +5925,9 @@ __alloc_extent_buffer(struct btrfs_fs_info *fs_info, = u64 start, struct extent_buffer *btrfs_clone_extent_buffer(const struct extent_buffer= *src) { int i; - struct page *p; struct extent_buffer *new; int num_pages =3D num_extent_pages(src); + int ret; =20 new =3D __alloc_extent_buffer(src->fs_info, src->start, src->len); if (new =3D=3D NULL) @@ -5913,22 +5940,23 @@ struct extent_buffer *btrfs_clone_extent_buffer(con= st struct extent_buffer *src) */ set_bit(EXTENT_BUFFER_UNMAPPED, &new->bflags); =20 + memset(new->pages, 0, sizeof(*new->pages) * num_pages); + ret =3D btrfs_alloc_page_array(num_pages, new->pages); + if (ret) { + btrfs_release_extent_buffer(new); + return NULL; + } + for (i =3D 0; i < num_pages; i++) { int ret; + struct page *p =3D new->pages[i]; =20 - p =3D alloc_page(GFP_NOFS); - if (!p) { - btrfs_release_extent_buffer(new); - return NULL; - } ret =3D attach_extent_buffer_page(new, p, NULL); if (ret < 0) { - put_page(p); btrfs_release_extent_buffer(new); return NULL; } WARN_ON(PageDirty(p)); - new->pages[i] =3D p; copy_page(page_address(p), page_address(src->pages[i])); } set_extent_buffer_uptodate(new); @@ -5942,31 +5970,36 @@ struct extent_buffer *__alloc_dummy_extent_buffer(s= truct btrfs_fs_info *fs_info, struct extent_buffer *eb; int num_pages; int i; + int ret; =20 eb =3D __alloc_extent_buffer(fs_info, start, len); if (!eb) return NULL; =20 num_pages =3D num_extent_pages(eb); + ret =3D btrfs_alloc_page_array(num_pages, eb->pages); + if (ret) + goto err; + for (i =3D 0; i < num_pages; i++) { - int ret; + struct page *p =3D eb->pages[i]; =20 - eb->pages[i] =3D alloc_page(GFP_NOFS); - if (!eb->pages[i]) - goto err; - ret =3D attach_extent_buffer_page(eb, eb->pages[i], NULL); + ret =3D attach_extent_buffer_page(eb, p, NULL); if (ret < 0) goto err; } + set_extent_buffer_uptodate(eb); btrfs_set_header_nritems(eb, 0); set_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags); =20 return eb; err: - for (; i > 0; i--) { - detach_extent_buffer_page(eb, eb->pages[i - 1]); - __free_page(eb->pages[i - 1]); + for (i =3D 0; i < num_pages; i++) { + if (eb->pages[i]) { + detach_extent_buffer_page(eb, eb->pages[i]); + __free_page(eb->pages[i]); + } } __free_extent_buffer(eb); return NULL; diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index 151e9da5da2d..464269e28941 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -277,6 +277,8 @@ void extent_range_redirty_for_io(struct inode *inode, u= 64 start, u64 end); void extent_clear_unlock_delalloc(struct btrfs_inode *inode, u64 start, u6= 4 end, struct page *locked_page, u32 bits_to_clear, unsigned long page_ops); + +int btrfs_alloc_page_array(unsigned long nr_pages, struct page **page_arra= y); struct bio *btrfs_bio_alloc(unsigned int nr_iovecs); struct bio *btrfs_bio_clone(struct bio *bio); struct bio *btrfs_bio_clone_partial(struct bio *orig, u64 offset, u64 size= ); diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index c7b15634fe70..121858652a09 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -10427,13 +10427,11 @@ static ssize_t btrfs_encoded_read_regular(struct = kiocb *iocb, pages =3D kcalloc(nr_pages, sizeof(struct page *), GFP_NOFS); if (!pages) return -ENOMEM; - for (i =3D 0; i < nr_pages; i++) { - pages[i] =3D alloc_page(GFP_NOFS); - if (!pages[i]) { - ret =3D -ENOMEM; - goto out; + ret =3D btrfs_alloc_page_array(nr_pages, pages); + if (ret) { + ret =3D -ENOMEM; + goto out; } - } =20 ret =3D btrfs_encoded_read_regular_fill_pages(inode, start, disk_bytenr, disk_io_size, pages); diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 0e239a4c3b26..ea7a9152b1cc 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -1026,37 +1026,15 @@ static struct btrfs_raid_bio *alloc_rbio(struct btr= fs_fs_info *fs_info, /* allocate pages for all the stripes in the bio, including parity */ static int alloc_rbio_pages(struct btrfs_raid_bio *rbio) { - int i; - struct page *page; - - for (i =3D 0; i < rbio->nr_pages; i++) { - if (rbio->stripe_pages[i]) - continue; - page =3D alloc_page(GFP_NOFS); - if (!page) - return -ENOMEM; - rbio->stripe_pages[i] =3D page; - } - return 0; + return btrfs_alloc_page_array(rbio->nr_pages, rbio->stripe_pages); } =20 /* only allocate pages for p/q stripes */ static int alloc_rbio_parity_pages(struct btrfs_raid_bio *rbio) { - int i; - struct page *page; - - i =3D rbio_stripe_page_index(rbio, rbio->nr_data, 0); - - for (; i < rbio->nr_pages; i++) { - if (rbio->stripe_pages[i]) - continue; - page =3D alloc_page(GFP_NOFS); - if (!page) - return -ENOMEM; - rbio->stripe_pages[i] =3D page; - } - return 0; + int data_pages =3D rbio_stripe_page_index(rbio, rbio->nr_data, 0); + return btrfs_alloc_page_array(rbio->nr_pages - data_pages, + rbio->stripe_pages + data_pages); } =20 /* --=20 2.35.1 From nobody Fri Jun 19 20:11:36 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 309C6C433F5 for ; Wed, 30 Mar 2022 20:11:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231941AbiC3UNZ (ORCPT ); Wed, 30 Mar 2022 16:13:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231761AbiC3UNP (ORCPT ); Wed, 30 Mar 2022 16:13:15 -0400 Received: from box.fidei.email (box.fidei.email [IPv6:2605:2700:0:2:a800:ff:feba:dc44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 226C26542A; Wed, 30 Mar 2022 13:11:30 -0700 (PDT) Received: from authenticated-user (box.fidei.email [71.19.144.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by box.fidei.email (Postfix) with ESMTPSA id 9F5038050E; Wed, 30 Mar 2022 16:11:29 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=dorminy.me; s=mail; t=1648671089; bh=U5dIQevY506VUTcHlKOqqtmto/0GP4VFy9OThN9O8gM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eW3U6GA4MKB5nVVfcawq/YhI8dpwuxKYCD4LS8h0yfyP5l38+hWLGEksHNn9v025q Dk/bp+vEmqDQWJYfUtZSMYXow25XpdIJyYeLCyp8wXJOvzj61aQKcr0I9LDkSH/eDE baBQLHh0qU3USpFjqTr8EFdabu9NHTTaujwKTSpYrYPQevOPcq/8LWOKZw8aVspt4A CuS3jhqkmrwiFXoTZzuJv12nsnqLebYyM4zRvJnP4KXDgGcfUfrW1Skipq2cxtbiG1 Wx/9TLtv6pVj9axFId77+fpykxsd56nLD38OERCSdfzPyM+1Cco4ZWh46VzicoDqev T3fE+SobYW6ng== From: Sweet Tea Dorminy To: Chris Mason , Josef Bacik , David Sterba , Nick Terrell , linux-kernel@vger.kernel.org, linux-btrfs@vger.kernel.org, kernel-team@fb.com Cc: Sweet Tea Dorminy , Nikolay Borisov Subject: [PATCH v3 2/2] btrfs: allocate page arrays using bulk page allocator Date: Wed, 30 Mar 2022 16:11:23 -0400 Message-Id: In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" While calling alloc_page() in a loop is an effective way to populate an array of pages, the kernel provides a method to allocate pages in bulk. alloc_pages_bulk_array() populates the NULL slots in a page array, trying to grab more than one page at a time. Unfortunately, it doesn't guarantee allocating all slots in the array, but it's easy to call it in a loop and return an error if no progress occurs. Similar code can be found in xfs/xfs_buf.c:xfs_buf_alloc_pages(). Signed-off-by: Sweet Tea Dorminy Reviewed-by: Nikolay Borisov --- Changes in v3: - Added a newline after variable declaration Changes in v2: - Moved from ctree.c to extent_io.c --- fs/btrfs/extent_io.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index ab4c1c4d1b59..b268e47aa2b7 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -3144,19 +3144,25 @@ static void end_bio_extent_readpage(struct bio *bio) */ int btrfs_alloc_page_array(unsigned long nr_pages, struct page **page_arra= y) { - int i; + long allocated =3D 0; + + for (;;) { + long last =3D allocated; =20 - for (i =3D 0; i < nr_pages; i++) { - struct page *page; + allocated =3D alloc_pages_bulk_array(GFP_NOFS, nr_pages, + page_array); + if (allocated =3D=3D nr_pages) + return 0; =20 - if (page_array[i]) + if (allocated !=3D last) continue; - page =3D alloc_page(GFP_NOFS); - if (!page) - return -ENOMEM; - page_array[i] =3D page; + /* + * During this iteration, no page could be allocated, even + * though alloc_pages_bulk_array() falls back to alloc_page() + * if it could not bulk-allocate. So we must be out of memory. + */ + return -ENOMEM; } - return 0; } =20 /* --=20 2.35.1