From nobody Sat Feb 7 19:41:45 2026 Received: from mail-il1-f177.google.com (mail-il1-f177.google.com [209.85.166.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDF851E1C26 for ; Fri, 13 Dec 2024 15:56:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105368; cv=none; b=fNTGDRRU9ecKJFNOR6NewCt7N5yk0B4Bfne32PPCHwzszaU9XYdjclLrblYoiZ4hAsXOTPST0ahbCfnitbYnifX6RJkZRubW/JMgvrwK821nAfAYjn3j3vlbTfQKM3Wdt/EDmLmJNV/RrE6eSMQxvKeJHMLaQNMDwXK3QacCOwk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105368; c=relaxed/simple; bh=CQHljKL0HvF5DjNsL/7Q8SjdRe1DhHG7gJBg0F4F8y0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Rfe9OrZdU6j68OImPwA0V5jCw9TNhoJz9zZED2rZE4hkjyoQnsC3+v67m8zY90IvFB/BA9xRNXQGl/I6HLrOZE9lnLsE0rd1nAU+Bfc41Ys5gMioyktnkZHCAD7MD3MqqQQ5hX0mM2kZdOiI6TXNAUeQkomxZEihaWfYybU7VSU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=rA2TyAiT; arc=none smtp.client-ip=209.85.166.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="rA2TyAiT" Received: by mail-il1-f177.google.com with SMTP id e9e14a558f8ab-3a817ee9936so6488485ab.2 for ; Fri, 13 Dec 2024 07:56:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105365; x=1734710165; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aJTsU8FOCztsZChrgynI/se4Hg526H6WFOVyatkHjPA=; b=rA2TyAiTEnZDuJUEzMLUg7pzz4q8wsMtXMQ2nso5c9/p5yCYWDwsrcSh98ibhdDugo QYV5xGrQopJndH/3afNqdFjHohTuwzsGMkT761zJSNSavteA9lV4LtZy3bVeWeEALQC7 y2U5OgpUZOM/ilJDz/2vlhIHaKxNy7YNbaHa0Y6V1CvSFxFSqL0Zed5VpY7KA2Mbm13b RIPhV06nHYQpDmJvWZLtjsFn9pkXVnRcIJ/wY0tu4icjeX7eaBnpKohIA8TsfsH9ZdF4 g8AAiof2EMeyErrAX30lLtyacgcuzUBw4bCRybt59rq1NmeuU/eLUcwz7Tcm0Zh4oVim BvWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105365; x=1734710165; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aJTsU8FOCztsZChrgynI/se4Hg526H6WFOVyatkHjPA=; b=S8jtvIyRW2vSC4oZWi5l+FmosTt4ef0CA0wRXq3H9VnJOF5lzWl2v1dCrxLe2wuO6p Cq+aKfaSG5Nm+bEytR7sqrWK8Ewaz1OpuJbQnuArGIuNuBt/wbA/RZQKFkV3CChaxCl7 hqYrKqAZCoQZ2bEXf3WfxzM00Tc2y4tGwKHQH5xy8+f7w+uR4JXecKHZsn/F/oNrn1zj FhAiOREP/tZwkAlWCdRDmSKgIiRy4+yiuBPB8v+4CsoYpMrUEtxveQU+JRyZ79SjkIjM Esn0snv+sMSKRTNlJkw2tEbso3p++GwrBEWnfoKU8fW6Y7tkZoMsQMQXfuYw2E+FDc2d 6Ckw== X-Forwarded-Encrypted: i=1; AJvYcCUZjR/KLEzsRm3mSOKwBFzmLSkuTfd4ZVNZ+6B/IenQwb7QQlnHDZOyei7+ly4cmCeeLgg20rwHnBB5UBQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yz+RJ8JuGbd4/pwx3sKRDcE0DQvJ353cv6HDnRrwXDzvDqnFkme sPWh8BEgo5t40hrCvXu4DYpNbDkOij2785JqUk8EFH9Rpq8GfnUQaUInUs9VNLs= X-Gm-Gg: ASbGnctXHGt5u4z+DcCZItzBW141Dl+tQ+tZAkMqcWxnVzQy9r2t9vpwGs7vCypw4A2 qQQBcX3pYV5wOTBg9Oj18VlChV9/Q5FklATidmK97agm+TaYxiu1Mhz1rWazxZTm+ZMUBhbZ5DN VfseSnTbO8OOf1fUU3g5gDDhGnBu3S78SnFbdnqhdAiYp2+4Zuyz46W4GWJFfhjidIbYQE5YrNT FTYEN0tiBO1k56aNuSYzSIUInC8Q9mEy9paHq2Wb0Gr9UjHbXc/TsfWBXmd X-Google-Smtp-Source: AGHT+IHzvgK4Rw98UmNxw7TWkyvAABSCjKuo4ks1FFrTwghvSS0Ll84bbCaOcol8J/RzXzpmHkduIA== X-Received: by 2002:a05:6e02:1a8b:b0:3a7:7124:bd2c with SMTP id e9e14a558f8ab-3affab62206mr35203095ab.19.1734105365052; Fri, 13 Dec 2024 07:56:05 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:03 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 01/11] mm/filemap: change filemap_create_folio() to take a struct kiocb Date: Fri, 13 Dec 2024 08:55:15 -0700 Message-ID: <20241213155557.105419-2-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rather than pass in both the file and position directly from the kiocb, just take a struct kiocb instead. With the kiocb being passed in, skip passing in the address_space separately as well. While doing so, move the ki_flags checking into filemap_create_folio() as well. In preparation for actually needing the kiocb in the function. No functional changes in this patch. Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- mm/filemap.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index f61cf51c2238..8b29323b15d7 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2459,15 +2459,17 @@ static int filemap_update_page(struct kiocb *iocb, return error; } =20 -static int filemap_create_folio(struct file *file, - struct address_space *mapping, loff_t pos, - struct folio_batch *fbatch) +static int filemap_create_folio(struct kiocb *iocb, struct folio_batch *fb= atch) { + struct address_space *mapping =3D iocb->ki_filp->f_mapping; struct folio *folio; int error; unsigned int min_order =3D mapping_min_folio_order(mapping); pgoff_t index; =20 + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + folio =3D filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; @@ -2486,7 +2488,7 @@ static int filemap_create_folio(struct file *file, * well to keep locking rules simple. */ filemap_invalidate_lock_shared(mapping); - index =3D (pos >> (PAGE_SHIFT + min_order)) << min_order; + index =3D (iocb->ki_pos >> (PAGE_SHIFT + min_order)) << min_order; error =3D filemap_add_folio(mapping, folio, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); if (error =3D=3D -EEXIST) @@ -2494,7 +2496,8 @@ static int filemap_create_folio(struct file *file, if (error) goto error; =20 - error =3D filemap_read_folio(file, mapping->a_ops->read_folio, folio); + error =3D filemap_read_folio(iocb->ki_filp, mapping->a_ops->read_folio, + folio); if (error) goto error; =20 @@ -2550,9 +2553,7 @@ static int filemap_get_pages(struct kiocb *iocb, size= _t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); } if (!folio_batch_count(fbatch)) { - if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) - return -EAGAIN; - err =3D filemap_create_folio(filp, mapping, iocb->ki_pos, fbatch); + err =3D filemap_create_folio(iocb, fbatch); if (err =3D=3D AOP_TRUNCATED_PAGE) goto retry; return err; --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 589E11E376C for ; Fri, 13 Dec 2024 15:56:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105371; cv=none; b=Lelt4Cz4ygSn5f3UBumiKc1KdgU9pt4NPZuXH1wj7ntGhidZtbLctEbOCJl7ZxyVULuoe6Um5coraixmF14hwA7wBcf7mYYG6HXwaQei41nyLtpVQDc/IXiSEcAGRz5PQrafia1jqScGFzSTtu/MXOkS3jm4EfvlXG/N7Wd7k4E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105371; c=relaxed/simple; bh=QAWYGVCQ4XEJ3pXTfPeihz8wMx0MW0Etgn545GbAEGI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hTPKCILfXKZpfj0iRSwnF9ZZHIXdThqWDqiOrgBDae8PlyVgcIQKsSLyvlU7G1V7zctCnrpP58Q4TvkAEVIcuQtthaHNB1EDPD5mZ5KWFN+eXcQJypbr884eUHHVSnu5EjC6Jh7h27MCl6NigCkJPa8y/vVW7alZ10YhoY7a8Z8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=t+ltFlte; arc=none smtp.client-ip=209.85.166.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="t+ltFlte" Received: by mail-io1-f44.google.com with SMTP id ca18e2360f4ac-844d555491eso65541339f.0 for ; Fri, 13 Dec 2024 07:56:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105366; x=1734710166; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hANs0KTrn4AbTh3ILmowq5bBaAQVBAkU7MnXkJd6Hic=; b=t+ltFlteBGcA0UWWo45gjFMi5F6YPSar7cOq/L5NtqarY+9zPmveE/IW4vEek6yAUb QlstFsbxAIu9kEt0zwXjNGD88a9t3fWpCAZpiZUmrG49S4IaVM1WmlpK85VlP3rg5XV/ kMwNchTRlDJ12UPl0JzyglzdszaKdN9d81I9v7cFM02VYogMGZAtiTOvqbgnIkb1iN1e AGmN29i9WDOTh0IGhZqjmiUgwP/YGL7zkZdN0tf0mN8GKf9/vgeDbKYExIw4tkFb15EY AdPfmvt+7GErjc501iosLF4jFlpzJPiyobSzkfAB68bBhxVlpp0WLJLXaOrVlhxZAwuh 3wpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105366; x=1734710166; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hANs0KTrn4AbTh3ILmowq5bBaAQVBAkU7MnXkJd6Hic=; b=rItDcPjokpllH9OzUYkfG2RUHuynAVH13Pb19wqvUsc+z9RWOO4OTcUC4fXA1YX+ot Rt+ZBqirK4jgqjXzrgf2IBfIopf6QH5UwthhP1mFro/z0X9IOakzeSlJPY8j1dgq0p8T 7p+3pdrhPmjgTlP0peecfMDX+8XAyeKqRslkHQE9uO5yVc9PTy1Kf+GPGT2H+IcUTv4f fBlKYygmyGSRu6uXn+jWp2Y+Glo2i8rxOlSfR8pemHjQFgB4IwTTxO/mC38DoiHD7ryZ 955fztjilfHwtC6GcwXmt7UHtgjVr0sO0WiIVVMj/ziSDNr0WzGnYotrsF/SQ7Oho/YE raJw== X-Forwarded-Encrypted: i=1; AJvYcCU/EA6MeASgo2FZUEBtiPrVCdQEg72IMB/lmJF3/AFYUDDKZJQVlN1ekggzypJ6/KxmwxFvVvnGKyX/sHY=@vger.kernel.org X-Gm-Message-State: AOJu0YxFFLnUP40yR6oFR/teVclDcPYvjdhWzvcUIHQV+o8ApQkVkHyB 2r6urgmNBM0mrkqXF+L038FSfASo9PxipdsJgiM7Y2scuKxEP3+vWcSpprPzeGI= X-Gm-Gg: ASbGncu2WP6Go9njtAEiRdt+k4uVKRpvREtj5DvJ63nXpc1kpRRxmSVq1VUyD8Yn0dO 0vN6GU/LCJFvOyJ3e0WIod06oGxr4n0G5l8DqijWnwiy+NHACSJG1L+PO0AblhI8nLJ71WeKo4y Tm0eaNj7nKtCO+fAwe8kI2MRqV38DZ+Z5heJs07ShdR2prH9/MSdcawNlzyiCrb4zmnD2VGB9md aOz92SDYaQuLWbSYdf4ikwFf2+bdKxU7g9uLGbxRqjV26uLUxlQuX9taY1o X-Google-Smtp-Source: AGHT+IFVLcWwM0MIr7oz/Yvc9CHKROeMGY95g0ljJ88B+/rBynMSTGl1ysoTz/aE+AP7fUiTK21pEQ== X-Received: by 2002:a05:6e02:1a07:b0:3a7:8040:595b with SMTP id e9e14a558f8ab-3aff50b64a0mr26251175ab.9.1734105366472; Fri, 13 Dec 2024 07:56:06 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:05 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , Christoph Hellwig Subject: [PATCH 02/11] mm/filemap: use page_cache_sync_ra() to kick off read-ahead Date: Fri, 13 Dec 2024 08:55:16 -0700 Message-ID: <20241213155557.105419-3-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rather than use the page_cache_sync_readahead() helper, define our own ractl and use page_cache_sync_ra() directly. In preparation for needing to modify ractl inside filemap_get_pages(). No functional changes in this patch. Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- mm/filemap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 8b29323b15d7..220dc7c6e12f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2527,7 +2527,6 @@ static int filemap_get_pages(struct kiocb *iocb, size= _t count, { struct file *filp =3D iocb->ki_filp; struct address_space *mapping =3D filp->f_mapping; - struct file_ra_state *ra =3D &filp->f_ra; pgoff_t index =3D iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; struct folio *folio; @@ -2542,12 +2541,13 @@ static int filemap_get_pages(struct kiocb *iocb, si= ze_t count, =20 filemap_get_read_batch(mapping, index, last_index - 1, fbatch); if (!folio_batch_count(fbatch)) { + DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index); + if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags =3D memalloc_noio_save(); - page_cache_sync_readahead(mapping, ra, filp, index, - last_index - index); + page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); filemap_get_read_batch(mapping, index, last_index - 1, fbatch); --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-il1-f175.google.com (mail-il1-f175.google.com [209.85.166.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 876D61E3791 for ; Fri, 13 Dec 2024 15:56:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105370; cv=none; b=h6/FAafojJRZYerLslEUq0KEi7Gels1PrlhCTUH7nNieUVTFfF7/Yd+5GgCZ/8PVn8CmVw8RwwamKEABtVqF+MUCuuJzBEZZpF46G9pcquABCS40+8XZTF1CeMgbHQo5XCxFMHGPdEb0JBNhFbxYp4zJRx6pd10oAz9hmFg+MOI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105370; c=relaxed/simple; bh=VkQhoZOrHyZykna0hJTs0oX+MEpAmUB5ySWSsyus/GI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TU6BI/OYE8Gf2K12zci94HMSqrwcx1KN0eCFAlzywynPe8sgx5P26TCHWKBHTMgHSdNLYKN4JQE9ngZITs1kYMc/Rr/wHOWnNQJ76+fyDxqlILnmm10S0cP3HjW4QdNVRFLM9BKg4C1j/CfuCHNN4J+LMsG3kqNY8iN1f+zN0OE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=gn/n6u5x; arc=none smtp.client-ip=209.85.166.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="gn/n6u5x" Received: by mail-il1-f175.google.com with SMTP id e9e14a558f8ab-3a8f1c97ef1so5750355ab.2 for ; Fri, 13 Dec 2024 07:56:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105367; x=1734710167; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BpcNvbMq6OW5fN6dnGRgZf80nNArt3mFYuYvhVaSlK4=; b=gn/n6u5x+8hDMuHR9n/5QfPXMLgtc2zKUHy8C+LfAqaUiOzmGwTZOzXpER2DEDaeCp aCAv0YyN8aZKftlj6NdlPXwUmC1l+aa8oYNf+33hv3rMpHWk9kwA503Yt9fZc5iX3jXj dQLybLF63c9Dn8jA2HcluDYFbK6zH4FA/7484s0VSlYmd3YMq9pYKVyuYzFMK00haI+4 y8QHfpiUia/RXDAakqyqO2974gWkFeBOMDHuObf5MJuKoa9/gXHUxOFeVAFCqeySQR8y u5rgZrAp3Y9cIkD0chUiE628vi1g/Kqo1BzRRoI3EBGmWvBPOOL6zOQfq4b4RdzMU44S 6uqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105367; x=1734710167; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BpcNvbMq6OW5fN6dnGRgZf80nNArt3mFYuYvhVaSlK4=; b=TCrFXpmwiTOIs5f9HulNOESB4KijAbx+svIXG3Jk1i+ZSJdC4i3F7QldKpKYfQgz7G 5E5vDdP2c/OEagpHCvwn3kXY/4m29unN+31t1cxZ0UMRJxwGBg33YzAVa1jKGyBnsRfs sfT/abllLeGuAU95hMTQGSC5QVbV+Rk1iBjaq8zjAQ1ZchreDKuf8i+meOInLA6VzPwu TpJMkAHQTCHrMj0qYsH/5c9kIC50ejMIInvMV37TTmqRZtiDr3CpmU6oO/JoRD8/ZcF2 ADNoPxth1QS2IBDQG10Uet7EHml0Bz2JaWqQgdq2yV3hscw68QVT3OQK4BULlQneVU9T YWSg== X-Forwarded-Encrypted: i=1; AJvYcCWyQuI51NlxaUunXr/jmrVcU6eE+oTdT2UJCqrTQBJHHrsf/VXi5uFArsKYsyG39mOi5lzBH/aExTlME5o=@vger.kernel.org X-Gm-Message-State: AOJu0YxzFeoynZovgUFo00Rc6NEPC1L0khl5lKUrCnqQj4ljyShh+toX RvW2/q9fmer/f/r1OjvPYTaNbKMHtUkZuGZKgBGsrH8o+o2TdSmBTo0waGU/Hw4= X-Gm-Gg: ASbGnct0mGYJrS74oqYi1LGqg2sUcLRIWjBeuB7ssWSftQ5/lScJbUEADOqJ/T7Y1ik gmIWIGxB+2RLemYjr/aR3k7Tbup8Py2ueFwUUAXN/x10hcig74tj535vSf3s34UDxQXMsc/ykyV yIGamsWU2SQEzTJQSA8bkrRL8l6uCxB0byVk7EUbi3j3FvlhMdEbNmJrIaS1q74DfH9EIZB8NJE WYCtOBWuXnxLkpxOPMuB69Ojm5pWeVrV89FVqmCcnxd2iotAAjJVzUlQOVo X-Google-Smtp-Source: AGHT+IFWkhf2olmYWP9+u8WH9KgVfslThGYwEFiU2/KIyOObE8orPa/AadsJR4yt5f2xYXvlvdRoMg== X-Received: by 2002:a05:6e02:1a07:b0:3a7:e4c7:ad18 with SMTP id e9e14a558f8ab-3aff80107a2mr31899055ab.18.1734105367708; Fri, 13 Dec 2024 07:56:07 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:07 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 03/11] mm/readahead: add folio allocation helper Date: Fri, 13 Dec 2024 08:55:17 -0700 Message-ID: <20241213155557.105419-4-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Just a wrapper around filemap_alloc_folio() for now, but add it in preparation for modifying the folio based on the 'ractl' being passed in. No functional changes in this patch. Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- mm/readahead.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index ea650b8b02fb..8a62ad4106ff 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -188,6 +188,12 @@ static void read_pages(struct readahead_control *rac) BUG_ON(readahead_count(rac)); } =20 +static struct folio *ractl_alloc_folio(struct readahead_control *ractl, + gfp_t gfp_mask, unsigned int order) +{ + return filemap_alloc_folio(gfp_mask, order); +} + /** * page_cache_ra_unbounded - Start unchecked readahead. * @ractl: Readahead control. @@ -265,8 +271,8 @@ void page_cache_ra_unbounded(struct readahead_control *= ractl, continue; } =20 - folio =3D filemap_alloc_folio(gfp_mask, - mapping_min_folio_order(mapping)); + folio =3D ractl_alloc_folio(ractl, gfp_mask, + mapping_min_folio_order(mapping)); if (!folio) break; =20 @@ -436,7 +442,7 @@ static inline int ra_alloc_folio(struct readahead_contr= ol *ractl, pgoff_t index, pgoff_t mark, unsigned int order, gfp_t gfp) { int err; - struct folio *folio =3D filemap_alloc_folio(gfp, order); + struct folio *folio =3D ractl_alloc_folio(ractl, gfp, order); =20 if (!folio) return -ENOMEM; @@ -750,7 +756,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ =20 - folio =3D filemap_alloc_folio(gfp_mask, min_order); + folio =3D ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; =20 @@ -779,7 +785,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ =20 - folio =3D filemap_alloc_folio(gfp_mask, min_order); + folio =3D ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; =20 --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-io1-f41.google.com (mail-io1-f41.google.com [209.85.166.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C68E51E3DFE for ; Fri, 13 Dec 2024 15:56:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105371; cv=none; b=ZT2pBLHA7cRZO1NMsNQHxjnds2wI/YF2LnLwnPOdkIQjKcBS92jtP9cikfDNnyU2RLTc8CZQ/QZGw3rQIby0cPBSD8yyzt+04spgh1zZIgv16fXuywK3JXKuEH63SM/YyUSxDrMnxg5oVveKo4wYza/qX8ndAYHzW1qNXaHhLN0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105371; c=relaxed/simple; bh=sPnR3qBnsWLGEr22pRK2TCMcNSO2RW9SlspmNOIT9ew=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ek+LzIUrAiT1jXOTns5m3HO0X1f4tAZ+srJGDLdEWwY86mW9JN4F2e0cun6tkLNefX2Ts/SjF+07RNTEfSe6786o+OHYAVWQ/xUCEF3VvZk8k8Ag8AUsVyn+xCyLM8lIzvjvPJRCaCemjnCDUxjgTYGm0/Zy2AjwCb75Sr321z0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=obyOaWIl; arc=none smtp.client-ip=209.85.166.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="obyOaWIl" Received: by mail-io1-f41.google.com with SMTP id ca18e2360f4ac-84435dbda4bso104648139f.0 for ; Fri, 13 Dec 2024 07:56:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105369; x=1734710169; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HOQaUrSMrbMaSJ8tKHsfpEHyiDws1YYfXq+2bH3/e/Y=; b=obyOaWIlLgEsnjXV7gMx/G3rChEhT3ViesKVO0mDuP1ASnoUTdHT/MOdQLx6tFeFCS 68sfgPX2jZBlKMS6C1TnsGBTyAlkwhdeL3L0rpJqF8gdynDHZGWrxPkH3Z0w+n2gyjuG AYUcKJFnFw4qGqcXUGxleGgzUSU9eaAtaubpt4iTHYuofBvbY9MU7xOPAkeV4kBcACny WpuiSxbjIxGn89SGOWcguKS7+2MN2HjdezPSoIzCp/88od/7d3eN0ImQtejzPHwvlb7B RHO9uDxi72CCUPgseMsdr+YW6Sqjrk22kbaCmHlG3X3OZBs91D1Bjd2tkQqWVvNqv/Py kpjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105369; x=1734710169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HOQaUrSMrbMaSJ8tKHsfpEHyiDws1YYfXq+2bH3/e/Y=; b=C6KFMABgei48U7tE4F9cg5RRygAXM7OK9ItMca0ZZrd2dvxchRtNYkz+nMI1a6Q43N ifzaTewV8nXBAdQZAUesNxzetSCKlomPDSBWJedtqv/1xGBu41tOr/0APP7w+zxmJQtB 17xvyb8ENz2a46OTf4uQiDUoUuDS4lzl1easM9T/ff3mB9VV29rSiyVkhJxMyrs2Z+Of 7u/b8LjtJOw85TxX0Jrp1d0g5xEJ/ghKFJXAD8l6bmlAfZ35eRV23pnC0JyPpSGgEH/Z 4yB2uYYSwWiYCjSMF5bBJO5dhwh3IOZqYqWE3VTltI25PcD1te5erY7DeVZqRFx0P3om AygA== X-Forwarded-Encrypted: i=1; AJvYcCURePgvA0qghRn8E8OSyseetQide98aiLULVnqMkehlU5rc09k8ZHQaUsf7Sp0/2qC73T1xGHBjlZhB+fU=@vger.kernel.org X-Gm-Message-State: AOJu0YyB0Pqiwi8dYXnteNxWldGqFJAR6SenBQtdZQm4Tgzo2V5l/F1a pRGozDoxH9J/PLSQ3mOoQuistO6zXA8+xiVY8QTUhxcTsuf9K6Qc7wA786HapvMB7P3hTkeJn+Q J X-Gm-Gg: ASbGncsjHupvJ6xtU+m9T2ggX3sA77ORhbnmQ36vHN8qpRYz7/x/JU7RVF6E8dGcO94 3s+N2m+UzNpuJ8s28mU/kl7H55jJuRkMTlYhf77oXMidAMquRZ1z+Tis1zVDEM/5F0NXUJ2DELl gJ9JyBVObDN/LqFhS/jHjBxf1alYbcCvgEUC+BvQepCEAACfsguJl/zyOrTjhMoOadMshgOXM9r 9evze38RjyAwe9o89fjdr/lcj6HcX03Rnk5DiUUoeTMfliwo/AdEnUlFhrM X-Google-Smtp-Source: AGHT+IFBVlz5yZz/k+skWWprsCIycLK8jgnRkqf8RE4HqF0RJQv2gkIr0yeXfbClfRWoooTGgqlEcg== X-Received: by 2002:a05:6e02:156f:b0:3a7:e8df:3fcb with SMTP id e9e14a558f8ab-3b02d788a63mr20845625ab.7.1734105369001; Fri, 13 Dec 2024 07:56:09 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:08 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 04/11] mm: add PG_dropbehind folio flag Date: Fri, 13 Dec 2024 08:55:18 -0700 Message-ID: <20241213155557.105419-5-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a folio flag that file IO can use to indicate that the cached IO being done should be dropped from the page cache upon completion. Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- include/linux/page-flags.h | 5 +++++ include/trace/events/mmflags.h | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cf46ac720802..16607f02abd0 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -110,6 +110,7 @@ enum pageflags { PG_reclaim, /* To be reclaimed asap */ PG_swapbacked, /* Page is backed by RAM/swap */ PG_unevictable, /* Page is "unevictable" */ + PG_dropbehind, /* drop pages on IO completion */ #ifdef CONFIG_MMU PG_mlocked, /* Page is vma mlocked */ #endif @@ -562,6 +563,10 @@ PAGEFLAG(Reclaim, reclaim, PF_NO_TAIL) FOLIO_FLAG(readahead, FOLIO_HEAD_PAGE) FOLIO_TEST_CLEAR_FLAG(readahead, FOLIO_HEAD_PAGE) =20 +FOLIO_FLAG(dropbehind, FOLIO_HEAD_PAGE) + FOLIO_TEST_CLEAR_FLAG(dropbehind, FOLIO_HEAD_PAGE) + __FOLIO_SET_FLAG(dropbehind, FOLIO_HEAD_PAGE) + #ifdef CONFIG_HIGHMEM /* * Must use a macro here due to header dependency issues. page_zone() is n= ot diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index bb8a59c6caa2..3bc8656c8359 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -116,7 +116,8 @@ DEF_PAGEFLAG_NAME(head), \ DEF_PAGEFLAG_NAME(reclaim), \ DEF_PAGEFLAG_NAME(swapbacked), \ - DEF_PAGEFLAG_NAME(unevictable) \ + DEF_PAGEFLAG_NAME(unevictable), \ + DEF_PAGEFLAG_NAME(dropbehind) \ IF_HAVE_PG_MLOCK(mlocked) \ IF_HAVE_PG_HWPOISON(hwpoison) \ IF_HAVE_PG_IDLE(idle) \ --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-io1-f41.google.com (mail-io1-f41.google.com [209.85.166.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25B961E47D7 for ; Fri, 13 Dec 2024 15:56:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105372; cv=none; b=KKSTVmfWNfKDp0ZKAb80L9AagiPoxxKv/H2+walI7tKaERThbcIF0wN/FJaHQN7Mzq8deed8vgrm4W/mJYVxyiZFUMYGxx+Qhajg+slXHZujZbEg/xoFu3tsSLL7BzvSmsTpi/UOALTtVgw6PTcuBFEXoQZP/MYpXCb9pc2AFN4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105372; c=relaxed/simple; bh=SKl6hxJdy3uyJbhZpOOdAiYhB+0FB2yQzq5XqnnCoF8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KBEOr0Lo3w4tqdw8NjL9VL6O6K7TQQrHb4r5skCYyDZwGiZVj9g19r23iqIhhvud56wa8gVuUrmTKGKnboQILDcL2X2/OwaJFBNBVfpZB6C86aiZ1AR1TZxaW82ToWfy5o1IM+FNG+ER9h1tYeU3f5AoVyDicAb9Pq1RKulNVFo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=AAeHJeZd; arc=none smtp.client-ip=209.85.166.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="AAeHJeZd" Received: by mail-io1-f41.google.com with SMTP id ca18e2360f4ac-844e6d1283aso39025139f.1 for ; Fri, 13 Dec 2024 07:56:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105370; x=1734710170; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kz0Pdg9jqnEJMYLqH2tprv5bkemx7uetlhB6shaJiPY=; b=AAeHJeZdtfP/2H1iy+ldIoEDGS54o7//Dw0LyEhQdwB2FK9yFFGFcrabdhflnp0wSE r4/OU71VRA2b2GHywltcJnJlEL1ybczlTCu7MiRyPZL4gHKbl9uq8e/I+jYAtiAR0p8b /HSiDCDdIwWN2KJ1ZA7CVvimVvrQK8YBnpEeKzchSpnBKlVDSH4xXiwlMNSs+rj7CWQn /ZPx9n6tpXNmdHn+5S2uFX+cZJDFowx0rXuu50y3DH2XqOyofqO4v32vjRTUT1OuleQG PEMbCUTiTt15tdYbNozMlbAvfmBcw+52Y+IVA1bOp+woVNHYK3HQ5w9p/BDSOI1UnKxN QM/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105370; x=1734710170; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kz0Pdg9jqnEJMYLqH2tprv5bkemx7uetlhB6shaJiPY=; b=KEYuPxSy9raQ54kvkeRTQzg/dPzAsKFlx30uNjaAzl3JeNtokihiLkBXvus0ijyAKS uH2DerJkyC7su3XnIxOQe7WR8NzgLxKbLtigyXQJBXUW/rP/RK0uOp7Bm6zLjmU+jAYO tY7bFIwPzEKSh8tGw9GOyIhyBEz6I49TXLrzMiy9wvDELguLEVXJN+i5BpWYCc7s4HgR zZEiO8rVWap5UlY1zmYAEGer1crUblKy+8uUXg4CScCh6LyhdRnvdbtgPst0uWzNQGeg 2rud+YC7itSClNYUGjZLzy+IvhlxBdVWD2zFI4LCvYcvUrBU5ViKwSjZ710bB1ZU0S6C qCng== X-Forwarded-Encrypted: i=1; AJvYcCXoikyW+M5kyYyHBYmzb7ygIvXjzx9zlfdITiB0QMkpGj0w00ET11T4fTDEiMYUEuR9m3Yo8Bu6705dqL8=@vger.kernel.org X-Gm-Message-State: AOJu0YyN3F9300gnUyyLDtrlDreoLX0qIXgh4p4XT+wP9E3QJqD7R1B+ kwBjV/mIHdtnDxSPv1DqYNYH1hzNr7JAVdNhuLJFtSaHSjEyKxWNL/HEFxwhGrE= X-Gm-Gg: ASbGncsCFvuWUpyQXbLb9UbSJlDLbIBNhxz+AXuo+1O0XdLU2NJiCVdyAHehn3+Rkoi MuSV7nJDx919dXxM38+dw5uXT9nz0vRq9pfemsVtWsQ1hbnIIWu6nlLyXpeddvCArqrBSTEcg6/ eE1J6kOyhr+Ue3lc+blals9nEMw7ZVA/LE5YxCoxYuu5zJzD50vsLcQQqP4VcnbVurpn0xOA9af VTkREspdYHzYh+w44CWwmKASCD4y0FihmtjWxij5bVLZVezxF0SA2gIAgyr X-Google-Smtp-Source: AGHT+IHcUplw3Q6ECt5DnQxOZIMw6sSCxN/kZ+qm1VNkzK8N15/r675oe1NRLkEoEeFprb07mxKCyg== X-Received: by 2002:a92:d10c:0:b0:3a8:13d5:bd2c with SMTP id e9e14a558f8ab-3ae61f23bb3mr46583235ab.2.1734105370422; Fri, 13 Dec 2024 07:56:10 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:09 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 05/11] mm/readahead: add readahead_control->dropbehind member Date: Fri, 13 Dec 2024 08:55:19 -0700 Message-ID: <20241213155557.105419-6-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If ractl->dropbehind is set to true, then folios created are marked as dropbehind as well. Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- include/linux/pagemap.h | 1 + mm/readahead.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index bcf0865a38ae..5da4b6d42fae 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1353,6 +1353,7 @@ struct readahead_control { pgoff_t _index; unsigned int _nr_pages; unsigned int _batch_count; + bool dropbehind; bool _workingset; unsigned long _pflags; }; diff --git a/mm/readahead.c b/mm/readahead.c index 8a62ad4106ff..c0a6dc5d5686 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -191,7 +191,13 @@ static void read_pages(struct readahead_control *rac) static struct folio *ractl_alloc_folio(struct readahead_control *ractl, gfp_t gfp_mask, unsigned int order) { - return filemap_alloc_folio(gfp_mask, order); + struct folio *folio; + + folio =3D filemap_alloc_folio(gfp_mask, order); + if (folio && ractl->dropbehind) + __folio_set_dropbehind(folio); + + return folio; } =20 /** --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-il1-f174.google.com (mail-il1-f174.google.com [209.85.166.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDA001E7668 for ; Fri, 13 Dec 2024 15:56:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105374; cv=none; b=jW7jhoPJPg5ZzsJQthDP9EFWQ3VJ5H0L0geLILHQM7CxV/Tqy4c4/YfDox8oETx4H8UMYzyn3cjbx9V0ADAYtHk7O7OXDU5Qkxucw1gzHrSyxnS3C4AvSHXj8GdOdpiH/V9ghVWNORbwv/S/9AgbxqMXUbwDpWIbYITFP1d9k2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105374; c=relaxed/simple; bh=EZq/FArZwXgCBs4pZJDUXjfhlPafSJVT3vDdAzxjcuc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nk/OWhURQCZJncaQZiZy5n8XwQoNWGBOZNdzorLiq5YCmtHInOcKpW/JpwhBe+EAORiFWHxrN9lxW25nZVaWeufF89Y1ExnpDFtXvEIzhNJYzQenrn76Ib7lMbRFzaAR0ef/SYP78N6xl62WjU8IpY3qI0S8R1ZMbAF+BVuUvdw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=weU06r9Y; arc=none smtp.client-ip=209.85.166.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="weU06r9Y" Received: by mail-il1-f174.google.com with SMTP id e9e14a558f8ab-3a8160382d4so6010695ab.0 for ; Fri, 13 Dec 2024 07:56:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105372; x=1734710172; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S6x6BO6+6Zb8b9h/4S7enLTX5qkkGgQROhVyQCWWxzU=; b=weU06r9YxvdS1WDcTqD0yw/mJvNIBBFPFHoPtJ/ANqgU/1+YICPLCz/p5ZaoKtruH/ wcQYTGUmqBhkZM0QqifHRu75JqEB8tRvlMYkS5D5yBzooFTvJSPUF27t4MaPau+QkGg4 /qyTert9WI/G7MGDy+CYNgtFiGFtopXWxGhOXvuohTQPNjrdIF+Nhw5pCmxo/tnf8ggT i4TNbQRfY1laF49TLG486wRYWdhfAHccLPai/Nql99tzfax0WISR2/70Kx29+FRQwHbr jS8ALKfGQ8HEsTvfuSIryOM2yYQhCpdLJYDGYNTv0NlG0ONSwxfkg3wYewuVRQpwFVis C37g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105372; x=1734710172; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S6x6BO6+6Zb8b9h/4S7enLTX5qkkGgQROhVyQCWWxzU=; b=iql26/olM3U3xavRsg5LKbwtHvRxG1zOc6Y/2UZYTcBTo8Tq8/iHCkAUNH5wl1N2fL f8l9wq8XBFN+M7iTLYbLUeAv9m5w3e9uur/ODTEXky0k0p7EGNLwDQif7oZwz30jmVUm UZd5XigB/fN5vazMuIc4pZeJMXSaXrGAPUmNlISoDH+Rrnnz0pjfmMckbs0rhHd0ERQs o1MDR35LwI8apW+JxRC0FKGKLRUWPuixsL6uPpjSw2vfJ60PXZU+da0PiPJAzlH4tyek yl0/54r6SillZM0fg3FNIrQ8EtsQT/uXAYeWW20z5jTyAB7MFYhHs0gbyuWidEuSxRGG VRIQ== X-Forwarded-Encrypted: i=1; AJvYcCXDkZuhfLxKMBhoJleBxbFfD+DHqCBstM/OQXjOGJANJuvhe3qLaDmwoCK4yE0E8Ckmlf3L+4yRpnDuEmg=@vger.kernel.org X-Gm-Message-State: AOJu0YyINFpJNZLRgxSRc/jiqMn3YzC2KifAsUiDVkMZA4AM9dCr2kv+ p491YOK7dZGyYuOzz9zPzH8TzUH1C1nqU9PCIkXDlSbnJvmzlIx+/gEruQxCVGk= X-Gm-Gg: ASbGncuwRrYzEyJIAJe9vozARwpeMMVBQiuJiszmV9EOPo3wqTqUWsTsx6rEkwr8Mj7 ScAI+VSeH/g3sR4kM4b/ARhD1XUTYcoPOCjB4C4tFyUsiefcnAWDTQt9lA0nwvY8pvKBhQNLA7u hkLAAi36/+bAtvMxeo1+JPlpmkPBexFU6/mIDKuoZJmCi5UbXHLYakgqFPXk4rBqAh1h0c7HXmC y3UpD0c8w9tgwUClBbcimJuuOGLQ/BEj2z4KwRT8PXWalreoRoci/Rj82AS X-Google-Smtp-Source: AGHT+IEynku8b05ugapn+xW5MO4tP+/urn6SKqialSRpaetMXPe/mw9YxvkBEr2Hl0/nNihyBVcVWg== X-Received: by 2002:a05:6e02:164e:b0:3a7:7ded:521b with SMTP id e9e14a558f8ab-3aff800f100mr28298385ab.17.1734105371849; Fri, 13 Dec 2024 07:56:11 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:11 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 06/11] mm/truncate: add folio_unmap_invalidate() helper Date: Fri, 13 Dec 2024 08:55:20 -0700 Message-ID: <20241213155557.105419-7-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a folio_unmap_invalidate() helper, which unmaps and invalidates a given folio. The caller must already have locked the folio. Embed the old invalidate_complete_folio2() helper in there as well, as nobody else calls it. Use this new helper in invalidate_inode_pages2_range(), rather than duplicate the code there. In preparation for using this elsewhere as well, have it take a gfp_t mask rather than assume GFP_KERNEL is the right choice. This bubbles back to invalidate_complete_folio2() as well. Signed-off-by: Jens Axboe --- mm/internal.h | 2 ++ mm/truncate.c | 54 ++++++++++++++++++++++++++------------------------- 2 files changed, 30 insertions(+), 26 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index cb8d8e8e3ffa..ed3c3690eb03 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -392,6 +392,8 @@ void unmap_page_range(struct mmu_gather *tlb, struct vm_area_struct *vma, unsigned long addr, unsigned long end, struct zap_details *details); +int folio_unmap_invalidate(struct address_space *mapping, struct folio *fo= lio, + gfp_t gfp); =20 void page_cache_ra_order(struct readahead_control *, struct file_ra_state = *, unsigned int order); diff --git a/mm/truncate.c b/mm/truncate.c index 7c304d2f0052..7b6f4399319f 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -525,6 +525,15 @@ unsigned long invalidate_mapping_pages(struct address_= space *mapping, } EXPORT_SYMBOL(invalidate_mapping_pages); =20 +static int folio_launder(struct address_space *mapping, struct folio *foli= o) +{ + if (!folio_test_dirty(folio)) + return 0; + if (folio->mapping !=3D mapping || mapping->a_ops->launder_folio =3D=3D N= ULL) + return 0; + return mapping->a_ops->launder_folio(folio); +} + /* * This is like mapping_evict_folio(), except it ignores the folio's * refcount. We do this because invalidate_inode_pages2() needs stronger @@ -532,14 +541,26 @@ EXPORT_SYMBOL(invalidate_mapping_pages); * shrink_folio_list() has a temp ref on them, or because they're transien= tly * sitting in the folio_add_lru() caches. */ -static int invalidate_complete_folio2(struct address_space *mapping, - struct folio *folio) +int folio_unmap_invalidate(struct address_space *mapping, struct folio *fo= lio, + gfp_t gfp) { - if (folio->mapping !=3D mapping) - return 0; + int ret; + + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); =20 - if (!filemap_release_folio(folio, GFP_KERNEL)) + if (folio_test_dirty(folio)) return 0; + if (folio_mapped(folio)) + unmap_mapping_folio(folio); + BUG_ON(folio_mapped(folio)); + + ret =3D folio_launder(mapping, folio); + if (ret) + return ret; + if (folio->mapping !=3D mapping) + return -EBUSY; + if (!filemap_release_folio(folio, gfp)) + return -EBUSY; =20 spin_lock(&mapping->host->i_lock); xa_lock_irq(&mapping->i_pages); @@ -558,16 +579,7 @@ static int invalidate_complete_folio2(struct address_s= pace *mapping, failed: xa_unlock_irq(&mapping->i_pages); spin_unlock(&mapping->host->i_lock); - return 0; -} - -static int folio_launder(struct address_space *mapping, struct folio *foli= o) -{ - if (!folio_test_dirty(folio)) - return 0; - if (folio->mapping !=3D mapping || mapping->a_ops->launder_folio =3D=3D N= ULL) - return 0; - return mapping->a_ops->launder_folio(folio); + return -EBUSY; } =20 /** @@ -629,18 +641,8 @@ int invalidate_inode_pages2_range(struct address_space= *mapping, folio_unlock(folio); continue; } - VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio); folio_wait_writeback(folio); - - if (folio_mapped(folio)) - unmap_mapping_folio(folio); - BUG_ON(folio_mapped(folio)); - - ret2 =3D folio_launder(mapping, folio); - if (ret2 =3D=3D 0) { - if (!invalidate_complete_folio2(mapping, folio)) - ret2 =3D -EBUSY; - } + ret2 =3D folio_unmap_invalidate(mapping, folio, GFP_KERNEL); if (ret2 < 0) ret =3D ret2; folio_unlock(folio); --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE8C61EBA02 for ; Fri, 13 Dec 2024 15:56:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105375; cv=none; b=VWF7jierJ2ZEtP75huMiQjGSSSxrXkMtElmbFxxHM6eZYlbtY54eAmOPUtQm7ngLzhXF8VhfIVQPqLsRNN65Y/tIm1OJfVq7S2ZeR/jwBArtbjsP4SjcnqrlcJb1nfDGk1V7dYiSfWjniuFPeE9NH6rvdQ/6rAMKt6XflBXK+ug= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105375; c=relaxed/simple; bh=2SEcDojzAqUnPgfjf8E9nRmr9dou/inG3/DhaNUqq8I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QKzkPgh8RuAdr6slTzsLI7V0lx1sB0HbIa9ymBcyU/2oMJSXtIR9ekEjAa+4GF5ZW13XkQ0TkoPLG+xE8F1SBEuXlsS9hJqP2EK7TDeSrkzUZRz4qNDl6mKtMEtuSOMmQTe3mVx09kjPdHQipCrLnjcopDBqim1/EWwR6KKHnYQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=01Hs9enO; arc=none smtp.client-ip=209.85.166.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="01Hs9enO" Received: by mail-il1-f170.google.com with SMTP id e9e14a558f8ab-3a9cdcec53fso15576755ab.1 for ; Fri, 13 Dec 2024 07:56:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105373; x=1734710173; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4w74VD3kuQguRINknUSjUP4VNHVLoGClDPYL5LdL0oM=; b=01Hs9enOnkwKRmuqv8OPi5cm5KXxcKFhJlkctvjFgT6DFRRQIS/rAYbdV/lvyTW3Lw sbiSSuGpUO1nmFNR1hLh0BtKs7xfKfvEiwQpiVm7nVqSGJ/hT/5+Plb7PGC/Bt2vI4WY QAJWdCaYyEo26sZrPOlYErQq+dlX5WObrlqnCgBy06zFkj/9Bc+Df/BCuPbFkMe2/Yj/ tF5zSPUbf59oWTKv+YkkQaZ4RSoBu4MFQv2/IPc8VTUa46726BfPG2gwdtYP+YDDWYi4 DPcJQlGxRyu15Gt2gSwceQ80X6hY6iH355vg8dJhHAmEwxsA7CAsVybF5RLXrMEzEiAj DV7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105373; x=1734710173; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4w74VD3kuQguRINknUSjUP4VNHVLoGClDPYL5LdL0oM=; b=i7ILgZ44sWBtkuCA1WzVyGermIfvJEEDvqe+G2+HSWfsK0PdWwafdy6B0k63ycptZ4 I8p12uXnymnYC6VC2zEgmKD/zz6iEe4zFMCl2xceyIWheQM5f0Qw0aNjmq36bE7e222G desLSB0h7aEEu11hgQr0AEyyOh02NXxp2wk4uKVFqEYH+AnaJtfmC2iFo+coL4wczO0r hj12zhVbdt2x+jtq9JtqvAnEIyi9ZIGConp6yFZYw5WGCVuxULPidMkG88xgFBbIBZZD gpwuE16h0ZhiIkLMNcC5QLMpz+8zhnSGRFh/c5InO4b/nTFHdjJ1AiWmMMyNKnYrHjpR FN8A== X-Forwarded-Encrypted: i=1; AJvYcCUR+3gbGZgYwFxjHKGIEnaillNPzyfLU1rG2s2tldhpPXI8pJLPpYGAkdBte44thQL9lVvHUjg0zF/NbDc=@vger.kernel.org X-Gm-Message-State: AOJu0YyFrto4z8JrfPB/Jsd/tw43+nm42Nr/u4FIRuK9JyQzzGX0oIco PkPB5DJAx+0qlSmctjgWZra2Xi6+Slg/EFX3I90Q3sjmZtQpXDG+qS0HqHJoWGs= X-Gm-Gg: ASbGncuuIU2Hx8RSysy8oXD7AmBxabpqg6J7Szu6XqsF1p4BTcoYPkag+4qdrEnQbeG fnjBGSORT+0Fc4sixyXKbSvNnvMujPsX9whuhXYgcSUiH4TLvP/JZbQALQfBLSuAK+Rtt7xekoo 9Kpb5pQQKwcZ7k4gxEyZN54YrCaLzIDS1d74AQxVlDvcFCk+URACO9uLmlmlgPkYs82bdODdb3W 0wj+wo3IirAKQGB5/JVNalpB+kENLNxZIjJXMegztNYTHVNVTdr8si9otAo X-Google-Smtp-Source: AGHT+IHLruDe6Zg9uYRMCFFj0re5LQoOZLn3MM3DvSbmN1NLcJuI0/2En87svfvn51ITwB5/NqHQ+A== X-Received: by 2002:a05:6e02:3042:b0:3a7:8720:9e9f with SMTP id e9e14a558f8ab-3aff4619d10mr35119105ab.2.1734105373195; Fri, 13 Dec 2024 07:56:13 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:12 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 07/11] fs: add RWF_DONTCACHE iocb and FOP_DONTCACHE file_operations flag Date: Fri, 13 Dec 2024 08:55:21 -0700 Message-ID: <20241213155557.105419-8-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If a file system supports uncached buffered IO, it may set FOP_DONTCACHE and enable support for RWF_DONTCACHE. If RWF_DONTCACHE is attempted without the file system supporting it, it'll get errored with -EOPNOTSUPP. Signed-off-by: Jens Axboe --- include/linux/fs.h | 14 +++++++++++++- include/uapi/linux/fs.h | 6 +++++- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 7e29433c5ecc..6a838b5479a6 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -322,6 +322,7 @@ struct readahead_control; #define IOCB_NOWAIT (__force int) RWF_NOWAIT #define IOCB_APPEND (__force int) RWF_APPEND #define IOCB_ATOMIC (__force int) RWF_ATOMIC +#define IOCB_DONTCACHE (__force int) RWF_DONTCACHE =20 /* non-RWF related bits - start at 16 */ #define IOCB_EVENTFD (1 << 16) @@ -356,7 +357,8 @@ struct readahead_control; { IOCB_SYNC, "SYNC" }, \ { IOCB_NOWAIT, "NOWAIT" }, \ { IOCB_APPEND, "APPEND" }, \ - { IOCB_ATOMIC, "ATOMIC"}, \ + { IOCB_ATOMIC, "ATOMIC" }, \ + { IOCB_DONTCACHE, "DONTCACHE" }, \ { IOCB_EVENTFD, "EVENTFD"}, \ { IOCB_DIRECT, "DIRECT" }, \ { IOCB_WRITE, "WRITE" }, \ @@ -2127,6 +2129,8 @@ struct file_operations { #define FOP_UNSIGNED_OFFSET ((__force fop_flags_t)(1 << 5)) /* Supports asynchronous lock callbacks */ #define FOP_ASYNC_LOCK ((__force fop_flags_t)(1 << 6)) +/* File system supports uncached read/write buffered IO */ +#define FOP_DONTCACHE ((__force fop_flags_t)(1 << 7)) =20 /* Wrap a directory iterator that needs exclusive inode access */ int wrap_directory_iterator(struct file *, struct dir_context *, @@ -3614,6 +3618,14 @@ static inline int kiocb_set_rw_flags(struct kiocb *k= i, rwf_t flags, if (!(ki->ki_filp->f_mode & FMODE_CAN_ATOMIC_WRITE)) return -EOPNOTSUPP; } + if (flags & RWF_DONTCACHE) { + /* file system must support it */ + if (!(ki->ki_filp->f_op->fop_flags & FOP_DONTCACHE)) + return -EOPNOTSUPP; + /* DAX mappings not supported */ + if (IS_DAX(ki->ki_filp->f_mapping->host)) + return -EOPNOTSUPP; + } kiocb_flags |=3D (__force int) (flags & RWF_SUPPORTED); if (flags & RWF_SYNC) kiocb_flags |=3D IOCB_DSYNC; diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..56a4f93a08f4 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -332,9 +332,13 @@ typedef int __bitwise __kernel_rwf_t; /* Atomic Write */ #define RWF_ATOMIC ((__force __kernel_rwf_t)0x00000040) =20 +/* buffered IO that drops the cache after reading or writing data */ +#define RWF_DONTCACHE ((__force __kernel_rwf_t)0x00000080) + /* mask of flags supported by the kernel */ #define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\ - RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC) + RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC |\ + RWF_DONTCACHE) =20 #define PROCFS_IOCTL_MAGIC 'f' =20 --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-il1-f171.google.com (mail-il1-f171.google.com [209.85.166.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DE621EC4D0 for ; Fri, 13 Dec 2024 15:56:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105377; cv=none; b=SoNLc3B0/C5gwU/lX1sMbwhiOQiwlktf+3R9DWyKSUK5PhW2gFjff8mevPR6pUnC83vTPwkukSJARX5a7lty23Ze4ugcDVHt1xN+Pf7SUC2BTDUfJ/jcOY5aUfZ6bxa8gp/RXL/k7EjymOBvPBzXsqiB1n0ON8X+lMvj1nGf6OM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105377; c=relaxed/simple; bh=vDmNnPIlxbQFCnastmzOm8F+RELQsnyCtSC4KgXE2I0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bHYlyLLo4eL/DxMztcmLH/VEFrV9mjFMKy+gCxttgBE/8rGjmk/2Q1ccUHGfFs6lTQAMz/HXO4fJAPUUUvUsx7r3WONCNYTiXvSlf73wMM0bbN/fgfmn0oYvhtnrivOpc7+9MGSiLpcurJvAs+NQyj7UidgBclEJlxZOWEs4AIM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=w4XdXwHr; arc=none smtp.client-ip=209.85.166.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="w4XdXwHr" Received: by mail-il1-f171.google.com with SMTP id e9e14a558f8ab-3a7dd54af4bso7236675ab.2 for ; Fri, 13 Dec 2024 07:56:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105374; x=1734710174; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IP/eZVPa/Uul8epuK9rEl1xWlFc7aGT1EOVSfKN8LMY=; b=w4XdXwHrYVwFTvJ/MqoLN+c2dm61xKQFuUZKIwd9uV037G2HBS2LFfxCmr03/Ijc/j Nk7Pg8xdZyUXbbdukjEFXskzc70Vt6AyrAa2OICUAUeoNLIc38lBfL7dcl0M6bdLRTB5 vgHreqf2XZ/BjsBZ7VEi6YH3OohKGNo4rUEp4ommh6ZZwRLTkwhspRKxquwO4vqbZot1 ZtAeAxZTV2ChkbXCzL/LZ/iFGR/BtBFpPrBstn0Z618+xcvsV4I0d2L3mOFswtZO6Vib omrWpihLU5nNR74wLtz6b7r9AqP8zNMqgvgP7UERlJtQv1582/d5JTge3yGOmWUHfFoT iX4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105374; x=1734710174; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IP/eZVPa/Uul8epuK9rEl1xWlFc7aGT1EOVSfKN8LMY=; b=fBvacxOrmVVD4PhjerIts6/IY59VkHpBt3/cBFBGR7WIvg+ozqjvEEJrbIG+14E+Rk J+ixXKxs7N+hvND/EKcdmFORbscqcZR36mcaB/oAsFR1cEUxEyEkEm9lJogLKk4l6yQc O03tl5yKgLfBzZ3VCuM2jmQrKApj8VJ9xuK12+0fsBXxfgUHuVsMDJ+dQm+Mta7sWa// zFeD8/jufBSEFCRdper2/WndpHLBsHaDz0qBnOvtdiZZUaTsMFMex8e6Mcdxjgow5g0G PpZbyu/0BpgsSFuIJzfSU0K7mvjk7Y/c8dmoaPm7GD/yWRTwY3qrEe1Vvz/L4iWrHENS nsfA== X-Forwarded-Encrypted: i=1; AJvYcCXLSpsee2Yv62ewORLmdr49WnZ7pHBQ8MEZiEyo4FQ2MTSWMZPxFVSvGM6N7xRtYiulAYXxejUT0CB0+yM=@vger.kernel.org X-Gm-Message-State: AOJu0YxiWz2cb8Q71PtDaW91QGMGtWMIedP0u8AYzUWs+PWmX4aOhwF+ xr/BDK2aPdaAkF1CSVF2QVU1qARyLc3BN4Dm+uKtx522jFTxnQLK8Xo+pvY9qRM= X-Gm-Gg: ASbGncsTGniEcdm6jPFjJgjT/R4yPwQxXSsTP/mjCzSnwrkyN0PpHlnHfRYVCiTB0G0 Yk1K6ORkcZmoxPAsNYqyyaRnFMhEX/ctNgGK7v02e2CyFnVXOeHGNKNZTdM0tVwb1T05smt/su+ 09wxkYC0qIF1q+dzsPIea2mEe8S8WzJXb0Uv3EqQ2ZnqZshe8PScCwnxvycrnTYO7RmmgioAonD Df/TVKS44MbetDfv+HxPqSbHFc1lFDKLnUpKQ/+dwrgvyL/osFFEcu2PnV5 X-Google-Smtp-Source: AGHT+IH9sg/APOeVcvmxNi3lc/CaFUy261G7dVBoU1ZPhJ36GVox2hlXSW2u68WXUBo94t1FWfRBpA== X-Received: by 2002:a05:6e02:168d:b0:3a7:c2ea:1095 with SMTP id e9e14a558f8ab-3aff4615779mr38827345ab.1.1734105374532; Fri, 13 Dec 2024 07:56:14 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:13 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 08/11] mm/filemap: add read support for RWF_DONTCACHE Date: Fri, 13 Dec 2024 08:55:22 -0700 Message-ID: <20241213155557.105419-9-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add RWF_DONTCACHE as a read operation flag, which means that any data read wil be removed from the page cache upon completion. Uses the page cache to synchronize, and simply prunes folios that were instantiated when the operation completes. While it would be possible to use private pages for this, using the page cache as synchronization is handy for a variety of reasons: 1) No special truncate magic is needed 2) Async buffered reads need some place to serialize, using the page cache is a lot easier than writing extra code for this 3) The pruning cost is pretty reasonable and the code to support this is much simpler as a result. You can think of uncached buffered IO as being the much more attractive cousin of O_DIRECT - it has none of the restrictions of O_DIRECT. Yes, it will copy the data, but unlike regular buffered IO, it doesn't run into the unpredictability of the page cache in terms of reclaim. As an example, on a test box with 32 drives, reading them with buffered IO looks as follows: Reading bs 65536, uncached 0 1s: 145945MB/sec 2s: 158067MB/sec 3s: 157007MB/sec 4s: 148622MB/sec 5s: 118824MB/sec 6s: 70494MB/sec 7s: 41754MB/sec 8s: 90811MB/sec 9s: 92204MB/sec 10s: 95178MB/sec 11s: 95488MB/sec 12s: 95552MB/sec 13s: 96275MB/sec where it's quite easy to see where the page cache filled up, and performance went from good to erratic, and finally settles at a much lower rate. Looking at top while this is ongoing, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7535 root 20 0 267004 0 0 S 3199 0.0 8:40.65 uncached 3326 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd4 3327 root 20 0 0 0 0 R 100.0 0.0 0:17.22 kswapd5 3328 root 20 0 0 0 0 R 100.0 0.0 0:13.29 kswapd6 3332 root 20 0 0 0 0 R 100.0 0.0 0:11.11 kswapd10 3339 root 20 0 0 0 0 R 100.0 0.0 0:16.25 kswapd17 3348 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd26 3343 root 20 0 0 0 0 R 100.0 0.0 0:16.30 kswapd21 3344 root 20 0 0 0 0 R 100.0 0.0 0:11.92 kswapd22 3349 root 20 0 0 0 0 R 100.0 0.0 0:16.28 kswapd27 3352 root 20 0 0 0 0 R 99.7 0.0 0:11.89 kswapd30 3353 root 20 0 0 0 0 R 96.7 0.0 0:16.04 kswapd31 3329 root 20 0 0 0 0 R 96.4 0.0 0:11.41 kswapd7 3345 root 20 0 0 0 0 R 96.4 0.0 0:13.40 kswapd23 3330 root 20 0 0 0 0 S 91.1 0.0 0:08.28 kswapd8 3350 root 20 0 0 0 0 S 86.8 0.0 0:11.13 kswapd28 3325 root 20 0 0 0 0 S 76.3 0.0 0:07.43 kswapd3 3341 root 20 0 0 0 0 S 74.7 0.0 0:08.85 kswapd19 3334 root 20 0 0 0 0 S 71.7 0.0 0:10.04 kswapd12 3351 root 20 0 0 0 0 R 60.5 0.0 0:09.59 kswapd29 3323 root 20 0 0 0 0 R 57.6 0.0 0:11.50 kswapd1 [...] which is just showing a partial list of the 32 kswapd threads that are running mostly full tilt, burning ~28 full CPU cores. If the same test case is run with RWF_DONTCACHE set for the buffered read, the output looks as follows: Reading bs 65536, uncached 0 1s: 153144MB/sec 2s: 156760MB/sec 3s: 158110MB/sec 4s: 158009MB/sec 5s: 158043MB/sec 6s: 157638MB/sec 7s: 157999MB/sec 8s: 158024MB/sec 9s: 157764MB/sec 10s: 157477MB/sec 11s: 157417MB/sec 12s: 157455MB/sec 13s: 157233MB/sec 14s: 156692MB/sec which is just chugging along at ~155GB/sec of read performance. Looking at top, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7961 root 20 0 267004 0 0 S 3180 0.0 5:37.95 uncached 8024 axboe 20 0 14292 4096 0 R 1.0 0.0 0:00.13 top where just the test app is using CPU, no reclaim is taking place outside of the main thread. Not only is performance 65% better, it's also using half the CPU to do it. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++-- mm/swap.c | 2 ++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 220dc7c6e12f..77290ac205dc 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2473,6 +2473,8 @@ static int filemap_create_folio(struct kiocb *iocb, s= truct folio_batch *fbatch) folio =3D filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; + if (iocb->ki_flags & IOCB_DONTCACHE) + __folio_set_dropbehind(folio); =20 /* * Protect against truncate / hole punch. Grabbing invalidate_lock @@ -2518,6 +2520,8 @@ static int filemap_readahead(struct kiocb *iocb, stru= ct file *file, =20 if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; + if (iocb->ki_flags & IOCB_DONTCACHE) + ractl.dropbehind =3D 1; page_cache_async_ra(&ractl, folio, last_index - folio->index); return 0; } @@ -2547,6 +2551,8 @@ static int filemap_get_pages(struct kiocb *iocb, size= _t count, return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags =3D memalloc_noio_save(); + if (iocb->ki_flags & IOCB_DONTCACHE) + ractl.dropbehind =3D 1; page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); @@ -2594,6 +2600,20 @@ static inline bool pos_same_folio(loff_t pos1, loff_= t pos2, struct folio *folio) return (pos1 >> shift =3D=3D pos2 >> shift); } =20 +static void filemap_uncached_read(struct address_space *mapping, + struct folio *folio) +{ + if (!folio_test_dropbehind(folio)) + return; + if (folio_test_writeback(folio) || folio_test_dirty(folio)) + return; + if (folio_trylock(folio)) { + if (folio_test_clear_dropbehind(folio)) + folio_unmap_invalidate(mapping, folio, 0); + folio_unlock(folio); + } +} + /** * filemap_read - Read data from the page cache. * @iocb: The iocb to read. @@ -2707,8 +2727,12 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_= iter *iter, } } put_folios: - for (i =3D 0; i < folio_batch_count(&fbatch); i++) - folio_put(fbatch.folios[i]); + for (i =3D 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio =3D fbatch.folios[i]; + + filemap_uncached_read(mapping, folio); + folio_put(folio); + } folio_batch_init(&fbatch); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); =20 diff --git a/mm/swap.c b/mm/swap.c index 10decd9dffa1..ba02bd5ba145 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -427,6 +427,8 @@ static void folio_inc_refs(struct folio *folio) */ void folio_mark_accessed(struct folio *folio) { + if (folio_test_dropbehind(folio)) + return; if (lru_gen_enabled()) { folio_inc_refs(folio); return; --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 919761EE00E for ; Fri, 13 Dec 2024 15:56:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105378; cv=none; b=g+zmbhz/gOqi7EHhl4EulK6VpSyAYlQX9yEiZ2DMepgQLUF4bNNLKfdvBLR7zD/tFLj2yWu5rgA98N/4WM4UdGlghl+5g7E0Zx5pq3hqh4xJDH2QlgvY8RM/Uj1JmSYJRcDkpnPJWCR8RGtkl0Ru6E2rbWMYJbQ3wjnNVWaeiTo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105378; c=relaxed/simple; bh=ZeMsDsphln50o1XFtLmHIP6lHmPNtmzQzlZFWxh1ZaQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cbS5wY7hBKZQ5/RCFnkJ+xjAs2BEmGOpGdu7zMehtCDoQRzP8L6/tIRowc/KwLgMcuUwfQ7ip58Bs0ezpTtfMj4iXZLBSCVWRREI3CBX/HN1aMtVOlqWg3sL2s+NF1FSGBu7u5mnbBG5tgYKXJYA8C7Lbdf8bBe1OraOOVg1i4s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=VY3watQ2; arc=none smtp.client-ip=209.85.166.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="VY3watQ2" Received: by mail-il1-f170.google.com with SMTP id e9e14a558f8ab-3a761a622dcso14691945ab.0 for ; Fri, 13 Dec 2024 07:56:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105376; x=1734710176; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eL7TH1+psm5CeVquxo+D9Ecvv3+RlpadAVoxegN1mFk=; b=VY3watQ2NR6jGnPZ6XQYSClGFKgxvsCoMHMZGD/dDEHNkXkum2+pHmho6aFmqJhf6h ehfSPV74UjBpmV85H8x0IbLd8VJCG2oLkYJctyquH7XYpePbvIZT5Vcyf6Rkm8Uw4LAH i25IJI0J1mFgRzMLo8fVstBkRzjnfxZCzu2PYl0pyTsNIBYlhMYqRwk1djlTCgWzZ6rk sFyMW/rBmnfw0yeawjS3wohXuRO/BKMJrRQqNaO+I0/ksVNiK9MKc/E4ofzGy5oAiuwO tHaCZKxU0TbTHUwDSt8rceIU2ttCHKamjds4J9/M3DRYMIlOljSUYiaUADvvqBbJMUZy sySA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105376; x=1734710176; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eL7TH1+psm5CeVquxo+D9Ecvv3+RlpadAVoxegN1mFk=; b=W8dwHPB83F/LHCppQvB8K3vrnFOQ/XKoZGxGl3f1EIJFCU1PCtO5gz3XaLk0Gh2hgo Xnu0vHmrB7o3qBCIYaLXJUSZBB+NDoYNq8aspYZUrAr7Ik3v+alPxRrIh2VJh5NUKy9l cD5WpF37PfyqAyj8VC/y2dk/QzJA6CNBDxYkclj7PMnFIM/OlZqt7qcKKOfyOXM0iwY4 vlAu3wNjXzygKPFk8uHpllBFxZhnZOiDHLmlqqV3HF5N9ymsiXHXYHXPliN58Awl+li3 1ky7nyaOWQXjI+aq+zM1shKcgw88071lTrbinVh8SE+i3Wpu1Fc+bYJ2rVzdE1+KbZTN wVEw== X-Forwarded-Encrypted: i=1; AJvYcCXltThygLCNzqXgmQTQE7tRpownCJkZTPFieu/P+cS9nK/YXDH4Osm+PwwkdaSd1irUlw4fMBUdSaX5CE0=@vger.kernel.org X-Gm-Message-State: AOJu0YzHG+jHI9xHgsn0C8E+9xCND5mJMMbHVrjZZdLwXChchkZqB6o0 2ufvghTOsbPzNQela7n+MPdbOtIc6UN2nTywPBCiSgvsoTY3T5awzj8Yh7J6KmE= X-Gm-Gg: ASbGncvoZh9Da92mCxK/UB47R+rOEscIx/+C0zSSIgRBXvnHHlMTrwuEC6JzQgzzK3x DWvI4KkrZ0jWaxaUsKb3kJZshOgJA6ZP5tj21ZoL60mIyTLp1AB46PncbtH9/q4cIPNCTzJsIZr +CEHuzGGs12zhx4inb0IozXKkK5J9T1d3ZqOVAKZetvVciR4VViN8tEEfKWV3Gf66QSRTHvnEzs qEo8FzhycXXwxRw92BLxbJN1et+CJQ6b2445kGf+JqKJ9/HevGC44CjDBwG X-Google-Smtp-Source: AGHT+IG5PcGGwL+OAF8BWXN0MfzrquLg0U95m3qvCvU2aqYiecyxUUHKeEBe+q/HNr+0PzMX/oeVbw== X-Received: by 2002:a05:6e02:12e2:b0:3a6:c98d:86bc with SMTP id e9e14a558f8ab-3aff4619e11mr40735545ab.1.1734105375832; Fri, 13 Dec 2024 07:56:15 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:15 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 09/11] mm/filemap: drop streaming/uncached pages when writeback completes Date: Fri, 13 Dec 2024 08:55:23 -0700 Message-ID: <20241213155557.105419-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If the folio is marked as streaming, drop pages when writeback completes. Intended to be used with RWF_DONTCACHE, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 77290ac205dc..ec087bab1c97 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1599,6 +1599,27 @@ int folio_wait_private_2_killable(struct folio *foli= o) } EXPORT_SYMBOL(folio_wait_private_2_killable); =20 +/* + * If folio was marked as dropbehind, then pages should be dropped when wr= iteback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset dropbehind for that case and latter completions should inval= idate. + */ +static void folio_end_dropbehind(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_DONTCACHE writeback, + * but can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1609,6 +1630,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_dropbehind =3D false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); =20 /* @@ -1630,9 +1653,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (!folio_test_dirty(folio)) + folio_dropbehind =3D folio_test_clear_dropbehind(folio); if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_dropbehind) + folio_end_dropbehind(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback); --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF0371EF088 for ; Fri, 13 Dec 2024 15:56:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105379; cv=none; b=Oming3qvMgxel8pbe1UQ7eeiqtKPJ/NyKxujXJhmd/vi1hHmiBRPzSdWgMYzZChHWkuCiFhtFMEwgitRfunOuCFDBgawxmcJQY2usvHeRFuyOqqbDwU10zgEmKrSsmk3J8aaI9snTR/xnjooLfmzqjFXycNp5ZPihVM5V8aVVrQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105379; c=relaxed/simple; bh=5nnCX/ugaux253bLZRpzYYnFsErlHtBtfq4h5ryDOj8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H0hp9ak9WnBlWM5O/JSmem7UvXw43xEiKeS4+xeaycXckjVwGb5DIAmd7B9cvIbrGQKlowxAO40ROMFT6pKiT61EUdVLBuQft5HKAOn4wPIJdj3pYXpOXA61Xxgr7+3ObAHuWor12HJAS5w38YP0dXV/oMvQr6k0UZJ3DiX5UpY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=xBsqQG4W; arc=none smtp.client-ip=209.85.166.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="xBsqQG4W" Received: by mail-il1-f182.google.com with SMTP id e9e14a558f8ab-3a8160382d4so6011025ab.0 for ; Fri, 13 Dec 2024 07:56:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105377; x=1734710177; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CJvpEUV5aCutO0n1vzbjd3IdvpDLlQuRWwd0hw2DTJ8=; b=xBsqQG4WWzJ40QYjSZYrM1RDXg+bzMi0GiUO4zqLy4cZRfW+o08/20ZGDrQY2xx9tf 2fAeWjLv/apNFx53ZsgOSVYfiGEZLt5NFsXDnQVtMBkp7FdI15rVO/VBgloMjPVb9Y8e F9aulmRj6oBbflzIRYO0Xo+jwlNYxQx0IoMqkyCRXZs+d7g4EkS7Ag/x7ZCSHTSHp6aL qHHHHQgJX0+3U21vc7pIVSiochjtnyM4woNvbvqAQasC52GPpCW4bPnXN9/GiwULZNpJ p7uaYm3Is+Kx952PvSZDTwAHZR/+q9OASOfMBkVMItN4iJJBOkhYumDdsl+m72lJ0XDD Rakw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105377; x=1734710177; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CJvpEUV5aCutO0n1vzbjd3IdvpDLlQuRWwd0hw2DTJ8=; b=IH/cDNFIrBYuEUg+DMDn6x49OXHN77rTyO0IALD50T/H5wAvA4XRHDqzvaGugNsx4W H2VS9udd0DcnGAnOu3Zc3aFqifz2VoFY6+UJGErWzbpbX3FSvV9vMYIgnq33N5/zUdE2 1c+s9KL3W5/cAkTMfUMn6VEsDMzzK/U74rlIgsWC81jaWk9MShpy37YRhAx3iBknbS71 rbDE9H2glezdy15u06VHx98PC6cv1y7WLfoTfTgt4DpvdvY9MTL4qM83cyhgFmKzigeU LaeFuueDfDstOHx1r/2SDAxNKDIWzIJ1FV8rQUkQHNIa5IrHUxl1mTUAnN0AHhsDNAgv xEyg== X-Forwarded-Encrypted: i=1; AJvYcCUtc8ni4Myzyjrk8yHuw/BBQwfe9Rv0ItnYscn8pj9XNH+m3g8l7vdRdw/Gp4sVccuo1/Q3KS8mVHlAyL0=@vger.kernel.org X-Gm-Message-State: AOJu0Yyj2HItANUuDxK3YkYVDeds9+8Pq1mq4Hl1UqzqYndX4GxlGkMD S3D8Rhlj5VflsNHBVhkxfK2TARQGTYy8tL9MqSxSaCPAC/y2cHcbs9zKdTExYtU= X-Gm-Gg: ASbGncsyUGHh5LUMOYa+b+LWBLNbg3exxpvFpst/qhgpdMuwSL9gZFf2hFC+lrCmXMa FkdyNXq5IY8uRS8/HeUbpIXkVQHUxjnrzK+1oKfHLpPCtD36buDmRgYpJVeDRUFMJvtLq2Wd3kK gQsWPMQ7AiqNHaa/EdK0J8T4ce7xJ1Js2QIhXxiRE/kaJpy5WaOaKrpMGmbf1TFXZPDOOAmPpmr E5gbHcIV0WFdMs5tOYvoo3W+EtYq3WAOBP/YXr5sHwHKG0Do4eRxkZCaEL9 X-Google-Smtp-Source: AGHT+IGauJ9uKAF5WxLAKMTF0QyH5oGnff/+E24GczIOzswP0NPnXSGxgq/rQ7IZv36KxS5lv0tqHw== X-Received: by 2002:a05:6e02:1564:b0:3a2:6cd7:3250 with SMTP id e9e14a558f8ab-3aff50b64c1mr29557815ab.10.1734105377176; Fri, 13 Dec 2024 07:56:17 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:16 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 10/11] mm/filemap: add filemap_fdatawrite_range_kick() helper Date: Fri, 13 Dec 2024 08:55:24 -0700 Message-ID: <20241213155557.105419-11-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Works like filemap_fdatawrite_range(), except it's a non-integrity data writeback and hence only starts writeback on the specified range. Will help facilitate generically starting uncached writeback from generic_write_sync(), as header dependencies preclude doing this inline from fs.h. Signed-off-by: Jens Axboe --- include/linux/fs.h | 2 ++ mm/filemap.c | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 6a838b5479a6..653b5efa3d3f 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2878,6 +2878,8 @@ extern int __must_check file_fdatawait_range(struct f= ile *file, loff_t lstart, extern int __must_check file_check_and_advance_wb_err(struct file *file); extern int __must_check file_write_and_wait_range(struct file *file, loff_t start, loff_t end); +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t st= art, + loff_t end); =20 static inline int file_write_and_wait(struct file *file) { diff --git a/mm/filemap.c b/mm/filemap.c index ec087bab1c97..7db995cb5179 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -449,6 +449,24 @@ int filemap_fdatawrite_range(struct address_space *map= ping, loff_t start, } EXPORT_SYMBOL(filemap_fdatawrite_range); =20 +/** + * filemap_fdatawrite_range_kick - start writeback on a range + * @mapping: target address_space + * @start: index to start writeback on + * @end: last (non-inclusive) index for writeback + * + * This is a non-integrity writeback helper, to start writing back folios + * for the indicated range. + * + * Return: %0 on success, negative error code otherwise. + */ +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t st= art, + loff_t end) +{ + return __filemap_fdatawrite_range(mapping, start, end, WB_SYNC_NONE); +} +EXPORT_SYMBOL_GPL(filemap_fdatawrite_range_kick); + /** * filemap_flush - mostly a non-blocking flush * @mapping: target address_space --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BED91F03C6 for ; Fri, 13 Dec 2024 15:56:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105380; cv=none; b=hbasF00HGdpYfJgd4Mbtogl1bdkiImpvGfWFjA6fPTW0FGJAzuDQcls5bdSZSYXm1Y6YYgZpNPEky6KMnqHnr6JVtmfzgelElSAw78+hanwC5Hyft/tqofXRfspX1dPLIVa2cenZJq8f7NpMVj2CiZ3qswHGUvf5HjEmRwbRhj0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105380; c=relaxed/simple; bh=zE0xjm5rhzd2cUhrSxtogiQia/migrwaLvBNVMhxtxQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T0kAsGm8N5wE4MeLDpZrujtQdcr+hf3sUgJU9jJhvVbpZ6H3P4vd9LcsqX+bb4CNoCAaNJuMf7SLsNEuewxzLlS6avwc7cbt2Pjd6XM0JRzrb5lxnMNFPpcYE3GXAHWqV4PQIWvGitidHr0H4gnTPVZGAZpk2d7USvXTVcnu/9A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=HA1gc6ir; arc=none smtp.client-ip=209.85.166.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="HA1gc6ir" Received: by mail-io1-f50.google.com with SMTP id ca18e2360f4ac-844eac51429so54904039f.2 for ; Fri, 13 Dec 2024 07:56:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105378; x=1734710178; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ol/7y93UZDywrWGNYi5AP2kj1AIRWUpJ2p+2aggyqzo=; b=HA1gc6irLN/yCOG18U3Q+66W/+z4/bTyZiSmJMqj1/w+cvYgQM7rHONzuluiRSQkBb cweE2VTc/HK/3DIwpjt1c2Nf+Gxv7SVYHiJHDaKlL6Ue4358GqyJqk4lFnNX4KOo53ku VmFbHFJypEDtZ/AoDel7A6O8JplP2uox9ONHtdpGANY26YTB9qjBVeCUpnTIPrYq0ADq LGEw+WA50yc2FslUOcedqLOjA5g2WjxE2ecuUhJGb5e4XK+bOgfINf86n4UFnP7qlARk Nc+mPBS0Zq36JNexBhw4nymkuB+o+0BvszRwuF2bqFwfAyaUj0YG3/g+Q75RwMt318lO 3Sog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105378; x=1734710178; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ol/7y93UZDywrWGNYi5AP2kj1AIRWUpJ2p+2aggyqzo=; b=Bj3WR7LxvOfbexZENNa/vG+iwqyQaG8jCdMJk4brYZENUIIDkT/rxQgX7T4ehUftMW MURowr1Szv6Ycb+AmtVCNuk09oIHlslDGOi4ImpsheUllOPTesj9oAD6VSC+3Npj3T6o mwykvb5AdA2acbEWxoheUv0DjQaCKBkL96NIQ59wWjPq5JnATvCNFzVTGZiZzahf+/to 0fshWUwlcFdHFJxU2eNbEOGPtVlwXmeOYrbCnZXqjFbccuelF2/3FyO+1vWAFJIRVnCm jvIKN4PnhQnY1eG0ViuZ7QIGWlW1nNIQlBFUAtIHfeqj8uqhhvFxIpAlFAfFKW8iiV8Y DVzQ== X-Forwarded-Encrypted: i=1; AJvYcCXj8aawRe9DMJd+X6xBAnkyeTIrcovENTPt0fUyNEI2PdeD1jdFbh/Lwgxq/V22Kpl5SvJbMwwYlvPE1gk=@vger.kernel.org X-Gm-Message-State: AOJu0YyPruv8PUUvpMa5lrnH27o2TgJy56NMAthrtajS4Zn941SBMK0r dXDIljKRQn1JuFQD9OILLP4g3KwYnIO5bFX1CEoUW+VdbN/D2G6AXKk1iremt5E= X-Gm-Gg: ASbGncvU8uJLihejCqtNZ3nEAwdyYHrtEF560Raqzf2POi5QgGbqUfuvxv1//kfThOC 56jDQsmrd4qCMITFcbvQ74dmiQMiW/RVAstrGUV2smVAf/4iyj6ussAz8YwujcJXXPV8cl1UaXS MTXevpsiBf1Q8OXiLxohp6JaAacTxotkFdxQYTD9FBZAfkeglC4eobT5Dm5AWMcK+WysjHxagqp hmcKbLGX1Ge/MTPiQUNPPBJx7JZyJ+e6M0RAfNu5EshLv3ISCp3oJQc/DTp X-Google-Smtp-Source: AGHT+IFA+GuHLn3+MK1ZeRZZP3C9VMVBGmb7rBcbA1fV4I0OGtHKvL+W7OoGXWUKvvS+Tr+1pcK1JQ== X-Received: by 2002:a05:6e02:2169:b0:3a7:7d26:4ce4 with SMTP id e9e14a558f8ab-3afee2d0367mr35186015ab.9.1734105378594; Fri, 13 Dec 2024 07:56:18 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:17 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 11/11] mm: call filemap_fdatawrite_range_kick() after IOCB_DONTCACHE issue Date: Fri, 13 Dec 2024 08:55:25 -0700 Message-ID: <20241213155557.105419-12-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a buffered write submitted with IOCB_DONTCACHE has been successfully submitted, call filemap_fdatawrite_range_kick() to kick off the IO. File systems call generic_write_sync() for any successful buffered write submission, hence add the logic here rather than needing to modify the file system. Signed-off-by: Jens Axboe --- include/linux/fs.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 653b5efa3d3f..58a618853574 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2912,6 +2912,11 @@ static inline ssize_t generic_write_sync(struct kioc= b *iocb, ssize_t count) (iocb->ki_flags & IOCB_SYNC) ? 0 : 1); if (ret) return ret; + } else if (iocb->ki_flags & IOCB_DONTCACHE) { + struct address_space *mapping =3D iocb->ki_filp->f_mapping; + + filemap_fdatawrite_range_kick(mapping, iocb->ki_pos, + iocb->ki_pos + count); } =20 return count; --=20 2.45.2 From nobody Sat Feb 7 19:41:45 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99A0E4A1D; Wed, 18 Dec 2024 17:16:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734542206; cv=none; b=TQA6sTH7J8ov4upAwqJNR5Ni244yxuLdOsFRl7XpGcEDHwBn7sndpwiyhOlR45lpTRpCGFluvIkFhtYHBEMW7RCbE7c6pZAd5bA1RycRq84RBzKgtwDrfkJ+IBh5m4ovrlluhE8sRSbObMN+1T2aFiFB+VNPlCJCl/RGyl3K4Rw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734542206; c=relaxed/simple; bh=vByKvfBxoRz2Ew61HCzPzSOCqlcnxTOUTykzOp1/1ME=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ntcXmMJf6taDtf68cQ0Ge3yfF+Yb8PJKG07mHl0i6FJuguopRQJZNLpFtOWmpVnVntdaqxSsV9It88JhQu31JclaY9WEFP/dsRpOlw3Hja21/mlsdrKeZ+/SiIVxQKAgK1HEilJR4ZlhNBp5ut9oOZY9HdWGaJKgpsiO7l4Y3u8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LaRZII0X; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LaRZII0X" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E4C71C4CECD; Wed, 18 Dec 2024 17:16:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734542206; bh=vByKvfBxoRz2Ew61HCzPzSOCqlcnxTOUTykzOp1/1ME=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=LaRZII0XV6az6sQ50USic51NDQt/fhWVoYaA0YSNVF8wUYpBxoJhSrjQCGTK6ajMD z44k3r2auvUrUCAQ2utziWzufJqHMxWXuyjupFw2hQDT4GtknGMBDIwv/yAYqbujzO 30CCkreSlXnX7bEvuNiMcv3X/KgZCvjfJ92T2tLmULVlTdo4UJ4eE8TL+mLSttbx8p 9mc3PfIVjxlmEs4uz9KMwzQaubt0ZtM0FbNMIwtm4XvZDZ0iIKywCsSTvbpiYkh8K8 ZMAkejSDmtJ60gpcz3HHFd7QMoSicq1hWSYwaIKB9HXHte41/NmPr73vJCAMZ5698y t9dJEp6sKqvDg== Date: Wed, 18 Dec 2024 12:16:44 -0500 From: Mike Snitzer To: Jens Axboe Cc: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, linux-nfs@vger.kernel.org Subject: [PATCH] nfs: flag as supporting FOP_DONTCACHE Message-ID: References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" On Fri, Dec 13, 2024 at 08:55:14AM -0700, Jens Axboe wrote: > Hi, >=20 > 5 years ago I posted patches adding support for RWF_UNCACHED, as a way > to do buffered IO that isn't page cache persistent. The approach back > then was to have private pages for IO, and then get rid of them once IO > was done. But that then runs into all the issues that O_DIRECT has, in > terms of synchronizing with the page cache. >=20 > So here's a new approach to the same concent, but using the page cache > as synchronization. Due to excessive bike shedding on the naming, this > is now named RWF_DONTCACHE, and is less special in that it's just page > cache IO, except it prunes the ranges once IO is completed. >=20 > Why do this, you may ask? The tldr is that device speeds are only > getting faster, while reclaim is not. Doing normal buffered IO can be > very unpredictable, and suck up a lot of resources on the reclaim side. > This leads people to use O_DIRECT as a work-around, which has its own > set of restrictions in terms of size, offset, and length of IO. It's > also inherently synchronous, and now you need async IO as well. While > the latter isn't necessarily a big problem as we have good options > available there, it also should not be a requirement when all you want > to do is read or write some data without caching. >=20 > Even on desktop type systems, a normal NVMe device can fill the entire > page cache in seconds. On the big system I used for testing, there's a > lot more RAM, but also a lot more devices. As can be seen in some of the > results in the following patches, you can still fill RAM in seconds even > when there's 1TB of it. Hence this problem isn't solely a "big > hyperscaler system" issue, it's common across the board. >=20 > Common for both reads and writes with RWF_DONTCACHE is that they use the > page cache for IO. Reads work just like a normal buffered read would, > with the only exception being that the touched ranges will get pruned > after data has been copied. For writes, the ranges will get writeback > kicked off before the syscall returns, and then writeback completion > will prune the range. Hence writes aren't synchronous, and it's easy to > pipeline writes using RWF_DONTCACHE. Folios that aren't instantiated by > RWF_DONTCACHE IO are left untouched. This means you that uncached IO > will take advantage of the page cache for uptodate data, but not leave > anything it instantiated/created in cache. >=20 > File systems need to support this. This patchset adds support for the > generic read path, which covers file systems like ext4. Patches exist to > add support for iomap/XFS and btrfs as well, which sit on top of this > series. If RWF_DONTCACHE IO is attempted on a file system that doesn't > support it, -EOPNOTSUPP is returned. Hence the user can rely on it > either working as designed, or flagging and error if that's not the > case. The intent here is to give the application a sensible fallback > path - eg, it may fall back to O_DIRECT if appropriate, or just live > with the fact that uncached IO isn't available and do normal buffered > IO. >=20 > Adding "support" to other file systems should be trivial, most of the > time just a one-liner adding FOP_DONTCACHE to the fop_flags in the > file_operations struct. >=20 > Performance results are in patch 8 for reads, and you can find the write > side results in the XFS patch adding support for DONTCACHE writes for > XFS: >=20 > ://git.kernel.dk/cgit/linux/commit/?h=3Dbuffered-uncached.9&id=3Dedd7b1c9= 10c5251941c6ba179f44b4c81a089019 >=20 > with the tldr being that I see about a 65% improvement in performance > for both, with fully predictable IO times. CPU reduction is substantial > as well, with no kswapd activity at all for reclaim when using > uncached IO. >=20 > Using it from applications is trivial - just set RWF_DONTCACHE for the > read or write, using pwritev2(2) or preadv2(2). For io_uring, same > thing, just set RWF_DONTCACHE in sqe->rw_flags for a buffered read/write > operation. And that's it. >=20 > Patches 1..7 are just prep patches, and should have no functional > changes at all. Patch 8 adds support for the filemap path for > RWF_DONTCACHE reads, and patches 9..11 are just prep patches for > supporting the write side of uncached writes. In the below mentioned > branch, there are then patches to adopt uncached reads and writes for > xfs, btrfs, and ext4. The latter currently relies on bit of a hack for > passing whether this is an uncached write or not through > ->write_begin(), which can hopefully go away once ext4 adopts iomap for > buffered writes. I say this is a hack as it's not the prettiest way to > do it, however it is fully solid and will work just fine. >=20 > Passes full xfstests and fsx overnight runs, no issues observed. That > includes the vm running the testing also using RWF_DONTCACHE on the > host. I'll post fsstress and fsx patches for RWF_DONTCACHE separately. > As far as I'm concerned, no further work needs doing here. >=20 > And git tree for the patches is here: >=20 > https://git.kernel.dk/cgit/linux/log/?h=3Dbuffered-uncached.9 >=20 > include/linux/fs.h | 21 +++++++- > include/linux/page-flags.h | 5 ++ > include/linux/pagemap.h | 1 + > include/trace/events/mmflags.h | 3 +- > include/uapi/linux/fs.h | 6 ++- > mm/filemap.c | 97 +++++++++++++++++++++++++++++----- > mm/internal.h | 2 + > mm/readahead.c | 22 ++++++-- > mm/swap.c | 2 + > mm/truncate.c | 54 ++++++++++--------- > 10 files changed, 166 insertions(+), 47 deletions(-) >=20 > Since v6 > - Rename the PG_uncached flag to PG_dropbehind > - Shuffle patches around a bit, most notably so the foliop_uncached > patch goes with the ext4 support > - Get rid of foliop_uncached hack for btrfs (Christoph) > - Get rid of passing in struct address_space to filemap_create_folio() > - Inline invalidate_complete_folio2() in folio_unmap_invalidate() rather > than keep it as a separate helper > - Rebase on top of current master >=20 > --=20 > Jens Axboe >=20 >=20 Hi Jens, You may recall I tested NFS to work with UNCACHED (now DONTCACHE). I've rebased the required small changes, feel free to append this to your series if you like. More work is needed to inform knfsd to selectively use DONTCACHE, but that will require more effort and coordination amongst the NFS kernel team. From: Mike Snitzer Date: Thu, 14 Nov 2024 22:09:01 +0000 Subject: [PATCH] nfs: flag as supporting FOP_DONTCACHE Jens says: "nfs just uses generic_file_read_iter(), so read side is fine with the flag added and generic_perform_write() for the write side, wrapped in some nfs jazz. So you can probably just set the flag and be done with it." Must also update nfs_write_begin() to set FGP_DONTCACHE in fgp flags passed to __filemap_get_folio(). Suggested-by: Jens Axboe Signed-off-by: Mike Snitzer --- fs/nfs/file.c | 3 +++ fs/nfs/nfs4file.c | 1 + 2 files changed, 4 insertions(+) diff --git a/fs/nfs/file.c b/fs/nfs/file.c index 1bb646752e466..ff5d4c97df494 100644 --- a/fs/nfs/file.c +++ b/fs/nfs/file.c @@ -354,6 +354,8 @@ static int nfs_write_begin(struct file *file, struct ad= dress_space *mapping, file, mapping->host->i_ino, len, (long long) pos); =20 fgp |=3D fgf_set_order(len); + if (foliop_is_dropbehind(foliop)) + fgp |=3D FGP_DONTCACHE; start: folio =3D __filemap_get_folio(mapping, pos >> PAGE_SHIFT, fgp, mapping_gfp_mask(mapping)); @@ -909,5 +911,6 @@ const struct file_operations nfs_file_operations =3D { .splice_write =3D iter_file_splice_write, .check_flags =3D nfs_check_flags, .setlease =3D simple_nosetlease, + .fop_flags =3D FOP_DONTCACHE, }; EXPORT_SYMBOL_GPL(nfs_file_operations); diff --git a/fs/nfs/nfs4file.c b/fs/nfs/nfs4file.c index 1cd9652f3c280..83e2467b7c66f 100644 --- a/fs/nfs/nfs4file.c +++ b/fs/nfs/nfs4file.c @@ -467,4 +467,5 @@ const struct file_operations nfs4_file_operations =3D { #else .llseek =3D nfs_file_llseek, #endif + .fop_flags =3D FOP_DONTCACHE, }; --=20 2.44.0