From nobody Sun Dec 14 13:38:53 2025 Received: from mail-il1-f178.google.com (mail-il1-f178.google.com [209.85.166.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BE54216397 for ; Fri, 20 Dec 2024 15:48:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709719; cv=none; b=GLdnSfBujeJLROAyIRktt3KvFhUYM+ok71/wlmJCbLCrnn+jheyLJwZdazQWaKRDnJzcVV75zinBg/DqrcCbKs98pMZXEr5KGEXOOuf0hQ8C+i/6BJvhzanXA+TSOEw5Pqm+33Cu6cAKMcYKoDzocosUJP2l2GxHjXtVKn+XoDE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709719; c=relaxed/simple; bh=b/y5rN0twqq/ec0xNXkKmKx5Y1U8KwWxvQTbJi27mNM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GmY7zgVnmcx5FFeDl2OoEA5kW+FP94/nLRGV6dvC0sVSWxbOANrt3BuuZ4ikaO8JaO4novAu63ZQ1Am5uniIkHG6HWzQ3ntoiL+tyToo715c3PoWvPEOaJvzml9ezun3Nkho+oWbLoTyNhjE6rTEHBstp0QACaEwcecZ/P9xKfE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=jkL83Moh; arc=none smtp.client-ip=209.85.166.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="jkL83Moh" Received: by mail-il1-f178.google.com with SMTP id e9e14a558f8ab-3a7dd54af4bso7539765ab.2 for ; Fri, 20 Dec 2024 07:48:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709716; x=1735314516; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1VCPzs6l+RtFDMTSDAiMfSmCwqht+PTO7Juc8Thzg3k=; b=jkL83MohDYhQY4bjlZQSU/5NzrH55KTXCTdEjCvqC5zm0NEGSuy+axb4CHoLYS0nw3 JXqDzqCPAezo31DUrtnls2EAQvQfLgiq30e+/5VUFDmd5imZQrpDb0z+tCq6llMaYR65 +/0SyHyRerEpNYWGu/H8L2gyTbX6eFxapOUk4VsJ7yNuxKj6qm293RAD+e+rYul672Xw HkgT1ta3CnX/hqtIrm0FAPNmRuts9AEg3nS5RzgGiJOpfcROP/zcauIbQAdRg7wu9pzs cGGpS2BBJ9DX8SehPlElyHtvi92P7y/lwDLGrLjoeyt3Gx5UkJXjCeGo7gNLEC9hcfJp RTdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709716; x=1735314516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1VCPzs6l+RtFDMTSDAiMfSmCwqht+PTO7Juc8Thzg3k=; b=oHPp/rDuTOi4D84ECcbFXOXjAVLZ0YVNLCNSn5X2vy1SJaGu5HY9PuSF5o/KJpSMlo T5Hrbtwu7FzHtIuiN3LZAZ6DFABNey/bJBEtqTqsmqdT6Q2yYtybWwUj439CeNvZ3SDG 0fDOKTNRFeoWojLrl6CFDbSpkHlhC+IPzkagburJAc5VFYaVn7FO08mP04KhMdDPWwXX 0H7LDhhDvKmZWomBqG+F2hF5QE+HFIzChmhf/P0m+dr5NlPHcQlVpZ5N+a8O3NeanVAh 3a3Of0d+RtEaiyetNncLjkhMene7Pcmz5pSMqzg5/aNX8xeXSXvSzjDag2jBa85w3Ocr Symw== X-Forwarded-Encrypted: i=1; AJvYcCUu0NTs3JPxwKHwWTI30iuSBIqayXS0QyuvWif8MyEMjkbX5da22A/cy84v0xFUFf+NDgVmpOReC4zaLxM=@vger.kernel.org X-Gm-Message-State: AOJu0YysmbKGdJYf6tvpbhVI0bJQ8YG0MeUk7qSdH5kcM2/89imDOkbb jRaU4lxWTwSSLesaUxoexum8tB4nPwGo7V4LN1sYwENp4Aby4jv7C2qgoctP/sE= X-Gm-Gg: ASbGncv1/2X+QvOlD5uF2tUUBSKjRycGPpG/vC7OkQFW6jgVTeyo991uJMpwIZaMAm2 eyurpYS4lS4jhYNIjGTy0X7gwmn3L3mo5mwH8qKNL1TsHon33v5ixz9OBMbBj2xCEYPGDZhHu57 UtTA4q8xE+EOs3pIbKcXgvPVtCXRVDbEwk0/4UOQPKiSshLvgl2ypd2UrGeW1JnU3h4cmfP2q0l wd6dkMh2rEEwge6yMGkJCMabOxv2u5xTf+hdP0yi64jQsZEpU8IK0UAJB7g X-Google-Smtp-Source: AGHT+IHcbw9zIKJRxdjYyzhRRbFlhvov/fILoOcF1iNDb9M6UqXSGlIvkLvTLfoaxOcUHsgp350Srg== X-Received: by 2002:a05:6e02:1e08:b0:3a7:8720:9deb with SMTP id e9e14a558f8ab-3c2d2d50993mr23190015ab.11.1734709716455; Fri, 20 Dec 2024 07:48:36 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:35 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" Subject: [PATCH 01/12] mm/filemap: change filemap_create_folio() to take a struct kiocb Date: Fri, 20 Dec 2024 08:47:39 -0700 Message-ID: <20241220154831.1086649-2-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rather than pass in both the file and position directly from the kiocb, just take a struct kiocb instead. With the kiocb being passed in, skip passing in the address_space separately as well. While doing so, move the ki_flags checking into filemap_create_folio() as well. In preparation for actually needing the kiocb in the function. No functional changes in this patch. Reviewed-by: Kirill A. Shutemov Signed-off-by: Jens Axboe Reviewed-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index f61cf51c2238..8b29323b15d7 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2459,15 +2459,17 @@ static int filemap_update_page(struct kiocb *iocb, return error; } =20 -static int filemap_create_folio(struct file *file, - struct address_space *mapping, loff_t pos, - struct folio_batch *fbatch) +static int filemap_create_folio(struct kiocb *iocb, struct folio_batch *fb= atch) { + struct address_space *mapping =3D iocb->ki_filp->f_mapping; struct folio *folio; int error; unsigned int min_order =3D mapping_min_folio_order(mapping); pgoff_t index; =20 + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + folio =3D filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; @@ -2486,7 +2488,7 @@ static int filemap_create_folio(struct file *file, * well to keep locking rules simple. */ filemap_invalidate_lock_shared(mapping); - index =3D (pos >> (PAGE_SHIFT + min_order)) << min_order; + index =3D (iocb->ki_pos >> (PAGE_SHIFT + min_order)) << min_order; error =3D filemap_add_folio(mapping, folio, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); if (error =3D=3D -EEXIST) @@ -2494,7 +2496,8 @@ static int filemap_create_folio(struct file *file, if (error) goto error; =20 - error =3D filemap_read_folio(file, mapping->a_ops->read_folio, folio); + error =3D filemap_read_folio(iocb->ki_filp, mapping->a_ops->read_folio, + folio); if (error) goto error; =20 @@ -2550,9 +2553,7 @@ static int filemap_get_pages(struct kiocb *iocb, size= _t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); } if (!folio_batch_count(fbatch)) { - if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) - return -EAGAIN; - err =3D filemap_create_folio(filp, mapping, iocb->ki_pos, fbatch); + err =3D filemap_create_folio(iocb, fbatch); if (err =3D=3D AOP_TRUNCATED_PAGE) goto retry; return err; --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C29B218EA2 for ; Fri, 20 Dec 2024 15:48:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709720; cv=none; b=YeR/qbCQhRUyzPwyVqe56xSgs3oiyMQqZ+fZ2jPKDGLHw42jRjib35Yi5Jj4SJJxhvGP4HS/n6NO1zofRPtNw/1sxs5wbh8qnxXfX4jwQQjzECJPcFc/ZS1pZAHSFEhZyoXNxBjcd8uQWX/cRlSiS+FPYhen7UnfckPx8MqbuRA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709720; c=relaxed/simple; bh=nP2D5ty4r2yjDcWsJ4kVfAz8c3OFcpVU3g/7tFCHj+w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Kc89i5PoV6T/ZPq+/xCFC3OPy9yERvfaRz2mfHp5Jk3Hx+MbvWCepDBlisYoojcav59lk3xcWD/x+iVpipuhcc4Zgyopr3q4fOPNiLWHiv3jL9D3VRS900JUvIf7cL2ydBODthxbVjdzeDf5Ed4j7DhPc4NB92MbqP34Pxv6e6s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=u0xMnDTF; arc=none smtp.client-ip=209.85.166.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="u0xMnDTF" Received: by mail-io1-f49.google.com with SMTP id ca18e2360f4ac-844e7409f8aso74948139f.1 for ; Fri, 20 Dec 2024 07:48:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709718; x=1735314518; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ymwG1n6nWD/uJHK94RoKI8EVCXuU9v7PVyju+T4v7w0=; b=u0xMnDTFqisiHf/UFpkVAqfaKJnXuM0CefNU15ODr/AWgfu4NslH7Njz41dtEJPEnz kZTbY6s6kfjRRBMWiN0eiYqEXuijdGCIQgrAQo7lyt/FAQC/HCwTzvaZ2NDBrTNY++vt i9AJha4TZTp769kHtXOEBdv/IIJMh7BrSeYAZpdvLLaoBgHsi1MPVa7H4Ji8GsGIevAC xlj5NOonwj0K4pAiALWP4iw8dkD//dGWZ904OdmJ84HXYPRgMFlUqZg+SgTvev21Yuhm 7DQu5IIRGXRCw9eTRuN2Iim7CaynNJ+ImZlrHkkD2BbgfwBpX9cuUy3X90vaOaQn3IZN wxyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709718; x=1735314518; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ymwG1n6nWD/uJHK94RoKI8EVCXuU9v7PVyju+T4v7w0=; b=C8/YW5aNtaZ8YBTMXMGxBSLqxDbBapFaFTRz8mSTP40erACl5W3HdDHIwlsYj6gQ57 BiTN2XX82kvtUUaepbnnQzecM6X2oq/W0zrczR3nGsCoYcXLnburt8zMohM7Pd6sslgL L6m6AALblfY7OlMUbSUwbaLTZCNHb+V8715EjJfSvdBfygIF8WEOGlZReuoycRcXQUnL Y3zurqBfiQUseX0ZX3q8B8cztfCwfpHTX5ga4DcT1LIXuhUvEEJxkipQWVN+sIODufMF hlIv1+Sl5/qYQgm3Xv06IE9uTM0Ic80GFmkSOT8zqKexZbnjU0WSq1mi6F+cn1xWpHuc NE6w== X-Forwarded-Encrypted: i=1; AJvYcCVxEbJEv1VBsKT/c2KB0h6xkDGtCjg5oFfgopA1VodGWO+SCfhUDJ80cHCU6X1Q7k7QF1GGTRSZmMEJxEA=@vger.kernel.org X-Gm-Message-State: AOJu0YzUHTcgt3LQvR7Bj44OYRqNPDEXBvL6OBXEpi+9HlXupMtbfonj W4eToPdtZyH8MX5NXKscDVwUb0B6eUiKejfLy5A1QvdJCW5biRJ7bb3fw5guNkw= X-Gm-Gg: ASbGncvCBzkBrieVnzhBW8BeFzGccIQbnTd4hAlBs07luKc0VZDqVARDuM3kubkUqvD USQrPBdrKr3Rbmd/g+V5tBzD7+EssRllyIg/yiPPHA/ZV3dPjPGBzkDAWgLb9LolsTYAex8j9Jy 8PEo7MSNqdwHbFR/p1yV4zDIriBmkVgtdfWbZhuVpfNcyBm80YdG26EmGZViXJnrNiXDBIIYE0i AqL5g69Va1Tm7LyEh//Sjtf9vlTYNeOefH6gbYF4UiK4250C1s0lk9Vrr5Z X-Google-Smtp-Source: AGHT+IG3fN9OsN4GGTbFHoS9tD5hs657TGN6y3Qqvn5pakOnrgdzlOR3L4/c6Fhv1UMBXZPo8R74Hg== X-Received: by 2002:a05:6e02:156d:b0:3ab:1b7a:593e with SMTP id e9e14a558f8ab-3c2d514f9ebmr38935955ab.19.1734709717719; Fri, 20 Dec 2024 07:48:37 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:36 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" , Christoph Hellwig Subject: [PATCH 02/12] mm/filemap: use page_cache_sync_ra() to kick off read-ahead Date: Fri, 20 Dec 2024 08:47:40 -0700 Message-ID: <20241220154831.1086649-3-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rather than use the page_cache_sync_readahead() helper, define our own ractl and use page_cache_sync_ra() directly. In preparation for needing to modify ractl inside filemap_get_pages(). No functional changes in this patch. Reviewed-by: Kirill A. Shutemov Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe Reviewed-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 8b29323b15d7..220dc7c6e12f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2527,7 +2527,6 @@ static int filemap_get_pages(struct kiocb *iocb, size= _t count, { struct file *filp =3D iocb->ki_filp; struct address_space *mapping =3D filp->f_mapping; - struct file_ra_state *ra =3D &filp->f_ra; pgoff_t index =3D iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; struct folio *folio; @@ -2542,12 +2541,13 @@ static int filemap_get_pages(struct kiocb *iocb, si= ze_t count, =20 filemap_get_read_batch(mapping, index, last_index - 1, fbatch); if (!folio_batch_count(fbatch)) { + DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index); + if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags =3D memalloc_noio_save(); - page_cache_sync_readahead(mapping, ra, filp, index, - last_index - index); + page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); filemap_get_read_batch(mapping, index, last_index - 1, fbatch); --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-il1-f169.google.com (mail-il1-f169.google.com [209.85.166.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0296B219A72 for ; Fri, 20 Dec 2024 15:48:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709721; cv=none; b=PaYGUa314R0sEYDztn+YA8JamShh7GiV9TbNFjimvk+/F0BYuVihJ1vpjqVRaOy0HpTQ78C5vJ71bFRIsQTQnq3RbozHYsIaV92QHl16M/5KQwqwdfacfvUqngcqQ/TArkQukglAHb3446NW9dhTb70l4fQ4PjKKK1dj7kU+uJo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709721; c=relaxed/simple; bh=gjkhKuIh/F81dbqkaKljfYc7jS3icLdyh4zFI+rhjmA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=inzUhWLBm1/FLzyfakfbirDnoL2WTeX8SVV0MhmZjJ0GBIiIHlKeEqQ0yb6Y8iVDXKovF532Cs3FNE13IiYd1hM4+TbLAlAU6olJPw6XlL3Yjq/7Ozvn2i3bwCADuDdcyC1t13p/xr7e4ViCTGv2VGy7OkCF1Nuziy06uKbA+QA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=nQV3wjDc; arc=none smtp.client-ip=209.85.166.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="nQV3wjDc" Received: by mail-il1-f169.google.com with SMTP id e9e14a558f8ab-3a8ed4c8647so14287955ab.1 for ; Fri, 20 Dec 2024 07:48:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709719; x=1735314519; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U1i1r3ArZkX2vgdPDoEq+WK9gDY1/MkPULwwiAP9Trg=; b=nQV3wjDcNTgJ4GchyVBCil7RWw3kEBcqovSCDdq407h+Fp4OUxJAHtkRzl7caMbOCW QUQRa9it6AvF+DPyJKC/Ng6Z4LxF7LYXb+fH1HP/8nLspS7vc3ZXjwC5ajUz6lVsIiCb waUtyWQBb+MkdhbP45bVZvLRmdt7yUnioSorXOs7kFcjTfDlw/5Z3RAkJRxGSPf0dXRx 4nL3ElUqrA7V22XHs9Ce1OtmnqPRTrETi53u+6od7ScPx3Yq+9UlkO+rMKBMzBN5N/ee GShvNHmHpXUyQfioJXe2VWYi+wLENkEgj/k3X2561CHf2sI0cTpR8qcX9UIxsHNGWxG9 JEYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709719; x=1735314519; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U1i1r3ArZkX2vgdPDoEq+WK9gDY1/MkPULwwiAP9Trg=; b=rylye8rqocf5H7eJDgqluNIve65Td7pgbd38z+/m5beo1yYU16Qv5Eui9Pf6g7ktIQ N7XXDctp4Yc9WoHjDfLzLHHgtBjhqBCH5T4w9aTopryxNjWTHPLjk3kd/dAJJqFhmogv 7uVWsAWoumS6Ch5sVybwITnPCuO6fVK0SpsK6imsnYvMbdaJIgOW6G2swtdzsxPMl9AF QrMr3DY32DtZw9Lh28/aWxgGt+ADq3q9p7+TDIZ830sD5McBtLIWtj0+n12qeSn3A6Em J5YPcIq8kcKpgmrRxHihn5yTlOw5joqUxH5BRDia6axfVm96pfZnrT6pyzFvtCuymv+5 hlvA== X-Forwarded-Encrypted: i=1; AJvYcCUuxm+hVCdOBqMyvfVdDGPe+FtyKJaQbHr2OawlDxAgCQUlLjsVcXVNfP8nqkks3Sc/Q7iJMTa9dIdzqH0=@vger.kernel.org X-Gm-Message-State: AOJu0YzaI6tl6xNAZvIZuD8Ji28sR6GRIQirieTbmka+Sm6DsNQmfQNO dSlASzYUBLJCoMEMHOVGexxhQzBIgz3AJlLo0E7WyqPKM6jk2Kn/66MeKEZozjnE/1Yaw50Q05L n X-Gm-Gg: ASbGncv1DcBWmLN/K2IcKnjgXJGSkobgvOI/Lx1+kZfOfIKtOxn3ZH/ctyKKvTdwWnl cB1Sr/br/LGqirxfZNAr8YRl1uxVW57RcDca/7fJcgriFKqwvDZ/+tX6PSFAoxSolW1/m4tN5kT B6ASgpaTnaRPLd/z3XS5y8TAvCJXkGsSxDCzIclQFP5Gpm4RzD14FTxNzJ4jsKyyUWYNRuh4BS9 eS3BxMzbDi6MykStGApKgLP08hEldrHJYI8OiALnVvzPorKwD01fDC5m6w2 X-Google-Smtp-Source: AGHT+IEQ+rQahAFsMNf0wBgiaHc3ybkwynnm3ue1l0Cc7XmizswXF3l+0I9aVhCYoLyroOrTRrUFJg== X-Received: by 2002:a05:6e02:1ca5:b0:3a7:88f2:cfa9 with SMTP id e9e14a558f8ab-3c2d2d50b1emr32075375ab.11.1734709719216; Fri, 20 Dec 2024 07:48:39 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:38 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" Subject: [PATCH 03/12] mm/readahead: add folio allocation helper Date: Fri, 20 Dec 2024 08:47:41 -0700 Message-ID: <20241220154831.1086649-4-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Just a wrapper around filemap_alloc_folio() for now, but add it in preparation for modifying the folio based on the 'ractl' being passed in. No functional changes in this patch. Reviewed-by: Kirill A. Shutemov Signed-off-by: Jens Axboe Reviewed-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index ea650b8b02fb..8a62ad4106ff 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -188,6 +188,12 @@ static void read_pages(struct readahead_control *rac) BUG_ON(readahead_count(rac)); } =20 +static struct folio *ractl_alloc_folio(struct readahead_control *ractl, + gfp_t gfp_mask, unsigned int order) +{ + return filemap_alloc_folio(gfp_mask, order); +} + /** * page_cache_ra_unbounded - Start unchecked readahead. * @ractl: Readahead control. @@ -265,8 +271,8 @@ void page_cache_ra_unbounded(struct readahead_control *= ractl, continue; } =20 - folio =3D filemap_alloc_folio(gfp_mask, - mapping_min_folio_order(mapping)); + folio =3D ractl_alloc_folio(ractl, gfp_mask, + mapping_min_folio_order(mapping)); if (!folio) break; =20 @@ -436,7 +442,7 @@ static inline int ra_alloc_folio(struct readahead_contr= ol *ractl, pgoff_t index, pgoff_t mark, unsigned int order, gfp_t gfp) { int err; - struct folio *folio =3D filemap_alloc_folio(gfp, order); + struct folio *folio =3D ractl_alloc_folio(ractl, gfp, order); =20 if (!folio) return -ENOMEM; @@ -750,7 +756,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ =20 - folio =3D filemap_alloc_folio(gfp_mask, min_order); + folio =3D ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; =20 @@ -779,7 +785,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ =20 - folio =3D filemap_alloc_folio(gfp_mask, min_order); + folio =3D ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; =20 --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E072219EAD for ; Fri, 20 Dec 2024 15:48:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709722; cv=none; b=JArdiriCJfQaJopChmRRRXIahjB5t5nNt6vRwvMkD2fiHHY+T5xezRvR0762EGIMUV28AU75yggPx5nqKeH/D29rOLQE60WAQ0OTf6TT15+Ao2m3XX7BDdisho1bpKv/8A5SRZQMil/aHoOX8j1k3cRWRsySAFxqxy7fMTbjCg0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709722; c=relaxed/simple; bh=JO6tsxSlP852WA3y5S797XWTlY9uJCLdRkQsjOqf/SY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IsbigaSKBTXajXOGcDnYLS1MkgZapn/Arok/om6IrEnX1zuKcaQ1WHsqX1V6KNUGfP3ar9zyeidTr+OEoEUX+PObKanVRXWM8WgBXep+u0XKSgSj+44iCxeV8jgGcC+0fTaQeA9MhtY4LwZ6MA95l5ysRVh1CLE5vQ3Ae+OfaNo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=Oc8efsLy; arc=none smtp.client-ip=209.85.166.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="Oc8efsLy" Received: by mail-il1-f170.google.com with SMTP id e9e14a558f8ab-3a814831760so6653505ab.1 for ; Fri, 20 Dec 2024 07:48:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709720; x=1735314520; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TZWY6vJPfqFGr1DBynm1KpCMDB4zdovTdnWkeolaIOo=; b=Oc8efsLyd512POrstMUgXa+FvteRc1oMKe9Ma51mEWEnjkPmq8lmhCK02N/JPMCUs5 fFoKBE9OyTb6NPJr46dyQ3rprc68njEKgQqczl1/azIDQpmO1tLR8mE0N5aaDPkJ6jtk Dz6fi0Z+6HAOco6MZtXj8mmPgNOzU8dGSTXFKqhdIX1DCxfprvoES1r6czX0z4OuH1Gw rqbPn8gvjVcq55EFFL/GpuZ1wMUuq7Ma1qkoCi2gh+IqV+vLh8PDYSivfeP25WQnE/Po SYelfNI1xK8fcErXUkdstKmaALvH/us9Lt9J1WETzro/u/inGdlUmTWf8pU1PvH2nGl/ EMCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709720; x=1735314520; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TZWY6vJPfqFGr1DBynm1KpCMDB4zdovTdnWkeolaIOo=; b=aE9LGa5b4tBmtBccrIWw+C2GeM5dxqVYcPV/CdFwj49gzKixLKYXbtm8h83X6o3Zry Joj8Qkc6xmteHValSXiqJ8j/w1Qh0l5SlTQKeEff6RFjszvjLb8dI2iCY/LQaAIsj0vt oRf5GUDlwrQpOaN2Yyq9xdrq4wDw7axZac+YnXw/V0eWhpaQrgPi7kiTShPWgnG7YRsQ /Jih9bjsHFR2FWEmnal9LLIT/HkINN3IVz36xrPFvBN2kggrAofV0OjY3gV5gavAf0l/ 3uE0r8w8Zvl9lx7R3dk4eqxYCh7GoTQjXPz9AubyygeDJPJVm0o1arntxWAgvWzgM63J fl1A== X-Forwarded-Encrypted: i=1; AJvYcCU/utfmViiqx1GRsUpxiqIU4NykQwEpBx991t3yLAWANDNU0XJn3HLqcPqws9Fqor60cC13BLh3uGUu3Sc=@vger.kernel.org X-Gm-Message-State: AOJu0YwX+8GH6jfK/SGDNEtolzF8vE3r2V34nMafcJyeEeGxWx/lTxh/ JLQgXe2D1pngTQR6bdVX2Hg8pdkMIKi8a/tOzF72hjIM+niXthommS9rva91Moo= X-Gm-Gg: ASbGncsU9nnW/CXDeMUH9sI8cyMuWqDJzS4y8Br5EPpw9X2v8fgP+wXFWptkrxOCCRX 05LaVD23dfauKXJiZz0EpHpbMrARaQcLi9m/dizFHmb1iJMbbp1tyr8xoafjKdV1vNsrQ5Cg3Qo BHqfAJXhTMpnkXPhFDr3rt2s9FhF+yG0Tji1PmkF/yxCtWrekJ3YXeYnpyNyXWdd1ltJQisjdWW bH+HD6NzjA6sZTNIkW4BVw6ye8rbS/RLV0gr5FPn/IVUuiUqngIDaKSWcTx X-Google-Smtp-Source: AGHT+IFvp8gxmeuG0px26ehs16GxVvbXtCVpdcyc+9Br4GzsJzwQZ198tT+fp2EHMgvEVKANcydJew== X-Received: by 2002:a92:cda6:0:b0:3a7:a738:d9c8 with SMTP id e9e14a558f8ab-3c2d14d1839mr28931745ab.2.1734709720617; Fri, 20 Dec 2024 07:48:40 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:39 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" Subject: [PATCH 04/12] mm: add PG_dropbehind folio flag Date: Fri, 20 Dec 2024 08:47:42 -0700 Message-ID: <20241220154831.1086649-5-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a folio flag that file IO can use to indicate that the cached IO being done should be dropped from the page cache upon completion. Reviewed-by: Kirill A. Shutemov Signed-off-by: Jens Axboe --- include/linux/page-flags.h | 5 +++++ include/trace/events/mmflags.h | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cf46ac720802..16607f02abd0 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -110,6 +110,7 @@ enum pageflags { PG_reclaim, /* To be reclaimed asap */ PG_swapbacked, /* Page is backed by RAM/swap */ PG_unevictable, /* Page is "unevictable" */ + PG_dropbehind, /* drop pages on IO completion */ #ifdef CONFIG_MMU PG_mlocked, /* Page is vma mlocked */ #endif @@ -562,6 +563,10 @@ PAGEFLAG(Reclaim, reclaim, PF_NO_TAIL) FOLIO_FLAG(readahead, FOLIO_HEAD_PAGE) FOLIO_TEST_CLEAR_FLAG(readahead, FOLIO_HEAD_PAGE) =20 +FOLIO_FLAG(dropbehind, FOLIO_HEAD_PAGE) + FOLIO_TEST_CLEAR_FLAG(dropbehind, FOLIO_HEAD_PAGE) + __FOLIO_SET_FLAG(dropbehind, FOLIO_HEAD_PAGE) + #ifdef CONFIG_HIGHMEM /* * Must use a macro here due to header dependency issues. page_zone() is n= ot diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index bb8a59c6caa2..3bc8656c8359 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -116,7 +116,8 @@ DEF_PAGEFLAG_NAME(head), \ DEF_PAGEFLAG_NAME(reclaim), \ DEF_PAGEFLAG_NAME(swapbacked), \ - DEF_PAGEFLAG_NAME(unevictable) \ + DEF_PAGEFLAG_NAME(unevictable), \ + DEF_PAGEFLAG_NAME(dropbehind) \ IF_HAVE_PG_MLOCK(mlocked) \ IF_HAVE_PG_HWPOISON(hwpoison) \ IF_HAVE_PG_IDLE(idle) \ --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-io1-f42.google.com (mail-io1-f42.google.com [209.85.166.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EE8D21A45B for ; Fri, 20 Dec 2024 15:48:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709724; cv=none; b=AKxjdhy2nrW+EiOsHJ7S2UNltwAlC/MlGSqKTOBD/dD980ccjM8JGHQ5YjNhV5vNaGrKXMF7lH67VF3wX0WycY7AA/fFPdjh4qTkl68DF41tSbLWUj/LWhy3cwrYVzbxWXFWr03c1QK7YFnuyPtn8FhuCxmFoJDJinyTcI8nidg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709724; c=relaxed/simple; bh=cl+oP5X8N7YKHDElO28Q8MeUv/ZKeS8cyPvyEaOZdN4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U7Kj4/nxP/YmV+tFEOpRO3SXOzKB0oa4kJCNqMcpTYSuFuR8amrSs+mWnTUs8m+dXGxdECFzupdxCkVDpLKLWFT22XBBh5rNG5AApBvDN5Ixg9zbNQQa+9+6jUH893Yee4wzEHu8NX1FbkFVgUkrjejrxxGWESep/AO+gTEItxQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=nqqzrpEc; arc=none smtp.client-ip=209.85.166.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="nqqzrpEc" Received: by mail-io1-f42.google.com with SMTP id ca18e2360f4ac-844bff5ba1dso149003939f.1 for ; Fri, 20 Dec 2024 07:48:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709722; x=1735314522; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qy66q6foyU2ZBCe4X6zLGxi2eTMkd0mxpiy36jLMGYw=; b=nqqzrpEccq1eo7LKwO1s4rU8vfiuRVkl3K3iumPJlThFajAbySOyap4/hIP9+X2wjV Bf8qUD3GSHB6JuvtBTOGC/sDN6Co3SXndULxWUdDGKZfa0STbdMXIe/5G4auM4+yGkbx 5In5SD7acOXTeVOTrWdkv/v450oWVB4lkWlyCqfcGwC3KZmTU7dQTu8PmsEK7YIPtxi6 nL0JVMzTa6kaaZ2g60OYPPlRSb3TAsOyzqd2e3MKHHDMoAj1La3uzdf+o1KA9CIdP0HV fHozHWEe5rNw+z5oTAnZaE42w6tnXZUbUK3ST6YX51RUT9/2gMMD2to0DdBgNZSe2me+ xHDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709722; x=1735314522; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qy66q6foyU2ZBCe4X6zLGxi2eTMkd0mxpiy36jLMGYw=; b=DrNO6pbKO/pzht4aFz42RBr4gHXOPEiroD5L04yPxa23kIlAIVfOQSO/vJACtYe1lc vmdl9HgXMSM2YIM+CDU7BjVuhrU9u2vFMO+QNg0eFy1239aueb9QT/Id7WdX2C0Fyq0P kQE1210FClsA0+UM/YqaQY9sHvJ2H1qru8CLqMqYrJ1J9BrUXtsloTR4+DX8pKADBE/K jFCI23auUnrnKkHrtqMgtWTuFQwDQaJvVOHWGw1JAvfxfISKViqDqebhMc+lqN5PMYZu ADltESAsw0VD48AFQ9/3LdNGbu4MR/KJqjpbv2IQClhnlIrEOvXGDbe1MFDl6XngqwqQ K1ww== X-Forwarded-Encrypted: i=1; AJvYcCWNe9lqW+EvZpBgdh9zSCEOdEQVv79QHPY0scbT/eXQXYml/hjXuzSzzf9BWE1130uYl+vmfakyeOeaMJI=@vger.kernel.org X-Gm-Message-State: AOJu0YzqJamqghN81XnmzN3av01/8ikbtOxvSqYvxq7bZQyAcrKu39n8 p0OXgNKDwsni+jH6wJaZo8aOZjC+8T5BaZMi6X2prfwz3N7WUrBCsYpk+ISSzNo= X-Gm-Gg: ASbGncs23ybFJy1SNsOYxagdC78wuoYt5Z9gqGhHj/HNa394ZPkIqP46QqO1gUMOm6A 8jL7DKeJLLWgmeq1SzjswhJG2tq6yo4lvMPeC9sF7J78xB0PE5Wf/8jCYDSQROl/IeSsvokGdv0 tVGpaQqJ3YGdZVKJj8uwAWNmJehIGzwvdou9s4rKXQUvcQROpErxb2MhzngVlI50VcuF+Fil4b4 QYLHtWc01MSrh+Bp2V5muWOygRnjnkBsJURyniO2aPKyYvdoGbvHuj83F9b X-Google-Smtp-Source: AGHT+IF2sIPl1WVDVbn7fgvbpII69l3j4cwL7QghFjhSCJ3SFTcAXuckA2Q7OzRuPKU0QQKKTwe1pQ== X-Received: by 2002:a05:6602:26c6:b0:841:81ef:70b9 with SMTP id ca18e2360f4ac-8499e605a2bmr323110339f.9.1734709721832; Fri, 20 Dec 2024 07:48:41 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:41 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" Subject: [PATCH 05/12] mm/readahead: add readahead_control->dropbehind member Date: Fri, 20 Dec 2024 08:47:43 -0700 Message-ID: <20241220154831.1086649-6-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If ractl->dropbehind is set to true, then folios created are marked as dropbehind as well. Reviewed-by: Kirill A. Shutemov Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 1 + mm/readahead.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index bcf0865a38ae..5da4b6d42fae 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1353,6 +1353,7 @@ struct readahead_control { pgoff_t _index; unsigned int _nr_pages; unsigned int _batch_count; + bool dropbehind; bool _workingset; unsigned long _pflags; }; diff --git a/mm/readahead.c b/mm/readahead.c index 8a62ad4106ff..c0a6dc5d5686 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -191,7 +191,13 @@ static void read_pages(struct readahead_control *rac) static struct folio *ractl_alloc_folio(struct readahead_control *ractl, gfp_t gfp_mask, unsigned int order) { - return filemap_alloc_folio(gfp_mask, order); + struct folio *folio; + + folio =3D filemap_alloc_folio(gfp_mask, order); + if (folio && ractl->dropbehind) + __folio_set_dropbehind(folio); + + return folio; } =20 /** --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0ED1D21B908 for ; Fri, 20 Dec 2024 15:48:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709725; cv=none; b=aL3rLm15GOV7uPCeZU82XPi9IAkp2kcK2nXq7GCj48oGa3NiVk8AAuvG4n3qBAjqBvRNcA763eEb+l2MZ3qxdTyuO/usZhqDKdZvwPXALyVgtaqT8SwtN5EUHrCE44vytvkNGrScJyAKsoPCtv2SatLfMh11jCZV1A1m8NnRmlY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709725; c=relaxed/simple; bh=hlD654noMspdeqksOMGzv5hrDtGKzL2oiFa3eBWzT6w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hJXLSj5fxCtSgXUYrQRngtaEv7ATTrYDWrTp4E9SsUpui0PqtF/wxV9nXHEdxzetYMMe3oqRuGUHh2Pjyc7z0qKVsHsv5+3UBWHEt8br5ZNtPnl214l6GaS9IaosCgnzLKfM9bXTtMUDqLcS5czzmnGoMpG78kYYIIo4SN/1asc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=GFIoSUsA; arc=none smtp.client-ip=209.85.166.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="GFIoSUsA" Received: by mail-io1-f52.google.com with SMTP id ca18e2360f4ac-844ef6275c5so70171539f.0 for ; Fri, 20 Dec 2024 07:48:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709723; x=1735314523; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hj114LsQAeZlZx1JIhp9yNhhSBSN5VfiZ1DE7mn9M1E=; b=GFIoSUsAw7RekpRQilDffDad1KbwSZM1hR0LU4Qn3qd5kH4bD6IlPJu6ZYVTgRnnJO OD36Oyg0QQoAICyGD8yqeDiGamkRD5sejSyB3N7ghYOf6U3+xdFTWsQsaaLSbvLy81ZP voY5m/+wX5ztbehFd1nKvmfI1KZmWRRldniEXZ7VMqJNi/naRHFQ/atyibjfZ28I0dNC 6fr0u4rVA/B1zBrnszuq4a1EIo+FyC/L/tS/p7QGhi/1IUPqlaAhdBQKZG55disGFjS7 Pxsia5sj1pcqLkPWF39CBq1ZGip4hwod6++xWDn/ca8awvF9pIIW/AfTgtnGpVdu65Cz SOkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709723; x=1735314523; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hj114LsQAeZlZx1JIhp9yNhhSBSN5VfiZ1DE7mn9M1E=; b=gaum+iJCAo5vEvI9PBLVuKBxm2tTf+OY1qEo8Brw+8krZdXWgLKeKWTbJYptvV6T1N sO+nKAJ7MS8XqFw675r9irj5RN1/hZza60FBrW83TSzFw6xqKwxtAUfpuJTJebzjW9Th icGjpWr8KS+xfcSW4QwtgFK9z6jEG6eRDRoOt7mQB1ALgVHcvXYqHzFzynQRGBwvpenL yhvkQoQT1Ct/QzG0uByfaq9leDDUWPO3G6AHF8XRjOkxZPPuat4eE/8sbm28WgxbMP1X tqu9SN2nBwUU3h1Xg4IgJSU4CawWlp/IR6u3e8PZTHbcJTEnGDq/qHgdOS0vfPP1gOvc sYiA== X-Forwarded-Encrypted: i=1; AJvYcCW5/7hdSCXTyim1wwLo7NLMUfpV3nziWopdDJx1RIoN5qM4w4HaYFQ4Ars8mJ2rHaXtqORv4WILYDO/950=@vger.kernel.org X-Gm-Message-State: AOJu0Yx1x3ylflmCdXBWOoZo2MRmAop3VWgNB5Sp1KuqdSUsJcJx8K7w K4PqeRUjUC5dPn1gNzRY7V8ti15Sqt0acYra/RO3PtkqZZXsJM1GZdUuY3ItmgI= X-Gm-Gg: ASbGncsopPW2HeZPdBTGVTt7uP3pXuAzDHfF9XWguY2oJa/aOR/IjMWDOewHs2YH7EC vlk7W2YwCNfbvNKadgfWytkfPc8H507pFdPnrcVAM1FJZukEOO80qQJBHPxoPiPPo4ucXEpD+ol 11L+wAirDiwNDX4wZaihB5inoR4dmBhGM/ORowShZOWPYSGHiflK/+5O/Asv0vWPmrePBbY50HL V1NwxO5f5ZDwWcXFduM3JMbcd3+BVSp4+XXQ5mXDzViK1bflMGya4wSf4P+ X-Google-Smtp-Source: AGHT+IECOzR43hLv/D5wLt0YmDzKJdUpeT0f1YGhsBnnrmOZn/oym4rW0bRMY26iQGlNbtfBjUSBgg== X-Received: by 2002:a05:6e02:3683:b0:3a7:8720:9de5 with SMTP id e9e14a558f8ab-3c2d1aa27e8mr34276355ab.1.1734709723099; Fri, 20 Dec 2024 07:48:43 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:42 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 06/12] mm/truncate: add folio_unmap_invalidate() helper Date: Fri, 20 Dec 2024 08:47:44 -0700 Message-ID: <20241220154831.1086649-7-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a folio_unmap_invalidate() helper, which unmaps and invalidates a given folio. The caller must already have locked the folio. Embed the old invalidate_complete_folio2() helper in there as well, as nobody else calls it. Use this new helper in invalidate_inode_pages2_range(), rather than duplicate the code there. In preparation for using this elsewhere as well, have it take a gfp_t mask rather than assume GFP_KERNEL is the right choice. This bubbles back to invalidate_complete_folio2() as well. Signed-off-by: Jens Axboe --- mm/internal.h | 2 ++ mm/truncate.c | 53 +++++++++++++++++++++++++++------------------------ 2 files changed, 30 insertions(+), 25 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index cb8d8e8e3ffa..ed3c3690eb03 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -392,6 +392,8 @@ void unmap_page_range(struct mmu_gather *tlb, struct vm_area_struct *vma, unsigned long addr, unsigned long end, struct zap_details *details); +int folio_unmap_invalidate(struct address_space *mapping, struct folio *fo= lio, + gfp_t gfp); =20 void page_cache_ra_order(struct readahead_control *, struct file_ra_state = *, unsigned int order); diff --git a/mm/truncate.c b/mm/truncate.c index 7c304d2f0052..e2e115adfbc5 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -525,6 +525,15 @@ unsigned long invalidate_mapping_pages(struct address_= space *mapping, } EXPORT_SYMBOL(invalidate_mapping_pages); =20 +static int folio_launder(struct address_space *mapping, struct folio *foli= o) +{ + if (!folio_test_dirty(folio)) + return 0; + if (folio->mapping !=3D mapping || mapping->a_ops->launder_folio =3D=3D N= ULL) + return 0; + return mapping->a_ops->launder_folio(folio); +} + /* * This is like mapping_evict_folio(), except it ignores the folio's * refcount. We do this because invalidate_inode_pages2() needs stronger @@ -532,14 +541,26 @@ EXPORT_SYMBOL(invalidate_mapping_pages); * shrink_folio_list() has a temp ref on them, or because they're transien= tly * sitting in the folio_add_lru() caches. */ -static int invalidate_complete_folio2(struct address_space *mapping, - struct folio *folio) +int folio_unmap_invalidate(struct address_space *mapping, struct folio *fo= lio, + gfp_t gfp) { - if (folio->mapping !=3D mapping) - return 0; + int ret; + + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); =20 - if (!filemap_release_folio(folio, GFP_KERNEL)) + if (folio_test_dirty(folio)) return 0; + if (folio_mapped(folio)) + unmap_mapping_folio(folio); + BUG_ON(folio_mapped(folio)); + + ret =3D folio_launder(mapping, folio); + if (ret) + return ret; + if (folio->mapping !=3D mapping) + return -EBUSY; + if (!filemap_release_folio(folio, gfp)) + return -EBUSY; =20 spin_lock(&mapping->host->i_lock); xa_lock_irq(&mapping->i_pages); @@ -558,16 +579,7 @@ static int invalidate_complete_folio2(struct address_s= pace *mapping, failed: xa_unlock_irq(&mapping->i_pages); spin_unlock(&mapping->host->i_lock); - return 0; -} - -static int folio_launder(struct address_space *mapping, struct folio *foli= o) -{ - if (!folio_test_dirty(folio)) - return 0; - if (folio->mapping !=3D mapping || mapping->a_ops->launder_folio =3D=3D N= ULL) - return 0; - return mapping->a_ops->launder_folio(folio); + return -EBUSY; } =20 /** @@ -631,16 +643,7 @@ int invalidate_inode_pages2_range(struct address_space= *mapping, } VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio); folio_wait_writeback(folio); - - if (folio_mapped(folio)) - unmap_mapping_folio(folio); - BUG_ON(folio_mapped(folio)); - - ret2 =3D folio_launder(mapping, folio); - if (ret2 =3D=3D 0) { - if (!invalidate_complete_folio2(mapping, folio)) - ret2 =3D -EBUSY; - } + ret2 =3D folio_unmap_invalidate(mapping, folio, GFP_KERNEL); if (ret2 < 0) ret =3D ret2; folio_unlock(folio); --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4FC9C21B199 for ; Fri, 20 Dec 2024 15:48:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709726; cv=none; b=BrOa8tJNUIKpKFEHHdsBYwyOOuas8ZkmLk14q4eX8J61nTwcSgUucI600sQY3dRpwjg+BAm3or2qpLe20khKlXLV7+b75xt8ARalSTKW+ATe1yIx/8vvI5iltKhtkdgFcvaj+vDNgi2mjIwdnktP6F53Iy+xTm54P72CghxE8QU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709726; c=relaxed/simple; bh=2SEcDojzAqUnPgfjf8E9nRmr9dou/inG3/DhaNUqq8I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U2hwHU2yFiEpj/bJP0S/WW2TwXNnNPZU0xCa5nqFx3lZ3ffC7SfXmObrj6mZZoKqW9ZHeWvFWc7S1RrioiVEui29oqxaVwkfrZvaeLd7iP+ShVDwx2iOvNjMocjUOv1gh6dDEez0azSMXKNrwFQzSeuC0Adtzr2iGa6oUK1AnRU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=pc4Oh3rK; arc=none smtp.client-ip=209.85.166.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="pc4Oh3rK" Received: by mail-io1-f52.google.com with SMTP id ca18e2360f4ac-844d67eb693so154328739f.3 for ; Fri, 20 Dec 2024 07:48:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709724; x=1735314524; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4w74VD3kuQguRINknUSjUP4VNHVLoGClDPYL5LdL0oM=; b=pc4Oh3rKs651mSGbJFsxIq7XbGM9DQE81MtSvSAEJpzIAX+GD09TqGFN4OhkXpEjCJ de8Afb3cdqyyMI9lAQtd6+O4c154QWEfsLMfmxdymGWhPMzRC2DkWsurRm5H0Mk2SnrP lKRyvz6Xj7dl/c4vLRMrRY9iACZVEzmyCQqOUJI7naeRq6IX92GHebPZV9Y6Exis68uW VtQdiGFo0NnEi8Fp0XQRrbBPgOuwUpzSlqkfycmGH8DNDkQL1WmoWvOTZsCUX7+t8TBx pF+VaWV8VsNbFr2Gb1X2SDUoXc8h2T4jUh1RYHODrzzak4DnlWqjlnuCjPLpXf4kT1z0 M8AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709724; x=1735314524; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4w74VD3kuQguRINknUSjUP4VNHVLoGClDPYL5LdL0oM=; b=rlpzD2HXNJyZzBHq/ZNA2gmhFZlUPDFG6tt9xoP7MXhTHKY07NGUolKA5eyY1/FrJJ IRFX0hhYinZ2LCVCI/fIYFRGKheblKifo1AmOL0VynPiH+PRyZPVZ+Hr8p8/lH0UXKB6 J+yOyfbhRhmVVXyFtB5+U5egfD+mJEWjNpVEXrtJbhYHRQsyHyS5Bto9Rrz0kGbHjLTj j7MeIcYoxjScWSZtIqS2B71MDH/bGtcMgwy+JpZ81BGVc1Zt2nEPY/D6mOJ0SbCDkBb2 tkfHmqORcv+zEX5x+qmivdaYxsbQT8k+jyaFiWi/bn5wkjh32OVbIyVCh3RsCEf1TEoB 8lTw== X-Forwarded-Encrypted: i=1; AJvYcCVb6mrH3e+W6ELC8huAh2mEYy865vgX+B34smbNKu/tmeUz2SnXGos1AZBgYUzlg0bF8zQ7HMFyuvcvrMU=@vger.kernel.org X-Gm-Message-State: AOJu0YzEApep8G7EGvGbTOUCGWywwDF0pPElgRVTI4+g/xad9Qw4ZFPm 1FcG6CSF81BeQqyj5WCxRusp0fPPUwD4KBLqFRpkD61DeLe6IeBQg0tjxLOyZ1o= X-Gm-Gg: ASbGncvUgr8tuiwocrtbKc5OIQIEFeQFywzgvt33gGzFA6bKb5rmi1GxyXWLSs+jwlL qiRY1p51/CbxOqGlVe2tc3OPrkjezlBrKKFxaS3Xgd5suQykMdkUfouyKXFVa3ckcIg9qz++Y6P sK/WRVg449e9V1fmAmUGkJ5mGDVFTxj4u9ChCntOV9lXCsBEgyZDyXzgNXlMfsBbg7c2ia3nPBy fB3j318FQNH7KAmzP498tBU7p/IoNs30XBnt0+Pv9pjea9gB2L8evrWfhDr X-Google-Smtp-Source: AGHT+IERkjVXfRiBHnc8FlfnVZJTgamzH1zCvDow9NijC4/j/GJhywdFRh297ssQYYJsqQo3rjiWFg== X-Received: by 2002:a05:6602:60c6:b0:83a:a746:68a6 with SMTP id ca18e2360f4ac-8499e4d888bmr383870539f.5.1734709724489; Fri, 20 Dec 2024 07:48:44 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:43 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 07/12] fs: add RWF_DONTCACHE iocb and FOP_DONTCACHE file_operations flag Date: Fri, 20 Dec 2024 08:47:45 -0700 Message-ID: <20241220154831.1086649-8-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If a file system supports uncached buffered IO, it may set FOP_DONTCACHE and enable support for RWF_DONTCACHE. If RWF_DONTCACHE is attempted without the file system supporting it, it'll get errored with -EOPNOTSUPP. Signed-off-by: Jens Axboe --- include/linux/fs.h | 14 +++++++++++++- include/uapi/linux/fs.h | 6 +++++- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 7e29433c5ecc..6a838b5479a6 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -322,6 +322,7 @@ struct readahead_control; #define IOCB_NOWAIT (__force int) RWF_NOWAIT #define IOCB_APPEND (__force int) RWF_APPEND #define IOCB_ATOMIC (__force int) RWF_ATOMIC +#define IOCB_DONTCACHE (__force int) RWF_DONTCACHE =20 /* non-RWF related bits - start at 16 */ #define IOCB_EVENTFD (1 << 16) @@ -356,7 +357,8 @@ struct readahead_control; { IOCB_SYNC, "SYNC" }, \ { IOCB_NOWAIT, "NOWAIT" }, \ { IOCB_APPEND, "APPEND" }, \ - { IOCB_ATOMIC, "ATOMIC"}, \ + { IOCB_ATOMIC, "ATOMIC" }, \ + { IOCB_DONTCACHE, "DONTCACHE" }, \ { IOCB_EVENTFD, "EVENTFD"}, \ { IOCB_DIRECT, "DIRECT" }, \ { IOCB_WRITE, "WRITE" }, \ @@ -2127,6 +2129,8 @@ struct file_operations { #define FOP_UNSIGNED_OFFSET ((__force fop_flags_t)(1 << 5)) /* Supports asynchronous lock callbacks */ #define FOP_ASYNC_LOCK ((__force fop_flags_t)(1 << 6)) +/* File system supports uncached read/write buffered IO */ +#define FOP_DONTCACHE ((__force fop_flags_t)(1 << 7)) =20 /* Wrap a directory iterator that needs exclusive inode access */ int wrap_directory_iterator(struct file *, struct dir_context *, @@ -3614,6 +3618,14 @@ static inline int kiocb_set_rw_flags(struct kiocb *k= i, rwf_t flags, if (!(ki->ki_filp->f_mode & FMODE_CAN_ATOMIC_WRITE)) return -EOPNOTSUPP; } + if (flags & RWF_DONTCACHE) { + /* file system must support it */ + if (!(ki->ki_filp->f_op->fop_flags & FOP_DONTCACHE)) + return -EOPNOTSUPP; + /* DAX mappings not supported */ + if (IS_DAX(ki->ki_filp->f_mapping->host)) + return -EOPNOTSUPP; + } kiocb_flags |=3D (__force int) (flags & RWF_SUPPORTED); if (flags & RWF_SYNC) kiocb_flags |=3D IOCB_DSYNC; diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..56a4f93a08f4 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -332,9 +332,13 @@ typedef int __bitwise __kernel_rwf_t; /* Atomic Write */ #define RWF_ATOMIC ((__force __kernel_rwf_t)0x00000040) =20 +/* buffered IO that drops the cache after reading or writing data */ +#define RWF_DONTCACHE ((__force __kernel_rwf_t)0x00000080) + /* mask of flags supported by the kernel */ #define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\ - RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC) + RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC |\ + RWF_DONTCACHE) =20 #define PROCFS_IOCTL_MAGIC 'f' =20 --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A883621C9E7 for ; Fri, 20 Dec 2024 15:48:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709728; cv=none; b=Kd74gB+ewyYCyYyL2VYiP/r7JoTohwTLiLOsTdXD6PxNwHumDXc2MoCjslBzBflx6CNFe2e7FNX5N3WigEn593cSnUzsj0uwVEGiqQ3o6PXC/YcYOjYwxFpxjQz4bOMkfc1YQQyTLSL2IiXqbDKGRhskEdn992BpyjFKtL5ZxMM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709728; c=relaxed/simple; bh=sLgD25Nhzbud83QHSsVSAK9UDdaoG1oUsOEMQEZOwUg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pCx1684YSeAVOyuyA9Y3bqxSaLiQH0NWoBwz8Wem11UGe5dYeR9aYPwxh9u+OMp0sNgYMKegM92gmMCJAlMvdKCpXgMkov5rt7W6jtmLhXNYFdV1vCQh4m9rT6dwXhVNMiaFifIeJ41ZwxfxU2JSdrP6NDBYlwPkRD62jRBe/PU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=pTwLuQKR; arc=none smtp.client-ip=209.85.166.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="pTwLuQKR" Received: by mail-il1-f170.google.com with SMTP id e9e14a558f8ab-3a7dd54af4bso7540915ab.2 for ; Fri, 20 Dec 2024 07:48:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709726; x=1735314526; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ks2mp2gCJ77yF3ZCGIs6WN32l2VaUGiu+2pglDT6eiI=; b=pTwLuQKRV9/Q3bq6gpmLR//R6ym34L5m6oW8S1GwoSYILJ9wTgck8/oITVcsgXUSCD AOe+Yn9S9UTOb61p7PakQV49SOm5f5Zh0torgzxBycAnzBFSPkUQ3RyhwmIzR484EQ1l HXsJMt9DQeyDf2YN0a6iDqkWCNiyK1CshRFATM84Nc/bzVIdX1QUcuqMpjLZ+aIfVfpJ xJCFHfxNzb7PrHh/G4gxmKippkxR4AeAvi9FydrVqS8XMbLrBhgjvXBu4CjgKuZGVqIH FssnBsxyRamd41o2MX9tkArMzab4R8c3DI1zWxfyxpTwVVD2PXqSrEuyVCeZR6sst1iA 8/bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709726; x=1735314526; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ks2mp2gCJ77yF3ZCGIs6WN32l2VaUGiu+2pglDT6eiI=; b=Jme4J45cVG8xrpcdSSGaOoj0vxyKMqX/g/1ljA8TB4pAdV8/Ov/lLt+XwpsoYoS/SH lM4+imqEivsBt/v3ydawTJI1aBlOYb+JD5+W8y4/MEPZuAt9np3f5ReTXGUCIEmq55fk 9K5V3y1Bn3Eekj2rZCiK8LYa+BKazfnZAnDduzZVoyNiuReUzgjWHgdw9M+Seofa8uwb L8yDCYPinm3eGyA8Cu43HOUw9cV0LZlIDxdlzuCc+6SxtUOZUM0+op6WmLKTWbVgCVM1 RsTn1dldNYglypvZ87a+dnqrMvL53OJtgPrifpIZwLtTT8+FSlr3OCW+KY+7Gw3Mhr+b 7erg== X-Forwarded-Encrypted: i=1; AJvYcCUVzxGqPQ0PA2o+Yu32Uzl80pdtcBMLnHTm4OvUDRv9/lNHU0jKaJaXyaCPFvFDpcB/t3zdEhI7JEbbDgM=@vger.kernel.org X-Gm-Message-State: AOJu0YygCWBylw3W5DD2+R3DDkPvFBvZIPeHdA9FAJ0Ci2ub7SfpQdgK 3v7Kyi+IsK3lS1OwYJRiS4u71W/nnHqd9n4RmJvBX4Ku++lVhOy1HPiA1K7AIoY= X-Gm-Gg: ASbGncs8yOzYxF2mUxwFTLx9stN7AESKpsCY+AFskPAx4S3s5viBMnRyv6gi9byo33T 9a4PVDM08vCW5ChOafNL2/QGFGLZnnkyYXxumPHMaHHaNAmOFILJcRp7kqCIrYqjM3kUmdZPnI9 Ti76fxkJa5aI4LJuMfs+Sd+Nwq+9KxRBk5d2vjdj/yQSflRcUstCkWa1rRfIFUTWxTASssOqzKo iWQKuecJvT+R5bllTpdcJGOqcp3qoNXWZo21Nry/cXvk8OwniWZ7v/alqNE X-Google-Smtp-Source: AGHT+IG1Q5MpgKTbxJc/Yphg2SqFQXSKzJLSB4ruy07feq+/t9lmMQDAi0QHm0P3EjCYdC27xoRmeQ== X-Received: by 2002:a05:6e02:3201:b0:3a7:be5e:e22d with SMTP id e9e14a558f8ab-3c2d1aa3e86mr30848135ab.2.1734709725732; Fri, 20 Dec 2024 07:48:45 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:44 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 08/12] mm/filemap: add read support for RWF_DONTCACHE Date: Fri, 20 Dec 2024 08:47:46 -0700 Message-ID: <20241220154831.1086649-9-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add RWF_DONTCACHE as a read operation flag, which means that any data read wil be removed from the page cache upon completion. Uses the page cache to synchronize, and simply prunes folios that were instantiated when the operation completes. While it would be possible to use private pages for this, using the page cache as synchronization is handy for a variety of reasons: 1) No special truncate magic is needed 2) Async buffered reads need some place to serialize, using the page cache is a lot easier than writing extra code for this 3) The pruning cost is pretty reasonable and the code to support this is much simpler as a result. You can think of uncached buffered IO as being the much more attractive cousin of O_DIRECT - it has none of the restrictions of O_DIRECT. Yes, it will copy the data, but unlike regular buffered IO, it doesn't run into the unpredictability of the page cache in terms of reclaim. As an example, on a test box with 32 drives, reading them with buffered IO looks as follows: Reading bs 65536, uncached 0 1s: 145945MB/sec 2s: 158067MB/sec 3s: 157007MB/sec 4s: 148622MB/sec 5s: 118824MB/sec 6s: 70494MB/sec 7s: 41754MB/sec 8s: 90811MB/sec 9s: 92204MB/sec 10s: 95178MB/sec 11s: 95488MB/sec 12s: 95552MB/sec 13s: 96275MB/sec where it's quite easy to see where the page cache filled up, and performance went from good to erratic, and finally settles at a much lower rate. Looking at top while this is ongoing, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7535 root 20 0 267004 0 0 S 3199 0.0 8:40.65 uncached 3326 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd4 3327 root 20 0 0 0 0 R 100.0 0.0 0:17.22 kswapd5 3328 root 20 0 0 0 0 R 100.0 0.0 0:13.29 kswapd6 3332 root 20 0 0 0 0 R 100.0 0.0 0:11.11 kswapd10 3339 root 20 0 0 0 0 R 100.0 0.0 0:16.25 kswapd17 3348 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd26 3343 root 20 0 0 0 0 R 100.0 0.0 0:16.30 kswapd21 3344 root 20 0 0 0 0 R 100.0 0.0 0:11.92 kswapd22 3349 root 20 0 0 0 0 R 100.0 0.0 0:16.28 kswapd27 3352 root 20 0 0 0 0 R 99.7 0.0 0:11.89 kswapd30 3353 root 20 0 0 0 0 R 96.7 0.0 0:16.04 kswapd31 3329 root 20 0 0 0 0 R 96.4 0.0 0:11.41 kswapd7 3345 root 20 0 0 0 0 R 96.4 0.0 0:13.40 kswapd23 3330 root 20 0 0 0 0 S 91.1 0.0 0:08.28 kswapd8 3350 root 20 0 0 0 0 S 86.8 0.0 0:11.13 kswapd28 3325 root 20 0 0 0 0 S 76.3 0.0 0:07.43 kswapd3 3341 root 20 0 0 0 0 S 74.7 0.0 0:08.85 kswapd19 3334 root 20 0 0 0 0 S 71.7 0.0 0:10.04 kswapd12 3351 root 20 0 0 0 0 R 60.5 0.0 0:09.59 kswapd29 3323 root 20 0 0 0 0 R 57.6 0.0 0:11.50 kswapd1 [...] which is just showing a partial list of the 32 kswapd threads that are running mostly full tilt, burning ~28 full CPU cores. If the same test case is run with RWF_DONTCACHE set for the buffered read, the output looks as follows: Reading bs 65536, uncached 0 1s: 153144MB/sec 2s: 156760MB/sec 3s: 158110MB/sec 4s: 158009MB/sec 5s: 158043MB/sec 6s: 157638MB/sec 7s: 157999MB/sec 8s: 158024MB/sec 9s: 157764MB/sec 10s: 157477MB/sec 11s: 157417MB/sec 12s: 157455MB/sec 13s: 157233MB/sec 14s: 156692MB/sec which is just chugging along at ~155GB/sec of read performance. Looking at top, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7961 root 20 0 267004 0 0 S 3180 0.0 5:37.95 uncached 8024 axboe 20 0 14292 4096 0 R 1.0 0.0 0:00.13 top where just the test app is using CPU, no reclaim is taking place outside of the main thread. Not only is performance 65% better, it's also using half the CPU to do it. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++-- mm/swap.c | 2 ++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 220dc7c6e12f..dd563208d09d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2473,6 +2473,8 @@ static int filemap_create_folio(struct kiocb *iocb, s= truct folio_batch *fbatch) folio =3D filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; + if (iocb->ki_flags & IOCB_DONTCACHE) + __folio_set_dropbehind(folio); =20 /* * Protect against truncate / hole punch. Grabbing invalidate_lock @@ -2518,6 +2520,8 @@ static int filemap_readahead(struct kiocb *iocb, stru= ct file *file, =20 if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; + if (iocb->ki_flags & IOCB_DONTCACHE) + ractl.dropbehind =3D 1; page_cache_async_ra(&ractl, folio, last_index - folio->index); return 0; } @@ -2547,6 +2551,8 @@ static int filemap_get_pages(struct kiocb *iocb, size= _t count, return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags =3D memalloc_noio_save(); + if (iocb->ki_flags & IOCB_DONTCACHE) + ractl.dropbehind =3D 1; page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); @@ -2594,6 +2600,20 @@ static inline bool pos_same_folio(loff_t pos1, loff_= t pos2, struct folio *folio) return (pos1 >> shift =3D=3D pos2 >> shift); } =20 +static void filemap_end_dropbehind_read(struct address_space *mapping, + struct folio *folio) +{ + if (!folio_test_dropbehind(folio)) + return; + if (folio_test_writeback(folio) || folio_test_dirty(folio)) + return; + if (folio_trylock(folio)) { + if (folio_test_clear_dropbehind(folio)) + folio_unmap_invalidate(mapping, folio, 0); + folio_unlock(folio); + } +} + /** * filemap_read - Read data from the page cache. * @iocb: The iocb to read. @@ -2707,8 +2727,12 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_= iter *iter, } } put_folios: - for (i =3D 0; i < folio_batch_count(&fbatch); i++) - folio_put(fbatch.folios[i]); + for (i =3D 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio =3D fbatch.folios[i]; + + filemap_end_dropbehind_read(mapping, folio); + folio_put(folio); + } folio_batch_init(&fbatch); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); =20 diff --git a/mm/swap.c b/mm/swap.c index 10decd9dffa1..ba02bd5ba145 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -427,6 +427,8 @@ static void folio_inc_refs(struct folio *folio) */ void folio_mark_accessed(struct folio *folio) { + if (folio_test_dropbehind(folio)) + return; if (lru_gen_enabled()) { folio_inc_refs(folio); return; --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-il1-f177.google.com (mail-il1-f177.google.com [209.85.166.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFEA121C9FE for ; Fri, 20 Dec 2024 15:48:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709729; cv=none; b=cG+IFKwtIYM++1NRYT58BQvmEpR39SbVVR1irCbGsCIzY6iYaVbNji+8OzTHn+anWPk6xeXZX1jYakwXPEb8vRzVdjp8aw3OUmo619zHXiP74d3IAfhXoG54QxNUMxvWKFH0DZNHlAeo6M0ANk8zYPM13AQSKyt3BZMUQGwbyE4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709729; c=relaxed/simple; bh=0z+Su/j7sFm3LNscSJlbortbXvVAgYiyFzcEVovFWT0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Z3Y7Xgd+0em+PwmXAPZlouYtUli0jnHcVpRWfxTXn/Af6HaDlOBy/mi0YBW0WZOkmNZ7zHToNe3iuqAXRIbK09MSCrWW8lWsoIpcn8lnipUHFwFT0D9uDMOYtmG3WcZ/R+5H5quM+AXd448xhYpn4dnfEL3LUsSjh0qPN2QqXlU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=FPp6vWfm; arc=none smtp.client-ip=209.85.166.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="FPp6vWfm" Received: by mail-il1-f177.google.com with SMTP id e9e14a558f8ab-3a8160382d4so5926675ab.0 for ; Fri, 20 Dec 2024 07:48:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709727; x=1735314527; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CWUKIERhmBif0xELkyRs9dlqXC97dRJQR9fjviiscOo=; b=FPp6vWfmmwe9NAWIBJmCxLtVD/hNP3BfSRpL38o0SUhrItLpx26f4Yg2aryhVcVvoM oPpp3NOtj7OKWZxGCnfwcG9g9XNK06KVogtbQHCDg8aY2+KQWK+Gml3annJycylGeLWM VQ1kjdOYDuMMIT7W2aVyIxb1yTomtke8VSfksgOXvvkFxe9p1Wv/YGuGwkhvJgu9kDY3 kxOwEwqwnebKAvLbCMZrqSUwIskcxGhWX71jrH+qY6XTE8r7WH7lFQhuo7xqJStur45R FdMuoeSXbu+PxhnUHTXiYz73dqknNbkjP1X3wZUw36JxSQNonJKRdhryqdAgT7BlbPd4 hl6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709727; x=1735314527; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CWUKIERhmBif0xELkyRs9dlqXC97dRJQR9fjviiscOo=; b=IGX/D112Q4YJNg5MnrrCtv+3bKbzrvH2AtafMF8BhHSbRTtQ/ayBMQ4v+pxUoCOJCF IpOzftm4LMDIVQlcM0IYcnnOs3qy1h/o3YH4wqmz48j9iFcvHzBAja1z18/3hRbf64qG 0tsJTUQIHxpc0mb6q1Cd5MVoATb7hTpGAN8C+VBie8SSC6Af/fLEHIZditnmEfGj19uS pg+Dk7NFf7teQqENetrm43aufMyt2Fn1bZfnXpyEVAZvmMNMqDQ5QztGR5f5PRAme49H RXu85v4J1J4VyvsL4W6PDKDX0/Ew2zH45q9ajCN8xVwiArpcGA8NED5uQT6YM/FZQTQv 7zYw== X-Forwarded-Encrypted: i=1; AJvYcCXHgdtp5hga1SzB6FCOqwDCucLXKLDRkJL//op5iiw3cxnLV5BWC5DKvRtc+j6yULEQuj6t4u5Df8ZHuLs=@vger.kernel.org X-Gm-Message-State: AOJu0YzeRJMtZhc9FF43nzL49BRpVjMueF20sFL1OSG6zP5Dy+0UBYkO OqLseQNDAxHFNxLDaK4ESibX8mmki6ERsCuugp4yydvR7JgeidcccjCsjAj6cHI= X-Gm-Gg: ASbGnctP72iNiLx2/O1VkT3KisoJ24h+ojwPUw2Izy+Z2cKKqwyzcE2nr5rkpZA2I+S fGJ0uHyG3yj8IsK6KS+UFe4AnsWWijAjaV8DpuQMn/GhpNz/loJlKwqyitnkMqgcsRZGYGBYn2j 3Wgpi/dVfedLgJMHIdikwfsEuzD5e3V9c1SrrwUo6/F5ol6npZZxwoW1mnmCfYgT+BP7QKecWbD UJY1Sz5N82h0okauecjOrVyGIZIoAXPI32oNC+p0kAtaf5UN+YcYyCb9+Nn X-Google-Smtp-Source: AGHT+IGjD+wBfW2QpC5lojTx62UJZa1o0QwQiIlqK6Hx/4wukTE9KhfBEOmSehD69MzvwKGp6r7FTw== X-Received: by 2002:a05:6e02:522:b0:3a7:e0a5:aa98 with SMTP id e9e14a558f8ab-3c2d2d509b6mr25164805ab.13.1734709727201; Fri, 20 Dec 2024 07:48:47 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:46 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 09/12] mm/filemap: drop streaming/uncached pages when writeback completes Date: Fri, 20 Dec 2024 08:47:47 -0700 Message-ID: <20241220154831.1086649-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If the folio is marked as streaming, drop pages when writeback completes. Intended to be used with RWF_DONTCACHE, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index dd563208d09d..aa0b3af6533d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1599,6 +1599,27 @@ int folio_wait_private_2_killable(struct folio *foli= o) } EXPORT_SYMBOL(folio_wait_private_2_killable); =20 +/* + * If folio was marked as dropbehind, then pages should be dropped when wr= iteback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset dropbehind for that case and latter completions should inval= idate. + */ +static void folio_end_dropbehind_write(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_DONTCACHE writeback, + * but can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1609,6 +1630,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_dropbehind =3D false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); =20 /* @@ -1630,9 +1653,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (!folio_test_dirty(folio)) + folio_dropbehind =3D folio_test_clear_dropbehind(folio); if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_dropbehind) + folio_end_dropbehind_write(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback); --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-io1-f51.google.com (mail-io1-f51.google.com [209.85.166.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C27521CA0E for ; Fri, 20 Dec 2024 15:48:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709730; cv=none; b=WoAY0E6TAOmA7ADe/IlceJnvOoCrDsF6Qg6iK+H34lPZ+wwLyq2aZsTFodK1litODuYmiccCzvekEGXpYV54MugX8EgeDwb9W58nsVWVYkkFTYGtKEjeSPdcisOSKiCwPTvlnAyQ/+c+4qYE6NxwzrsLArr19JRm1R6DymK08fw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709730; c=relaxed/simple; bh=GaN0/oBzXsAiWqwBFn1n2Fc++lywThntHtcNeWBfcz4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EzuSwI0UoeS5resPVOC6S71/q6kCcLvI0UGHIjc0AcI+w9WKjnlKH8/peg8Z9f5equ9NS5/Y8IReFFAtJ6PGM4nkhGTl6ZQrvAMsbKqPlmZ+4dYxXV9/bJhh9AnqvfmGLktP3yoVRdXyyon/1BGEHWNejfqqQsUZ18GtzID1s8Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=ncr/MD9e; arc=none smtp.client-ip=209.85.166.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="ncr/MD9e" Received: by mail-io1-f51.google.com with SMTP id ca18e2360f4ac-844e10ef3cfso154741439f.2 for ; Fri, 20 Dec 2024 07:48:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709728; x=1735314528; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z4qQwDJE6vdlPLyJQIpmMrltr90cAGUkJZOMxstE6os=; b=ncr/MD9eAO4S+4YZqT2PvSRUEUfRH0XRRnl8I516z8jRSkZ31sND98yfS6V+rRRspf 5Kf/Ih7ECLvuydILFZhyqjSWG5a0bFHOQuclk7Aa5IkNAXA9StMT23TpMMzuiaE8Reop s+/SZtvp/g63rRqlCEC4yu/eGk5P5iygxFE7q1pQGkGw2K1J91g8DrjgTMMq4WDIRrSD D0Fv68GrKAqWRL0VI0jnvmWPRajAKViBpycTXrWNDBoIKkqs0jJq5fWbWXhb1zrIdMDv OaUM3NEj+wa0kS9lobiBPDHTorWqD9w2cbpGIhaDA4lYoZVx4sZFwChBBXDGWiJfmQyw Z1zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709728; x=1735314528; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z4qQwDJE6vdlPLyJQIpmMrltr90cAGUkJZOMxstE6os=; b=OKyt0bGMmLY4x4fMb1UC7+ffKQtWyCwtM98ccMIU2aoTrdF70yXY301U8sGacrFJn1 q6sRoXvxjfpAPS5GVHaHBeCZYhQrMwXt/RVUlCoH3pZVQKEOIp2mygFEk6hbn4rxQq77 JSdZKE+fr90MTJgBHbD2+I0gkJh248asZJ/fbyaLxX2qGb5p3g4Y+ihj7uz5ni51HmCl rEYIkv5ONainlUD8YLnOTj7obZcJmH8HDpsgm/XYNkvsHrJyeX7dT52nT1A+LXT5yBEa 8puTJEQMDQr+7b6HwJnPV+YXzmgNKODEHH0yuTnVqrlpp57OZhXQE7S4zovs9nM5ORbk WWcA== X-Forwarded-Encrypted: i=1; AJvYcCXStDBE6/kIYDBPAD8YM00cObWYSky8se6dvC2dThDoXwOztm3kkHIqxlogi/kxwtwrit7crv263j8A3yA=@vger.kernel.org X-Gm-Message-State: AOJu0Yz2Y4TtD18meN/iv83iH24CccYEMLcTxu51bsJ2HfMtSirIxyJG XKesS8U+xltAvtQjkJLnZsE/9o8fGzAhxD+L7hdAYYctNSXk4882OJQC2LqswQw= X-Gm-Gg: ASbGncvQVteIO8d7RE8lNcWVenDAJhxDxZSCv65ZHph+MYzzHIz4khFyD4q0xSiA5oz vff1fLviLIyV0UBf0HrDWwHtoqn/9SowhWQQ/JJ7Ms7vK97hlFiz78p/77OabpRWlTwo8Cku7Cm 6C1+vFrDXLNKkeGiP+l8dMWo6gFdgpMFNYYEamPER5D4FWXBrb4FAApu5EOP5fl8h1BM0ft8dYC cpnBFiETFdB0sqKIaDO2zFg9b+F4+Es1T9ux7jSFU2oy67DRCM1o4owR3/U X-Google-Smtp-Source: AGHT+IFGv05g9yMPyknBeMlKwprAW5pwl3vrHXnYcOynrc2BQ0O5hQEIENaNx6lgdvRHQ9LzhuK/ng== X-Received: by 2002:a05:6602:158e:b0:844:e06e:53c5 with SMTP id ca18e2360f4ac-8499e4fafe3mr298729939f.8.1734709728444; Fri, 20 Dec 2024 07:48:48 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:47 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 10/12] mm/filemap: add filemap_fdatawrite_range_kick() helper Date: Fri, 20 Dec 2024 08:47:48 -0700 Message-ID: <20241220154831.1086649-11-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Works like filemap_fdatawrite_range(), except it's a non-integrity data writeback and hence only starts writeback on the specified range. Will help facilitate generically starting uncached writeback from generic_write_sync(), as header dependencies preclude doing this inline from fs.h. Signed-off-by: Jens Axboe --- include/linux/fs.h | 2 ++ mm/filemap.c | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 6a838b5479a6..653b5efa3d3f 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2878,6 +2878,8 @@ extern int __must_check file_fdatawait_range(struct f= ile *file, loff_t lstart, extern int __must_check file_check_and_advance_wb_err(struct file *file); extern int __must_check file_write_and_wait_range(struct file *file, loff_t start, loff_t end); +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t st= art, + loff_t end); =20 static inline int file_write_and_wait(struct file *file) { diff --git a/mm/filemap.c b/mm/filemap.c index aa0b3af6533d..9842258ba343 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -449,6 +449,24 @@ int filemap_fdatawrite_range(struct address_space *map= ping, loff_t start, } EXPORT_SYMBOL(filemap_fdatawrite_range); =20 +/** + * filemap_fdatawrite_range_kick - start writeback on a range + * @mapping: target address_space + * @start: index to start writeback on + * @end: last (non-inclusive) index for writeback + * + * This is a non-integrity writeback helper, to start writing back folios + * for the indicated range. + * + * Return: %0 on success, negative error code otherwise. + */ +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t st= art, + loff_t end) +{ + return __filemap_fdatawrite_range(mapping, start, end, WB_SYNC_NONE); +} +EXPORT_SYMBOL_GPL(filemap_fdatawrite_range_kick); + /** * filemap_flush - mostly a non-blocking flush * @mapping: target address_space --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A971F21D597 for ; Fri, 20 Dec 2024 15:48:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709732; cv=none; b=QIChTy+Icp4UB1XK/50QrmEW1iMJgx1yshG1sHyK76SxXcfcWLKunW0POAltzVPteuDBjVAg8Ka3mm7cp1D9uEV3aVX9ubi5KanZ6N5NsMCvfNPssRCAHIZ1GS/JbR97YqvdPhLJ1opHdHJNyFbDCbAiALHq9DyDKeCjtbvv7E0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709732; c=relaxed/simple; bh=zE0xjm5rhzd2cUhrSxtogiQia/migrwaLvBNVMhxtxQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZyY0YIZgq04nRsDSrkK2XWbAJ+Hskb18qNWLulJw01bxUPgZ99Wb0KjAHbc0w70gm62itnLZIEEUO3oNqOLGfy/+hmrcgJwTRpC+/HblUaPleorS/uUqVJBQedhbubLxr6Yrq6+MnGUgiWoS4alfgoJ43Hp9pQ3F2u4fMNTIN3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=XpNPsYgT; arc=none smtp.client-ip=209.85.166.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="XpNPsYgT" Received: by mail-io1-f54.google.com with SMTP id ca18e2360f4ac-844e61f3902so168720439f.0 for ; Fri, 20 Dec 2024 07:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709730; x=1735314530; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ol/7y93UZDywrWGNYi5AP2kj1AIRWUpJ2p+2aggyqzo=; b=XpNPsYgTpipkSpVs7LrbeF1UL0my+iixYP3Cr4OQkBjATc8tdVJhLIT476q7wihOZV BkB5ehPy16t6rAFUTluHuYjes5Gaf51+jjOZiXYKMf6qtE5GjeUIijoOA/ODjm8NdrLn 0XZ8WETcvLRc+t5zb707wNiMIptKwv6c0v2+0NfazgNqK9co/x4RAnB4QT+wpvvb6GmZ 5Xmm3bZIPMrEsJLXaN8u6rDnXrVegjcLy1lcYFDlE0OsZzbhB7rYdv49n1kpQ6M7o3D6 I9fNEVK3Mz1L9I6osQSY4N8BMInx4CeTCYQySqVBUI3Ntk5Ho718230kzdmgVMGjbNjS JvhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709730; x=1735314530; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ol/7y93UZDywrWGNYi5AP2kj1AIRWUpJ2p+2aggyqzo=; b=dckDSzaV2Epw5VSQ4GAlGadELW1zBYSrzD+SvVBx9YgQTOvEx6rXzkoPS1NlaV9K26 VFjxB0Rn/W2twOkfUEOdUTBdzmHds+NrN/9Q9fmtT8d13U/6ETon8deAuIAvhZG2UjJn tiQYpOYCIuodGLf4w7y5tnBRwZhS8gZmDdfcdP+Ayp5TYI80KLq/Bqdd1MYPY4dldnn1 JzPPE9fU8f3H7hND7gl4BEcQxmHiTRlSmTke+r6mMdxrWsOlllRX5uoRg9OP+7prX+bS 8nYmh3BhkZOp9NAq75kgPceSWL7QoyJXPdp9EQHP0BBWZXDQMddS/2R+b6hZCBqmooEt yPKQ== X-Forwarded-Encrypted: i=1; AJvYcCW+35bndcss4mWrt7E9Zn0R/ikZTiwhRe7wvsuc4aq/OIpgjF/bsg805nBjVz72dftMrUvkz3wOPaE0nnk=@vger.kernel.org X-Gm-Message-State: AOJu0Yxo1vyHlUO5p7v5MX84Ne8/Q8fcTLZEIuiFm2oTxHahDWEvuCbl jW9fbY0Z0+p3eAjJpF27ym3sq8zK2r8idi3nuXO6ENGqZSeQsV4RHeC2Mxul69lHh9VGDJR702p c X-Gm-Gg: ASbGncuZe0Zp6CnK1knh+7tu5t9ySRO9IR+uQLV2UroFpXXYlPmLCqAyVJ0P7jTQt5/ DUeVTZAZsHTgC5D/cBRq+t3P5WQhH+nRmUNuF5IsxZ/WiwfsJXXOs5nYWDxHUmCuuZDuvD5CDiI QzWPtMN9TSFniwL58PogewLjFgp+W71qTfX13MPn/A1XkctNruzv7LAgi9sCD6PeyuEzSUu7nn1 HefDsiN3Lu29kKaZixotI4k+cGZw/5+WJ19AOqBcGrMSdCkQfW7AUMQ93R0 X-Google-Smtp-Source: AGHT+IHBkJtXw0hIyGWIXWPfQ49T4FfZrn/htFVCpySmbejAxdhZ6x6C0BhRXfs8Jh6Ol55mQ+pxBw== X-Received: by 2002:a05:6e02:17cf:b0:3a7:70a4:6872 with SMTP id e9e14a558f8ab-3c2d257934dmr36428035ab.9.1734709729627; Fri, 20 Dec 2024 07:48:49 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:48 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 11/12] mm: call filemap_fdatawrite_range_kick() after IOCB_DONTCACHE issue Date: Fri, 20 Dec 2024 08:47:49 -0700 Message-ID: <20241220154831.1086649-12-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a buffered write submitted with IOCB_DONTCACHE has been successfully submitted, call filemap_fdatawrite_range_kick() to kick off the IO. File systems call generic_write_sync() for any successful buffered write submission, hence add the logic here rather than needing to modify the file system. Signed-off-by: Jens Axboe --- include/linux/fs.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 653b5efa3d3f..58a618853574 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2912,6 +2912,11 @@ static inline ssize_t generic_write_sync(struct kioc= b *iocb, ssize_t count) (iocb->ki_flags & IOCB_SYNC) ? 0 : 1); if (ret) return ret; + } else if (iocb->ki_flags & IOCB_DONTCACHE) { + struct address_space *mapping =3D iocb->ki_filp->f_mapping; + + filemap_fdatawrite_range_kick(mapping, iocb->ki_pos, + iocb->ki_pos + count); } =20 return count; --=20 2.45.2 From nobody Sun Dec 14 13:38:53 2025 Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D57F721CA1C for ; Fri, 20 Dec 2024 15:48:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709733; cv=none; b=gwJIwFJHPpyrO1rpYPiYOEq7hR6pUJsocTY7pQp1XlMkGraOnY8zwEJV6RcYCgnjpJHlMBSav4JANjaPAVst8qU6/AnYcL33YlR9tghvmxxMxmZavyjnkcESoNTaSUWF0ihfqfg0KgEzjyo2Uq+1jlbfb1xQ3XVYLbJQKtJ6f7I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709733; c=relaxed/simple; bh=nMl1WIMQHt2bPQz6aG6MCeSjhSpxjtnQ2LzuXFwS3EE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FBgqOj7GID5cfWzYMwKs3spJ8MJIFWA9/j1fg8aGTOfs2N7wHW7smSZAe2fjfMGwxzgybewh6jYq3s7tjTORCQzqMETqf8DB5JkIflrDg54xhoim1YajYBctce5pGmocQ7fEeusu8t+18/J0437a4rcGUDQ+70CYKTp8ojtuJYE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=rQoXGEdg; arc=none smtp.client-ip=209.85.166.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="rQoXGEdg" Received: by mail-io1-f44.google.com with SMTP id ca18e2360f4ac-844de072603so177572739f.0 for ; Fri, 20 Dec 2024 07:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709731; x=1735314531; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=86GdD84SMshyNZnB2LgwxYI5vb/3z8g/QfHnuRLr9vU=; b=rQoXGEdgBAzash2UkPI/h6AWqTly2sPjVW9b7hA4HKvUEDfPRY9hME8xK+5OHk9dPB imdsvJiZzoYLSP1vgailaI0VzLusxo0vLieDcqeeQsKtokxL9Y8ZP3QaILODAfuWpe8R 1StjFxmLHfw26Thl9noyWzN8NYsirgyQkUFyhe2nQAs4waoVe01LDIQsJOzfrAd1+4b4 reZMrtNARzwTYhjFHd+s3dWJ7Wjg9zBWH1SZ//2Lk3JMTczFKAFrBRdlDnf8vYolCv7N kynphxojBktG2JkHYrhoDacuNHMOPY9LODRFxN4kdxcuG4IeO6nP5L9fpm6gAs/Jm4sz 3vZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709731; x=1735314531; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=86GdD84SMshyNZnB2LgwxYI5vb/3z8g/QfHnuRLr9vU=; b=cEBtdXSh4kePOa+DkAVYTnyg/Id/B2tEz3ua05/QVY1Tsbg0flj44WNf3/ll7403Mw Q91Xe8RS232NfLuQtabsSLBJfljAr9oaMDIw1teYLIg5b4XUmq0Yw1ubWc4ifY+R4wFv od8L+l3dynZkqlx7V6mejFX2civqOYId73ivbemoBqxgWQNwGM0xE8+AcMokA+AMkEIi AaPgU7suu4bZh9yn320TreDeTr9E9YhDgePLpdLxnsh0FxkdaJJosx8FEVcxf3TT2X0B PzbA1kr6B+DI9e3Su2BuOCZJt2tVKPMFo5MtAe14r5CbqacRC1uX5jjDxtaoXu1rxsAM OooA== X-Forwarded-Encrypted: i=1; AJvYcCVNCRYSDwStbARFrRMfyphl4sGffhrNr1LxYQ3JGwQpwD7MkN3rbibMn8nVhP0jXwXCou/w06Nd0bSRPXY=@vger.kernel.org X-Gm-Message-State: AOJu0YwxBnFdCc46Eibyhtfd/NwvW80USV0borkV0qcGN7pPEISeIKNz yWS0pqR0M6RMqJLd8kwXFXl8epqsumrSwqkr11Rivgl23sXDU38/So1aU4BsiUI= X-Gm-Gg: ASbGnct5j77TW53SyrptYZVx0txPqpTY9MNvyILw1swZcHq8mJbRsX3+bbthrBeacQ8 AJBImDLuKgC0RPv0ipbOw3Foh4OUGq9h6gvaJiXOtfNUiUPwTULQFlROo5nOf9xrSIgUUhSefUA YOepzfnxhK073FxpZ+5rYeLMy5WoC7WK9RPOwGUFPpuLM8YRV4ZRi0bTwWpYha6a3Ga0oMr8YVO qxrYDRjAF9sAJaRfp4y+Q1piKkBwIDX6fPyln0rWHTRqKRxp5uy4/Gpexs1 X-Google-Smtp-Source: AGHT+IENwEpB78XkZV/FFvO7zp/8wAKDMGCn+bUr8XfGQ9+IvUs//gy2KTAuIDXDGGYGOcgB+kegLA== X-Received: by 2002:a05:6e02:20c1:b0:3a7:e3e3:bd57 with SMTP id e9e14a558f8ab-3c2d533e943mr33235325ab.15.1734709731154; Fri, 20 Dec 2024 07:48:51 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:50 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 12/12] mm: add FGP_DONTCACHE folio creation flag Date: Fri, 20 Dec 2024 08:47:50 -0700 Message-ID: <20241220154831.1086649-13-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Callers can pass this in for uncached folio creation, in which case if a folio is newly created it gets marked as uncached. If a folio exists for this index and lookup succeeds, then it will not get marked as uncached. If an !uncached lookup finds a cached folio, clear the flag. For that case, there are competeting uncached and cached users of the folio, and it should not get pruned. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 5da4b6d42fae..64c6dada837e 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -710,6 +710,7 @@ pgoff_t page_cache_prev_miss(struct address_space *mapp= ing, * * %FGP_NOFS - __GFP_FS will get cleared in gfp. * * %FGP_NOWAIT - Don't block on the folio lock. * * %FGP_STABLE - Wait for the folio to be stable (finished writeback) + * * %FGP_DONTCACHE - Uncached buffered IO * * %FGP_WRITEBEGIN - The flags to use in a filesystem write_begin() * implementation. */ @@ -723,6 +724,7 @@ typedef unsigned int __bitwise fgf_t; #define FGP_NOWAIT ((__force fgf_t)0x00000020) #define FGP_FOR_MMAP ((__force fgf_t)0x00000040) #define FGP_STABLE ((__force fgf_t)0x00000080) +#define FGP_DONTCACHE ((__force fgf_t)0x00000100) #define FGF_GET_ORDER(fgf) (((__force unsigned)fgf) >> 26) /* top 6 bits */ =20 #define FGP_WRITEBEGIN (FGP_LOCK | FGP_WRITE | FGP_CREAT | FGP_STABLE) diff --git a/mm/filemap.c b/mm/filemap.c index 9842258ba343..68bdfff4117e 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2001,6 +2001,8 @@ struct folio *__filemap_get_folio(struct address_spac= e *mapping, pgoff_t index, /* Init accessed so avoid atomic mark_page_accessed later */ if (fgp_flags & FGP_ACCESSED) __folio_set_referenced(folio); + if (fgp_flags & FGP_DONTCACHE) + __folio_set_dropbehind(folio); =20 err =3D filemap_add_folio(mapping, folio, index, gfp); if (!err) @@ -2023,6 +2025,9 @@ struct folio *__filemap_get_folio(struct address_spac= e *mapping, pgoff_t index, =20 if (!folio) return ERR_PTR(-ENOENT); + /* not an uncached lookup, clear uncached if set */ + if (folio_test_dropbehind(folio) && !(fgp_flags & FGP_DONTCACHE)) + folio_clear_dropbehind(folio); return folio; } EXPORT_SYMBOL(__filemap_get_folio); --=20 2.45.2