From nobody Tue Dec 16 16:22:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2954CC77B78 for ; Fri, 21 Apr 2023 22:13:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233773AbjDUWN3 (ORCPT ); Fri, 21 Apr 2023 18:13:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233661AbjDUWNY (ORCPT ); Fri, 21 Apr 2023 18:13:24 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4A47198B for ; Fri, 21 Apr 2023 15:13:22 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-63d4595d60fso17238593b3a.0 for ; Fri, 21 Apr 2023 15:13:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1682115202; x=1684707202; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=u/AkHMm2p+EP+a6U0DbEFYOuvSo1i83VDzuubdv1yU4=; b=gWPYUh/qJS3hKUZOK+9h4MWoL8QjpqBENZF1/R77mtm+LeEDasMRdSowMwEpPxxS78 EP2q9aDYBJHna1dXO/vRjzhgMH3xXjajRzj+Gem3tqcctYYdBwGqaQ29Kog5RyRfy0yp UjVmuXQ923zKnlvwr5/ZYrwLOML+t858ymiAc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682115202; x=1684707202; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u/AkHMm2p+EP+a6U0DbEFYOuvSo1i83VDzuubdv1yU4=; b=lWmzPLLu/PxABxqwhwajJkCOEjaCcViOUC0CRsffN2GAvsPZQLCCrPFAod7PvrsZWt pzormJHHoIgObT0c8m4KYzAr5uMt21RgsLQ3k47dJji4bZ1m1NdGVCYRJch4T8h9ps6q eR6erWPTUQuiwjxJVntxkaghKnTl0LnE7otybmZlR4SMsUTjYgFYkKNR5ZAm/6ijeoOE G4fIeW+939P/l6ESDgOpJ01talhySElp5yCa4Lz8PZcQB9/TduWOqU5YvF/lFCZCJSW9 II8NgMwsCJ9LE777DgKyq+ym0sA31RUl4QW/v/qtaLrzja/fmcnMp3Op+RReP/D6rC0F UfRw== X-Gm-Message-State: AAQBX9dmQatpdlUcfpn3jSjrt23QxhkBx3meXmX4+Xe2EJ0DByjuRDlS dS/231yHQZK45eTCkDkhdy7OAQ== X-Google-Smtp-Source: AKy350bcHsIIY4k+lLMfZH1OxrXb0oxc9pTN/Ivx+wmbrGQA//e/eiBuiaIqWiD0BPfrEISD1XsY2A== X-Received: by 2002:a05:6a20:8426:b0:f0:8708:2341 with SMTP id c38-20020a056a20842600b000f087082341mr7837599pzd.26.1682115202268; Fri, 21 Apr 2023 15:13:22 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:87cc:9018:e569:4a27]) by smtp.gmail.com with ESMTPSA id y72-20020a62644b000000b006372791d708sm3424715pfb.104.2023.04.21.15.13.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 15:13:21 -0700 (PDT) From: Douglas Anderson To: Andrew Morton , Mel Gorman , Vlastimil Babka , Ying , Alexander Viro , Christian Brauner Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Douglas Anderson Subject: [PATCH v2 1/4] mm/filemap: Add folio_lock_timeout() Date: Fri, 21 Apr 2023 15:12:45 -0700 Message-ID: <20230421151135.v2.1.I2b71e11264c5c214bc59744b9e13e4c353bc5714@changeid> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230421221249.1616168-1-dianders@chromium.org> References: <20230421221249.1616168-1-dianders@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a variant of folio_lock() that can timeout. This is useful to avoid unbounded waits for the page lock in kcompactd. Signed-off-by: Douglas Anderson --- Changes in v2: - "Add folio_lock_timeout()" new for v2. include/linux/pagemap.h | 16 ++++++++++++++ mm/filemap.c | 47 +++++++++++++++++++++++++++++------------ 2 files changed, 50 insertions(+), 13 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 0acb8e1fb7af..0f3ef9f79300 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -892,6 +892,7 @@ static inline bool wake_page_match(struct wait_page_que= ue *wait_page, } =20 void __folio_lock(struct folio *folio); +int __folio_lock_timeout(struct folio *folio, long timeout); int __folio_lock_killable(struct folio *folio); bool __folio_lock_or_retry(struct folio *folio, struct mm_struct *mm, unsigned int flags); @@ -952,6 +953,21 @@ static inline void folio_lock(struct folio *folio) __folio_lock(folio); } =20 +/** + * folio_lock_timeout() - Lock this folio, with a timeout. + * @folio: The folio to lock. + * @timeout: The timeout in jiffies; %MAX_SCHEDULE_TIMEOUT means wait fore= ver. + * + * Return: 0 upon success; -ETIMEDOUT upon failure. + */ +static inline int folio_lock_timeout(struct folio *folio, long timeout) +{ + might_sleep(); + if (!folio_trylock(folio)) + return __folio_lock_timeout(folio, timeout); + return 0; +} + /** * lock_page() - Lock the folio containing this page. * @page: The page to lock. diff --git a/mm/filemap.c b/mm/filemap.c index 2723104cc06a..c6056ec41284 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1220,7 +1220,7 @@ static inline bool folio_trylock_flag(struct folio *f= olio, int bit_nr, int sysctl_page_lock_unfairness =3D 5; =20 static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, - int state, enum behavior behavior) + int state, enum behavior behavior, long timeout) { wait_queue_head_t *q =3D folio_waitqueue(folio); int unfairness =3D sysctl_page_lock_unfairness; @@ -1229,6 +1229,7 @@ static inline int folio_wait_bit_common(struct folio = *folio, int bit_nr, bool thrashing =3D false; unsigned long pflags; bool in_thrashing; + int err; =20 if (bit_nr =3D=3D PG_locked && !folio_test_uptodate(folio) && folio_test_workingset(folio)) { @@ -1295,10 +1296,13 @@ static inline int folio_wait_bit_common(struct foli= o *folio, int bit_nr, /* Loop until we've been woken or interrupted */ flags =3D smp_load_acquire(&wait->flags); if (!(flags & WQ_FLAG_WOKEN)) { + if (!timeout) + break; + if (signal_pending_state(state, current)) break; =20 - io_schedule(); + timeout =3D io_schedule_timeout(timeout); continue; } =20 @@ -1324,10 +1328,10 @@ static inline int folio_wait_bit_common(struct foli= o *folio, int bit_nr, } =20 /* - * If a signal happened, this 'finish_wait()' may remove the last - * waiter from the wait-queues, but the folio waiters bit will remain - * set. That's ok. The next wakeup will take care of it, and trying - * to do it here would be difficult and prone to races. + * If a signal/timeout happened, this 'finish_wait()' may remove the + * last waiter from the wait-queues, but the folio waiters bit will + * remain set. That's ok. The next wakeup will take care of it, and + * trying to do it here would be difficult and prone to races. */ finish_wait(q, wait); =20 @@ -1336,6 +1340,13 @@ static inline int folio_wait_bit_common(struct folio= *folio, int bit_nr, psi_memstall_leave(&pflags); } =20 + /* + * If we don't meet the success criteria below then we've got an error + * of some sort. Differentiate between the two error cases. If there's + * no time left it must have been a timeout. + */ + err =3D !timeout ? -ETIMEDOUT : -EINTR; + /* * NOTE! The wait->flags weren't stable until we've done the * 'finish_wait()', and we could have exited the loop above due @@ -1350,9 +1361,9 @@ static inline int folio_wait_bit_common(struct folio = *folio, int bit_nr, * waiter, but an exclusive one requires WQ_FLAG_DONE. */ if (behavior =3D=3D EXCLUSIVE) - return wait->flags & WQ_FLAG_DONE ? 0 : -EINTR; + return wait->flags & WQ_FLAG_DONE ? 0 : err; =20 - return wait->flags & WQ_FLAG_WOKEN ? 0 : -EINTR; + return wait->flags & WQ_FLAG_WOKEN ? 0 : err; } =20 #ifdef CONFIG_MIGRATION @@ -1442,13 +1453,15 @@ void migration_entry_wait_on_locked(swp_entry_t ent= ry, pte_t *ptep, =20 void folio_wait_bit(struct folio *folio, int bit_nr) { - folio_wait_bit_common(folio, bit_nr, TASK_UNINTERRUPTIBLE, SHARED); + folio_wait_bit_common(folio, bit_nr, TASK_UNINTERRUPTIBLE, SHARED, + MAX_SCHEDULE_TIMEOUT); } EXPORT_SYMBOL(folio_wait_bit); =20 int folio_wait_bit_killable(struct folio *folio, int bit_nr) { - return folio_wait_bit_common(folio, bit_nr, TASK_KILLABLE, SHARED); + return folio_wait_bit_common(folio, bit_nr, TASK_KILLABLE, SHARED, + MAX_SCHEDULE_TIMEOUT); } EXPORT_SYMBOL(folio_wait_bit_killable); =20 @@ -1467,7 +1480,8 @@ EXPORT_SYMBOL(folio_wait_bit_killable); */ static int folio_put_wait_locked(struct folio *folio, int state) { - return folio_wait_bit_common(folio, PG_locked, state, DROP); + return folio_wait_bit_common(folio, PG_locked, state, DROP, + MAX_SCHEDULE_TIMEOUT); } =20 /** @@ -1662,17 +1676,24 @@ EXPORT_SYMBOL_GPL(page_endio); void __folio_lock(struct folio *folio) { folio_wait_bit_common(folio, PG_locked, TASK_UNINTERRUPTIBLE, - EXCLUSIVE); + EXCLUSIVE, MAX_SCHEDULE_TIMEOUT); } EXPORT_SYMBOL(__folio_lock); =20 int __folio_lock_killable(struct folio *folio) { return folio_wait_bit_common(folio, PG_locked, TASK_KILLABLE, - EXCLUSIVE); + EXCLUSIVE, MAX_SCHEDULE_TIMEOUT); } EXPORT_SYMBOL_GPL(__folio_lock_killable); =20 +int __folio_lock_timeout(struct folio *folio, long timeout) +{ + return folio_wait_bit_common(folio, PG_locked, TASK_KILLABLE, + EXCLUSIVE, timeout); +} +EXPORT_SYMBOL_GPL(__folio_lock_timeout); + static int __folio_lock_async(struct folio *folio, struct wait_page_queue = *wait) { struct wait_queue_head *q =3D folio_waitqueue(folio); --=20 2.40.0.634.g4ca3ef3211-goog From nobody Tue Dec 16 16:22:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13B3EC77B76 for ; Fri, 21 Apr 2023 22:13:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233512AbjDUWNe (ORCPT ); Fri, 21 Apr 2023 18:13:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233769AbjDUWN2 (ORCPT ); Fri, 21 Apr 2023 18:13:28 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D01751BD8 for ; Fri, 21 Apr 2023 15:13:26 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-63b70f0b320so3675036b3a.1 for ; Fri, 21 Apr 2023 15:13:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1682115205; x=1684707205; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q5AkNpnnC51XtdBu29L1YvEWhYLiet10/hNPEtnlk4k=; b=G4PoB7ojaq05ZoUvu0bGhZVqKkk7zuFsPAQngKsh3YkvxtolTqu1sE5QemG7mwI9bC +lav55U9fdGkURLbrLnJpkwmFYqt+QPV1H3FtmqsyKCWezQ4MHw4sDrMDuq7BBrcauHj vF1OEuZHKgdoBCROSFAfcnRZzV8U5N65M2Rkk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682115205; x=1684707205; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q5AkNpnnC51XtdBu29L1YvEWhYLiet10/hNPEtnlk4k=; b=BnKNXc8X4BwgpevIwREcDFnxXtiTPxh6ojULDK3BuLY9sWf1a+8vbmy9G6QXW+j4Nl /ap0QrZo4MlFW9sJkawxSbTrkVN+Aq9drO3CEGeUSQRe71UzDZPXB9gkv6EoBfnvdqsy GANeOighzDVRkI/WqFZux8BAp5oMpGSGdQt8KXUTwy2O0ljOgReRhrQ1xsZsI5+bruJN B9jWqpYct1noeEh4EuMSEwy5sr4f17h/jO/JcddvwxxLFf6hz+ZOtmoOgdLBOeIrgSR4 GEFxONbxQ9KNVtL1Uk4GsR7c4akffSime6o3oag9iWtMf1MgqVbEnaEGEW6fTqGubi3V lFdQ== X-Gm-Message-State: AAQBX9c4WBSFSIQcbqIhjjLWLVj368CtgZKR6XMObDXzhJ88Y5dSfIHQ 1HpYk9q9iq16AwGztijZ4YjuOw== X-Google-Smtp-Source: AKy350Zf5b8NMr8PvbXnnwZSXrydNzrK4lJ4UsMs/fDtsJnXPPkwv+LxUNSlTubSDT1jS6eyThlL9A== X-Received: by 2002:a05:6a00:1393:b0:63f:15cc:9c1a with SMTP id t19-20020a056a00139300b0063f15cc9c1amr6134194pfg.1.1682115205182; Fri, 21 Apr 2023 15:13:25 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:87cc:9018:e569:4a27]) by smtp.gmail.com with ESMTPSA id y72-20020a62644b000000b006372791d708sm3424715pfb.104.2023.04.21.15.13.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 15:13:24 -0700 (PDT) From: Douglas Anderson To: Andrew Morton , Mel Gorman , Vlastimil Babka , Ying , Alexander Viro , Christian Brauner Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Douglas Anderson , Bart Van Assche , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Ingo Molnar , Jan Kara , Juri Lelli , Mel Gorman , Mikulas Patocka , Peter Zijlstra , "Ritesh Harjani (IBM)" , Steven Rostedt , Valentin Schneider , Vincent Guittot , Will Deacon , Zhang Yi Subject: [PATCH v2 2/4] buffer: Add lock_buffer_timeout() Date: Fri, 21 Apr 2023 15:12:46 -0700 Message-ID: <20230421151135.v2.2.Ie146eec4d41480ebeb15f0cfdfb3bc9095e4ebd9@changeid> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230421221249.1616168-1-dianders@chromium.org> References: <20230421221249.1616168-1-dianders@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a variant of lock_buffer() that can timeout. This is useful to avoid unbounded waits for the page lock in kcompactd. Signed-off-by: Douglas Anderson --- Changes in v2: - "Add lock_buffer_timeout()" new for v2. fs/buffer.c | 7 +++++++ include/linux/buffer_head.h | 10 ++++++++++ include/linux/wait_bit.h | 24 ++++++++++++++++++++++++ kernel/sched/wait_bit.c | 14 ++++++++++++++ 4 files changed, 55 insertions(+) diff --git a/fs/buffer.c b/fs/buffer.c index 9e1e2add541e..fcd19c270024 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -71,6 +71,13 @@ void __lock_buffer(struct buffer_head *bh) } EXPORT_SYMBOL(__lock_buffer); =20 +int __lock_buffer_timeout(struct buffer_head *bh, unsigned long timeout) +{ + return wait_on_bit_lock_io_timeout(&bh->b_state, BH_Lock, + TASK_UNINTERRUPTIBLE, timeout); +} +EXPORT_SYMBOL(__lock_buffer_timeout); + void unlock_buffer(struct buffer_head *bh) { clear_bit_unlock(BH_Lock, &bh->b_state); diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 8f14dca5fed7..2bae464f89d5 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -237,6 +237,7 @@ struct buffer_head *alloc_buffer_head(gfp_t gfp_flags); void free_buffer_head(struct buffer_head * bh); void unlock_buffer(struct buffer_head *bh); void __lock_buffer(struct buffer_head *bh); +int __lock_buffer_timeout(struct buffer_head *bh, unsigned long timeout); int sync_dirty_buffer(struct buffer_head *bh); int __sync_dirty_buffer(struct buffer_head *bh, blk_opf_t op_flags); void write_dirty_buffer(struct buffer_head *bh, blk_opf_t op_flags); @@ -400,6 +401,15 @@ static inline void lock_buffer(struct buffer_head *bh) __lock_buffer(bh); } =20 +static inline int lock_buffer_timeout(struct buffer_head *bh, + unsigned long timeout) +{ + might_sleep(); + if (!trylock_buffer(bh)) + return __lock_buffer_timeout(bh, timeout); + return 0; +} + static inline struct buffer_head *getblk_unmovable(struct block_device *bd= ev, sector_t block, unsigned size) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 7725b7579b78..33f0f60b1c8c 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -30,6 +30,7 @@ void wake_up_bit(void *word, int bit); int out_of_line_wait_on_bit(void *word, int, wait_bit_action_f *action, un= signed int mode); int out_of_line_wait_on_bit_timeout(void *word, int, wait_bit_action_f *ac= tion, unsigned int mode, unsigned long timeout); int out_of_line_wait_on_bit_lock(void *word, int, wait_bit_action_f *actio= n, unsigned int mode); +int out_of_line_wait_on_bit_lock_timeout(void *word, int, wait_bit_action_= f *action, unsigned int mode, unsigned long timeout); struct wait_queue_head *bit_waitqueue(void *word, int bit); extern void __init wait_bit_init(void); =20 @@ -208,6 +209,29 @@ wait_on_bit_lock_io(unsigned long *word, int bit, unsi= gned mode) return out_of_line_wait_on_bit_lock(word, bit, bit_wait_io, mode); } =20 +/** + * wait_on_bit_lock_io_timeout - wait_on_bit_lock_io() with a timeout + * @word: the word being waited on, a kernel virtual address + * @bit: the bit of the word being waited on + * @mode: the task state to sleep in + * @timeout: the timeout in jiffies; %MAX_SCHEDULE_TIMEOUT means wait fore= ver + * + * Returns zero if the bit was (eventually) found to be clear and was + * set. Returns non-zero if a timeout happened or a signal was delivered = to + * the process and the @mode allows that signal to wake the process. + */ +static inline int +wait_on_bit_lock_io_timeout(unsigned long *word, int bit, unsigned mode, + unsigned long timeout) +{ + might_sleep(); + if (!test_and_set_bit(bit, word)) + return 0; + return out_of_line_wait_on_bit_lock_timeout(word, bit, + bit_wait_io_timeout, + mode, timeout); +} + /** * wait_on_bit_lock_action - wait for a bit to be cleared, when wanting to= set it * @word: the word being waited on, a kernel virtual address diff --git a/kernel/sched/wait_bit.c b/kernel/sched/wait_bit.c index 0b1cd985dc27..629acd1c6c79 100644 --- a/kernel/sched/wait_bit.c +++ b/kernel/sched/wait_bit.c @@ -118,6 +118,20 @@ int __sched out_of_line_wait_on_bit_lock(void *word, i= nt bit, } EXPORT_SYMBOL(out_of_line_wait_on_bit_lock); =20 +int __sched out_of_line_wait_on_bit_lock_timeout(void *word, int bit, + wait_bit_action_f *action, + unsigned mode, + unsigned long timeout) +{ + struct wait_queue_head *wq_head =3D bit_waitqueue(word, bit); + DEFINE_WAIT_BIT(wq_entry, word, bit); + + wq_entry.key.timeout =3D jiffies + timeout; + + return __wait_on_bit_lock(wq_head, &wq_entry, action, mode); +} +EXPORT_SYMBOL(out_of_line_wait_on_bit_lock_timeout); + void __wake_up_bit(struct wait_queue_head *wq_head, void *word, int bit) { struct wait_bit_key key =3D __WAIT_BIT_KEY_INITIALIZER(word, bit); --=20 2.40.0.634.g4ca3ef3211-goog From nobody Tue Dec 16 16:22:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F53DC77B7E for ; Fri, 21 Apr 2023 22:13:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233661AbjDUWNp (ORCPT ); Fri, 21 Apr 2023 18:13:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233777AbjDUWN3 (ORCPT ); Fri, 21 Apr 2023 18:13:29 -0400 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FB001FF0 for ; Fri, 21 Apr 2023 15:13:28 -0700 (PDT) Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-63b4a64c72bso2269470b3a.0 for ; Fri, 21 Apr 2023 15:13:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1682115208; x=1684707208; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=u93zuMvcW9crJ3xx/ibodoPbhN86+1UwtbTmDpeZKPY=; b=AkSpP2B6MEWZUl0frTvsfsvLWFK8V1GAoZ0xgM/K0816fyASmzFYyukMou4sJkPvvQ x8m+0LZ0Kley490iLqLstsoKAwAWY4snWPkkl+IHwzXeFi5Wamq/VJvtD6G0iFq0oHV0 OAD4MLYCsMD/2Dj11rG/SZNFI7U3EbP8CQc1w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682115208; x=1684707208; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u93zuMvcW9crJ3xx/ibodoPbhN86+1UwtbTmDpeZKPY=; b=FRhlM0nVlmuXXyJQGkf1Ih4s90Pp4BtZOU16pR0mFCgX0NfKb2WtjEo2oX/u0w6+CA xGfjktjkNRBN1Xnbqzpehv/GrYEpy5d/iwkR82i7bxCMQ18Yv1XgbwXA7+GG8/6aMvKc HDYu9O0cvaRW9xL28LuKpg7q9r9fW6xD/Tag5P3rShYQMmcxuVO27SBSbGd1MRRTva0E 9SM61PhW8GONeAg7E3PwM2wv3mUbXgO8ZrioYnPJU4GHH6a9WXZ5D8AzOhwUAbEk8aH8 oMATWCSTrCvCU76W3zq8KDayaJzbkm1S3/2yY1PW+rBHHN+9eKOgbTXb1ObJpcBl2aF6 DdBw== X-Gm-Message-State: AAQBX9f24o8zIh9Y0sLZvJfyEFPzhvHgfqvbpY0EGAUrHpMAhkbIJ6CA K67/mKyq4zA/PC1Qr3zbBa3nNA== X-Google-Smtp-Source: AKy350YerkDyIlCdlYz+2bbDYDMPvivuf/02m3z67Um6oKR/8X1Rqic07nKJ4+CvFtUj6ZUAdN+jUw== X-Received: by 2002:a05:6a00:2196:b0:63a:5bcd:e580 with SMTP id h22-20020a056a00219600b0063a5bcde580mr8793160pfi.9.1682115207703; Fri, 21 Apr 2023 15:13:27 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:87cc:9018:e569:4a27]) by smtp.gmail.com with ESMTPSA id y72-20020a62644b000000b006372791d708sm3424715pfb.104.2023.04.21.15.13.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 15:13:26 -0700 (PDT) From: Douglas Anderson To: Andrew Morton , Mel Gorman , Vlastimil Babka , Ying , Alexander Viro , Christian Brauner Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Douglas Anderson Subject: [PATCH v2 3/4] migrate_pages: Don't wait forever locking pages in MIGRATE_SYNC_LIGHT Date: Fri, 21 Apr 2023 15:12:47 -0700 Message-ID: <20230421151135.v2.3.Ia86ccac02a303154a0b8bc60567e7a95d34c96d3@changeid> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230421221249.1616168-1-dianders@chromium.org> References: <20230421221249.1616168-1-dianders@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The MIGRATE_SYNC_LIGHT mode is intended to block for things that will finish quickly but not for things that will take a long time. Exactly how long is too long is not well defined, but waits of tens of milliseconds is likely non-ideal. Waiting on the folio lock in isolate_movable_page() is something that usually is pretty quick, but is not officially bounded. Nothing stops another process from holding a folio lock while doing an expensive operation. Having an unbounded wait like this is not within the design goals of MIGRATE_SYNC_LIGHT. When putting a Chromebook under memory pressure (opening over 90 tabs on a 4GB machine) it was fairly easy to see delays waiting for the lock of > 100 ms. While the laptop wasn't amazingly usable in this state, it was still limping along and this state isn't something artificial. Sometimes we simply end up with a lot of memory pressure. Putting the same Chromebook under memory pressure while it was running Android apps (though not stressing them) showed a much worse result (NOTE: this was on a older kernel but the codepaths here are similar). Android apps on ChromeOS currently run from a 128K-block, zlib-compressed, loopback-mounted squashfs disk. If we get a page fault from something backed by the squashfs filesystem we could end up holding a folio lock while reading enough from disk to decompress 128K (and then decompressing it using the somewhat slow zlib algorithms). That reading goes through the ext4 subsystem (because it's a loopback mount) before eventually ending up in the block subsystem. This extra jaunt adds extra overhead. Without much work I could see cases where we ended up blocked on a folio lock for over a second. With more more extreme memory pressure I could see up to 25 seconds. Let's bound the amount of time we can wait for the folio lock. The SYNC_LIGHT migration mode can already handle failure for things that are slow, so adding this timeout in is fairly straightforward. With this timeout, it can be seen that kcompactd can move on to more productive tasks if it's taking a long time to acquire a lock. NOTE: The reason I stated digging into this isn't because some benchmark had gone awry, but because we've received in-the-field crash reports where we have a hung task waiting on the page lock (which is the equivalent code path on old kernels). While the root cause of those crashes is likely unrelated and won't be fixed by this patch, analyzing those crash reports did point out this unbounded wait and it seemed like something good to fix. ALSO NOTE: the timeout mechanism used here uses "jiffies" and we also will retry up to 7 times. That doesn't give us much accuracy in specifying the timeout. On 1000 Hz machines we'll end up timing out in 7-14 ms. On 100 Hz machines we'll end up in 70-140 ms. Given that we don't have a strong definition of how long "too long" is, this is probably OK. Suggested-by: Mel Gorman Signed-off-by: Douglas Anderson --- Changes in v2: - Keep unbounded delay in "SYNC", delay with a timeout in "SYNC_LIGHT" mm/migrate.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index db3f154446af..60982df71a93 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -58,6 +58,23 @@ =20 #include "internal.h" =20 +/* Returns the schedule timeout for a non-async mode */ +static long timeout_for_mode(enum migrate_mode mode) +{ + /* + * We'll always return 1 jiffy as the timeout. Since all places using + * this timeout are in a retry loop this means that the maximum time + * we might block is actually NR_MAX_MIGRATE_SYNC_RETRY jiffies. + * If a jiffy is 1 ms that's 7 ms, though with the accuracy of the + * timeouts it often ends up more like 14 ms; if a jiffy is 10 ms + * that's 70-140 ms. + */ + if (mode =3D=3D MIGRATE_SYNC_LIGHT) + return 1; + + return MAX_SCHEDULE_TIMEOUT; +} + bool isolate_movable_page(struct page *page, isolate_mode_t mode) { struct folio *folio =3D folio_get_nontail_page(page); @@ -1162,7 +1179,8 @@ static int migrate_folio_unmap(new_page_t get_new_pag= e, free_page_t put_new_page if (current->flags & PF_MEMALLOC) goto out; =20 - folio_lock(src); + if (folio_lock_timeout(src, timeout_for_mode(mode))) + goto out; } locked =3D true; =20 --=20 2.40.0.634.g4ca3ef3211-goog From nobody Tue Dec 16 16:22:59 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 161C5C77B76 for ; Fri, 21 Apr 2023 22:13:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233793AbjDUWNt (ORCPT ); Fri, 21 Apr 2023 18:13:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51918 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233770AbjDUWNb (ORCPT ); Fri, 21 Apr 2023 18:13:31 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EBBA2D52 for ; Fri, 21 Apr 2023 15:13:30 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-63b62d2f729so2293385b3a.1 for ; Fri, 21 Apr 2023 15:13:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1682115210; x=1684707210; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Dc5jbER8wEc7OW3/IotC+kA2A3o7OZfeQaN33ZlBlP4=; b=Bj7hb2g1Ygk+uI1Xxfx+Lf8COV76humjsOT5sJKw8qFdCUzNXODMPPSgSmVEXfumLP gUzdVnED6ugNaAcG2SKbDhlEqm5/AA1x3lyiaYtvzKe1z+7yY9FeeWZjZ4j8ZGNYMvpT nSEalWU0KsSv7vdtph2qjlryw/U/r62x/nUxA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682115210; x=1684707210; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Dc5jbER8wEc7OW3/IotC+kA2A3o7OZfeQaN33ZlBlP4=; b=RoSt24tej8oDzoSVoTbST6U8qNRvgTNTF1zdg/0P5FTZI/nIj1x759PaxXp4fOhpyH lp7ESlKbz4wCqLDN531LwsHRqP5witMe7tW/SjGYKGaVSkd/DrP6UI39fBroj/mXKpwK WE+Sjs9egFwBNngdrRM8BJcrZnJgvCOxENfYtmTO66POo8Es2hRYCifgVuWPZHipu6NJ DfMA3csEKc2/Ud06H3/fchlZq/bK/mIUdov/PglKtme/Py8ahhoX7/l7dWDHyoxKhWlX PH2Rgq+1niyaVIZL3uPMxFaZiRcJDs1uJaFzuWhzXq6azM+LBEgKvxRo27i8k9UqxStT uVvQ== X-Gm-Message-State: AAQBX9f7oOtTOuVxR0zyF5GGSIipCNinqBQprHNBhrv3YFDMcNwSKYSg cE8tbgBD8kKsvhIILVLQw3+Pow== X-Google-Smtp-Source: AKy350aBr9TvmZeKHCBJSqIpG2B/nWcBgQOHXkcvQWceE4r02qrPTXFu3nXc3bs+0O5+8kUMfAUyog== X-Received: by 2002:a05:6a00:1a56:b0:63d:3789:733f with SMTP id h22-20020a056a001a5600b0063d3789733fmr8752575pfv.15.1682115209980; Fri, 21 Apr 2023 15:13:29 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:87cc:9018:e569:4a27]) by smtp.gmail.com with ESMTPSA id y72-20020a62644b000000b006372791d708sm3424715pfb.104.2023.04.21.15.13.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 15:13:29 -0700 (PDT) From: Douglas Anderson To: Andrew Morton , Mel Gorman , Vlastimil Babka , Ying , Alexander Viro , Christian Brauner Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Douglas Anderson Subject: [PATCH v2 4/4] migrate_pages: Don't wait forever locking buffers in MIGRATE_SYNC_LIGHT Date: Fri, 21 Apr 2023 15:12:48 -0700 Message-ID: <20230421151135.v2.4.Ic39f0b16516acf4af1ce5d923150f93ee85a9398@changeid> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230421221249.1616168-1-dianders@chromium.org> References: <20230421221249.1616168-1-dianders@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Just as talked about in the patch ("migrate_pages: Don't wait forever locking pages in MIGRATE_SYNC_LIGHT"), we don't really want unbounded waits when we're running in MIGRATE_SYNC_LIGHT mode. Waiting on the buffer lock is a second such unbounded wait. Let's put a timeout on it. While measurement didn't show this wait to be quite as bad as the one waiting for the folio lock, it could still be measured to be over a second in some cases. Signed-off-by: Douglas Anderson --- Changes in v2: - "Don't wait forever locking buffers in MIGRATE_SYNC_LIGHT" new for v2. mm/migrate.c | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 60982df71a93..97c93604eb4c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -715,25 +715,16 @@ static bool buffer_migrate_lock_buffers(struct buffer= _head *head, enum migrate_mode mode) { struct buffer_head *bh =3D head; + bool locked; =20 - /* Simple case, sync compaction */ - if (mode !=3D MIGRATE_ASYNC) { - do { - lock_buffer(bh); - bh =3D bh->b_this_page; - - } while (bh !=3D head); - - return true; - } - - /* async case, we cannot block on lock_buffer so use trylock_buffer */ do { - if (!trylock_buffer(bh)) { - /* - * We failed to lock the buffer and cannot stall in - * async migration. Release the taken locks - */ + if (mode =3D=3D MIGRATE_ASYNC) + locked =3D trylock_buffer(bh); + else + locked =3D !lock_buffer_timeout(bh, timeout_for_mode(mode)); + + if (!locked) { + /* We failed to lock the buffer. Release the taken locks. */ struct buffer_head *failed_bh =3D bh; bh =3D head; while (bh !=3D failed_bh) { --=20 2.40.0.634.g4ca3ef3211-goog