From nobody Sat Jul 25 02:11:07 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2A54370AF6 for ; Mon, 20 Jul 2026 14:29:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557767; cv=none; b=LDAL7E4EE068wJnITmLyLxOGUq1qZmNDOtTL3OV2O20e0zijESUF3Zwb7iKW72fpH9Vp9hcnCnd2BtOmyVpr7GngSGkRm5VPSLTW5ZXfsO4WRS+5RJJUxaR9N2W601mRKUa0cB9TyC8CvGJtue+ImE01fVu2gsUdN41VyXckglg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557767; c=relaxed/simple; bh=aE2MltOlwv6YZavHICNQnNzHkLg3/U9Rzn4kbh58NNY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IvWPIHd7ch++oiIsCDEFIYE7HiZJuAoloXxK27hPiDWotDqtbBSxDl3RzKYu00cdQXIYhgmwbKVS3q1O7vHWpRQI5M/RELoNeCLZ98vWObneYKkFS6TQG7Hwg8frjtaNDgZgeOeVA0ykf2TSnc9KB8sBUmAqKTIBS4OLsqrJDt0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 3368C3E21; Mon, 20 Jul 2026 14:29:18 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2D919779AB; Mon, 20 Jul 2026 14:29:17 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 4HvYB70wXmrKTQAAD6G6ig (envelope-from ); Mon, 20 Jul 2026 14:29:17 +0000 From: Pedro Falcato To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Pedro Falcato , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/8] mm/khugepaged: separate out windy folio logic from collapse_file Date: Mon, 20 Jul 2026 15:29:06 +0100 Message-ID: <20260720142913.846902-2-pfalcato@suse.de> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260720142913.846902-1-pfalcato@suse.de> References: <20260720142913.846902-1-pfalcato@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 3368C3E21 X-Rspamd-Action: no action Content-Type: text/plain; charset="utf-8" Separate out complex folio-related logic from the main collapse_file() loop, and introduce a new helper struct to help marshal arguments back and forth. Signed-off-by: Pedro Falcato --- mm/khugepaged.c | 189 ++++++++++++++++++++++++++++-------------------- 1 file changed, 112 insertions(+), 77 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 27e8f3077e80..d4de507ac001 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2218,6 +2218,92 @@ static void retract_page_tables(struct address_space= *mapping, pgoff_t pgoff) i_mmap_unlock_read(mapping); } =20 +struct collapse_file_state { + /* in-out parameter */ + struct folio *folio; + /* in parameters */ + struct address_space *mapping; + struct file *file; + struct xa_state *xas; + /* collapse end index */ + pgoff_t end; + unsigned int is_shmem : 1; +}; + +static enum scan_result prepare_collapse_file_folio(pgoff_t index, struct = collapse_file_state *state) +{ + struct address_space *mapping =3D state->mapping; + enum scan_result result =3D SCAN_SUCCEED; + const int is_shmem =3D state->is_shmem; + struct folio *folio =3D state->folio; + + if (is_shmem) { + if (xa_is_value(folio) || !folio_test_uptodate(folio)) { + xas_unlock_irq(state->xas); + /* swap in or instantiate fallocated page */ + if (shmem_get_folio(mapping->host, index, 0, + &folio, SGP_NOALLOC)) + result =3D SCAN_FAIL; + /* drain lru cache to help folio_isolate_lru() */ + lru_add_drain(); + goto xa_unlocked; + } else if (folio_trylock(folio)) { + folio_get(folio); + } else { + result =3D SCAN_PAGE_LOCK; + goto xa_locked; + } + } else { /* !is_shmem */ + if (!folio || xa_is_value(folio)) { + xas_unlock_irq(state->xas); + page_cache_sync_readahead(mapping, &state->file->f_ra, + state->file, index, + state->end - index); + /* drain lru cache to help folio_isolate_lru() */ + lru_add_drain(); + folio =3D filemap_lock_folio(mapping, index); + if (IS_ERR(folio)) + result =3D SCAN_FAIL; + goto xa_unlocked; + } else if (folio_test_dirty(folio)) { + /* + * This page is dirty because it hasn't + * been flushed since first write. + * + * Trigger async flush for read-only files and + * hope the writeback is done when khugepaged + * revisits this page. Writable files can have + * their folios dirty at any time; blindly + * flushing them would cause undesirable + * system-wide writeback. + * + * This is a one-off situation. We are not + * forcing writeback in loop. + */ + xas_unlock_irq(state->xas); + if (!inode_is_open_for_write(mapping->host)) + filemap_flush(mapping); + result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; + goto xa_unlocked; + } else if (folio_test_writeback(folio)) { + xas_unlock_irq(state->xas); + result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; + goto xa_unlocked; + } else if (folio_trylock(folio)) { + folio_get(folio); + } else { + result =3D SCAN_PAGE_LOCK; + goto xa_locked; + } + } + +xa_locked: + xas_unlock_irq(state->xas); +xa_unlocked: + state->folio =3D folio; + return result; +} + /** * collapse_file - collapse filemap/tmpfs/shmem pages into huge one. * @@ -2255,6 +2341,13 @@ static enum scan_result collapse_file(struct mm_stru= ct *mm, unsigned long addr, enum scan_result result =3D SCAN_SUCCEED; int nr_none =3D 0; bool is_shmem =3D shmem_file(file); + struct collapse_file_state state =3D { + .is_shmem =3D is_shmem, + .xas =3D &xas, + .mapping =3D mapping, + .file =3D file, + .end =3D end, + }; =20 /* * MADV_COLLAPSE ignores shmem huge config, so do not check shmem @@ -2298,87 +2391,29 @@ static enum scan_result collapse_file(struct mm_str= uct *mm, unsigned long addr, folio =3D xas_load(&xas); =20 VM_BUG_ON(index !=3D xas.xa_index); - if (is_shmem) { - if (!folio) { - /* - * Stop if extent has been truncated or - * hole-punched, and is now completely - * empty. - */ - if (index =3D=3D start) { - if (!xas_next_entry(&xas, end - 1)) { - result =3D SCAN_TRUNCATED; - goto xa_locked; - } - } - nr_none++; - index++; - continue; - } - - if (xa_is_value(folio) || !folio_test_uptodate(folio)) { - xas_unlock_irq(&xas); - /* swap in or instantiate fallocated page */ - if (shmem_get_folio(mapping->host, index, 0, - &folio, SGP_NOALLOC)) { - result =3D SCAN_FAIL; - goto xa_unlocked; - } - /* drain lru cache to help folio_isolate_lru() */ - lru_add_drain(); - } else if (folio_trylock(folio)) { - folio_get(folio); - xas_unlock_irq(&xas); - } else { - result =3D SCAN_PAGE_LOCK; - goto xa_locked; - } - } else { /* !is_shmem */ - if (!folio || xa_is_value(folio)) { - xas_unlock_irq(&xas); - page_cache_sync_readahead(mapping, &file->f_ra, - file, index, - end - index); - /* drain lru cache to help folio_isolate_lru() */ - lru_add_drain(); - folio =3D filemap_lock_folio(mapping, index); - if (IS_ERR(folio)) { - result =3D SCAN_FAIL; - goto xa_unlocked; + if (is_shmem && !folio) { + /* + * Stop if extent has been truncated or + * hole-punched, and is now completely + * empty. + */ + if (index =3D=3D start) { + if (!xas_next_entry(&xas, end - 1)) { + result =3D SCAN_TRUNCATED; + goto xa_locked; } - } else if (folio_test_dirty(folio)) { - /* - * This page is dirty because it hasn't - * been flushed since first write. - * - * Trigger async flush for read-only files and - * hope the writeback is done when khugepaged - * revisits this page. Writable files can have - * their folios dirty at any time; blindly - * flushing them would cause undesirable - * system-wide writeback. - * - * This is a one-off situation. We are not - * forcing writeback in loop. - */ - xas_unlock_irq(&xas); - if (!inode_is_open_for_write(mapping->host)) - filemap_flush(mapping); - result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; - goto xa_unlocked; - } else if (folio_test_writeback(folio)) { - xas_unlock_irq(&xas); - result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; - goto xa_unlocked; - } else if (folio_trylock(folio)) { - folio_get(folio); - xas_unlock_irq(&xas); - } else { - result =3D SCAN_PAGE_LOCK; - goto xa_locked; } + nr_none++; + index++; + continue; } =20 + /* At this point folio can be NULL, or a value. */ + state.folio =3D folio; + result =3D prepare_collapse_file_folio(index, &state); + folio =3D state.folio; + if (result !=3D SCAN_SUCCEED) + goto xa_unlocked; /* * The folio must be locked, so we can drop the i_pages lock * without racing with truncate. --=20 2.55.0 From nobody Sat Jul 25 02:11:07 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B67C3E556C for ; Mon, 20 Jul 2026 14:29:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557763; cv=none; b=XXE0AqB9xsI2UyfNbzNxz6c6brFDk3JFMnttme7QcOBoJe901f1foOqNwHvOgWFNxSbYb0w51J8iBAs1dm77Ca3yEn+ROB+QLrhLv1geZHdhMYYxmy6l9lxOKuohtmFb9hgCjfNWMJwVq20TBsukR8i5UvjYcAz/vN0gOz9y9HQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557763; c=relaxed/simple; bh=gyvSiIGuRLHd6UDTi0IQLcFYLN1XH4/td0kN6u5KqKQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h2YOAxvcZ0gT4MtbootGfdEW1Dn5e/G3w7Smek2clwqasl0HFquLA0chDhCF20wIQwR2GjE9oQTzwhgHEpAMhJCFc5pXh72pg0UtBJraTE/M3uTLQuuR4GhR0mwrPGO7tU/ra9DqTNdR/YB5AviZTcEE3WvCHzVMf9sjBlYMrI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5229D79FCB; Mon, 20 Jul 2026 14:29:19 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4B27D779AA; Mon, 20 Jul 2026 14:29:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id wC8BD74wXmrKTQAAD6G6ig (envelope-from ); Mon, 20 Jul 2026 14:29:18 +0000 From: Pedro Falcato To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Pedro Falcato , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/8] mm/khugepaged: factor out page cache folio reading Date: Mon, 20 Jul 2026 15:29:07 +0100 Message-ID: <20260720142913.846902-3-pfalcato@suse.de> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260720142913.846902-1-pfalcato@suse.de> References: <20260720142913.846902-1-pfalcato@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 5229D79FCB X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="utf-8" The two mechanisms (shmem, filemap) are quite similar and only change in subtleties. There is no need to have two variants. Factor this out into its own separate function, shared by both paths. Signed-off-by: Pedro Falcato --- mm/khugepaged.c | 67 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 44 insertions(+), 23 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index d4de507ac001..4cc6917a55c7 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2230,42 +2230,63 @@ struct collapse_file_state { unsigned int is_shmem : 1; }; =20 +static struct folio *collapse_read_folio(pgoff_t index, struct collapse_fi= le_state *state) +{ + const bool may_bring_uptodate =3D state->is_shmem; + struct folio *folio =3D state->folio; + + if (!folio || xa_is_value(folio) || + (may_bring_uptodate && !folio_test_uptodate(folio))) { + xas_unlock_irq(state->xas); + if (state->is_shmem) { + /* swap in or instantiate fallocated page */ + if (shmem_get_folio(state->mapping->host, index, 0, + &folio, SGP_NOALLOC)) + return NULL; + } else { + page_cache_sync_readahead(state->mapping, &state->file->f_ra, + state->file, index, + state->end - index); + folio =3D filemap_lock_folio(state->mapping, index); + if (IS_ERR(folio)) + return NULL; + } + } + + return folio; +} + static enum scan_result prepare_collapse_file_folio(pgoff_t index, struct = collapse_file_state *state) { struct address_space *mapping =3D state->mapping; enum scan_result result =3D SCAN_SUCCEED; const int is_shmem =3D state->is_shmem; - struct folio *folio =3D state->folio; + struct folio *folio; + + folio =3D collapse_read_folio(index, state); + if (!folio) + return SCAN_FAIL; + if (folio !=3D state->folio) { + /* + * collapse_read_folio() got a new folio. This folio is locked + * and ref'd up. Nothing tricky (dirty, writeback, etc) can + * happen, so bail now. But before that, drain the local LRU + * add batch. Otherwise, it's very possible folio_isolate_lru() + * will not succeed. + */ + lru_add_drain(); + goto xa_unlocked; + } =20 if (is_shmem) { - if (xa_is_value(folio) || !folio_test_uptodate(folio)) { - xas_unlock_irq(state->xas); - /* swap in or instantiate fallocated page */ - if (shmem_get_folio(mapping->host, index, 0, - &folio, SGP_NOALLOC)) - result =3D SCAN_FAIL; - /* drain lru cache to help folio_isolate_lru() */ - lru_add_drain(); - goto xa_unlocked; - } else if (folio_trylock(folio)) { + if (folio_trylock(folio)) { folio_get(folio); } else { result =3D SCAN_PAGE_LOCK; goto xa_locked; } } else { /* !is_shmem */ - if (!folio || xa_is_value(folio)) { - xas_unlock_irq(state->xas); - page_cache_sync_readahead(mapping, &state->file->f_ra, - state->file, index, - state->end - index); - /* drain lru cache to help folio_isolate_lru() */ - lru_add_drain(); - folio =3D filemap_lock_folio(mapping, index); - if (IS_ERR(folio)) - result =3D SCAN_FAIL; - goto xa_unlocked; - } else if (folio_test_dirty(folio)) { + if (folio_test_dirty(folio)) { /* * This page is dirty because it hasn't * been flushed since first write. --=20 2.55.0 From nobody Sat Jul 25 02:11:07 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C692A42B732 for ; Mon, 20 Jul 2026 14:29:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557769; cv=none; b=QJBsK06QlCu2z4fRYUwvgGiy2ytIL10HwsaR/VvSETO3607a4qqwzI9INkf900g0nRdmXZpQDaeMFaWWfanLlkEWC5Sow66C3qOFbyMpUmX5xnGhQqqG3zEQuCRFr6fQq+IlJMWWAyaJs3IaN9l8OZRQ+/ilHVAKf5n6AAexc0Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557769; c=relaxed/simple; bh=lvDcDspryRvBxxLCYm2W4oriv+PudjLNC0uD4dOVw6A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tyGn1C0zPRKjLk8WwU72l3SmfBmQeP7juwFxiqyR8FUqXGrv/JtdWQJPy1ND4V6ZHwY0Mu2pmsDpNhTJl7MxWbY3zDGG7Vfg1K/qD5IkAhjwUSyKDzU+q25jYcxqVhzQiIDrNglENQiQjIsC7cJWxhrH+rSKjlx9XfLo9HBSOVk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=r40uyrRo; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=y4Jm4JzQ; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=r40uyrRo; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=y4Jm4JzQ; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="r40uyrRo"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="y4Jm4JzQ"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="r40uyrRo"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="y4Jm4JzQ" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 671B679FCD; Mon, 20 Jul 2026 14:29:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557760; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tOHlMuSJ0FI7SQpr2VHqCuu+f1bO4v5hqudfd2KAok4=; b=r40uyrRoiqYJ5w6S9nx2T3R+WF0RHP5QmYJqEeckMXPikZ6+AZKWmnj68J3Y0QIgvXOwEC El8mJyYZ8SY6PLgH1k+vUVZK6ay/sG5j1i5QRQ00ZCWo4EnkKnIKKy8tp0snW6byxdiOZN Z6uH16Rb/t596hIIkccQu4M9XmD2X+A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557760; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tOHlMuSJ0FI7SQpr2VHqCuu+f1bO4v5hqudfd2KAok4=; b=y4Jm4JzQQMIayljn52lr6HptDGAg2ypunSHsK2VmiluQ0ISRluZh4giM0O4P5f8ldfzdbx vGFugCDgeOSecFDg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557760; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tOHlMuSJ0FI7SQpr2VHqCuu+f1bO4v5hqudfd2KAok4=; b=r40uyrRoiqYJ5w6S9nx2T3R+WF0RHP5QmYJqEeckMXPikZ6+AZKWmnj68J3Y0QIgvXOwEC El8mJyYZ8SY6PLgH1k+vUVZK6ay/sG5j1i5QRQ00ZCWo4EnkKnIKKy8tp0snW6byxdiOZN Z6uH16Rb/t596hIIkccQu4M9XmD2X+A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557760; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tOHlMuSJ0FI7SQpr2VHqCuu+f1bO4v5hqudfd2KAok4=; b=y4Jm4JzQQMIayljn52lr6HptDGAg2ypunSHsK2VmiluQ0ISRluZh4giM0O4P5f8ldfzdbx vGFugCDgeOSecFDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 69BCE779AA; Mon, 20 Jul 2026 14:29:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WFyAFr8wXmrKTQAAD6G6ig (envelope-from ); Mon, 20 Jul 2026 14:29:19 +0000 From: Pedro Falcato To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Pedro Falcato , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/8] mm/khugepaged: factor out and simplify dirty/writeback handling Date: Mon, 20 Jul 2026 15:29:08 +0100 Message-ID: <20260720142913.846902-4-pfalcato@suse.de> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260720142913.846902-1-pfalcato@suse.de> References: <20260720142913.846902-1-pfalcato@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.992]; MIME_GOOD(-0.10)[text/plain]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[15]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; R_RATELIMIT(0.00)[to_ip_from(RL8d8xedm6iu8o66torxsk6bwd)]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[] X-Spam-Level: X-Spam-Score: -6.80 Content-Type: text/plain; charset="utf-8" Dirty/writeback folios can easily share most of the branch. These checks are still gated under !is_shmem due to shmem not really having issues with dirty folios. While here, reflow the comment. Signed-off-by: Pedro Falcato --- mm/khugepaged.c | 46 ++++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 22 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 4cc6917a55c7..ed5d23317177 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2262,6 +2262,7 @@ static enum scan_result prepare_collapse_file_folio(p= goff_t index, struct collap enum scan_result result =3D SCAN_SUCCEED; const int is_shmem =3D state->is_shmem; struct folio *folio; + bool dirty; =20 folio =3D collapse_read_folio(index, state); if (!folio) @@ -2278,39 +2279,40 @@ static enum scan_result prepare_collapse_file_folio= (pgoff_t index, struct collap goto xa_unlocked; } =20 - if (is_shmem) { - if (folio_trylock(folio)) { - folio_get(folio); - } else { - result =3D SCAN_PAGE_LOCK; - goto xa_locked; - } - } else { /* !is_shmem */ - if (folio_test_dirty(folio)) { + if (!is_shmem) { + dirty =3D folio_test_dirty(folio); + if (dirty || folio_test_writeback(folio)) { /* - * This page is dirty because it hasn't - * been flushed since first write. + * This folio is either dirty or under writeback. + * khugepaged cannot operate on such folios. * - * Trigger async flush for read-only files and - * hope the writeback is done when khugepaged - * revisits this page. Writable files can have - * their folios dirty at any time; blindly - * flushing them would cause undesirable + * For dirty folios, trigger async flush for + * read-only files and hope the writeback is done + * when khugepaged revisits this page. Writable + * files can have their folios dirty at any time; + * blindly flushing them would cause undesirable * system-wide writeback. * * This is a one-off situation. We are not * forcing writeback in loop. */ xas_unlock_irq(state->xas); - if (!inode_is_open_for_write(mapping->host)) + if (dirty && !inode_is_open_for_write(mapping->host)) filemap_flush(mapping); result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; goto xa_unlocked; - } else if (folio_test_writeback(folio)) { - xas_unlock_irq(state->xas); - result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; - goto xa_unlocked; - } else if (folio_trylock(folio)) { + } + } + + if (is_shmem) { + if (folio_trylock(folio)) { + folio_get(folio); + } else { + result =3D SCAN_PAGE_LOCK; + goto xa_locked; + } + } else { /* !is_shmem */ + if (folio_trylock(folio)) { folio_get(folio); } else { result =3D SCAN_PAGE_LOCK; --=20 2.55.0 From nobody Sat Jul 25 02:11:07 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3E5842A7B1 for ; Mon, 20 Jul 2026 14:29:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557775; cv=none; b=sJE+mbfbHo/pjzwRWZuovyjLfcT3giXOI7vWm6MEu76g7VPssDxMvwQjTnwYAes8AtyZrzw5WWvsfF85c4Dp1ibLZqUtbiyx+FhNS0aXqr3B1TweQni6i/AUWXiELQiLqqixc9HwBtQzXdX8zigP0h6JVkDacGbQ+lwiVONvFf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557775; c=relaxed/simple; bh=RW1rEmwRZWZWBRI317Ab8uK7CcPVT5OUBLsJ1QvtzPo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GS0vpJF9w9bCoeOLbtbcudB6PUtpalP3bnp6utv9jXZjWSp6BRvr1F56QD0qw7OVOB4uInFIUlZxwRF7g04bMogbpaEpbZngo/f3dooWFq8VbzeDzlbxyY+lT8Gz6Uj/zEcMkI9sy0JAlwqKnUO7ointSWs/+UF5VJ4E0tJXU8o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=ISgXyunc; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=P4vFvQAE; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=ISgXyunc; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=P4vFvQAE; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="ISgXyunc"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="P4vFvQAE"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="ISgXyunc"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="P4vFvQAE" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8451079FCF; Mon, 20 Jul 2026 14:29:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557761; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mtAUlclE6KOqACLOJXzP3lbBnDD0ozmS9uuandpFwC4=; b=ISgXyuncxsOmNKEHpWbgZmlvEcLQ7a+uufeHsKQudQufx67G8VWSVaD2ybnvNXc6BLZdGM nvA/dmS03+4HW23nDhogiUsTMuN5AhKbB00uz39J/GLh8ICMv4ozeevk+raa9DCBqoQg2x rHHgYWqXCwo0bYskTomMc21Pht3cNK0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557761; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mtAUlclE6KOqACLOJXzP3lbBnDD0ozmS9uuandpFwC4=; b=P4vFvQAEX0x+BYfyrYZe4HJzHZZgHIfdqyAgELWzHgq7047zE2Ff67MCsXxhfv3NKKdZv0 EYQqZk9cYQb4AFAg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557761; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mtAUlclE6KOqACLOJXzP3lbBnDD0ozmS9uuandpFwC4=; b=ISgXyuncxsOmNKEHpWbgZmlvEcLQ7a+uufeHsKQudQufx67G8VWSVaD2ybnvNXc6BLZdGM nvA/dmS03+4HW23nDhogiUsTMuN5AhKbB00uz39J/GLh8ICMv4ozeevk+raa9DCBqoQg2x rHHgYWqXCwo0bYskTomMc21Pht3cNK0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557761; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mtAUlclE6KOqACLOJXzP3lbBnDD0ozmS9uuandpFwC4=; b=P4vFvQAEX0x+BYfyrYZe4HJzHZZgHIfdqyAgELWzHgq7047zE2Ff67MCsXxhfv3NKKdZv0 EYQqZk9cYQb4AFAg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 80362779AA; Mon, 20 Jul 2026 14:29:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id COzlG8AwXmrKTQAAD6G6ig (envelope-from ); Mon, 20 Jul 2026 14:29:20 +0000 From: Pedro Falcato To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Pedro Falcato , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/8] mm/khugepaged: simplify prepare folio locking and exit paths Date: Mon, 20 Jul 2026 15:29:09 +0100 Message-ID: <20260720142913.846902-5-pfalcato@suse.de> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260720142913.846902-1-pfalcato@suse.de> References: <20260720142913.846902-1-pfalcato@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.992]; MIME_GOOD(-0.10)[text/plain]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[15]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; R_RATELIMIT(0.00)[to_ip_from(RL8d8xedm6iu8o66torxsk6bwd)]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[] X-Spam-Level: X-Spam-Score: -6.80 Content-Type: text/plain; charset="utf-8" folio_trylock() + folio_get() do the exact same thing on both is_shmem and !is_shmem paths. Unify them into one path. At the end of the day, there are two large invariants for the function: - xarray lock is not held at exit. Either it was dropped on error, or dropped on success/IO. - success always means a locked-and-ref'd-up folio. After this transformation, it's trivial to notice how simple the error paths are, so do away with the gotos and hoist some code. Signed-off-by: Pedro Falcato --- mm/khugepaged.c | 36 +++++++++++++----------------------- 1 file changed, 13 insertions(+), 23 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ed5d23317177..d170b64378e6 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2259,8 +2259,8 @@ static struct folio *collapse_read_folio(pgoff_t inde= x, struct collapse_file_sta static enum scan_result prepare_collapse_file_folio(pgoff_t index, struct = collapse_file_state *state) { struct address_space *mapping =3D state->mapping; - enum scan_result result =3D SCAN_SUCCEED; const int is_shmem =3D state->is_shmem; + enum scan_result ret =3D SCAN_SUCCEED; struct folio *folio; bool dirty; =20 @@ -2276,7 +2276,8 @@ static enum scan_result prepare_collapse_file_folio(p= goff_t index, struct collap * will not succeed. */ lru_add_drain(); - goto xa_unlocked; + state->folio =3D folio; + return SCAN_SUCCEED; } =20 if (!is_shmem) { @@ -2299,32 +2300,21 @@ static enum scan_result prepare_collapse_file_folio= (pgoff_t index, struct collap xas_unlock_irq(state->xas); if (dirty && !inode_is_open_for_write(mapping->host)) filemap_flush(mapping); - result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; - goto xa_unlocked; + return SCAN_PAGE_DIRTY_OR_WRITEBACK; } } =20 - if (is_shmem) { - if (folio_trylock(folio)) { - folio_get(folio); - } else { - result =3D SCAN_PAGE_LOCK; - goto xa_locked; - } - } else { /* !is_shmem */ - if (folio_trylock(folio)) { - folio_get(folio); - } else { - result =3D SCAN_PAGE_LOCK; - goto xa_locked; - } - } =20 -xa_locked: + /* + * Note: trylock + simple folio_get() is safe here due to the i_pages + * lock being held; this excludes truncation happening in parallel. + */ + if (!folio_trylock(folio)) + ret =3D SCAN_PAGE_LOCK; + else + folio_get(folio); xas_unlock_irq(state->xas); -xa_unlocked: - state->folio =3D folio; - return result; + return ret; } =20 /** --=20 2.55.0 From nobody Sat Jul 25 02:11:07 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CDE842DFEA for ; Mon, 20 Jul 2026 14:29:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557781; cv=none; b=pMVifG4aHFX1egPvvrVlVsazUUID5Z8mK1zws0hIW15imnVJX/WMaK6fT/aWlqplXju2ZMyMkVARgTapukpgmwwHNV2qrixHa0AMn4Z49z1rQ5uV/lZiNXYK8A+YUHFf2esHAFQwu8F0cWZAJP9J15Pqrgm35TdvNPpOStaL3hk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557781; c=relaxed/simple; bh=jBaElGI7/ZUem0tlEQYzHl+JM0gXbRr3gGXOCdSfUdw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=E/u8171sQC9HRrtcXGDo5TECQzBz6hpgh014EzDHH/8NbAiUF3hTGO6TVEctto0K7IRxgedG2os6jNlCkRewUGb2bu5SVXt8TPkwRtS0/S9bTmh1Mc/xCvjkoEE7/EiIUyBsONKIGzSaRm6cT9lGjM/vWCyEYBD68WAbZHqOICQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=CXeOidXW; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=In7xaeto; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=CXeOidXW; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=In7xaeto; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="CXeOidXW"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="In7xaeto"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="CXeOidXW"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="In7xaeto" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A1F7E79FD1; Mon, 20 Jul 2026 14:29:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557762; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wMzkkAq1z2Y80PcJnVoa9xD0YS9/V3IeR5uFiuqgrfM=; b=CXeOidXWIbvLnTUq+SygjrSElZGh74hLlnQs02aGIQzCcp+CtKjgvYqU7fAA2oZG7D8RIH 74CJOB1GIKVgKf2bMgMwZwvDHdjckJe4kA6mFNrbHpkSh43uYTgPP/3Igmwon/bW/u89eI acIwvWc38EfYNBZUO9fEfn2ZIUj+bpI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557762; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wMzkkAq1z2Y80PcJnVoa9xD0YS9/V3IeR5uFiuqgrfM=; b=In7xaetoG+Iy1MF6i3VsDFgW8+ruD5cI7ODn8GIJ6Xk37B2EWcPFWmvTR7+1zx2rzWbx8M fXGMc1nfoULf8YCw== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557762; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wMzkkAq1z2Y80PcJnVoa9xD0YS9/V3IeR5uFiuqgrfM=; b=CXeOidXWIbvLnTUq+SygjrSElZGh74hLlnQs02aGIQzCcp+CtKjgvYqU7fAA2oZG7D8RIH 74CJOB1GIKVgKf2bMgMwZwvDHdjckJe4kA6mFNrbHpkSh43uYTgPP/3Igmwon/bW/u89eI acIwvWc38EfYNBZUO9fEfn2ZIUj+bpI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557762; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wMzkkAq1z2Y80PcJnVoa9xD0YS9/V3IeR5uFiuqgrfM=; b=In7xaetoG+Iy1MF6i3VsDFgW8+ruD5cI7ODn8GIJ6Xk37B2EWcPFWmvTR7+1zx2rzWbx8M fXGMc1nfoULf8YCw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9D51E779AA; Mon, 20 Jul 2026 14:29:21 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QIsWI8EwXmrKTQAAD6G6ig (envelope-from ); Mon, 20 Jul 2026 14:29:21 +0000 From: Pedro Falcato To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Pedro Falcato , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/8] mm/khugepaged: add kerneldoc to prepare_collapse_file_folio() Date: Mon, 20 Jul 2026 15:29:10 +0100 Message-ID: <20260720142913.846902-6-pfalcato@suse.de> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260720142913.846902-1-pfalcato@suse.de> References: <20260720142913.846902-1-pfalcato@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.992]; MIME_GOOD(-0.10)[text/plain]; FUZZY_RATELIMITED(0.00)[rspamd.com]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_TWELVE(0.00)[15]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; R_RATELIMIT(0.00)[to_ip_from(RL8d8xedm6iu8o66torxsk6bwd)]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid] X-Spam-Level: Content-Type: text/plain; charset="utf-8" The input-output expectations are not entirely clear, thus document it. Signed-off-by: Pedro Falcato --- mm/khugepaged.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index d170b64378e6..db1f765b6ead 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2256,6 +2256,21 @@ static struct folio *collapse_read_folio(pgoff_t ind= ex, struct collapse_file_sta return folio; } =20 + +/** + * prepare_collapse_file_folio() - Prepare the collapsing of a single file= folio + * @index: folio index in the page cache + * @state: helper struct of state being passed back and forth. + * + * prepare_collapse_file_folio() helps prepare the THP collapsing of file = folios + * (both shmem and !shmem), including the reading of folios, initial tests= for + * dirty/writeback, locking, etc. + * + * Context: Expects @state->mapping->i_pages->xa_lock to be held. Releases= the same xa_lock. + * On success, returns a refcounted and locked folio in @state->f= olio (that may + * not be the same that was passed in). + * Return: Appropriate scan_result value (on success, SCAN_SUCCEED). + */ static enum scan_result prepare_collapse_file_folio(pgoff_t index, struct = collapse_file_state *state) { struct address_space *mapping =3D state->mapping; --=20 2.55.0 From nobody Sat Jul 25 02:11:07 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DA6D3E5EDA for ; Mon, 20 Jul 2026 14:29:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557773; cv=none; b=g68eXt8fLYhMzNQO6IDKoumwm5c1bS9z1vgjwPXvDL0Snq8ZW0Bi9BW5HpUbPYCCl2i/WlI+ikOmowe10u584Udf9RJ5jOtiq/rOVf7wsMLXCT+pyyDtYPAQUrQ/LLEGOBCi2cgD8l/WxlmYcRXR4RCflVfHCvW8LMtjzp6DvZs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557773; c=relaxed/simple; bh=pAlioVGcAe/e3f7WyV06lzak5Q7e6yUDJt0T3QvBv2A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iwfXQ/3k6VXwJNiovAH7MCUYhDxQpOX2Bh1k+R4HMK9NltYboxbmbsdHpp/nFleUHBcy3Itzn+N1cS+lq/a0bdF4N8rt3M2toDTUC6sKOjnUiiM96ucUwgaYCdYDgZdVEJUARUmt3sGSqdmGxkLXkLWp0kPg9yI4Rc4/tyVoqO8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id B95653E18; Mon, 20 Jul 2026 14:29:23 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BA09F779AA; Mon, 20 Jul 2026 14:29:22 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cLYiKsIwXmrKTQAAD6G6ig (envelope-from ); Mon, 20 Jul 2026 14:29:22 +0000 From: Pedro Falcato To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Pedro Falcato , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/8] mm/khugepaged: hoist isolation into collapse_isolate_folio() Date: Mon, 20 Jul 2026 15:29:11 +0100 Message-ID: <20260720142913.846902-7-pfalcato@suse.de> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260720142913.846902-1-pfalcato@suse.de> References: <20260720142913.846902-1-pfalcato@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: B95653E18 X-Rspamd-Action: no action Content-Type: text/plain; charset="utf-8" Signed-off-by: Pedro Falcato --- mm/khugepaged.c | 118 +++++++++++++++++++++++++++--------------------- 1 file changed, 66 insertions(+), 52 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index db1f765b6ead..429e2c5833d0 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2332,6 +2332,69 @@ static enum scan_result prepare_collapse_file_folio(= pgoff_t index, struct collap return ret; } =20 +static enum scan_result collapse_isolate_folio(struct collapse_file_state = *state) +{ + struct folio *folio =3D state->folio; + enum scan_result result; + + /* + * The folio must be locked, so we can drop the i_pages lock + * without racing with truncate. + */ + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + + /* make sure the folio is up to date */ + if (unlikely(!folio_test_uptodate(folio))) { + result =3D SCAN_FAIL; + goto out_unlock; + } + + /* + * If file was truncated then extended, or hole-punched, before + * we locked the first folio, then a THP might be there already. + * This will be discovered on the first iteration. + */ + if (is_pmd_order(folio_order(folio))) { + result =3D SCAN_PTE_MAPPED_HUGEPAGE; + goto out_unlock; + } + + if (folio_mapping(folio) !=3D state->mapping) { + result =3D SCAN_TRUNCATED; + goto out_unlock; + } + + if (!state->is_shmem && (folio_test_dirty(folio) || + folio_test_writeback(folio))) { + /* + * khugepaged only works on clean file-backed folios, + * so this folio is dirty because it hasn't been flushed + * since first write. + */ + result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; + goto out_unlock; + } + + if (!folio_isolate_lru(folio)) { + result =3D SCAN_DEL_PAGE_LRU; + goto out_unlock; + } + + if (!filemap_release_folio(folio, GFP_KERNEL)) { + result =3D SCAN_PAGE_HAS_PRIVATE; + folio_putback_lru(folio); + goto out_unlock; + } + + if (folio_mapped(folio)) + try_to_unmap(folio, TTU_IGNORE_MLOCK | TTU_BATCH_FLUSH); + return SCAN_SUCCEED; +out_unlock: + folio_unlock(folio); + folio_put(folio); + return result; +} + /** * collapse_file - collapse filemap/tmpfs/shmem pages into huge one. * @@ -2442,61 +2505,12 @@ static enum scan_result collapse_file(struct mm_str= uct *mm, unsigned long addr, folio =3D state.folio; if (result !=3D SCAN_SUCCEED) goto xa_unlocked; - /* - * The folio must be locked, so we can drop the i_pages lock - * without racing with truncate. - */ - VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); - - /* make sure the folio is up to date */ - if (unlikely(!folio_test_uptodate(folio))) { - result =3D SCAN_FAIL; - goto out_unlock; - } - - /* - * If file was truncated then extended, or hole-punched, before - * we locked the first folio, then a THP might be there already. - * This will be discovered on the first iteration. - */ - if (is_pmd_order(folio_order(folio))) { - result =3D SCAN_PTE_MAPPED_HUGEPAGE; - goto out_unlock; - } - - if (folio_mapping(folio) !=3D mapping) { - result =3D SCAN_TRUNCATED; - goto out_unlock; - } - - if (!is_shmem && (folio_test_dirty(folio) || - folio_test_writeback(folio))) { - /* - * khugepaged only works on clean file-backed folios, - * so this folio is dirty because it hasn't been flushed - * since first write. - */ - result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; - goto out_unlock; - } =20 - if (!folio_isolate_lru(folio)) { - result =3D SCAN_DEL_PAGE_LRU; - goto out_unlock; - } - - if (!filemap_release_folio(folio, GFP_KERNEL)) { - result =3D SCAN_PAGE_HAS_PRIVATE; - folio_putback_lru(folio); - goto out_unlock; - } - - if (folio_mapped(folio)) - try_to_unmap(folio, - TTU_IGNORE_MLOCK | TTU_BATCH_FLUSH); + result =3D collapse_isolate_folio(&state); + if (result !=3D SCAN_SUCCEED) + goto xa_unlocked; =20 xas_lock_irq(&xas); - VM_BUG_ON_FOLIO(folio !=3D xa_load(xas.xa, index), folio); =20 /* --=20 2.55.0 From nobody Sat Jul 25 02:11:07 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5953C42A7B1 for ; Mon, 20 Jul 2026 14:29:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557779; cv=none; b=V/Js3bYncNDcFQOEi9YoU2lUbjNG0jbuMyEK9rBebqtVYQLa36JZzlxxdGjbbnI9ENcQqggHGG+pi8VPrtdQ2fF02lCsL8jlNLmTVmc+1+iTmPUO781v08g4Cdcm6gEonAlmAM0Swi8DtbpF1rnVwVNtoJcIvFx+4DAoj27mDgI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557779; c=relaxed/simple; bh=+BIRuKu8qqAY1LVaoR6nr009QzucxOO0f8gD0aYY4LM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FVlkMt28bXkTxGbFeQagHiIw8g2YBTEgzrA1U32fn3Vca98vlAD4L5YUWLawuhPqVQ3PpNjFVawwfN0fSqw5aUKrxHiASdVdBY8Nv4axtGeHsguBczdyor9olyk5ZEUmZ9zTyoNfwtS/yp4OeBLjmy2fIBn9xYM1YzdnTemYSaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=I8f093Gt; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=zIPot4lI; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=I8f093Gt; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=zIPot4lI; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="I8f093Gt"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="zIPot4lI"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="I8f093Gt"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="zIPot4lI" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D8F663E22; Mon, 20 Jul 2026 14:29:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557764; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QwWtCS1crk1JPnVWYwUvcYFHRLHis6CsOgoX49n95Ww=; b=I8f093GtNZzJw23a862NbIXxjnZL0gLD4DhsL4eieV5dsdC9VLY0iNBweLhQtwx8BksDQZ tvyfKJmwjTK7+L3H3JEd+qIG6Qyx59u81vbdBKiE2An3hNI98zWpiJNYZT4H7Y7hLm4X/E qsZ3y4+nukAPhchkdttG7sRYkNXevbA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557764; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QwWtCS1crk1JPnVWYwUvcYFHRLHis6CsOgoX49n95Ww=; b=zIPot4lIM66SF1r38ioJl2NT6cNaDwQdluP8gN4lhBmyRhMnDwkgzXXG6ktKmg9iV/aNCM ntUAYPOOEj4Z25AA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557764; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QwWtCS1crk1JPnVWYwUvcYFHRLHis6CsOgoX49n95Ww=; b=I8f093GtNZzJw23a862NbIXxjnZL0gLD4DhsL4eieV5dsdC9VLY0iNBweLhQtwx8BksDQZ tvyfKJmwjTK7+L3H3JEd+qIG6Qyx59u81vbdBKiE2An3hNI98zWpiJNYZT4H7Y7hLm4X/E qsZ3y4+nukAPhchkdttG7sRYkNXevbA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557764; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QwWtCS1crk1JPnVWYwUvcYFHRLHis6CsOgoX49n95Ww=; b=zIPot4lIM66SF1r38ioJl2NT6cNaDwQdluP8gN4lhBmyRhMnDwkgzXXG6ktKmg9iV/aNCM ntUAYPOOEj4Z25AA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D4C21779AA; Mon, 20 Jul 2026 14:29:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id +PvML8MwXmrKTQAAD6G6ig (envelope-from ); Mon, 20 Jul 2026 14:29:23 +0000 From: Pedro Falcato To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Pedro Falcato , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 7/8] mm/khugepaged: hoist more code into collapse_isolate_folio() Date: Mon, 20 Jul 2026 15:29:12 +0100 Message-ID: <20260720142913.846902-8-pfalcato@suse.de> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260720142913.846902-1-pfalcato@suse.de> References: <20260720142913.846902-1-pfalcato@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.994]; MIME_GOOD(-0.10)[text/plain]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid,imap1.dmz-prg2.suse.org:helo]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[15]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; R_RATELIMIT(0.00)[to_ip_from(RL8d8xedm6iu8o66torxsk6bwd)]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[] X-Spam-Level: X-Spam-Score: -6.80 Content-Type: text/plain; charset="utf-8" Checking expected refcounts or dirtyness has no need for the i_pages xarray lock. Signed-off-by: Pedro Falcato --- mm/khugepaged.c | 83 ++++++++++++++++++++++--------------------------- 1 file changed, 37 insertions(+), 46 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 429e2c5833d0..a09e4e4e1943 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2382,13 +2382,48 @@ static enum scan_result collapse_isolate_folio(stru= ct collapse_file_state *state =20 if (!filemap_release_folio(folio, GFP_KERNEL)) { result =3D SCAN_PAGE_HAS_PRIVATE; - folio_putback_lru(folio); - goto out_unlock; + goto out_putback; } =20 if (folio_mapped(folio)) try_to_unmap(folio, TTU_IGNORE_MLOCK | TTU_BATCH_FLUSH); + /* + * We control 2 + nr_pages references to the folio: + * - we hold a pin on it; + * - nr_pages reference from page cache; + * - one from lru_isolate_folio; + * If those are the only references, then any new usage + * of the folio will have to fetch it from the page + * cache. That requires locking the folio to handle + * truncate, so any new usage will be blocked until we + * unlock folio after collapse/during rollback. + */ + if (folio_ref_count(folio) !=3D 2 + folio_nr_pages(folio)) { + result =3D SCAN_PAGE_COUNT; + goto out_putback; + } + + /* + * At this point, the folio is locked and unmapped. If the PTE + * was dirty, try_to_unmap() has transferred the dirty bit to + * the folio and we must not collapse it into a clean + * file-backed folio. + * + * If the folio is clean here, no one can write it until we + * drop the folio lock. A write through a stale TLB entry came + * from a clean PTE and must fault because the PTE has been + * cleared; the fault path has to take the folio lock before + * installing a writable mapping. Buffered write paths also + * have to take the folio lock before modifying file contents + * without a mapping, typically via write_begin_get_folio(). + */ + if (!state->is_shmem && folio_test_dirty(folio)) { + result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; + goto out_putback; + } return SCAN_SUCCEED; +out_putback: + folio_putback_lru(folio); out_unlock: folio_unlock(folio); folio_put(folio); @@ -2513,55 +2548,11 @@ static enum scan_result collapse_file(struct mm_str= uct *mm, unsigned long addr, xas_lock_irq(&xas); VM_BUG_ON_FOLIO(folio !=3D xa_load(xas.xa, index), folio); =20 - /* - * We control 2 + nr_pages references to the folio: - * - we hold a pin on it; - * - nr_pages reference from page cache; - * - one from lru_isolate_folio; - * If those are the only references, then any new usage - * of the folio will have to fetch it from the page - * cache. That requires locking the folio to handle - * truncate, so any new usage will be blocked until we - * unlock folio after collapse/during rollback. - */ - if (folio_ref_count(folio) !=3D 2 + folio_nr_pages(folio)) { - result =3D SCAN_PAGE_COUNT; - xas_unlock_irq(&xas); - folio_putback_lru(folio); - goto out_unlock; - } - - /* - * At this point, the folio is locked and unmapped. If the PTE - * was dirty, try_to_unmap() has transferred the dirty bit to - * the folio and we must not collapse it into a clean - * file-backed folio. - * - * If the folio is clean here, no one can write it until we - * drop the folio lock. A write through a stale TLB entry came - * from a clean PTE and must fault because the PTE has been - * cleared; the fault path has to take the folio lock before - * installing a writable mapping. Buffered write paths also - * have to take the folio lock before modifying file contents - * without a mapping, typically via write_begin_get_folio(). - */ - if (!is_shmem && folio_test_dirty(folio)) { - result =3D SCAN_PAGE_DIRTY_OR_WRITEBACK; - xas_unlock_irq(&xas); - folio_putback_lru(folio); - goto out_unlock; - } - /* * Accumulate the folios that are being collapsed. */ list_add_tail(&folio->lru, &pagelist); index +=3D folio_nr_pages(folio); - continue; -out_unlock: - folio_unlock(folio); - folio_put(folio); - goto xa_unlocked; } =20 xa_locked: --=20 2.55.0 From nobody Sat Jul 25 02:11:07 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 490CB42BEAC for ; Mon, 20 Jul 2026 14:29:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557787; cv=none; b=sO9KrH1g0kbaFcKyz/FOyHux1HqPmv8FVOWrKpfY1eyNTvIzs4RYzMUzasXTwyRVvt3UXcHgfPACQ2ydbINI5dA6tXgWe4HBRpC39kF3iwpYwvAZK14ea6Gd7Q9MPsX6ereYC4J6kbN81y2GNyxYlVojLCTRCL+uTLn+3wgjgTE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784557787; c=relaxed/simple; bh=QIY2FuBZwKbrg4tCz9eDtqpjRLWlMZZc+HKHuSgsoYI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Rce58BOVNJTUtTuRgJDy6ya/i1QluI/hglM8jVB9ta/jePvVZLaDjxgxXCvuravsiOs+nM9chF40N/jj9G2t/T8k9lo/cuoIwGvuqd5RWzR0iDVLTt3CDmoU3JCZuSL+dC9gNqx7LHDlL/a/Zep4Z1l86Zw5qgwMcK5ggRp0Vvc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=F4/kNHtf; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=IVH/Cz9M; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=F4/kNHtf; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=IVH/Cz9M; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="F4/kNHtf"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="IVH/Cz9M"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="F4/kNHtf"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="IVH/Cz9M" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0046279FCB; Mon, 20 Jul 2026 14:29:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557766; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OJE+9lZzjBh6Jglu9uhAVEd2RMCplNuIFAAggzteniA=; b=F4/kNHtfryO0CG7lEx/YsClXVlnOiVsDbFjQ/58/kXndEa6KCtm+IFQCf+5y2AREyh6VQS wPTwa9HJdaXtCLUA2wsz9hCjDhjyVxapNzXoBrI69AgD8mwQ2mEhtajas2wd4cJ3TgV1TQ vPjeA9riOZENzmXrgQrrD6NSBM2uvdg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557766; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OJE+9lZzjBh6Jglu9uhAVEd2RMCplNuIFAAggzteniA=; b=IVH/Cz9McsHkxDnzOacIPsRLqvfjMtn8k/LvzEI9ixGJjN2BtmnL9OevStXAjFEWQGl0aP 2yxEgaresH93GYCg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1784557766; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OJE+9lZzjBh6Jglu9uhAVEd2RMCplNuIFAAggzteniA=; b=F4/kNHtfryO0CG7lEx/YsClXVlnOiVsDbFjQ/58/kXndEa6KCtm+IFQCf+5y2AREyh6VQS wPTwa9HJdaXtCLUA2wsz9hCjDhjyVxapNzXoBrI69AgD8mwQ2mEhtajas2wd4cJ3TgV1TQ vPjeA9riOZENzmXrgQrrD6NSBM2uvdg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1784557766; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OJE+9lZzjBh6Jglu9uhAVEd2RMCplNuIFAAggzteniA=; b=IVH/Cz9McsHkxDnzOacIPsRLqvfjMtn8k/LvzEI9ixGJjN2BtmnL9OevStXAjFEWQGl0aP 2yxEgaresH93GYCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id F3576779AA; Mon, 20 Jul 2026 14:29:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kOipN8QwXmrKTQAAD6G6ig (envelope-from ); Mon, 20 Jul 2026 14:29:24 +0000 From: Pedro Falcato To: Andrew Morton , David Hildenbrand , Lorenzo Stoakes Cc: Pedro Falcato , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 8/8] mm/khugepaged: fix and flesh out try_to_unmap_flush() comment Date: Mon, 20 Jul 2026 15:29:13 +0100 Message-ID: <20260720142913.846902-9-pfalcato@suse.de> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260720142913.846902-1-pfalcato@suse.de> References: <20260720142913.846902-1-pfalcato@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.992]; MIME_GOOD(-0.10)[text/plain]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[15]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; R_RATELIMIT(0.00)[to_ip_from(RL8d8xedm6iu8o66torxsk6bwd)]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_TLS_ALL(0.00)[] X-Spam-Level: Content-Type: text/plain; charset="utf-8" Instead of vaguely handwaving that it should probably be done, flesh out any correctness concerns with try_to_unmap batching, flushing and its interaction with the rest of collapse_file(). Signed-off-by: Pedro Falcato --- mm/khugepaged.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a09e4e4e1943..1999e094df20 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2560,15 +2560,19 @@ static enum scan_result collapse_file(struct mm_str= uct *mm, unsigned long addr, xa_unlocked: =20 /* - * If collapse is successful, flush must be done now before copying. - * If collapse is unsuccessful, does flush actually need to be done? - * Do it anyway, to clear the state. + * try_to_unmap() flush must be done now before copying, regardless + * of success or not. In case of success, folios are about to be + * copied and collapsed onto a single large folio. In that case, + * stale TLB entries need to be flushed out, so no racing write may + * get lost. In case of failure, stale TLB entries need to be flushed + * out before putting the folio (which can possibly free it). */ try_to_unmap_flush(); =20 if (result =3D=3D SCAN_SUCCEED && nr_none && !shmem_charge(mapping->host, nr_none)) result =3D SCAN_FAIL; + if (result !=3D SCAN_SUCCEED) { nr_none =3D 0; goto rollback; --=20 2.55.0