From nobody Wed Dec 17 10:20:52 2025 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 271D3C133 for ; Thu, 11 Jul 2024 02:13:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664007; cv=none; b=FowVCXe6rbzatTNTzaU0l44wr4JUhXD9/VR0jSUeyl2zmQNe+2w5Hkw9aXuDnRgjR2TlAh5hDmDrzV2PDsK38SrPwyEmjYnbcmnfbv6jl7hlhEBW1vesFWJsnwMDdozpWe4jhO3UkD6ug3zjdocGvdjk8tsA+JbsVyJmj9Vn+9c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664007; c=relaxed/simple; bh=Ub6aJebnv3sNKRUMRdjFJPK9vCRxRBndWeWnvuCQDyU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QtVyf90040z0Fw0WMpME0to2vvx7zQzk+ROHWV1bs42eLKFlBneYMCH9XxRgEmNRJrCUhK0nlFGs1vvmZpDG/RmEz7rPKp6gP8MejpQ5X4hkODPbHRuPC6nCMPdC/p+zvxNjjR4f3/wxGWDPuo8bsYQ8SNiulcqLx51jEKS7QaU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WEcL1jGt; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WEcL1jGt" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e039fe0346eso724408276.1 for ; Wed, 10 Jul 2024 19:13:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664005; x=1721268805; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Q2KA7HqAEDJDFjninnbueVy/cjuPGw3T2Kp1IRta9cE=; b=WEcL1jGt0B5lxrpVYyXtjL8Xz3hcyx6lFtwCIglP6L3VVLf2/gYCSpPi1NDawQzK1A uLR88RsL6KDyT/k31NoVV4U4cfJU5GfmKe4cEgBlbIkbRicK8teK3QHH0qKAlzfk6MFY oivkEI/QW8djgEsadrg34pyBcQjmXn5tXRm0PscCNBq2XwVTrVEKbLUP0+3ecihppUEl zK4qaltwswzqDN6vzc9jHTW88MfRxhn+xGK2Gl/yV4NqeKcalU5YlzepvP6ff4PunGEZ FMc2kcjS+ogPrd7OnOhin0Bp0tV7k5Wpc0o46Ai/MyX5b9DUbm1sGr1zWKN+akSFp2lr VEjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664005; x=1721268805; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Q2KA7HqAEDJDFjninnbueVy/cjuPGw3T2Kp1IRta9cE=; b=l0dw2TTUaLXUTPwzEYv1Yh639WJe7BdPeTwvvxmd2i7UY/v2VTS9D3v9snG+71T6z/ 6cdj5TjHa1AcRQqvQXniOiLHDBcRaXUjvg4h0eGE0smn4wyAUojYSNyOn7Djq2up4VQh B0nAkRSVln7AXvyecrY1ndjm+m43XG6qf97nfNT2schxdJu7iAH9849IL8faeFlqiYlu DVZlW7yYEs5iSRipTH4qjJImkoQgNXsjOLTlTHkwL3Q8lIWuxwlj59paxTzJRfo6m2X8 cXqwvEUcpLgsg5XkD3D8cb4gW2p8tXM/VRK/S8/t/bu1G0ChkyOsvKidGx36zXz0J2mB TlAg== X-Forwarded-Encrypted: i=1; AJvYcCX1Uid9I/9ARSxDJSPFEWgXDV1NlyZ6z7yIEZFK8sJ7+NibTBJFNs3l+Em3mVcIdKBRmiE62KCPrBAI0VbxkkloyUj4NgaC12ZxI/dP X-Gm-Message-State: AOJu0Yy/RaylEIc+UjJVZ7idHMRFPkmfm63Nr+mnMPKB/e6sY0lOVNPL OGs5wAxgA4VKTbi/8jS2fUsNJGvZvxGyECE5sVaISkVV4ynINeOMTlMrxLqvjNho64FQkhdDlTe eTg== X-Google-Smtp-Source: AGHT+IH/Rp/IjlPFZgIAEzdNYzL8aBM88d5tP9iD3BV755kvnBakj7CsVrD7R8NFWV17wOfNsmdxA2dCn38= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:1701:b0:e02:c478:c8b9 with SMTP id 3f1490d57ef6-e041b1449e9mr15176276.12.1720664004897; Wed, 10 Jul 2024 19:13:24 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:13 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-2-yuzhao@google.com> Subject: [PATCH mm-unstable v1 1/5] mm/swap: reduce indentation level From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Reduce indentation level by returning directly when there is no cleanup needed, i.e., if (condition) { | if (condition) { do_this(); | do_this(); return; | return; } else { | } do_that(); | } | do_that(); and if (condition) { | if (!condition) do_this(); | return; do_that(); | } | do_this(); return; | do_that(); Presumably the old style became repetitive as the result of copy and paste. Signed-off-by: Yu Zhao --- mm/swap.c | 209 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 109 insertions(+), 100 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 9caf6b017cf0..952e4aac6eb1 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -117,7 +117,9 @@ void __folio_put(struct folio *folio) if (unlikely(folio_is_zone_device(folio))) { free_zone_device_folio(folio); return; - } else if (folio_test_hugetlb(folio)) { + } + + if (folio_test_hugetlb(folio)) { free_huge_folio(folio); return; } @@ -228,17 +230,19 @@ static void folio_batch_add_and_move(struct folio_bat= ch *fbatch, if (folio_batch_add(fbatch, folio) && !folio_test_large(folio) && !lru_cache_disabled()) return; + folio_batch_move_lru(fbatch, move_fn); } =20 static void lru_move_tail_fn(struct lruvec *lruvec, struct folio *folio) { - if (!folio_test_unevictable(folio)) { - lruvec_del_folio(lruvec, folio); - folio_clear_active(folio); - lruvec_add_folio_tail(lruvec, folio); - __count_vm_events(PGROTATED, folio_nr_pages(folio)); - } + if (folio_test_unevictable(folio)) + return; + + lruvec_del_folio(lruvec, folio); + folio_clear_active(folio); + lruvec_add_folio_tail(lruvec, folio); + __count_vm_events(PGROTATED, folio_nr_pages(folio)); } =20 /* @@ -250,22 +254,23 @@ static void lru_move_tail_fn(struct lruvec *lruvec, s= truct folio *folio) */ void folio_rotate_reclaimable(struct folio *folio) { - if (!folio_test_locked(folio) && !folio_test_dirty(folio) && - !folio_test_unevictable(folio)) { - struct folio_batch *fbatch; - unsigned long flags; + struct folio_batch *fbatch; + unsigned long flags; =20 - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } + if (folio_test_locked(folio) || folio_test_dirty(folio) || + folio_test_unevictable(folio)) + return; =20 - local_lock_irqsave(&lru_rotate.lock, flags); - fbatch =3D this_cpu_ptr(&lru_rotate.fbatch); - folio_batch_add_and_move(fbatch, folio, lru_move_tail_fn); - local_unlock_irqrestore(&lru_rotate.lock, flags); + folio_get(folio); + if (!folio_test_clear_lru(folio)) { + folio_put(folio); + return; } + + local_lock_irqsave(&lru_rotate.lock, flags); + fbatch =3D this_cpu_ptr(&lru_rotate.fbatch); + folio_batch_add_and_move(fbatch, folio, lru_move_tail_fn); + local_unlock_irqrestore(&lru_rotate.lock, flags); } =20 void lru_note_cost(struct lruvec *lruvec, bool file, @@ -328,18 +333,19 @@ void lru_note_cost_refault(struct folio *folio) =20 static void folio_activate_fn(struct lruvec *lruvec, struct folio *folio) { - if (!folio_test_active(folio) && !folio_test_unevictable(folio)) { - long nr_pages =3D folio_nr_pages(folio); - - lruvec_del_folio(lruvec, folio); - folio_set_active(folio); - lruvec_add_folio(lruvec, folio); - trace_mm_lru_activate(folio); - - __count_vm_events(PGACTIVATE, nr_pages); - __count_memcg_events(lruvec_memcg(lruvec), PGACTIVATE, - nr_pages); - } + long nr_pages =3D folio_nr_pages(folio); + + if (folio_test_active(folio) || folio_test_unevictable(folio)) + return; + + + lruvec_del_folio(lruvec, folio); + folio_set_active(folio); + lruvec_add_folio(lruvec, folio); + trace_mm_lru_activate(folio); + + __count_vm_events(PGACTIVATE, nr_pages); + __count_memcg_events(lruvec_memcg(lruvec), PGACTIVATE, nr_pages); } =20 #ifdef CONFIG_SMP @@ -353,20 +359,21 @@ static void folio_activate_drain(int cpu) =20 void folio_activate(struct folio *folio) { - if (!folio_test_active(folio) && !folio_test_unevictable(folio)) { - struct folio_batch *fbatch; + struct folio_batch *fbatch; =20 - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } + if (folio_test_active(folio) || folio_test_unevictable(folio)) + return; =20 - local_lock(&cpu_fbatches.lock); - fbatch =3D this_cpu_ptr(&cpu_fbatches.activate); - folio_batch_add_and_move(fbatch, folio, folio_activate_fn); - local_unlock(&cpu_fbatches.lock); + folio_get(folio); + if (!folio_test_clear_lru(folio)) { + folio_put(folio); + return; } + + local_lock(&cpu_fbatches.lock); + fbatch =3D this_cpu_ptr(&cpu_fbatches.activate); + folio_batch_add_and_move(fbatch, folio, folio_activate_fn); + local_unlock(&cpu_fbatches.lock); } =20 #else @@ -378,12 +385,13 @@ void folio_activate(struct folio *folio) { struct lruvec *lruvec; =20 - if (folio_test_clear_lru(folio)) { - lruvec =3D folio_lruvec_lock_irq(folio); - folio_activate_fn(lruvec, folio); - unlock_page_lruvec_irq(lruvec); - folio_set_lru(folio); - } + if (!folio_test_clear_lru(folio)) + return; + + lruvec =3D folio_lruvec_lock_irq(folio); + folio_activate_fn(lruvec, folio); + unlock_page_lruvec_irq(lruvec); + folio_set_lru(folio); } #endif =20 @@ -610,41 +618,41 @@ static void lru_deactivate_file_fn(struct lruvec *lru= vec, struct folio *folio) =20 static void lru_deactivate_fn(struct lruvec *lruvec, struct folio *folio) { - if (!folio_test_unevictable(folio) && (folio_test_active(folio) || lru_ge= n_enabled())) { - long nr_pages =3D folio_nr_pages(folio); + long nr_pages =3D folio_nr_pages(folio); =20 - lruvec_del_folio(lruvec, folio); - folio_clear_active(folio); - folio_clear_referenced(folio); - lruvec_add_folio(lruvec, folio); + if (folio_test_unevictable(folio) || !(folio_test_active(folio) || lru_ge= n_enabled())) + return; =20 - __count_vm_events(PGDEACTIVATE, nr_pages); - __count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, - nr_pages); - } + lruvec_del_folio(lruvec, folio); + folio_clear_active(folio); + folio_clear_referenced(folio); + lruvec_add_folio(lruvec, folio); + + __count_vm_events(PGDEACTIVATE, nr_pages); + __count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, nr_pages); } =20 static void lru_lazyfree_fn(struct lruvec *lruvec, struct folio *folio) { - if (folio_test_anon(folio) && folio_test_swapbacked(folio) && - !folio_test_swapcache(folio) && !folio_test_unevictable(folio)) { - long nr_pages =3D folio_nr_pages(folio); + long nr_pages =3D folio_nr_pages(folio); =20 - lruvec_del_folio(lruvec, folio); - folio_clear_active(folio); - folio_clear_referenced(folio); - /* - * Lazyfree folios are clean anonymous folios. They have - * the swapbacked flag cleared, to distinguish them from normal - * anonymous folios - */ - folio_clear_swapbacked(folio); - lruvec_add_folio(lruvec, folio); + if (!folio_test_anon(folio) || !folio_test_swapbacked(folio) || + folio_test_swapcache(folio) || folio_test_unevictable(folio)) + return; =20 - __count_vm_events(PGLAZYFREE, nr_pages); - __count_memcg_events(lruvec_memcg(lruvec), PGLAZYFREE, - nr_pages); - } + lruvec_del_folio(lruvec, folio); + folio_clear_active(folio); + folio_clear_referenced(folio); + /* + * Lazyfree folios are clean anonymous folios. They have + * the swapbacked flag cleared, to distinguish them from normal + * anonymous folios + */ + folio_clear_swapbacked(folio); + lruvec_add_folio(lruvec, folio); + + __count_vm_events(PGLAZYFREE, nr_pages); + __count_memcg_events(lruvec_memcg(lruvec), PGLAZYFREE, nr_pages); } =20 /* @@ -726,21 +734,21 @@ void deactivate_file_folio(struct folio *folio) */ void folio_deactivate(struct folio *folio) { - if (!folio_test_unevictable(folio) && (folio_test_active(folio) || - lru_gen_enabled())) { - struct folio_batch *fbatch; + struct folio_batch *fbatch; =20 - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } + if (folio_test_unevictable(folio) || !(folio_test_active(folio) || lru_ge= n_enabled())) + return; =20 - local_lock(&cpu_fbatches.lock); - fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_deactivate); - folio_batch_add_and_move(fbatch, folio, lru_deactivate_fn); - local_unlock(&cpu_fbatches.lock); + folio_get(folio); + if (!folio_test_clear_lru(folio)) { + folio_put(folio); + return; } + + local_lock(&cpu_fbatches.lock); + fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_deactivate); + folio_batch_add_and_move(fbatch, folio, lru_deactivate_fn); + local_unlock(&cpu_fbatches.lock); } =20 /** @@ -752,21 +760,22 @@ void folio_deactivate(struct folio *folio) */ void folio_mark_lazyfree(struct folio *folio) { - if (folio_test_anon(folio) && folio_test_swapbacked(folio) && - !folio_test_swapcache(folio) && !folio_test_unevictable(folio)) { - struct folio_batch *fbatch; + struct folio_batch *fbatch; =20 - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } + if (!folio_test_anon(folio) || !folio_test_swapbacked(folio) || + folio_test_swapcache(folio) || folio_test_unevictable(folio)) + return; =20 - local_lock(&cpu_fbatches.lock); - fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_lazyfree); - folio_batch_add_and_move(fbatch, folio, lru_lazyfree_fn); - local_unlock(&cpu_fbatches.lock); + folio_get(folio); + if (!folio_test_clear_lru(folio)) { + folio_put(folio); + return; } + + local_lock(&cpu_fbatches.lock); + fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_lazyfree); + folio_batch_add_and_move(fbatch, folio, lru_lazyfree_fn); + local_unlock(&cpu_fbatches.lock); } =20 void lru_add_drain(void) --=20 2.45.2.803.g4e1b14247a-goog From nobody Wed Dec 17 10:20:52 2025 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A02FDDF78 for ; Thu, 11 Jul 2024 02:13:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664010; cv=none; b=olDJCTAodIGYQ0YP6o5qmklWHIcwVD8A2qHTWBFGi7WeaGFsB+dYhijo3RDzdUZOuiW/7r//TKTgoUDozFjWC2w1nAqU/LWgTtd6t/cUuvzrQWMjZ8O7Z7VH/G5/oXAGPwstxNW+NwBA6gbMGJTd+hNghuCGXvB8QYx+Y2l+Qsg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664010; c=relaxed/simple; bh=swqsDkPvw6N6omPNGa5/j5p0uymSXIdVOTnWFKLeiCw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=UVQYM1+H36+IFJ7otU+ody3XwU3EQncQvccLCG49FpLoZ5q2//c+KJ4LrXigrpPHhTOKRBIID69XFpRd7TWGDuyE/fy1Xn/boK3AGwvRRDy2UlwdeR7t3U3Cu9EOKSCWM6YxWO9iFOYl1jgQjbMWFJ132zXpLNWP9xm9jDuG7QU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EteHRKx/; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EteHRKx/" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e02a4de4f4eso698997276.1 for ; Wed, 10 Jul 2024 19:13:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664007; x=1721268807; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Mz1u87ev8DYclZ/2H0TBqU+M5VpYJmHKMXbGaF77G5w=; b=EteHRKx/HrpKN7yEzpos7730bj3Hd6fxCLzbD+kcUDDRtTN/mbZtm2GeVNU5nXQ8mY blt2f0noTjfPBPEqPMMrywBNTn/i03EzdCVaEwy70kp3JK5SX1lvMYckttYflpqIs4+Z 7W/eJ4KoDxoCBFbiJ32Y9SORoCrIgmqsH9F+VYrmTTDTELhPnAnHy+t74eusWgcvC/zs 3IYcUpK7Ekhfh3vzjMUt4GY7OwvlVoxkwR6gaiXq5a9FvLBVTcWHnqlo6GW7SJ1nMcyd qwJfzemaTru1/7zJW7dEG/gM6DTxLBxusUxA6ToQaI0kN2s6C0TaH3hkfvBrFUt8QxLX Pdlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664007; x=1721268807; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Mz1u87ev8DYclZ/2H0TBqU+M5VpYJmHKMXbGaF77G5w=; b=JX071JufJbLQfWU0uwWlw4ltJm/cmpHpuVLmxEyDbhYSnwekjse+xo4g45d/XyiM24 HkpJwg2FZUz2+QSFxbkOKvLI+LdwhNZGf1ff8NXt/6bCaLWHIVCjXwFxg3TPaaQ9HaGW tvpjM+mOmnbuWAlujg2h1SRhXGj47kdMaxiYGQUCe0lw25TFxR3nA6xy9v/zVti2xJag 3dWEGyRNudQan1Ra6xkjWd5tDa87EzRhSmbJr92jPfsw9tBofewYrKzlfazmyaJPZYqH erJMcRyog4vtTjLzqqag722owcp4J3A239fIt40YaS+IywcRs6sqPgYURKA9Uowdvhs4 nK6w== X-Forwarded-Encrypted: i=1; AJvYcCWBbBTMB55K/KVM9aXuz9R4+WKX3ReuYqOkFVIrFrXTyzJkmdG/4NzVyWiFAK9z1CzM9p1mQFbgwXEjCIgFH8qrQcgsHndys9wyTlce X-Gm-Message-State: AOJu0Ywm/4w+oWS1ZrmubDK7jPnl5yDwJdfr4AHHo9Yd2GctnWERHtfd DGTbZqjlqvMj3h1w9H/5NfESgVg0PoVOif/SP8zv1RsGnZu664fD+7JTdvPXlHf3FhHnnhsswvA 7Ug== X-Google-Smtp-Source: AGHT+IEFx7WEmcVLtwhV7MLkiuoJz3OGus58zTubJgwlkdvY2QBLN+8BvOJfH7aoBGbgNkKVhQae3bL0CsI= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:729:b0:e01:f998:fea2 with SMTP id 3f1490d57ef6-e041b14337emr15713276.11.1720664007468; Wed, 10 Jul 2024 19:13:27 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:14 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-3-yuzhao@google.com> Subject: [PATCH mm-unstable v1 2/5] mm/swap: rename cpu_fbatches->activate From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename cpu_fbatches->activate to cpu_fbatches->lru_activate, and its handler folio_activate_fn() to lru_activate() so that all the boilerplate can be removed at the end of this series. Signed-off-by: Yu Zhao --- mm/swap.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 952e4aac6eb1..e4745b88a964 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -67,7 +67,7 @@ struct cpu_fbatches { struct folio_batch lru_deactivate; struct folio_batch lru_lazyfree; #ifdef CONFIG_SMP - struct folio_batch activate; + struct folio_batch lru_activate; #endif }; static DEFINE_PER_CPU(struct cpu_fbatches, cpu_fbatches) =3D { @@ -331,7 +331,7 @@ void lru_note_cost_refault(struct folio *folio) folio_nr_pages(folio), 0); } =20 -static void folio_activate_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_activate(struct lruvec *lruvec, struct folio *folio) { long nr_pages =3D folio_nr_pages(folio); =20 @@ -351,10 +351,10 @@ static void folio_activate_fn(struct lruvec *lruvec, = struct folio *folio) #ifdef CONFIG_SMP static void folio_activate_drain(int cpu) { - struct folio_batch *fbatch =3D &per_cpu(cpu_fbatches.activate, cpu); + struct folio_batch *fbatch =3D &per_cpu(cpu_fbatches.lru_activate, cpu); =20 if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, folio_activate_fn); + folio_batch_move_lru(fbatch, lru_activate); } =20 void folio_activate(struct folio *folio) @@ -371,8 +371,8 @@ void folio_activate(struct folio *folio) } =20 local_lock(&cpu_fbatches.lock); - fbatch =3D this_cpu_ptr(&cpu_fbatches.activate); - folio_batch_add_and_move(fbatch, folio, folio_activate_fn); + fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_activate); + folio_batch_add_and_move(fbatch, folio, lru_activate); local_unlock(&cpu_fbatches.lock); } =20 @@ -389,7 +389,7 @@ void folio_activate(struct folio *folio) return; =20 lruvec =3D folio_lruvec_lock_irq(folio); - folio_activate_fn(lruvec, folio); + lru_activate(lruvec, folio); unlock_page_lruvec_irq(lruvec); folio_set_lru(folio); } @@ -490,7 +490,7 @@ void folio_mark_accessed(struct folio *folio) } else if (!folio_test_active(folio)) { /* * If the folio is on the LRU, queue it for activation via - * cpu_fbatches.activate. Otherwise, assume the folio is in a + * cpu_fbatches.lru_activate. Otherwise, assume the folio is in a * folio_batch, mark it active and it'll be moved to the active * LRU on the next drain. */ @@ -829,7 +829,7 @@ static bool cpu_needs_drain(unsigned int cpu) folio_batch_count(&fbatches->lru_deactivate_file) || folio_batch_count(&fbatches->lru_deactivate) || folio_batch_count(&fbatches->lru_lazyfree) || - folio_batch_count(&fbatches->activate) || + folio_batch_count(&fbatches->lru_activate) || need_mlock_drain(cpu) || has_bh_in_lru(cpu, NULL); } --=20 2.45.2.803.g4e1b14247a-goog From nobody Wed Dec 17 10:20:52 2025 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F009C12E7C for ; Thu, 11 Jul 2024 02:13:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664012; cv=none; b=C0ILRFW1A19IGPkjB1KJXDHRTdirwZxIYvOqloBVbP4ODJF+Z4ZPiusHzQFu6HRoJl/bsSYjM7WPFsJNJPxcX+TsEF+kHD6SeZAyx1gHZQjNxIg7bLfednPgolLfYPyMBsAK6GYgO3ejFev5uM0BDzBdxlAl//VBmX/5/wXgRNQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664012; c=relaxed/simple; bh=iP/ws+ycbMf6bFYw1o6xwD5xNp9ysgzyHcMAdbVxrCI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LqtBobZRIA+IGK1J4H1NJh6z19pxriFdtCPialnG2byTywj9+3Y5Y3z4G9SUarYgchgWugg1OuQU3xkOsRc1gV36ZFtu9+UuBOupUI5PbqYkAMCXUij7kAkhRfyw9W+2iWMj2ooxABcuCxqICU9IrS2Knof+KvFiupJvurMI9iU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bwACksfq; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bwACksfq" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e035f7b5976so889978276.0 for ; Wed, 10 Jul 2024 19:13:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664010; x=1721268810; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WhNUH7SAP/MNFXjYcDX49dOz4gKNgHUZEXTQwt2QQyA=; b=bwACksfqADURhXe3ZpGx5vk82DATvkoud8XnfKC5vdB9ugqFu1arCVUSgwd3VPInl8 lCScRksq7iyedqNZtnQyXxbxlzpvfP1IJJiKKOWGM0ov0JyqEmNxgOVqxzGOSOd74iRm Ni+edC9oGCCO8qmGjBLXaDCW4nZIARKc6sohgduAk53N3mUKDeBlRTB00JLt0i2UQdit XsJpqjqvtb66NRSUv6VCJ1YHJgazXPy5vncRK4SfeA2gb70RtGD6GjwAOiAf4D3jReoR oOhnJ32C4m5AzqMYvi6cQlVr9adxgOoTGD3kZsyNz2TkchCyCZVV8opva0G6OwPDQa/n +rpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664010; x=1721268810; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WhNUH7SAP/MNFXjYcDX49dOz4gKNgHUZEXTQwt2QQyA=; b=BaabCgXleJ6zRYKIlnUe4w27VjyLCR6D2gq8l1dIREc5RNXmWn8S5voVQzQWRYYi6P hFKh0JPtRVwrD9a6CAThh/L2UdgkvWI73GKqjBBxMcRjvAXkUqb748F0Q0Thhyeu0AB2 XaG81qh53mTtqGrVvrv3n2CldPMMTgIxWCebXdX3D69lZGNag2m9Jr53wO1gBs0auOKB sfrxogVrG1p68FppfFHgaALG/mvgbbqQ+dKwdEuw1JL2xa5cXIkSIkNujVHQU3n9jOLW JDfTwVDLg6dOQTJHLkeTQsytv1FpDDn+afarNH8D4qP1xwnTvQ6yThWQpGbDCYkQkF3a 3YhQ== X-Forwarded-Encrypted: i=1; AJvYcCXmDDmI3T4TlztEcuIrp5RnXodiV8gHgvjvvl1TNCUfroDsUI/hWdfW5iDrPue6qkabwtm0r/bLSDqGprZkfQX1MMt9aIoDXUNA7NtN X-Gm-Message-State: AOJu0Yx50vgMDRAsGeXl0qzVvBz6VP4yKXKDkX+xuj+2NFjDqJPy9I9K JD3QTA+/y8exUGrwvuUds01LR7JWPYFiZVXmp/yeJt19OvVrIUUOEEdW8QbrNXY9OxjLnUL2oln pQg== X-Google-Smtp-Source: AGHT+IGkIwS4bVyfM5ByoYpmZQq2iqjOHx1IN9uZ+1eS+2/SoQ8AfYNeitOzBddCLqNu0G2MAu021nNjM8U= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:32a6:b0:e03:61bb:6032 with SMTP id 3f1490d57ef6-e0577fa164amr20670276.1.1720664010014; Wed, 10 Jul 2024 19:13:30 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:15 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-4-yuzhao@google.com> Subject: [PATCH mm-unstable v1 3/5] mm/swap: fold lru_rotate into cpu_fbatches From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fold lru_rotate into cpu_fbatches, and rename the folio_batch and the lock protecting it to lru_move_tail and lock_irq respectively so that all the boilerplate can be removed at the end of this series. Also remove data_race() around folio_batch_count(), which is out of place: all folio_batch_count() calls on remote cpu_fbatches are subject to data_race(), and therefore data_race() should be inside folio_batch_count(). Signed-off-by: Yu Zhao --- mm/swap.c | 36 ++++++++++++++++-------------------- 1 file changed, 16 insertions(+), 20 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index e4745b88a964..774ae9eab1e6 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -47,20 +47,11 @@ int page_cluster; const int page_cluster_max =3D 31; =20 -/* Protecting only lru_rotate.fbatch which requires disabling interrupts */ -struct lru_rotate { - local_lock_t lock; - struct folio_batch fbatch; -}; -static DEFINE_PER_CPU(struct lru_rotate, lru_rotate) =3D { - .lock =3D INIT_LOCAL_LOCK(lock), -}; - -/* - * The following folio batches are grouped together because they are prote= cted - * by disabling preemption (and interrupts remain enabled). - */ struct cpu_fbatches { + /* + * The following folio batches are grouped together because they are prot= ected + * by disabling preemption (and interrupts remain enabled). + */ local_lock_t lock; struct folio_batch lru_add; struct folio_batch lru_deactivate_file; @@ -69,9 +60,14 @@ struct cpu_fbatches { #ifdef CONFIG_SMP struct folio_batch lru_activate; #endif + /* Protecting the following batches which require disabling interrupts */ + local_lock_t lock_irq; + struct folio_batch lru_move_tail; }; + static DEFINE_PER_CPU(struct cpu_fbatches, cpu_fbatches) =3D { .lock =3D INIT_LOCAL_LOCK(lock), + .lock_irq =3D INIT_LOCAL_LOCK(lock_irq), }; =20 static void __page_cache_release(struct folio *folio, struct lruvec **lruv= ecp, @@ -267,10 +263,10 @@ void folio_rotate_reclaimable(struct folio *folio) return; } =20 - local_lock_irqsave(&lru_rotate.lock, flags); - fbatch =3D this_cpu_ptr(&lru_rotate.fbatch); + local_lock_irqsave(&cpu_fbatches.lock_irq, flags); + fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_move_tail); folio_batch_add_and_move(fbatch, folio, lru_move_tail_fn); - local_unlock_irqrestore(&lru_rotate.lock, flags); + local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); } =20 void lru_note_cost(struct lruvec *lruvec, bool file, @@ -668,15 +664,15 @@ void lru_add_drain_cpu(int cpu) if (folio_batch_count(fbatch)) folio_batch_move_lru(fbatch, lru_add_fn); =20 - fbatch =3D &per_cpu(lru_rotate.fbatch, cpu); + fbatch =3D &fbatches->lru_move_tail; /* Disabling interrupts below acts as a compiler barrier. */ if (data_race(folio_batch_count(fbatch))) { unsigned long flags; =20 /* No harm done if a racing interrupt already did this */ - local_lock_irqsave(&lru_rotate.lock, flags); + local_lock_irqsave(&cpu_fbatches.lock_irq, flags); folio_batch_move_lru(fbatch, lru_move_tail_fn); - local_unlock_irqrestore(&lru_rotate.lock, flags); + local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); } =20 fbatch =3D &fbatches->lru_deactivate_file; @@ -825,7 +821,7 @@ static bool cpu_needs_drain(unsigned int cpu) =20 /* Check these in order of likelihood that they're not zero */ return folio_batch_count(&fbatches->lru_add) || - data_race(folio_batch_count(&per_cpu(lru_rotate.fbatch, cpu))) || + folio_batch_count(&fbatches->lru_move_tail) || folio_batch_count(&fbatches->lru_deactivate_file) || folio_batch_count(&fbatches->lru_deactivate) || folio_batch_count(&fbatches->lru_lazyfree) || --=20 2.45.2.803.g4e1b14247a-goog From nobody Wed Dec 17 10:20:52 2025 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A04901C686 for ; Thu, 11 Jul 2024 02:13:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664015; cv=none; b=kVSPKLRY1cUGdLsQmNNsZDkYuf6fW6nz5lFqwFhQw0qCL+c10uNSdPYbM5/WFSIbttjAwIhL8heU7S24V83t93LMdwLNvvH+qubKXzeMHh2BWXBvUFTkTjdgzoP6veCbQDcmS3H/OAzPNTiYo5dazoqhjD9Xwu8PgIrzJdOyHfQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664015; c=relaxed/simple; bh=DW9P12l/Vxqw6YA4KHi8AXt6xBkY2xccCQhFjXmnCoY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HwcjmGp+QXZRr4jH2VkJPkBCeh15C6bWF4/VPJS5rgftugVM3pRfcNDjaNLApbubyrjO84XciGx9QeA8kLqeJEL7PZWJu1LJuw4pXZtHrTUczer9XLl4AwI+MWqhGe5AczIfxMMqx387Gq3HWkhRmwUQ1DN4uWnNpzO/VMHN95Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=PnWirced; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="PnWirced" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-64a5503e253so6866607b3.0 for ; Wed, 10 Jul 2024 19:13:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664012; x=1721268812; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6RCNHIc0/84Yq5SbDOD6xZQS6zwZu2n7K7QhObKOUEk=; b=PnWircedP1xM/JbRBTbBGQcKK3CIMVl9ehb2282jutMLdqS339LgcTshxysRD3W980 sLW59G2PreqEFeosJjIz4wbtXo3K5M2HcTZSo1ddhpJ7C1xiv9xNDuPnvLQmP6XCakIE BdQIbbHaoUqdJweBJn+tg6GDLv7BvvTDnAk15Y569HJHkAonxfmTwqHu2ydv7c1+2LNR SPcCE8iRVcFWazAvBux4hpOC0PJMEmQQIxav/uWXFNv6hA/5m0TbvqnbTP3KvYW2eocx 44CY2afudo/VJ1WlRmJ9UcWVxwGbIQ5db/16GY3tnaDrUF3RNGToWiVgFey9A3uN+RoP NLMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664012; x=1721268812; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6RCNHIc0/84Yq5SbDOD6xZQS6zwZu2n7K7QhObKOUEk=; b=U+hRGeinE8dGQx/DP+NIlSBe+HF6Zfov3YIgNONX9bOudRcWr7G7VrHYr2GiYpLcSb W5uMJtvqP13Qgsxqx3SxmGfPW865vg868AhzJvm1G6OM/AoJP8IYKvKwmLWxgqKUxh8U Jzs7tzMhUGBBtH1//T17BHnNKnjb6Nugqqmlf0m+w5hGGf2Ll6ThnSn0edWp5iGSst9o /BKOOOJFcEk7SPYtEn2ZO9Qh4aCdFEP+70OMI/4JnaDhKtv3sfknabd6gVLsqAb7Fx5u vMqwoeOIhpBigYrhRingA/9cA4OG3ssZL1c8wtTbaMtJywtMzpVhlUyILjdFqw/aJxYj ZjrA== X-Forwarded-Encrypted: i=1; AJvYcCX4gJe8EAy3faa/MRzZPXeN31iiZJTjYVAzsTt4ZuEhYbZMQ53o3L+nH8+NRUZDhrpleXg4Q7a4gF01hI6JZOZom66SIXK0+8o2lif/ X-Gm-Message-State: AOJu0YwQSydRHVWXbhbeTgmu0NQuVnEbpl2LZ/YFdtwJRnFNyQ36A8XU 8wgBW1U0sPVzveJ+MsdzzNrZTedmOmb8Xeg6ENLdAMmgm5yes+0QD+PqG2YDbHFtzOYaHdHwSF2 qWQ== X-Google-Smtp-Source: AGHT+IGs4gFVi/kxatEGxz/B1lm3LDZuehH0XgpGsn3CIFnEqcrCXAfD8VgVg/nDjjGVQUAHTzXmxtBJ/OQ= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:2101:b0:e03:31ec:8a24 with SMTP id 3f1490d57ef6-e041b17dfb5mr397954276.8.1720664012653; Wed, 10 Jul 2024 19:13:32 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:16 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-5-yuzhao@google.com> Subject: [PATCH mm-unstable v1 4/5] mm/swap: remove remaining _fn suffix From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove remaining _fn suffix from cpu_fbatches handlers, which are already self-explanatory. Signed-off-by: Yu Zhao --- mm/swap.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 774ae9eab1e6..4a66d2f87f26 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -160,7 +160,7 @@ EXPORT_SYMBOL(put_pages_list); =20 typedef void (*move_fn_t)(struct lruvec *lruvec, struct folio *folio); =20 -static void lru_add_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_add(struct lruvec *lruvec, struct folio *folio) { int was_unevictable =3D folio_test_clear_unevictable(folio); long nr_pages =3D folio_nr_pages(folio); @@ -230,7 +230,7 @@ static void folio_batch_add_and_move(struct folio_batch= *fbatch, folio_batch_move_lru(fbatch, move_fn); } =20 -static void lru_move_tail_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_move_tail(struct lruvec *lruvec, struct folio *folio) { if (folio_test_unevictable(folio)) return; @@ -265,7 +265,7 @@ void folio_rotate_reclaimable(struct folio *folio) =20 local_lock_irqsave(&cpu_fbatches.lock_irq, flags); fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_move_tail); - folio_batch_add_and_move(fbatch, folio, lru_move_tail_fn); + folio_batch_add_and_move(fbatch, folio, lru_move_tail); local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); } =20 @@ -527,7 +527,7 @@ void folio_add_lru(struct folio *folio) folio_get(folio); local_lock(&cpu_fbatches.lock); fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_add); - folio_batch_add_and_move(fbatch, folio, lru_add_fn); + folio_batch_add_and_move(fbatch, folio, lru_add); local_unlock(&cpu_fbatches.lock); } EXPORT_SYMBOL(folio_add_lru); @@ -571,7 +571,7 @@ void folio_add_lru_vma(struct folio *folio, struct vm_a= rea_struct *vma) * written out by flusher threads as this is much more efficient * than the single-page writeout from reclaim. */ -static void lru_deactivate_file_fn(struct lruvec *lruvec, struct folio *fo= lio) +static void lru_deactivate_file(struct lruvec *lruvec, struct folio *folio) { bool active =3D folio_test_active(folio); long nr_pages =3D folio_nr_pages(folio); @@ -612,7 +612,7 @@ static void lru_deactivate_file_fn(struct lruvec *lruve= c, struct folio *folio) } } =20 -static void lru_deactivate_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_deactivate(struct lruvec *lruvec, struct folio *folio) { long nr_pages =3D folio_nr_pages(folio); =20 @@ -628,7 +628,7 @@ static void lru_deactivate_fn(struct lruvec *lruvec, st= ruct folio *folio) __count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, nr_pages); } =20 -static void lru_lazyfree_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) { long nr_pages =3D folio_nr_pages(folio); =20 @@ -662,7 +662,7 @@ void lru_add_drain_cpu(int cpu) struct folio_batch *fbatch =3D &fbatches->lru_add; =20 if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, lru_add_fn); + folio_batch_move_lru(fbatch, lru_add); =20 fbatch =3D &fbatches->lru_move_tail; /* Disabling interrupts below acts as a compiler barrier. */ @@ -671,21 +671,21 @@ void lru_add_drain_cpu(int cpu) =20 /* No harm done if a racing interrupt already did this */ local_lock_irqsave(&cpu_fbatches.lock_irq, flags); - folio_batch_move_lru(fbatch, lru_move_tail_fn); + folio_batch_move_lru(fbatch, lru_move_tail); local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); } =20 fbatch =3D &fbatches->lru_deactivate_file; if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, lru_deactivate_file_fn); + folio_batch_move_lru(fbatch, lru_deactivate_file); =20 fbatch =3D &fbatches->lru_deactivate; if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, lru_deactivate_fn); + folio_batch_move_lru(fbatch, lru_deactivate); =20 fbatch =3D &fbatches->lru_lazyfree; if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, lru_lazyfree_fn); + folio_batch_move_lru(fbatch, lru_lazyfree); =20 folio_activate_drain(cpu); } @@ -716,7 +716,7 @@ void deactivate_file_folio(struct folio *folio) =20 local_lock(&cpu_fbatches.lock); fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_deactivate_file); - folio_batch_add_and_move(fbatch, folio, lru_deactivate_file_fn); + folio_batch_add_and_move(fbatch, folio, lru_deactivate_file); local_unlock(&cpu_fbatches.lock); } =20 @@ -743,7 +743,7 @@ void folio_deactivate(struct folio *folio) =20 local_lock(&cpu_fbatches.lock); fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_deactivate); - folio_batch_add_and_move(fbatch, folio, lru_deactivate_fn); + folio_batch_add_and_move(fbatch, folio, lru_deactivate); local_unlock(&cpu_fbatches.lock); } =20 @@ -770,7 +770,7 @@ void folio_mark_lazyfree(struct folio *folio) =20 local_lock(&cpu_fbatches.lock); fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_lazyfree); - folio_batch_add_and_move(fbatch, folio, lru_lazyfree_fn); + folio_batch_add_and_move(fbatch, folio, lru_lazyfree); local_unlock(&cpu_fbatches.lock); } =20 --=20 2.45.2.803.g4e1b14247a-goog From nobody Wed Dec 17 10:20:52 2025 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D436D2E6 for ; Thu, 11 Jul 2024 02:13:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664017; cv=none; b=ndaWYKltVXzch0MvrAvFVdF8mm28fTM+QG8txOaxG8XcfAYM+613oAJ5uHNWpi3vDclt/os4ncMmDjeZfcanJiiYsErzpmgXYSueN2HHDY/8Hs6Ui5AaTOYaQDi7Dh2+bcJ8BA7i5tqHErqLi7HBWIN4FiCWUxuBSdlJVrKTMw8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720664017; c=relaxed/simple; bh=swOwdDH1NV4Az5FcwJyGuyop9Sm6BF+VY5nxNejLdHE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rZ8iShtYPw4FMqD3T5LZITa8X3VIxn8QCobC6sHAcBbU/7hQVP5+W4jPaLArbvIA9vtGA9X/6cUJ3MmP19z9NU47kDmhKEDY8UAKM6JSv1owKFHYhLAq1Mnwek+twhJsipFBmHvT+xuWTwTXBbGNmAbUaqm5FVFarL4Opy/tq9g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ua92oq3Y; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yuzhao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ua92oq3Y" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-65bbd01d146so5835557b3.3 for ; Wed, 10 Jul 2024 19:13:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664015; x=1721268815; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NwEn3rgyWV9MRh72z1BbFtM83NQgy5RTa5P2H131skI=; b=ua92oq3Y77AfCbPrwb5EqH9nNKbsISkAysHWn53oNsTgzG3SdrPRnM4Twgwe8gUtGM sot65fMC/sobEO66w7jYdoli1LhzRltythIgqiQyzZpK7NxxhQzir5t4ZSEy4ZvSfG7N h5p/n0f2txfWxjwl3uLOlUBtrBdnQWgs7Mzql7JAAaU3AE3HG6dAKsVUL7Pnu3a/JQhp QNFsEJ7i7oWu3u+1tpe/B3+KuV8FUb9+U9Q1pmD7NX03k3CIm8WpkQXEc//mBLc9gBKQ 5lRCzVv2xj5gSkqNwnw4VC1NSlAz/qpkBSqH48CT/7imOPrVxQ5lagjvGAyPPdlQ6VRD LW9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664015; x=1721268815; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NwEn3rgyWV9MRh72z1BbFtM83NQgy5RTa5P2H131skI=; b=j/XVObqjmkGG+wBLOv8Z0+QqgiNZn83MYWzlm6WpzC8HZb83B/a+rZa0wB5LuF0H2q 9WSBKH2D2wwQOnnP+su0y8htsKd+NxmHBnBGEiL+hAjbMTocHD6jUp+3rAPXSyGdQO26 oBkGJI1FrfUW6y91xZ1cUK01e+sesV/FHwfH8KTrYd4DjQCjQTQFUNoivFR2jyyTCGpm 1bNCKWsHNJ530Uw5gAfURwMzei905NPEz+sY6EWFm8oo41mgAt9eS5D5VywMb7jXykxG W2uY63X+H+8ZFgKk6NWm7XxHRdObsw7myzISRT5W9HKQnw/KTo/aUVp28uA84ctAIpg6 P9oQ== X-Forwarded-Encrypted: i=1; AJvYcCXt9mxX2sLWJlBf04SkhqTUiF09vIsNJknX4BNbL+xYENpXt7GHYLRit/l4JPqlBkCjD24GipSZAVQe8hDIiOenPm/hR3fr4MZBYZfe X-Gm-Message-State: AOJu0Yy2z4qDrFvJc/hNGobQeucLi9mezGge91VaBS4vHgiVWZpMXCn6 XZlFczdokTAGFYXZcbWI5ttPH3T49rodUaTse+KJz/TXvS6GAuZBC5OTI09USovqvV/D87JSVFb UEg== X-Google-Smtp-Source: AGHT+IEbVZmYW6DpRm49c6xD0RPJKOVNNtKwNJhKE+00HKB2nAyQq5npS5jud2Z6RbIqjV4/yQeeTz+gpoU= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:158c:b0:e03:a0b2:f73 with SMTP id 3f1490d57ef6-e041b05c952mr151870276.6.1720664015346; Wed, 10 Jul 2024 19:13:35 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:17 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-6-yuzhao@google.com> Subject: [PATCH mm-unstable v1 5/5] mm/swap: remove boilerplate From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove boilerplate by using a macro to choose the corresponding lock and handler for each folio_batch in cpu_fbatches. Signed-off-by: Yu Zhao --- mm/swap.c | 107 +++++++++++++++++++----------------------------------- 1 file changed, 37 insertions(+), 70 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 4a66d2f87f26..342ff4e39ba4 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -220,16 +220,45 @@ static void folio_batch_move_lru(struct folio_batch *= fbatch, move_fn_t move_fn) folios_put(fbatch); } =20 -static void folio_batch_add_and_move(struct folio_batch *fbatch, - struct folio *folio, move_fn_t move_fn) +static void __folio_batch_add_and_move(struct folio_batch *fbatch, + struct folio *folio, move_fn_t move_fn, + bool on_lru, bool disable_irq) { + unsigned long flags; + + folio_get(folio); + + if (on_lru && !folio_test_clear_lru(folio)) { + folio_put(folio); + return; + } + if (folio_batch_add(fbatch, folio) && !folio_test_large(folio) && !lru_cache_disabled()) return; =20 + if (disable_irq) + local_lock_irqsave(&cpu_fbatches.lock_irq, flags); + else + local_lock(&cpu_fbatches.lock); + folio_batch_move_lru(fbatch, move_fn); + + if (disable_irq) + local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); + else + local_unlock(&cpu_fbatches.lock); } =20 +#define folio_batch_add_and_move(folio, op, on_lru) \ + __folio_batch_add_and_move( \ + this_cpu_ptr(&cpu_fbatches.op), \ + folio, \ + op, \ + on_lru, \ + offsetof(struct cpu_fbatches, op) > offsetof(struct cpu_fbatches, lock_i= rq) \ + ) + static void lru_move_tail(struct lruvec *lruvec, struct folio *folio) { if (folio_test_unevictable(folio)) @@ -250,23 +279,11 @@ static void lru_move_tail(struct lruvec *lruvec, stru= ct folio *folio) */ void folio_rotate_reclaimable(struct folio *folio) { - struct folio_batch *fbatch; - unsigned long flags; - if (folio_test_locked(folio) || folio_test_dirty(folio) || folio_test_unevictable(folio)) return; =20 - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock_irqsave(&cpu_fbatches.lock_irq, flags); - fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_move_tail); - folio_batch_add_and_move(fbatch, folio, lru_move_tail); - local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); + folio_batch_add_and_move(folio, lru_move_tail, true); } =20 void lru_note_cost(struct lruvec *lruvec, bool file, @@ -355,21 +372,10 @@ static void folio_activate_drain(int cpu) =20 void folio_activate(struct folio *folio) { - struct folio_batch *fbatch; - if (folio_test_active(folio) || folio_test_unevictable(folio)) return; =20 - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock(&cpu_fbatches.lock); - fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_activate); - folio_batch_add_and_move(fbatch, folio, lru_activate); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_activate, true); } =20 #else @@ -513,8 +519,6 @@ EXPORT_SYMBOL(folio_mark_accessed); */ void folio_add_lru(struct folio *folio) { - struct folio_batch *fbatch; - VM_BUG_ON_FOLIO(folio_test_active(folio) && folio_test_unevictable(folio), folio); VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); @@ -524,11 +528,7 @@ void folio_add_lru(struct folio *folio) lru_gen_in_fault() && !(current->flags & PF_MEMALLOC)) folio_set_active(folio); =20 - folio_get(folio); - local_lock(&cpu_fbatches.lock); - fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_add); - folio_batch_add_and_move(fbatch, folio, lru_add); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_add, false); } EXPORT_SYMBOL(folio_add_lru); =20 @@ -702,22 +702,11 @@ void lru_add_drain_cpu(int cpu) */ void deactivate_file_folio(struct folio *folio) { - struct folio_batch *fbatch; - /* Deactivating an unevictable folio will not accelerate reclaim */ if (folio_test_unevictable(folio)) return; =20 - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock(&cpu_fbatches.lock); - fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_deactivate_file); - folio_batch_add_and_move(fbatch, folio, lru_deactivate_file); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_deactivate_file, true); } =20 /* @@ -730,21 +719,10 @@ void deactivate_file_folio(struct folio *folio) */ void folio_deactivate(struct folio *folio) { - struct folio_batch *fbatch; - if (folio_test_unevictable(folio) || !(folio_test_active(folio) || lru_ge= n_enabled())) return; =20 - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock(&cpu_fbatches.lock); - fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_deactivate); - folio_batch_add_and_move(fbatch, folio, lru_deactivate); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_deactivate, true); } =20 /** @@ -756,22 +734,11 @@ void folio_deactivate(struct folio *folio) */ void folio_mark_lazyfree(struct folio *folio) { - struct folio_batch *fbatch; - if (!folio_test_anon(folio) || !folio_test_swapbacked(folio) || folio_test_swapcache(folio) || folio_test_unevictable(folio)) return; =20 - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock(&cpu_fbatches.lock); - fbatch =3D this_cpu_ptr(&cpu_fbatches.lru_lazyfree); - folio_batch_add_and_move(fbatch, folio, lru_lazyfree); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_lazyfree, true); } =20 void lru_add_drain(void) --=20 2.45.2.803.g4e1b14247a-goog