From nobody Thu Dec 18 20:36:30 2025 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E288F1C3F00 for ; Mon, 13 Jan 2025 18:00:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736791224; cv=none; b=mSupBUyU20E4z1R5eu7grCw6M/9AWiLBAM90lJ65C1i+zk5J3XPzaFy+LizJs1qEU+1M3om5OU+KdvDFfwn57xNPxamNZLiaUd5MGZ+bWwm+oI6egkaKH3I+GsJZHsGwHfMa9LuVbAbYRjk7cPEDIoFbxgsbq38XHXBlwcT5/ek= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736791224; c=relaxed/simple; bh=HIejNutHqYjSTnhgFSzWtGIWy4SvmLyElRTFel63dxU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EZv7BvzCaEhmXDwy7II4xhmoY4oH6j3EFiSIRS+Wc6ziCVx9FW8fN3iRp3XF81O0Eyqe2Hjq6ZcAlRizXUwaebxg2v3XnIe4rWGPQBbYGB5gZVxuXSvYdTPWsUkklsG0k36PRLlBn7rIBb97MJh6VDyX241+UH7/E2LL2cINw5s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=O4j7ScUn; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O4j7ScUn" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-2eeb4d643a5so8058854a91.3 for ; Mon, 13 Jan 2025 10:00:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736791222; x=1737396022; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ZcBqrejXjDN/SI+43XhdviH2TahNKIN+YXabcnMV+tI=; b=O4j7ScUn3a5682HI49MfSSG/jyiLljutD2bgXDaiuO7cEy4DNVDYV5v0vZ26cW65gw i/r5x4L3waRq4LbIDThKNmpLygUPk2olL85vqx96BWL3MYADZSqyqKJcHjNXunZXyeDZ QJz5fYqO7jYeHaWRaCjbjXL8M6QvLmx0P4eOu1Z3ad/Yp++PQR7yYoDEm72/Ib+PaiTJ GJjAAqXRECUMypl8Iw635ZVyb6w+hQg8E4ja+gPuJh/WppabeF0zA31Klu0jjPxpbXK4 C6+sotmk1m6BAKvz9l31xWZ/Ix866qhUiAEb/T88cn/m749V3EHFZY/TOa9aVtI8mS/j kewg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736791222; x=1737396022; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ZcBqrejXjDN/SI+43XhdviH2TahNKIN+YXabcnMV+tI=; b=Ysew/jSMajlqjN/n2XNPv1Zu/dxL4zXq59q/Fqlwm/mBdQBesbK+TklYGvmC8wsK5O +xThO8jQqFa2Eu7Nuqz+L98p9g4JfPeQqZIVGBSiiFvDar9T1P2lx4Ul1+sOMd7mHq28 LQtpy9brzdENMKXPasP+Fhn20loqH9r/iOUbBrSLao644LkXVQ/SqkTdbn3lttakPTi4 QGTqeJ4svcZH/qGbLr7NqvzlP+/SmORSQ9HxF+AbLX0EJQkWJm4If1SZSO9V7WIFt11M UbijHV/tc067en66gLmMsErOimqVv7iNL2Db5Xy9cqiIgYAfDnecqFR4kpw39NFsZkJD ZFYQ== X-Forwarded-Encrypted: i=1; AJvYcCXjvGg/yfg2kr/9sX+ARjreovuj1vuKzUyxExYp+nVsWGSJYSIY2qqQumsWPsUvwU/dlDwuKR5m5xVIk6w=@vger.kernel.org X-Gm-Message-State: AOJu0Yw7TnCiaMGMUoxnUWMinlsjnz/3Gs585JhoF6w1Nssat07cndIX yOz+Pb+EOiffHnZYCPEN5cz8t1eq/x/BAQYIMrHNrBZTbvcHaoM7 X-Gm-Gg: ASbGncuRLEk8HzkYLPYGdFJ3s2gWbSuM34x9Hth9Z76Js2+mt9Pd0tNgJYSDh0XIcl6 a4BKwiX6D/A3DZ7OBq1eQzRYxo40vgOL+7d8Ix5NkRJptmYgxpzBc2GtLK6gTqu04Z0Pw+jYCDl Q21eOUDOEqWiCzDyu824A4qlkF3I4AEF35gbIy5nrA37F+OAk4oWMrZidR5iy/RBMXFz9xzTyqv nXCB0IE3eSSbIO2Dlo9sQrb7Ct5f35R7uMl1hwkTIXH2/7XHTKkMX8VRenLaLOg8L9f4UjWDTxm +A== X-Google-Smtp-Source: AGHT+IF7Lv6p+QRH7O5OPr3Tvh76Vf7drCt/uBv+BS+KwCQv0OckMgkTAS/vMPDOVtUVb/e8JFn2VQ== X-Received: by 2002:a17:90b:54c4:b0:2ee:8ea0:6b9c with SMTP id 98e67ed59e1d1-2f548f33220mr37465138a91.12.1736791221839; Mon, 13 Jan 2025 10:00:21 -0800 (PST) Received: from KASONG-MC4.tencent.com ([115.171.41.132]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21a9f21aba7sm57023635ad.113.2025.01.13.10.00.18 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 13 Jan 2025 10:00:21 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Barry Song , Ryan Roberts , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Baoquan He , Nhat Pham , Johannes Weiner , Kalesh Singh , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v4 08/13] mm, swap: use an enum to define all cluster flags and wrap flags changes Date: Tue, 14 Jan 2025 01:57:27 +0800 Message-ID: <20250113175732.48099-9-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250113175732.48099-1-ryncsn@gmail.com> References: <20250113175732.48099-1-ryncsn@gmail.com> Reply-To: Kairui Song Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kairui Song Currently, we are only using flags to indicate which list the cluster is on. Using one bit for each list type might be a waste, as the list type grows, we will consume too many bits. Additionally, the current mixed usage of '&' and '=3D=3D' is a bit confusing. Make it clean by using an enum to define all possible cluster statuses. Only an off-list cluster will have the NONE (0) flag. And use a wrapper to annotate and sanitize all flag settings and list movements. Suggested-by: Chris Li Signed-off-by: Kairui Song --- include/linux/swap.h | 17 +++++++--- mm/swapfile.c | 76 +++++++++++++++++++++++--------------------- 2 files changed, 53 insertions(+), 40 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 02120f1005d5..339d7f0192ff 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -257,10 +257,19 @@ struct swap_cluster_info { u8 order; struct list_head list; }; -#define CLUSTER_FLAG_FREE 1 /* This cluster is free */ -#define CLUSTER_FLAG_NONFULL 2 /* This cluster is on nonfull list */ -#define CLUSTER_FLAG_FRAG 4 /* This cluster is on nonfull list */ -#define CLUSTER_FLAG_FULL 8 /* This cluster is on full list */ + +/* All on-list cluster must have a non-zero flag. */ +enum swap_cluster_flags { + CLUSTER_FLAG_NONE =3D 0, /* For temporary off-list cluster */ + CLUSTER_FLAG_FREE, + CLUSTER_FLAG_NONFULL, + CLUSTER_FLAG_FRAG, + /* Clusters with flags above are allocatable */ + CLUSTER_FLAG_USABLE =3D CLUSTER_FLAG_FRAG, + CLUSTER_FLAG_FULL, + CLUSTER_FLAG_DISCARD, + CLUSTER_FLAG_MAX, +}; =20 /* * The first page in the swap file is the swap header, which is always mar= ked diff --git a/mm/swapfile.c b/mm/swapfile.c index 3898576f947a..b754c9e16c3b 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -403,7 +403,7 @@ static void discard_swap_cluster(struct swap_info_struc= t *si, =20 static inline bool cluster_is_free(struct swap_cluster_info *info) { - return info->flags & CLUSTER_FLAG_FREE; + return info->flags =3D=3D CLUSTER_FLAG_FREE; } =20 static inline unsigned int cluster_index(struct swap_info_struct *si, @@ -434,6 +434,28 @@ static inline void unlock_cluster(struct swap_cluster_= info *ci) spin_unlock(&ci->lock); } =20 +static void move_cluster(struct swap_info_struct *si, + struct swap_cluster_info *ci, struct list_head *list, + enum swap_cluster_flags new_flags) +{ + VM_WARN_ON(ci->flags =3D=3D new_flags); + + BUILD_BUG_ON(1 << sizeof(ci->flags) * BITS_PER_BYTE < CLUSTER_FLAG_MAX); + + if (ci->flags =3D=3D CLUSTER_FLAG_NONE) { + list_add_tail(&ci->list, list); + } else { + if (ci->flags =3D=3D CLUSTER_FLAG_FRAG) { + VM_WARN_ON(!si->frag_cluster_nr[ci->order]); + si->frag_cluster_nr[ci->order]--; + } + list_move_tail(&ci->list, list); + } + ci->flags =3D new_flags; + if (new_flags =3D=3D CLUSTER_FLAG_FRAG) + si->frag_cluster_nr[ci->order]++; +} + /* Add a cluster to discard list and schedule it to do discard */ static void swap_cluster_schedule_discard(struct swap_info_struct *si, struct swap_cluster_info *ci) @@ -447,10 +469,8 @@ static void swap_cluster_schedule_discard(struct swap_= info_struct *si, */ memset(si->swap_map + idx * SWAPFILE_CLUSTER, SWAP_MAP_BAD, SWAPFILE_CLUSTER); - - VM_BUG_ON(ci->flags & CLUSTER_FLAG_FREE); - list_move_tail(&ci->list, &si->discard_clusters); - ci->flags =3D 0; + VM_BUG_ON(ci->flags =3D=3D CLUSTER_FLAG_FREE); + move_cluster(si, ci, &si->discard_clusters, CLUSTER_FLAG_DISCARD); schedule_work(&si->discard_work); } =20 @@ -458,12 +478,7 @@ static void __free_cluster(struct swap_info_struct *si= , struct swap_cluster_info { lockdep_assert_held(&si->lock); lockdep_assert_held(&ci->lock); - - if (ci->flags) - list_move_tail(&ci->list, &si->free_clusters); - else - list_add_tail(&ci->list, &si->free_clusters); - ci->flags =3D CLUSTER_FLAG_FREE; + move_cluster(si, ci, &si->free_clusters, CLUSTER_FLAG_FREE); ci->order =3D 0; } =20 @@ -479,6 +494,8 @@ static void swap_do_scheduled_discard(struct swap_info_= struct *si) while (!list_empty(&si->discard_clusters)) { ci =3D list_first_entry(&si->discard_clusters, struct swap_cluster_info,= list); list_del(&ci->list); + /* Must clear flag when taking a cluster off-list */ + ci->flags =3D CLUSTER_FLAG_NONE; idx =3D cluster_index(si, ci); spin_unlock(&si->lock); =20 @@ -519,9 +536,6 @@ static void free_cluster(struct swap_info_struct *si, s= truct swap_cluster_info * lockdep_assert_held(&si->lock); lockdep_assert_held(&ci->lock); =20 - if (ci->flags & CLUSTER_FLAG_FRAG) - si->frag_cluster_nr[ci->order]--; - /* * If the swap is discardable, prepare discard the cluster * instead of free it immediately. The cluster will be freed @@ -573,13 +587,9 @@ static void dec_cluster_info_page(struct swap_info_str= uct *si, return; } =20 - if (!(ci->flags & CLUSTER_FLAG_NONFULL)) { - VM_BUG_ON(ci->flags & CLUSTER_FLAG_FREE); - if (ci->flags & CLUSTER_FLAG_FRAG) - si->frag_cluster_nr[ci->order]--; - list_move_tail(&ci->list, &si->nonfull_clusters[ci->order]); - ci->flags =3D CLUSTER_FLAG_NONFULL; - } + if (ci->flags !=3D CLUSTER_FLAG_NONFULL) + move_cluster(si, ci, &si->nonfull_clusters[ci->order], + CLUSTER_FLAG_NONFULL); } =20 static bool cluster_reclaim_range(struct swap_info_struct *si, @@ -663,11 +673,13 @@ static bool cluster_alloc_range(struct swap_info_stru= ct *si, struct swap_cluster if (!(si->flags & SWP_WRITEOK)) return false; =20 + VM_BUG_ON(ci->flags =3D=3D CLUSTER_FLAG_NONE); + VM_BUG_ON(ci->flags > CLUSTER_FLAG_USABLE); + if (cluster_is_free(ci)) { - if (nr_pages < SWAPFILE_CLUSTER) { - list_move_tail(&ci->list, &si->nonfull_clusters[order]); - ci->flags =3D CLUSTER_FLAG_NONFULL; - } + if (nr_pages < SWAPFILE_CLUSTER) + move_cluster(si, ci, &si->nonfull_clusters[order], + CLUSTER_FLAG_NONFULL); ci->order =3D order; } =20 @@ -675,14 +687,8 @@ static bool cluster_alloc_range(struct swap_info_struc= t *si, struct swap_cluster swap_range_alloc(si, nr_pages); ci->count +=3D nr_pages; =20 - if (ci->count =3D=3D SWAPFILE_CLUSTER) { - VM_BUG_ON(!(ci->flags & - (CLUSTER_FLAG_FREE | CLUSTER_FLAG_NONFULL | CLUSTER_FLAG_FRAG))); - if (ci->flags & CLUSTER_FLAG_FRAG) - si->frag_cluster_nr[ci->order]--; - list_move_tail(&ci->list, &si->full_clusters); - ci->flags =3D CLUSTER_FLAG_FULL; - } + if (ci->count =3D=3D SWAPFILE_CLUSTER) + move_cluster(si, ci, &si->full_clusters, CLUSTER_FLAG_FULL); =20 return true; } @@ -821,9 +827,7 @@ static unsigned long cluster_alloc_swap_entry(struct sw= ap_info_struct *si, int o while (!list_empty(&si->nonfull_clusters[order])) { ci =3D list_first_entry(&si->nonfull_clusters[order], struct swap_cluster_info, list); - list_move_tail(&ci->list, &si->frag_clusters[order]); - ci->flags =3D CLUSTER_FLAG_FRAG; - si->frag_cluster_nr[order]++; + move_cluster(si, ci, &si->frag_clusters[order], CLUSTER_FLAG_FRAG); offset =3D alloc_swap_scan_cluster(si, cluster_offset(si, ci), &found, order, usage); frags++; --=20 2.47.1