From nobody Sat Feb 7 13:45:50 2026 Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B1EF1DE3AE for ; Mon, 2 Dec 2024 18:42:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733164934; cv=none; b=o1msoLf5NgPxbISg0uhlBe0QDd+NBMHH4ApxzZJvy6PBdJHRf+0RYqqAeMt7vg6pB2agxAsX2x8sNxPtt/jxjLGhwMPONdeCk0Inw9XGeqwqcvuq9Uep1HFST3B5KhQxoFo90ksSsO1hBs2RwaQFMIIUo/o98V4Ga3/HlUYzFvc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733164934; c=relaxed/simple; bh=c8ZDldLXg0Q+M0U0cvWmOKA0bxBuSKo/xc83hM0pd2k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ohf21h1hC+EYATNSZSUyYZQlAo/ghOGeI4OWyOe35Tmjb+QbMYsCeMieV3QdBuj7CRm3KYeIvxmCZEFheS4oAsGUi7UyTJKbr+1RBEX/I5RewJzpePnMdpZ9zIE+j9v598JEEgDwC5/WaKIr9TzYSYKdRNwyjiChzPVTXauc1Fw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hn6DZHe3; arc=none smtp.client-ip=209.85.215.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hn6DZHe3" Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-7fbbe0fb0b8so3390779a12.0 for ; Mon, 02 Dec 2024 10:42:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733164932; x=1733769732; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=zPi/40j9fg7YjPYY/QjOr+/MPbDnr/kKPaDCNQF5gDU=; b=hn6DZHe3bXWyhjZBsB9Un6UIXx2rVpx58rlKiSRyz1kB2cD2BT1RwTSKvz3X5KjXzH +5JtxtXjrawMNGPpFsshLEUzIfC/pli3bCTquRfKsbwk/SBPTL8/dOHrLqGgy2ib3xI0 e8n2nVTe8KZ2jzHiFpazhwu8JXW/+t3ZwC0+Ztg3OgpwIDJd4671vmQUOaYEdKpGxuNi jRuRFSDQubAP4rrtzl2p5NBkxHKBozImkMGsAyt+K/MJQ3TTGxcC5rQxf6chKBQCokE2 BHQWXmv+U87NUft72EUgdW5aEEkGbEWccbbPTVhlRXrQoLl3ikFANCBPhGG6VT3sGOEY jIdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733164932; x=1733769732; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zPi/40j9fg7YjPYY/QjOr+/MPbDnr/kKPaDCNQF5gDU=; b=vTlGm2a6CHqGoYYY4leaFScqguX4PGym5DPni2w9U9xGu5R/h/mkbZq8+iTHfVp8C+ cRlC1nXz/gISR6wx1gEazuCOhy/WQS7LHcaqMJmGyqN42KPxaYKsVn7l8WM1rZuYv2Qj xFbMlB/alyVkN9v7UBb7wTw5P2VC4VwsgX9WRWpWodPiIQL/VelXSRzM8aqoNVpNZbru AsN1zF2jZ6g9KV2UBoHPbM7ckNN8+mDxHRPYnpLHlT9SuzsAwafxd0+mEZ+Jbn1Tcjvd 4UPxLYOn0imoIB/5dC4cbGcZKokmnove4ZoQqYlDFHRLFlp2RkywXTbUU03KfR9pLfey uX0A== X-Forwarded-Encrypted: i=1; AJvYcCWptr7/s/1nuWhpqho1ziXwFKOpyKcIwpqDqZNRyCfSQxHQpxxfTF2kk+wOvwzbAfjd4iXCDzPPps9RT1M=@vger.kernel.org X-Gm-Message-State: AOJu0YwOPvY5NPcHXB2nOm4A8vx7CRQwthEE9r+mDJYLoTntOoQCz20L zqsrZLw9WLWTMNFe1n8/zSSdMawyiiSQuYgNq77w1A6QEzEcrWoE X-Gm-Gg: ASbGncvakUCyd18eARIVqSU3pmLZaPIZevxbma/SY7cGfKdTmmmeAUJIzI11GkaHpXe b7TGsWtz2ulSaV/plRdK3TZU1lDjOt626LqH3ZtKE+4AhVmNsaruXvUhSl5+H2oX7IkA/STEnma A6bD4gJHJH1Efv+jk1pvDlEQrde1PyLg2/F79xrdkseWdWlErYXI7VW604DpCL6ylSvXLO9Qyx9 6oBJe2/3QCaZUXj42fCxhUK0tVrNnGWL/J0YbBHjxSEl+wiI4YYN7pDLGYoa2Wohc6N9UY= X-Google-Smtp-Source: AGHT+IE35vF16FjRydL5iBcJInocVBicSo1ZjyULb2W4jf/KJQsy63zCROjVa8zaUbfc3UsLb+zrGQ== X-Received: by 2002:a05:6a20:7350:b0:1e0:c378:6f5b with SMTP id adf61e73a8af0-1e0e0b71f30mr31209157637.38.1733164931986; Mon, 02 Dec 2024 10:42:11 -0800 (PST) Received: from KASONG-MC4.tencent.com ([106.37.120.120]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417fb7d5sm8849781b3a.94.2024.12.02.10.42.08 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 02 Dec 2024 10:42:11 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Hugh Dickins , "Huang, Ying" , Yosry Ahmed , Roman Gushchin , Shakeel Butt , Johannes Weiner , Barry Song , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 1/4] mm, memcontrol: avoid duplicated memcg enable check Date: Tue, 3 Dec 2024 02:41:51 +0800 Message-ID: <20241202184154.19321-2-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241202184154.19321-1-ryncsn@gmail.com> References: <20241202184154.19321-1-ryncsn@gmail.com> Reply-To: Kairui Song Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kairui Song mem_cgroup_uncharge_swap() implies a mem_cgroup_disabled() check, which is already checked by the caller here. Skip it by calling __mem_cgroup_uncharge_swap() directly. Signed-off-by: Kairui Song Acked-by: Chris Li Acked-by: Shakeel Butt Reviewed-by: Barry Song Reviewed-by: Roman Gushchin Reviewed-by: Yosry Ahmed --- mm/memcontrol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b3503d12aaf..d3d1eb506eee 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4615,7 +4615,7 @@ void mem_cgroup_swapin_uncharge_swap(swp_entry_t entr= y, unsigned int nr_pages) * let's not wait for it. The page already received a * memory+swap charge, drop the swap entry duplicate. */ - mem_cgroup_uncharge_swap(entry, nr_pages); + __mem_cgroup_uncharge_swap(entry, nr_pages); } } =20 --=20 2.47.0 From nobody Sat Feb 7 13:45:50 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D23A51DE2B9 for ; Mon, 2 Dec 2024 18:42:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733164938; cv=none; b=UmP27MkjBiiHPorpqkcrym96b9I8C+GAh2c7Fr16laVlXOoBoHLTv8YbsiLfvhyoYvamx8zAtlsQK/zJJvF/SkdhqgzCYkU+phq5MoZ+qjo07/JFwo8SHvzJbA+bYGD6QevPiQ3xIapq8JGtq7Yob6Xr+xybC4ZQUfhCp3bdasM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733164938; c=relaxed/simple; bh=7xMcIoyfw2WhDIAFKEZLat3eNgZVj3iRjhG+H08KVv4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DYcHoJBKGah5U+RJRUe+SaJvvy+/2oN8OfcXbKddS5aRhWPSuZmMIrqV0Mr9HaXBFdmZ7ajGsWlDcVEFh97r36Bs0gHKjVNp4dNyUtuI2jb2rfvK0jo3u5ychSPjABngMDmehAx23UfwPzGaqTPgF/rwOCj8O76+25AVl8Qrlr0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JAJ7DNVH; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JAJ7DNVH" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-724fc1aaa91so4186490b3a.3 for ; Mon, 02 Dec 2024 10:42:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733164936; x=1733769736; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=bgfYlxq8M7pj/EQVq83uOmq930B/5yWrzKvSMuqYhjs=; b=JAJ7DNVHcSGWICQsHg/KNKBfta+N95ILEholS4MSG+W92ato4WjQAuun24cIGSTy7I teMozxFj7kieQMeE88RdUEBUBwymXfo+Gja2hTDJiwsIZLQGMWA6hKALMtOZJghuHqHe 7bwRve59z8ZsHiVC7XXItswo3MpoEhmE6R5CiTp/FH0f9Gf4IHp3RF/oZL3JHh8GpcCc E/PVjmsB8IWF7Be+jonbhkQxxdNxu3mpmKW96wn2IOTtNCEWHgTKAWhivtdQIuLZ02/f 0jXiLVnmwaNduMMokdNqIUGw3I/+ZZhPxsWAS3ruHCfaVsnHzE0aEjSLJWWXRvevrKJr iG2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733164936; x=1733769736; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=bgfYlxq8M7pj/EQVq83uOmq930B/5yWrzKvSMuqYhjs=; b=KAT/U15iJ9CGf50YGXJVzFsTHUrOUyZKRDHYRBD006rzB/36XRvQbjYaDK3lVuBX51 gjBCaQ1jS/0QKuWRZH7yExD9dVQYAv2OqAmaWEeDnYVRva34dJMuE0N9QHd/CWcIV78d 9K7ADMeVmjNbhXj+NrATv5f2/gizYsb2Yk8FTtx2vtGLkpEuT2dVYso+m4v4NkJPz4Cy UvR1LTsZJrz33q/q4KZ4IcKxrhQRkJkGMf8dL381dfln0hNkA6SVtVA31ozhjU0idjYT LYdWu/gGZjw255K1uqqOe7rmj8BS48ufEjCQdQoVt2WZ0rQboBmlbAmpJVjKcBRwcO+8 0PCA== X-Forwarded-Encrypted: i=1; AJvYcCVcya69si12NAi4wKdyEv1+Fp6BNfNWraZhk0QKmt8pZ33bjHNrAqBuu2C1Fitw1uyzWJeH8H2HrE6CjOE=@vger.kernel.org X-Gm-Message-State: AOJu0Yy+wHHvltjX4p1pKdQSg37i0vvZPslM+8x0MlcA+X5w3Gc402x8 Kab6dhWl1G5v/LaLlJgrjlvz6BzTjz8c8q/t2L8pREKmHcwC112l X-Gm-Gg: ASbGnctcAJuV1SSYfdmd/IDNvFHrUtAizXHbKFNvKl0D0k3q207TTnW/6zD5nmIOmrW GcScMitnx7f8wwpw7uNvYjstvigpHJRxbcraWAD8d3n4JeeMNAKf/LiCSM1tSiEVrWrHjAX1RzN VA9zFGFnA9ljyJPI1cZRXzluo7zdI9mCx4LGjHeyjJEYZllm+qkkbx6WO7iVel2sdmWz8c2iBsf zKs6qFuN4mvZguUpVDHfbbG2BYcG7smPT8sLXwGiob6xUgUdEXcmJG/6JERnoWZ5P7AXtA= X-Google-Smtp-Source: AGHT+IEeppgHPbYXo/4IKwmclLHEdajSYeynVAjocWeV2X5vj8HNw3SKD3ti5afx7GbXPWkD1M5KNQ== X-Received: by 2002:a05:6a00:1c8d:b0:725:3bd4:9b56 with SMTP id d2e1a72fcca58-7253bd49c85mr31675949b3a.5.1733164936073; Mon, 02 Dec 2024 10:42:16 -0800 (PST) Received: from KASONG-MC4.tencent.com ([106.37.120.120]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417fb7d5sm8849781b3a.94.2024.12.02.10.42.12 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 02 Dec 2024 10:42:15 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Hugh Dickins , "Huang, Ying" , Yosry Ahmed , Roman Gushchin , Shakeel Butt , Johannes Weiner , Barry Song , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 2/4] mm/swap_cgroup: remove swap_cgroup_cmpxchg Date: Tue, 3 Dec 2024 02:41:52 +0800 Message-ID: <20241202184154.19321-3-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241202184154.19321-1-ryncsn@gmail.com> References: <20241202184154.19321-1-ryncsn@gmail.com> Reply-To: Kairui Song Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kairui Song This function is never used after commit 6b611388b626 ("memcg-v1: remove charge move code"). Signed-off-by: Kairui Song Acked-by: Chris Li Acked-by: Shakeel Butt Reviewed-by: Roman Gushchin Reviewed-by: Yosry Ahmed --- include/linux/swap_cgroup.h | 2 -- mm/swap_cgroup.c | 29 ----------------------------- 2 files changed, 31 deletions(-) diff --git a/include/linux/swap_cgroup.h b/include/linux/swap_cgroup.h index ae73a87775b3..d521ad1c4164 100644 --- a/include/linux/swap_cgroup.h +++ b/include/linux/swap_cgroup.h @@ -6,8 +6,6 @@ =20 #if defined(CONFIG_MEMCG) && defined(CONFIG_SWAP) =20 -extern unsigned short swap_cgroup_cmpxchg(swp_entry_t ent, - unsigned short old, unsigned short new); extern unsigned short swap_cgroup_record(swp_entry_t ent, unsigned short i= d, unsigned int nr_ents); extern unsigned short lookup_swap_cgroup_id(swp_entry_t ent); diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c index f63d1aa072a1..1770b076f6b7 100644 --- a/mm/swap_cgroup.c +++ b/mm/swap_cgroup.c @@ -45,35 +45,6 @@ static struct swap_cgroup *lookup_swap_cgroup(swp_entry_= t ent, return &ctrl->map[offset]; } =20 -/** - * swap_cgroup_cmpxchg - cmpxchg mem_cgroup's id for this swp_entry. - * @ent: swap entry to be cmpxchged - * @old: old id - * @new: new id - * - * Returns old id at success, 0 at failure. - * (There is no mem_cgroup using 0 as its id) - */ -unsigned short swap_cgroup_cmpxchg(swp_entry_t ent, - unsigned short old, unsigned short new) -{ - struct swap_cgroup_ctrl *ctrl; - struct swap_cgroup *sc; - unsigned long flags; - unsigned short retval; - - sc =3D lookup_swap_cgroup(ent, &ctrl); - - spin_lock_irqsave(&ctrl->lock, flags); - retval =3D sc->id; - if (retval =3D=3D old) - sc->id =3D new; - else - retval =3D 0; - spin_unlock_irqrestore(&ctrl->lock, flags); - return retval; -} - /** * swap_cgroup_record - record mem_cgroup for a set of swap entries * @ent: the first swap entry to be recorded into --=20 2.47.0 From nobody Sat Feb 7 13:45:50 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1861E1DE3A6 for ; Mon, 2 Dec 2024 18:42:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733164943; cv=none; b=EeTydwHWWWdtWJdNvm2fNGyU8HJKf9GSAnAF7u0Kbhu5yA6197jbTIlodjL1KnGNdqe2EN5Odm+14eCWnW9lcmGlnAOD8FSPqBD84A3ICMsMjiLRxu5i+Dh+LIonPJnB2gYJD+WZ0RdMilwKGfFkqWHYnBGc6SZp5ua+uGDc7cw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733164943; c=relaxed/simple; bh=kohIEs6g/0WO80fMbekc0VeT6TBBEae4RbP8+DGDZrU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WduKyws/9imjTL9KYb8pAMQBNWBTEpINvrxut4eeQRjKUPzVUZKE+4ty/sGdxOa7poxDblZcrFfbvTMSOVsPdxFy5S2wMwDuuiGyod8p7/dgYr8GsZoaFywzzjRIEH7xUpVs178yvuHSFz5rVUEz6v9IJ62Y4xviebYq9Mh3xmE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=McTFBxXp; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="McTFBxXp" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-724d57a9f7cso3895488b3a.3 for ; Mon, 02 Dec 2024 10:42:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733164940; x=1733769740; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=1aeU1SawrmUs4i99mcy6Gmav34YptGDoV51/MymT7EU=; b=McTFBxXp0+ybvd8OlR4iRofdkvFq09vmJm9hdC/ykKwb5/K09ctDLsipSPtpvEeUS4 XpgQ65K4l1AVSSrKzzAdDI8QhrM1FoFSojBr/jlsUi1jXT4hf3sijNUJpH6oR+eRkeAD DqpSXvIJEhdwPn474kH9z9U7uzqPn0oMcLIqMQnUSj4Gndv0fZKedSgtkQtVVv3L/ZVP hjnp+3zXBtYJbOxNU9Gw3ZzkGU3zDon7AP35sJfnB7aNdANqc+QEzQcGiml74JKYT9Dv oI4ouHFoFL2EX0/td59Hq/Ek+GDcNyjvShRzSL7beogJv+w9bcFNTMMJgRhVXivLu51t EbFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733164940; x=1733769740; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=1aeU1SawrmUs4i99mcy6Gmav34YptGDoV51/MymT7EU=; b=mVLSAo8OJd/nmn/2K6scLZ3I35hv4Wm+PWrovWwxPoSG0H8tlsMr0PPPQV95gKKycL uqXN8q1ZiBSUrABLLy94GOZQ8jHmw90UuhWD7Oo2rTEArm3hvFzZYZdJgLfa94EjPL05 UmScx+4Urf5lba7tsVEtT3G6uVEk5NagUx5gwFdMW4xFcocc7A+l3GR81Lxkc9GvzX0J Tg9MkuD2BGfc5T4pvhhIuPFwyGctag51OsBjiWxW+qKYyg2q+oQNuZV4lgKLLecWSs60 rJ3O7yXbA9eucvBjeuINInHoTLkFaRKLkaobto81mxrSP888JTdrc4oX8Yn/DdPFkMfR FEzg== X-Forwarded-Encrypted: i=1; AJvYcCVqDoZOZK2uoSSIsmv2cytNZW1dpnCapV47Ux9XZib3HTUlbN4VGvhExADif2OovCDvFz9bn4cKhh718wk=@vger.kernel.org X-Gm-Message-State: AOJu0YynsJJ2QyysEzbSorrHObvBPrultvQy9lhC/CBv7oJfGC0NShmV GqVxpdIlYq5m8wR+4tRVGytcSVTwx25RjNdPc8q4UncJBaQy3QVK X-Gm-Gg: ASbGncvJzgMNocQLGZ+ikG/WO72SPZZPt7iTd6BspIm2+upQ7BvNqpnYSxj2MJ909l+ Miky8s53WGq9nSDAJld51kY+CmLyCHmkWix5+6wrqttS70/rfp2ITaoaXCG4YfBqL5OBHu7o0sK 4BWmegOMpd237vAwwXSGiKaEtpVMkO5SRNH/SDSJLqvSDXL6Iln1i+Ovm9YVAA4knmGRXsgtF99 sYmeCsq2fu9l+J4D2cE3aKP95hXfE/541f8crIu3mY6UxsQlIWfuklV6f+HMvbyGlavgBE= X-Google-Smtp-Source: AGHT+IGfiyQ6y7KACpekf7bdygxRsz/ROBwam32XtlPd4Ct4vBnneUD3+Zdg4Z2+z0w6o72q8gMZxw== X-Received: by 2002:a05:6a00:1a8b:b0:724:fcff:505b with SMTP id d2e1a72fcca58-7253007218fmr32371685b3a.14.1733164940196; Mon, 02 Dec 2024 10:42:20 -0800 (PST) Received: from KASONG-MC4.tencent.com ([106.37.120.120]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417fb7d5sm8849781b3a.94.2024.12.02.10.42.16 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 02 Dec 2024 10:42:19 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Hugh Dickins , "Huang, Ying" , Yosry Ahmed , Roman Gushchin , Shakeel Butt , Johannes Weiner , Barry Song , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 3/4] mm/swap_cgroup: simplify swap cgroup definitions Date: Tue, 3 Dec 2024 02:41:53 +0800 Message-ID: <20241202184154.19321-4-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241202184154.19321-1-ryncsn@gmail.com> References: <20241202184154.19321-1-ryncsn@gmail.com> Reply-To: Kairui Song Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kairui Song Remove the intermediate struct swap_cgroup, it just a unsigned short wrapper, simplify the code. Also zero the map on initialization to prevent unexpected behaviour as swap cgroup helpers are suppose to return 0 on error. Signed-off-by: Kairui Song --- mm/swap_cgroup.c | 45 +++++++++++++++++++-------------------------- 1 file changed, 19 insertions(+), 26 deletions(-) diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c index 1770b076f6b7..a76afdc3666a 100644 --- a/mm/swap_cgroup.c +++ b/mm/swap_cgroup.c @@ -12,14 +12,12 @@ struct swap_cgroup { }; =20 struct swap_cgroup_ctrl { - struct swap_cgroup *map; + unsigned short *map; spinlock_t lock; }; =20 static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAPFILES]; =20 -#define SC_PER_PAGE (PAGE_SIZE/sizeof(struct swap_cgroup)) - /* * SwapCgroup implements "lookup" and "exchange" operations. * In typical usage, this swap_cgroup is accessed via memcg's charge/uncha= rge @@ -33,18 +31,6 @@ static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAP= FILES]; * * TODO: we can push these buffers out to HIGHMEM. */ -static struct swap_cgroup *lookup_swap_cgroup(swp_entry_t ent, - struct swap_cgroup_ctrl **ctrlp) -{ - pgoff_t offset =3D swp_offset(ent); - struct swap_cgroup_ctrl *ctrl; - - ctrl =3D &swap_cgroup_ctrl[swp_type(ent)]; - if (ctrlp) - *ctrlp =3D ctrl; - return &ctrl->map[offset]; -} - /** * swap_cgroup_record - record mem_cgroup for a set of swap entries * @ent: the first swap entry to be recorded into @@ -58,20 +44,21 @@ unsigned short swap_cgroup_record(swp_entry_t ent, unsi= gned short id, unsigned int nr_ents) { struct swap_cgroup_ctrl *ctrl; - struct swap_cgroup *sc; + unsigned short *map; unsigned short old; unsigned long flags; pgoff_t offset =3D swp_offset(ent); pgoff_t end =3D offset + nr_ents; =20 - sc =3D lookup_swap_cgroup(ent, &ctrl); + ctrl =3D &swap_cgroup_ctrl[swp_type(ent)]; + map =3D ctrl->map; =20 spin_lock_irqsave(&ctrl->lock, flags); - old =3D sc->id; - for (; offset < end; offset++, sc++) { - VM_BUG_ON(sc->id !=3D old); - sc->id =3D id; - } + old =3D map[offset]; + do { + VM_BUG_ON(map[offset] !=3D old); + map[offset] =3D id; + } while (++offset !=3D end); spin_unlock_irqrestore(&ctrl->lock, flags); =20 return old; @@ -85,20 +72,26 @@ unsigned short swap_cgroup_record(swp_entry_t ent, unsi= gned short id, */ unsigned short lookup_swap_cgroup_id(swp_entry_t ent) { + struct swap_cgroup_ctrl *ctrl; + if (mem_cgroup_disabled()) return 0; - return lookup_swap_cgroup(ent, NULL)->id; + + ctrl =3D &swap_cgroup_ctrl[swp_type(ent)]; + pgoff_t offset =3D swp_offset(ent); + + return READ_ONCE(ctrl->map[offset]); } =20 int swap_cgroup_swapon(int type, unsigned long max_pages) { - struct swap_cgroup *map; + void *map; struct swap_cgroup_ctrl *ctrl; =20 if (mem_cgroup_disabled()) return 0; =20 - map =3D vcalloc(max_pages, sizeof(struct swap_cgroup)); + map =3D vzalloc(max_pages * sizeof(unsigned short)); if (!map) goto nomem; =20 @@ -117,7 +110,7 @@ int swap_cgroup_swapon(int type, unsigned long max_page= s) =20 void swap_cgroup_swapoff(int type) { - struct swap_cgroup *map; + void *map; struct swap_cgroup_ctrl *ctrl; =20 if (mem_cgroup_disabled()) --=20 2.47.0 From nobody Sat Feb 7 13:45:50 2026 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 455A01DE8A5 for ; Mon, 2 Dec 2024 18:42:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733164947; cv=none; b=RnfTYUfepldc/N3TZ9cY5N5uIxDLkmlRRPXR1NdRuTPwhpYZmdw5bAggsdKj5vkRvv349IaEycoDhL3ZBVI9v3IlLdGfuD0OOsW4bs4zaVWDLpt9+nWV4RttsNwTqWH7nfh9EIXFeL99xzKSzBFV/IdZGKNJfWm+W7Lw1YL3gU0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733164947; c=relaxed/simple; bh=03Tp5Bvvbel16Ml166yBp6r99cBxzKBlFQMhKZVMWzg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ncl585SQGZ/18hiRNre3I8xjVpfzndb4WBea2hDKVzjwWY6WHxLRY02fWOqQLMslD6q4deaQUhcsqCalTu4HHarqzN/PYdoV+rTDskaMnXP/YmlOrC+m6TUzVmq1sgMiB9D4kL1uhp8ZOF7uRyX7MK9R+4nuhvBQmBRcD+wS0cs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WQVgWGbr; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WQVgWGbr" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-7251abe0e69so3952241b3a.0 for ; Mon, 02 Dec 2024 10:42:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733164944; x=1733769744; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=n486kN9w2/p4kQRIxzmOz9BLiT1ewnt6ZG6JQNAAsl4=; b=WQVgWGbrIfwnnPLFx0F7wRrV9UhKNqyviiiLjrzlzNnFpoXEbRmbHe3qIbgbkHu0kq dVJKSReAKfibzFNPytdhvjPR0ukR1s/R3e5L9dxBo1uB2cE/FUWTlS7IUpkKpoWlXT1V j0rObOZ9JKq3yXT4stxbud4au5L8yVDZSjVLD2vVZL1s6pTW0a8trUj/YWJKCAb+7R2Y rsRdRwKULUX0xhca81PFYTyWtnzWkdPzJ0mVE+B1/cJ4D2uruLRtleENN14cXSNMpXTx MS0EMeOfIlw2zvvFuFt+C2VX2xkCTA4oirGYeVeupeKiPkHlWtWDjgtmoaTumpNHEao/ W6UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733164944; x=1733769744; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=n486kN9w2/p4kQRIxzmOz9BLiT1ewnt6ZG6JQNAAsl4=; b=uvbiKX8Y31qP2ph2WhiBxnfxJ2dJqO9+2TSSTCbjjVcGjwmw3bbZsY9YHLKngV8djj GFP0IA1ur8Q9zvLCiQ/nqjjQHn/htCvXeLroagCNVbYcrq3oF9uQhhwfAESsUoXGPROm DRtj4U+K6oO3eBq4l51hDefqWFCuM1kUpCrXbr7VXbVLaD1FFoMBznLfyx6M4Wwr8WF5 xqZ6r76R5SSfuoCLpujTdKa9H4hGoEMnXWwax+2OpLjCvrdEKsVlFKP5vj4yylLtWjlV aeYaD/r34HbtCIwyXFeBbeSO4qOmfh9W9gvKKq7uyhQFAyalqF5/No2NIb7hjTeJwVoj i8UA== X-Forwarded-Encrypted: i=1; AJvYcCWpeD9yVKr6t6SJtiC+S7KupolV2G1eOHmAvcsCNyHWA6EJyjSR0fbxVPwCcyS9pVqh1NGFGEr0Fb1dafU=@vger.kernel.org X-Gm-Message-State: AOJu0YzUzYIZiH/6Tcic8QLA2KhiUkEnf9kvq+8j3kpaYbWdXPGSSx/p 1poDNMpA4J8ABwOwU3NdpFWwxQfedXMZ3x2okb7sNu+Tyj2GotjQPQwESM3sWHP1LA== X-Gm-Gg: ASbGncvOwPwHC1t7+uqs1dhRtekiF+wID4k4kIiiQNKs9++ctoyBW+aSE6dn9i8id+G v/M9HlI5FCs7+oprwfoXYzqV37F8yWROXmbkoUO4QdchuIKDFRx5zrGXi+jOzKe4acC2iy5dXIk UUqceFh+CtOoRgT7rlw+W11FNUaTRBMQoPT90JISpPY9e2enFq8Tlo+I78auV9v+scnTEoHIDZ1 6hl8mBrOYepm+XRwdtre3SquFrCpMa4ZTZq0D5me8MIfvYPs4D0Rdef6vTH/qVrCcCPlKg= X-Google-Smtp-Source: AGHT+IE6F7f6lJrUVeUD4BRVPb3PuBX80oqzE3+WCYBtL1H232mPA2ctT8z6mSlYDWpdAGJGBit8ww== X-Received: by 2002:a05:6a00:4f81:b0:71d:eb7d:20d5 with SMTP id d2e1a72fcca58-72530045b6emr37194340b3a.8.1733164944304; Mon, 02 Dec 2024 10:42:24 -0800 (PST) Received: from KASONG-MC4.tencent.com ([106.37.120.120]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725417fb7d5sm8849781b3a.94.2024.12.02.10.42.20 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 02 Dec 2024 10:42:23 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Hugh Dickins , "Huang, Ying" , Yosry Ahmed , Roman Gushchin , Shakeel Butt , Johannes Weiner , Barry Song , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 4/4] mm, swap_cgroup: remove global swap cgroup lock Date: Tue, 3 Dec 2024 02:41:54 +0800 Message-ID: <20241202184154.19321-5-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241202184154.19321-1-ryncsn@gmail.com> References: <20241202184154.19321-1-ryncsn@gmail.com> Reply-To: Kairui Song Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kairui Song commit e9e58a4ec3b1 ("memcg: avoid use cmpxchg in swap cgroup maintainance") replaced the cmpxchg/xchg with a global irq spinlock because some archs doesn't support 2 bytes cmpxchg/xchg. Clearly this won't scale well. And as commented in swap_cgroup.c, this lock is not needed for map synchronization. Emulation of 2 bytes cmpxchg/xchg with atomic isn't hard, so implement it to get rid of this lock. Testing using 64G brd and build with build kernel with make -j96 in 1.5G memory cgroup using 4k folios showed below improvement (10 test run): Before this series: Sys time: 10730.08 (stdev 49.030728) Real time: 171.03 (stdev 0.850355) After this commit: Sys time: 9612.24 (stdev 66.310789), -10.42% Real time: 159.78 (stdev 0.577193), -6.57% With 64k folios and 2G memcg: Before this series: Sys time: 7626.77 (stdev 43.545517) Real time: 136.22 (stdev 1.265544) After this commit: Sys time: 6936.03 (stdev 39.996280), -9.06% Real time: 129.65 (stdev 0.880039), -4.82% Sequential swapout of 8G 4k zero folios (24 test run): Before this series: 5461409.12 us (stdev 183957.827084) After this commit: 5420447.26 us (stdev 196419.240317) Sequential swapin of 8G 4k zero folios (24 test run): Before this series: 19736958.916667 us (stdev 189027.246676) After this commit: 19662182.629630 us (stdev 172717.640614) Performance is better or at least not worse for all tests above. Signed-off-by: Kairui Song --- mm/swap_cgroup.c | 56 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 41 insertions(+), 15 deletions(-) diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c index a76afdc3666a..028f5e6be3f0 100644 --- a/mm/swap_cgroup.c +++ b/mm/swap_cgroup.c @@ -5,6 +5,15 @@ =20 #include /* depends on mm.h include */ =20 +#define ID_PER_UNIT (sizeof(atomic_t) / sizeof(unsigned short)) +struct swap_cgroup_unit { + union { + int raw; + atomic_t val; + unsigned short __id[ID_PER_UNIT]; + }; +}; + static DEFINE_MUTEX(swap_cgroup_mutex); =20 struct swap_cgroup { @@ -12,8 +21,10 @@ struct swap_cgroup { }; =20 struct swap_cgroup_ctrl { - unsigned short *map; - spinlock_t lock; + union { + struct swap_cgroup_unit *units; + unsigned short *map; + }; }; =20 static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAPFILES]; @@ -31,6 +42,24 @@ static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAP= FILES]; * * TODO: we can push these buffers out to HIGHMEM. */ +static unsigned short __swap_cgroup_xchg(void *map, + pgoff_t offset, + unsigned int new_id) +{ + unsigned int old_id; + struct swap_cgroup_unit *units =3D map; + struct swap_cgroup_unit *unit =3D &units[offset / ID_PER_UNIT]; + struct swap_cgroup_unit new, old =3D { .raw =3D atomic_read(&unit->val) }; + + do { + new.raw =3D old.raw; + old_id =3D old.__id[offset % ID_PER_UNIT]; + new.__id[offset % ID_PER_UNIT] =3D new_id; + } while (!atomic_try_cmpxchg(&unit->val, &old.raw, new.raw)); + + return old_id; +} + /** * swap_cgroup_record - record mem_cgroup for a set of swap entries * @ent: the first swap entry to be recorded into @@ -44,22 +73,19 @@ unsigned short swap_cgroup_record(swp_entry_t ent, unsi= gned short id, unsigned int nr_ents) { struct swap_cgroup_ctrl *ctrl; - unsigned short *map; - unsigned short old; - unsigned long flags; pgoff_t offset =3D swp_offset(ent); pgoff_t end =3D offset + nr_ents; + unsigned short old, iter; + unsigned short *map; =20 ctrl =3D &swap_cgroup_ctrl[swp_type(ent)]; map =3D ctrl->map; =20 - spin_lock_irqsave(&ctrl->lock, flags); - old =3D map[offset]; + old =3D READ_ONCE(map[offset]); do { - VM_BUG_ON(map[offset] !=3D old); - map[offset] =3D id; + iter =3D __swap_cgroup_xchg(map, offset, id); + VM_BUG_ON(iter !=3D old); } while (++offset !=3D end); - spin_unlock_irqrestore(&ctrl->lock, flags); =20 return old; } @@ -85,20 +111,20 @@ unsigned short lookup_swap_cgroup_id(swp_entry_t ent) =20 int swap_cgroup_swapon(int type, unsigned long max_pages) { - void *map; + struct swap_cgroup_unit *units; struct swap_cgroup_ctrl *ctrl; =20 if (mem_cgroup_disabled()) return 0; =20 - map =3D vzalloc(max_pages * sizeof(unsigned short)); - if (!map) + units =3D vzalloc(DIV_ROUND_UP(max_pages, ID_PER_UNIT) * + sizeof(struct swap_cgroup_unit)); + if (!units) goto nomem; =20 ctrl =3D &swap_cgroup_ctrl[type]; mutex_lock(&swap_cgroup_mutex); - ctrl->map =3D map; - spin_lock_init(&ctrl->lock); + ctrl->units =3D units; mutex_unlock(&swap_cgroup_mutex); =20 return 0; --=20 2.47.0