From nobody Thu Dec 18 12:16:10 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4D384C83 for ; Thu, 5 Sep 2024 00:31:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496311; cv=none; b=SaU0+wAZnXwkaLUh55KmuHm7DihNB2GzP9UGj3sVzxU1ArIGSXKb/TkgEvxpog4ogWxtTdD5t86S6eayNXX/QuWtER+aUQCdYiafw0j0gCqj21ZQZYVWhDGPYLLigyf1MqD0fpaoIJM2yaL5Nh1Ijaxtu995sE90LXQ6Z4RWgWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496311; c=relaxed/simple; bh=Lx4+ZLTwtckuXYhwvmNIbZVz3klX1ad8ibwPx6HLoxw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=CTxCin8tkLSwzOKuUe5GUSXIqw4KsqEanDiCokovPpArnf8q88ibe9bpAWbyebVkx/HvssKXhjFIKVWzIDfXz/6y75muTfipieOieGfQxZLxgYg9FbjfsMzkFudSC31vQIBSO0IDKn1aacdKJkxmIb04NPlOLJ82NRBfzAjxmQ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=tHAAEmpM; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tHAAEmpM" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7143d66b510so189656b3a.3 for ; Wed, 04 Sep 2024 17:31:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496308; x=1726101108; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=OCIYE2bxjzqInsqy6M77OBXOLSiNi2ahaVS1P1UY4eQ=; b=tHAAEmpM8iACFaPoO36jgqLHqxQCsNM/xco0SOS5V2CbUSySLdrEyFPpwdQ8v4ecX5 Y3khsfalPyNPLLj1d//pG68p3InRyPIyBY0fZzRJvKZPoqe8WsESVK3qDhOTpX2SgckS NA3tqlJtqbdCWKxH3bfoad0p+zRVyrmRpfn959No3FOdLSK/8LnFsXwmBjiHRHdEz/TE tWZcXCwUQ1OO+SvkJVpa6tzeS/uhsdqcdqyLKvYddGofZAEnezANYisd6yXQHsEqwzOk A8SSO0ccIBDM28kMiu2EqqakAzO+HHQo3IiBvS3kuDHUneKeh0Z6IpzHRLTpCp1rRb6O 4TJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496308; x=1726101108; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=OCIYE2bxjzqInsqy6M77OBXOLSiNi2ahaVS1P1UY4eQ=; b=c3NAsWkdqAPKtAc2gOxC9cChf5HUIfP1SL15f/OmGWDEqQX1LIhzvB8Oilu3tD1DwR WQ+hCW1xkK7n9swznuiyNW8NhiaSggYEalpCMZ3E8Fyz3RcMU1vxqpUF+RUWihr7FCqw mUsrKJCFEANfmpNiSjDyev1BGnYWX1hf3H4rOfB4VX6yLXJH9BNulNNa92fWb4QPhXcm AIFt9n6FXMcJ4xwng8QmkJg7F5eNJS/oMPO0W6gRZYGI0WuhL6Kgq1sJ05BeVYL3SGzV LF9wITUPGJaKEQHH/zmsVWTsaWqSdz47OkXWCPgXSRRh6HxPO7oAOCH+QzDqzemIVvrb n2jg== X-Forwarded-Encrypted: i=1; AJvYcCVxMUIIwUFMW2NeBSvhzVIzc1oyA08+DRIv4pYbrVG/Pz9kG0BaL4Fn68laE4MwDcyBXqpRzhG9kq6Aq/c=@vger.kernel.org X-Gm-Message-State: AOJu0YxLLy5KE1oCz+yzL4OxyMN1jHtvlsPF0ZN2xM3zGofRHRBjqbcE dyYUlN6mEyMl1C9wxc2m0wA7zIDTW29kvInj78ncZ85MZNb8j9aRVzImn5hpfgAUUoN4uCBS79y VFn1A45u8Ug== X-Google-Smtp-Source: AGHT+IHUPQSbmeHjhZigLeW58QOY18ICrs203NRPh39wz9VYTS5JC1PdHx3HOB6u+ncLsUa1QDiBpyCBj6pKLQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a05:6a00:8682:b0:714:1e0f:4477 with SMTP id d2e1a72fcca58-71730791c72mr43937b3a.5.1725496307872; Wed, 04 Sep 2024 17:31:47 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:50 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-2-kinseyho@google.com> Subject: [PATCH mm-unstable v4 1/5] cgroup: clarify css sibling linkage is protected by cgroup_mutex or RCU From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly document that css sibling/descendant linkage is protected by cgroup_mutex or RCU. Also, document in css_next_descendant_pre() and similar functions that it isn't necessary to hold a ref on @pos. The following changes in this patchset rely on this clarification for simplification in memcg iteration code. Suggested-by: Yosry Ahmed Reviewed-by: Michal Koutn=C3=BD Signed-off-by: Kinsey Ho Cc: Johannes Weiner Cc: Michal Hocko Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Zefan Li Cc: Hugh Dickins Cc: T.J. Mercier --- include/linux/cgroup-defs.h | 6 +++++- kernel/cgroup/cgroup.c | 16 +++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 7fc2d0195f56..ca7e912b8355 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -172,7 +172,11 @@ struct cgroup_subsys_state { /* reference count - access via css_[try]get() and css_put() */ struct percpu_ref refcnt; =20 - /* siblings list anchored at the parent's ->children */ + /* + * siblings list anchored at the parent's ->children + * + * linkage is protected by cgroup_mutex or RCU + */ struct list_head sibling; struct list_head children; =20 diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 0a97cb2ef124..ece2316e2bca 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4602,8 +4602,9 @@ struct cgroup_subsys_state *css_next_child(struct cgr= oup_subsys_state *pos, * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct next descendant as long - * as both @pos and @root are accessible and @pos is a descendant of @root. + * section. Additionally, it isn't necessary to hold onto a reference to @= pos. + * This function will return the correct next descendant as long as both @= pos + * and @root are accessible and @pos is a descendant of @root. * * If a subsystem synchronizes ->css_online() and the start of iteration, a * css which finished ->css_online() is guaranteed to be visible in the @@ -4651,8 +4652,9 @@ EXPORT_SYMBOL_GPL(css_next_descendant_pre); * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct rightmost descendant as - * long as @pos is accessible. + * section. Additionally, it isn't necessary to hold onto a reference to @= pos. + * This function will return the correct rightmost descendant as long as @= pos + * is accessible. */ struct cgroup_subsys_state * css_rightmost_descendant(struct cgroup_subsys_state *pos) @@ -4696,9 +4698,9 @@ css_leftmost_descendant(struct cgroup_subsys_state *p= os) * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct next descendant as long - * as both @pos and @cgroup are accessible and @pos is a descendant of - * @cgroup. + * section. Additionally, it isn't necessary to hold onto a reference to @= pos. + * This function will return the correct next descendant as long as both @= pos + * and @cgroup are accessible and @pos is a descendant of @cgroup. * * If a subsystem synchronizes ->css_online() and the start of iteration, a * css which finished ->css_online() is guaranteed to be visible in the --=20 2.46.0.469.g59c65b2a67-goog From nobody Thu Dec 18 12:16:10 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B993A8F58 for ; Thu, 5 Sep 2024 00:31:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496312; cv=none; b=LD00j1Pv2OWWmZUi61VNv1Q9BeueMyKgqPWLTPryf1gbyY+O0/sIIzhpYNnsXao3VDHtW9lRgFxF6nJJeYL+lDeHVWs8cFdn+KhuXP1HdjaUn1/QN2XHZl+tz/6c6B4D8Unx31MjSzapueWx2RHK/K7D2I1lMYMduoMxKcyEFj0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496312; c=relaxed/simple; bh=fk2/lGT/4GsriM/5ZGMKLoZ8zzkgaZL8fCE8hrnHHzY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MwUa8ASrMrsZc4/AobLU2hSra3syEAJfLuZauxB0mne31R4j41sH6Aeeh2NKIzAIM29R2Id71USbtYYz8/ZFJ/wXcH2bXk8u5smBtJRaEBffdkgwA1QOTYfVabdT5smzB3WuKxlK6UjLxow6bLgjEmCg6Ph4naKhsHdxKMG/hXI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hGwvOgz2; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hGwvOgz2" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7cd98f27becso285541a12.3 for ; Wed, 04 Sep 2024 17:31:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496310; x=1726101110; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=sqTDHTCRiwsvIU2qvmmvg5mPCxzlppgliuAtX0a739I=; b=hGwvOgz2Okmct2aS5wPnYk+3JMzJDdVKq9LpGVZbThH/5uzZVGOnjnNea4TDiU6css yufw96/rt124h8rgD8cbFQNAK8UjxF2fW7ImIE/iybSA6L+7aE6W0rxjotuTD/yvicGH 6XMgNdWGvWQQDL2hheChL1N4p72iHSkqCZw9Dc68+VgQEYQ+Z79eoPYFD49JaVyJPTZ+ kzrzmTuLLNh8/GyaqYigqpEWUJ3yI2GbcR/EEVk3vboZo9xRte+kIQuXelckWsm2EHRY yaBeWTLw0D6wYjXXOKe06uGpnWpLXJMYIQzxdGZkYqbjsowslDz4WB2ewOGzP2souVfh HYyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496310; x=1726101110; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=sqTDHTCRiwsvIU2qvmmvg5mPCxzlppgliuAtX0a739I=; b=ZA3dGjwVdo9HDPiOohQbD6tSZ5+W//6RJXMG2iRXNZAl8hSUzuq8iCOYAu9WlcgxfL 1W7ldb2xGlOe4Jqe+/zMAoigh0j+UNSLZd4fokNMVS1pa68uB7y990XFVDWMjToQzl5t S3giMdqK0pQLi9KDLE/VYU3sBayDtjDfewkJkzUh+jdbpT/C573QE3wRaFDEQNEq7+X+ tDABj3GeFaTyJyEnkX4jy6dUGRJ1AnTa4/VyJUVnk7h3+uRi0c3ev+61RrpIk5m7k3Kl dz1llBNcxhUyn8BYM9hWKD1wub6kXFMm34zpqyGd4tVypH5DgGoeVH3+eyeDTNP+eEy5 xeOg== X-Forwarded-Encrypted: i=1; AJvYcCVLhQ4wXach1u4QV4eEkB2KX8PbP9Aff5o3pfjKqcds7S4kZTdZspMTvxrG1QhdcJ6R1xtWVP948Y0WLe8=@vger.kernel.org X-Gm-Message-State: AOJu0Yy275t70sM+8R30/QqOPCRUUD4OXfVYs93uoLbBSKMctrWejpEx nm6sOqUv6WZCvEmSYnRq8Zfd+cfOLMXxkdNRP3uTemX6r2P1hqeSoe5jeFLraSR47rnWGwMMcCG X7nCC8ZcYMQ== X-Google-Smtp-Source: AGHT+IEOy0bHsy4chODc/dIKYmpQwZgXe72gHtWSiqNJNS/vQMmLAYyUUqkUY8wRavMe2EY8pkrILzNoQwJIVw== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:e34e:0:b0:6ea:d240:23b0 with SMTP id 41be03b00d2f7-7d4c1019e5emr56861a12.4.1725496309928; Wed, 04 Sep 2024 17:31:49 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:51 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-3-kinseyho@google.com> Subject: [PATCH mm-unstable v4 2/5] mm: don't hold css->refcnt during traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To obtain the pointer to the next memcg position, mem_cgroup_iter() currently holds css->refcnt during memcg traversal only to put css->refcnt at the end of the routine. This isn't necessary as an rcu_read_lock is already held throughout the function. The use of the RCU read lock with css_next_descendant_pre() guarantees that sibling linkage is safe without holding a ref on the passed-in @css. Remove css->refcnt usage during traversal by leveraging RCU. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier Cc: Johannes Weiner Cc: Michal Hocko Cc: Michal Koutn=C3=BD Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Yosry Ahmed Cc: Zefan Li Cc: Hugh Dickins --- mm/memcontrol.c | 18 +----------------- 1 file changed, 1 insertion(+), 17 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index dbc1d43a5c4c..cdd324812b55 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1015,20 +1015,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup= *root, else if (reclaim->generation !=3D iter->generation) goto out_unlock; =20 - while (1) { - pos =3D READ_ONCE(iter->position); - if (!pos || css_tryget(&pos->css)) - break; - /* - * css reference reached zero, so iter->position will - * be cleared by ->css_released. However, we should not - * rely on this happening soon, because ->css_released - * is called from a work queue, and by busy-waiting we - * might block it. So we clear iter->position right - * away. - */ - (void)cmpxchg(&iter->position, pos, NULL); - } + pos =3D READ_ONCE(iter->position); } else if (prev) { pos =3D prev; } @@ -1069,9 +1056,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup = *root, */ (void)cmpxchg(&iter->position, pos, memcg); =20 - if (pos) - css_put(&pos->css); - if (!memcg) iter->generation++; } --=20 2.46.0.469.g59c65b2a67-goog From nobody Thu Dec 18 12:16:10 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB2FCDDA9 for ; Thu, 5 Sep 2024 00:31:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496315; cv=none; b=lRoB+VQ6QxFNIQ+I38Rd00Ol3dhPd9K5lEUNB+puSag3jA3s9zzh2Zmn8Mz5EJjq+QR9xRRsvhZP64cG2nMh7C7VuoKIm0w7uwZesOHg+kp2a/ikRWIfv7EiKi9YnZv/JIvqIf4AywvTUyRHX2yxDxCCX7tZT7Hs1cKakcTdb70= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496315; c=relaxed/simple; bh=ZLKOWxT3NuOir/w+fbRaVKkoQ7BQs6qbrCt9McP4q70=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=p7Q3ckrUKhnYOVe7+N96yvdL0mFEzuSsK//BnCQyGx0zzeZKPYQnDkLJu31JONGAF6XUzcWAsf/0qK5YIVn0aVh4zAqFZAKkVNSeSu3ctbetERvrhxCOFqAZPdFK5FrjS4Ep2DUCnSX89XQHG218ZEum7yktjuX7AdqyCy9o1Hk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mMngY00S; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mMngY00S" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7d4fb78fe49so269303a12.2 for ; Wed, 04 Sep 2024 17:31:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496313; x=1726101113; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=Ac9f2JvhNnpWVv2ZKYegshHvMbbmkBcnbuU73G1cYt8=; b=mMngY00Sf9O5z89pMRe/B1BCUmu+O7z3z4Hm/s7H5elhaCSimN4eIHfMqPp9CfJKxS Zj+Knf2OoRFYQ9XC3j/g6a5fBnQiD3LebFY2dor/Z9WGlU+lY5XtdH3iLYUTYE0QhsCc zrnLDfJ5XYAfuyOAW0DMBxFcq4K7gVpakK1q5DrZaDZPs6QFydEhysTsitl10egHS4z+ 6NZLrm/at1/zlPPWCFzXiN6jGkLUMT6wrHlNTJlr8yKtl1ISwL4jAyH8yQSZs15TLduy /i5Q0J/gJdah5tnR0UgOqYqGpsozNissMCC6EagyvOJKbJRZqN98HNsYqt6/sHezwQkz rmkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496313; x=1726101113; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=Ac9f2JvhNnpWVv2ZKYegshHvMbbmkBcnbuU73G1cYt8=; b=bciZ4KGF6P4ZNPzOnnTC7OwWHZnlJnGA3sPDfC0WrYWPSzU4OxIUgkk1P6GzY4poXZ V+zYjV1mP/41nevAxc4I6pYB7duREZuGJadGapMY2D+e4Avy+suQmWJhZSi79ofLsMNy vnH222P4CNIay+lsgbw4x5zYjZkACzBX2g8GwMBhU1OGbbwW61mfFx2yJFD1cIIq33jg +k46Nu62tY8nRspvnOsPhDDJT/Sjg45/hEvdRdE/kAJDwclhiwniX5UlBhsmCFt/CAAn /XvbZCtTgH6igQZRu2md5wHR80X4d85NZJ+cNz0NAJwf0KA2zxOzc3++iarnqp8eovOZ 7h6A== X-Forwarded-Encrypted: i=1; AJvYcCVCA+EFX7SXZDzTOC+T52gDUepGK+lM0rh+86LpZlWdD+HFeRqwbPQ8Jao+3rS2TaQ+Z05qaBEQbj5MA/I=@vger.kernel.org X-Gm-Message-State: AOJu0Yx7zt/S8I1EwQw3R2wH9dmFLUK+L3YFSgv1LedbLLN0Y29igpmg r1i12J7VFj8abQoYnc7Jq3jiFLqQYY0ORKc1GFzOQKcJLN3lIg9ONR8EDstZH4HQhbwLOLXTOEx W5dkzjOH1qA== X-Google-Smtp-Source: AGHT+IHuJuEjgofhNfeZ87JSGyMalBhwo29t0H9DuDw9GmIw/cnVXOLyLUXVF5+Ftrg6N3vF7BBT4Qhm6qOSmQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:f303:0:b0:7a1:f561:8c99 with SMTP id 41be03b00d2f7-7d4c1087417mr46719a12.6.1725496311779; Wed, 04 Sep 2024 17:31:51 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:52 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-4-kinseyho@google.com> Subject: [PATCH mm-unstable v4 3/5] mm: increment gen # before restarting traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The generation number in struct mem_cgroup_reclaim_iter should be incremented on every round-trip. Currently, it is possible for a concurrent reclaimer to jump in at the end of the hierarchy, causing a traversal restart (resetting the iteration position) without incrementing the generation number. By resetting the position without incrementing the generation, it's possible for another ongoing mem_cgroup_iter() thread to walk the tree twice. Move the traversal restart such that the generation number is incremented before the restart. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier Cc: Johannes Weiner Cc: Michal Hocko Cc: Michal Koutn=C3=BD Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Yosry Ahmed Cc: Zefan Li Cc: Hugh Dickins --- mm/memcontrol.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index cdd324812b55..c24ef6a106e0 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -999,7 +999,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *r= oot, root =3D root_mem_cgroup; =20 rcu_read_lock(); - +restart: if (reclaim) { struct mem_cgroup_per_node *mz; =20 @@ -1026,14 +1026,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup= *root, for (;;) { css =3D css_next_descendant_pre(css, &root->css); if (!css) { - /* - * Reclaimers share the hierarchy walk, and a - * new one might jump in right at the end of - * the hierarchy - make sure they see at least - * one group and restart from the beginning. - */ - if (!prev) - continue; break; } =20 @@ -1056,8 +1048,18 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup= *root, */ (void)cmpxchg(&iter->position, pos, memcg); =20 - if (!memcg) + if (!memcg) { iter->generation++; + + /* + * Reclaimers share the hierarchy walk, and a + * new one might jump in right at the end of + * the hierarchy - make sure they see at least + * one group and restart from the beginning. + */ + if (!prev) + goto restart; + } } =20 out_unlock: --=20 2.46.0.469.g59c65b2a67-goog From nobody Thu Dec 18 12:16:10 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0D0712B63 for ; Thu, 5 Sep 2024 00:31:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496317; cv=none; b=UAS6tmVucjiAqHTT2DHkvO+Hb+bQcAipJQ3LrkTTO2CsuGANNxwtbLx+8qhooQUUr14l7661luC9oOE+aJhWLo6uIVcG5KkA6JisR5EwiG7KS+GPn4sjyGTAP7TVB985HSD0LPkqV9rasf1D1m9OSCoVvXFT2rpVRen1eYk69eg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496317; c=relaxed/simple; bh=tNdPgcpIqADtqnC02rHoIq3VZI89MnwpugBq5dQEs0U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jDRzhmVAXZXIknGR+oVveOsWP0NaS1C4m7WUNRGjHBR17nv/0US71sz1K1GzdBC3UDf34fexvX6iYeYYsX97e6nXrKekgaN1BsTO5eTp7MO8csB8w01v3ILTCrCqJmxU+zgHhOp2H8gCGPpWvkbLzUP8IzIXzn2vNAHOhlI0fJA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Bcd8t2hv; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Bcd8t2hv" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7cf603d9ffaso164731a12.0 for ; Wed, 04 Sep 2024 17:31:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496315; x=1726101115; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=C+NIQ3hruAHgC9xMj3fhofnWDiiLf7+Cp7wyaHwiSJ4=; b=Bcd8t2hv6uy7PvHGY8hp73vQwhNH7Nw9dAP8Wv4NcGSnB+GU+fcl1f0BrnoNqhLqiz n84PTlQ7aXAFMfdHpCyyx8h6l4Wsl4hkHoE7jBK+g3JY+c6bYieKX4dbmI3D1jIX3cTK bJoAVcoWVsUjPK0AITkN430mvWFVVATHMlAF9TYtgdzY3IFvaKR8q2ToRHXRD2909aLs 4Rlg2mrQAboAUsA9P4fRWczdXwgFyRJbhdoutOzi6VXBJddLlDgHGbcSfM1yFNTHE1yi rXTiZzrsslhOZrlZ8CktFn6IAyRbymNienNK0qU9SOS9K2v7tSSotB5AIi31GgbQHELX S3UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496315; x=1726101115; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=C+NIQ3hruAHgC9xMj3fhofnWDiiLf7+Cp7wyaHwiSJ4=; b=QBIQ8gMUIB4zimDMjIzOOdAEiWme7ZbWwUlRmHb5olFX1qIAXmidwmdK7dcNhCiSzI uZgPSjp8O/8BLkMvxXkS2+RwnyLWqzuaGNBcnjL2za8MGDrSTt6qY8TlUVFXmIEVmYWt 1FMuWCKySmyuv9HC0lkmP9oDfuOAknfRM3VKYUk7YxT4iOQX8KGNFEt5u93b36NY+ofn rCw8tofP/0uyuyfU9O1Q/glzeUPZodlIUwhCGhyxGgVyMjGzeHGLWUj6IC8eGDAnW6gs AwttDQn5Mvzf6tAy5SZ8S1eYnXSNQaVSI6QyoSubmzRS/i2UNgBCvBtfjRC+xzVS4vMB AB8Q== X-Forwarded-Encrypted: i=1; AJvYcCU84b33cqQbz7XL4KUmnF1SudzZJwja+Pqq3q7oPGdvFPnkxv96j/DQtRVo1kNwib3dnMeoBT3ruTGOclA=@vger.kernel.org X-Gm-Message-State: AOJu0YwKZV3DbXCFggClALM6oevGrOgi/YBmARO8oCBw2xa9B+R412LU 581SLkgOGx3XOA5X/ViJ6kXI3ERJfBQPFV6XjA7N3ZWY65F+Nlm3nr9etAJj63TT0UKasSxbxM4 Ihk5266rIjg== X-Google-Smtp-Source: AGHT+IFE9E8HfCvWJ7p3tjF6dU79+RYxA1wZ2PaXCVQ7+B7BuvDX9ZOeiHESUdZETs6+j99tFDtqjKU0xWP7eQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:1c16:0:b0:7cd:8558:9151 with SMTP id 41be03b00d2f7-7d4c1094218mr69531a12.10.1725496314864; Wed, 04 Sep 2024 17:31:54 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:53 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-5-kinseyho@google.com> Subject: [PATCH mm-unstable v4 4/5] mm: restart if multiple traversals raced From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho , syzbot+e099d407346c45275ce9@syzkaller.appspotmail.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, if multiple reclaimers raced on the same position, the reclaimers which detect the race will still reclaim from the same memcg. Instead, the reclaimers which detect the race should move on to the next memcg in the hierarchy. So, in the case where multiple traversals race, jump back to the start of the mem_cgroup_iter() function to find the next memcg in the hierarchy to reclaim from. Reported-by: syzbot+e099d407346c45275ce9@syzkaller.appspotmail.com Closes: https://lore.kernel.org/000000000000817cf10620e20d33@google.com/ Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier Cc: Johannes Weiner Cc: Michal Hocko Cc: Michal Koutn=C3=BD Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Yosry Ahmed Cc: Zefan Li Cc: Hugh Dickins --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 26 +++++++++++++++++--------- 2 files changed, 19 insertions(+), 11 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index fe05fdb92779..2ef94c74847d 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -57,7 +57,7 @@ enum memcg_memory_event { =20 struct mem_cgroup_reclaim_cookie { pg_data_t *pgdat; - unsigned int generation; + int generation; }; =20 #ifdef CONFIG_MEMCG @@ -78,7 +78,7 @@ struct lruvec_stats; struct mem_cgroup_reclaim_iter { struct mem_cgroup *position; /* scan generation, increased every round-trip */ - unsigned int generation; + atomic_t generation; }; =20 /* diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c24ef6a106e0..ce5d066393c9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -988,8 +988,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *r= oot, struct mem_cgroup_reclaim_cookie *reclaim) { struct mem_cgroup_reclaim_iter *iter; - struct cgroup_subsys_state *css =3D NULL; - struct mem_cgroup *memcg =3D NULL; + struct cgroup_subsys_state *css; + struct mem_cgroup *memcg; struct mem_cgroup *pos =3D NULL; =20 if (mem_cgroup_disabled()) @@ -1000,19 +1000,23 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgrou= p *root, =20 rcu_read_lock(); restart: + memcg =3D NULL; + if (reclaim) { + int gen; struct mem_cgroup_per_node *mz; =20 mz =3D root->nodeinfo[reclaim->pgdat->node_id]; iter =3D &mz->iter; + gen =3D atomic_read(&iter->generation); =20 /* * On start, join the current reclaim iteration cycle. * Exit when a concurrent walker completes it. */ if (!prev) - reclaim->generation =3D iter->generation; - else if (reclaim->generation !=3D iter->generation) + reclaim->generation =3D gen; + else if (reclaim->generation !=3D gen) goto out_unlock; =20 pos =3D READ_ONCE(iter->position); @@ -1020,8 +1024,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup = *root, pos =3D prev; } =20 - if (pos) - css =3D &pos->css; + css =3D pos ? &pos->css : NULL; =20 for (;;) { css =3D css_next_descendant_pre(css, &root->css); @@ -1035,21 +1038,26 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgrou= p *root, * and kicking, and don't take an extra reference. */ if (css =3D=3D &root->css || css_tryget(css)) { - memcg =3D mem_cgroup_from_css(css); break; } } =20 + memcg =3D mem_cgroup_from_css(css); + if (reclaim) { /* * The position could have already been updated by a competing * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - (void)cmpxchg(&iter->position, pos, memcg); + if (cmpxchg(&iter->position, pos, memcg) !=3D pos) { + if (css && css !=3D &root->css) + css_put(css); + goto restart; + } =20 if (!memcg) { - iter->generation++; + atomic_inc(&iter->generation); =20 /* * Reclaimers share the hierarchy walk, and a --=20 2.46.0.469.g59c65b2a67-goog From nobody Thu Dec 18 12:16:10 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5020DDDAB for ; Thu, 5 Sep 2024 00:31:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496318; cv=none; b=CJt8PCi6yUyVF0PrBPOLxnLkn+As0G9XywXYYJapb4P6fFb4XF/s9PyYxPkSvBpww6eMx+t6ck786qneTCvc/GOMLEkIt536qD20y4799q1/lZs7LUW5e+Xrffad+jLC9qlcS7/v4+m+QDCkfyeOLNHThfXJLy0u/J/Un0RgWfU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725496318; c=relaxed/simple; bh=rNTFMsSf4ipv9IfY2q02hT078vD9+pCTekO9b6Df+h4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DKjx58kP9gd+X7A6LKnFmWQMA3nASHTAYfDO5YsJYGRyxbE/2kBJKrS/nny8JukaK3q5N5Ewy+oUS9T5/KUmC2J8reJTOrw5ctMH9x3Ms7kW4xL31ObZHnw9eBeUQh9lVuw9hELmG3/hfE+XxnxemjbgK7QktpZ3WLsjnICsqLI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=aLHhJAhB; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="aLHhJAhB" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7cf546bbcffso289074a12.1 for ; Wed, 04 Sep 2024 17:31:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496317; x=1726101117; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=UFlJk+bdMFXrL5bkGb5l4QnCCI5p7k/K3Pj8oKz1txU=; b=aLHhJAhBI8aHQsrfn5T2wGgEdvoqYMAihfPSqMf7j71JfQQf95Qjhr9OUaPxkBNqm7 bIbw2SWiHwkDTXHEiKVFfHT2XQXJVvNQE+fGpPR2eFoAG903g9rsECuXJ6xAda67ZdZO zwabj2j1YQGv7G7BNlScoTqGE9eWH7+3HKohzIGxvFa4iFjd2OO10G6AaaOh7xFwLkFk cvByu+MdmIhbg8+FLZ1oQl5qkcMw6gEq+Oq2VFFQSJIej0MBQ2g/gCdKqw/6JUauL2WH pPLcUeNJw5Vwxg4K6wEoK1KYQ0oycGwzWOUpuENJ+HeESptf9/nlwgCKLHwPvEPaH89w qnAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496317; x=1726101117; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=UFlJk+bdMFXrL5bkGb5l4QnCCI5p7k/K3Pj8oKz1txU=; b=lPH9mDfdS1rSnArV451AZdR1lWy3f8GpN4Qt8+RPNHwACUFJGgjZ+knENRGADIjBpA aV+GGO0XUn0ydeY2i/JM4qCGYlJEC8qPRU3JwpQ+X36aKMFjJnSCdl3LAV+nB0zRMWbt Be76hjFeI3DrP+KB0a2aaQojzK6Put3hUGxCIFw09rPVFau57uqHzV8d7N4pbdDa8Xqi PBKiwkCOtp8IGhs7Mr1/fDK7RZByUZlOka8IpKR+chO8OjdfINIGpU0BQZNmj4xeZgjG 2mkRKazRJkTfig2O/XmVPvPE74yQ4VE/+5TKGBV4+fRdo0Ck8aMjC7A5hrUwqu+87bs8 PWTA== X-Forwarded-Encrypted: i=1; AJvYcCUGaKaHsg8lC2osFK1wBm+AqdtOi99Ht3QEqqE4yy491SX1Pdj/c0MTCFi6o3SiP9zNVx7zRziAhHkhdvY=@vger.kernel.org X-Gm-Message-State: AOJu0Yx/IPgWZbAwbDo/n1Lg5piZ3Gg4CyrTnO/lpqAge6SKeLJFbMwx ugX04YeKqxLESXPuOYUIRcvygdiNRckt5rlqeSYEx1LIzxVdJmPsiX08NPz2qI+nQ1Z0wEZ1+XH wR+XpVXkrMA== X-Google-Smtp-Source: AGHT+IFCmyZz1oeuwdzh9ujb+lyKCdb7Lv33IMyUDlAfA9IgJETASTtABlliPQg/ISAiLVSGlddqDM0VzXnGuQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a05:6a02:2385:b0:6c4:5b35:c986 with SMTP id 41be03b00d2f7-7d50176fca8mr4225a12.8.1725496316657; Wed, 04 Sep 2024 17:31:56 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:54 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-6-kinseyho@google.com> Subject: [PATCH mm-unstable v4 5/5] mm: clean up mem_cgroup_iter() From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A clean up to make variable names more clear and to improve code readability. No functional change. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier Cc: Johannes Weiner Cc: Michal Hocko Cc: Michal Koutn=C3=BD Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Yosry Ahmed Cc: Zefan Li Cc: Hugh Dickins --- mm/memcontrol.c | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ce5d066393c9..31db8467f19d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -989,8 +989,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *r= oot, { struct mem_cgroup_reclaim_iter *iter; struct cgroup_subsys_state *css; - struct mem_cgroup *memcg; - struct mem_cgroup *pos =3D NULL; + struct mem_cgroup *pos; + struct mem_cgroup *next; =20 if (mem_cgroup_disabled()) return NULL; @@ -1000,14 +1000,13 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgrou= p *root, =20 rcu_read_lock(); restart: - memcg =3D NULL; + next =3D NULL; =20 if (reclaim) { int gen; - struct mem_cgroup_per_node *mz; + int nid =3D reclaim->pgdat->node_id; =20 - mz =3D root->nodeinfo[reclaim->pgdat->node_id]; - iter =3D &mz->iter; + iter =3D &root->nodeinfo[nid]->iter; gen =3D atomic_read(&iter->generation); =20 /* @@ -1020,29 +1019,22 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgrou= p *root, goto out_unlock; =20 pos =3D READ_ONCE(iter->position); - } else if (prev) { + } else pos =3D prev; - } =20 css =3D pos ? &pos->css : NULL; =20 - for (;;) { - css =3D css_next_descendant_pre(css, &root->css); - if (!css) { - break; - } - + while ((css =3D css_next_descendant_pre(css, &root->css))) { /* * Verify the css and acquire a reference. The root * is provided by the caller, so we know it's alive * and kicking, and don't take an extra reference. */ - if (css =3D=3D &root->css || css_tryget(css)) { + if (css =3D=3D &root->css || css_tryget(css)) break; - } } =20 - memcg =3D mem_cgroup_from_css(css); + next =3D mem_cgroup_from_css(css); =20 if (reclaim) { /* @@ -1050,13 +1042,13 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgrou= p *root, * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - if (cmpxchg(&iter->position, pos, memcg) !=3D pos) { + if (cmpxchg(&iter->position, pos, next) !=3D pos) { if (css && css !=3D &root->css) css_put(css); goto restart; } =20 - if (!memcg) { + if (!next) { atomic_inc(&iter->generation); =20 /* @@ -1075,7 +1067,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup = *root, if (prev && prev !=3D root) css_put(&prev->css); =20 - return memcg; + return next; } =20 /** --=20 2.46.0.469.g59c65b2a67-goog