From nobody Fri Dec 19 00:05:22 2025 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12D1B289E05; Fri, 6 Jun 2025 16:18:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749226735; cv=none; b=Blh0JsjdMCrxblE4soSAkQ3/Mz/VyFHM6bpTa8tfUmAVE5MJpbqZhWhtPQr7KFrGHFRaCjKE5oe1fDIxDb3l6++gneuZrF6v/d36hxZyWieTydREFv7N3ZZQSR3KIhFogfm3OC4tzy1oo7XIqXrKRHQoGLovB6WH4ukTCVABbFU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749226735; c=relaxed/simple; bh=Ys6phxAi1yCzadfLuMIMyyXTOESVHhFvpOJF7AGYjSk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vr7ifI0g1gYu1FGQfmIuwo5aewNPg0kjVZSVMz+b2+zSvWOZ1u2YknH6hwrDMn7a2ziYR1sCB8bL0LqsxbrKxB1aG7XrLBQUEBn8TizTsz4baAC4CD2l0GWSQ/ody8UzIOlEcS6zaehrJpXvaHeP3Ubd1S9qsl1Ogl++v5fn788= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NC11wwJK; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NC11wwJK" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-747fc7506d4so2370869b3a.0; Fri, 06 Jun 2025 09:18:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1749226733; x=1749831533; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B1D/UwBfO9rb1qi8d6FbfGYivA4b5vNq0TisU6WUgjI=; b=NC11wwJK+oN7ZRJ6VurdDcWObuKIGMy8+yg86LRol7Tto12d+bFTeiLT83iU5vQLcH TEC28Bw6WIBUNpz+IMuH/a9rptnDxsMcI2ZBnS33fNkbMIEt5l48Xage3Tjk8T15iQb7 ngKe2FK8S32Ln3utrPnCGrrgJhcNqwveFvxVz2J+B7/rWxQa6ruGLwlMhumqNeisvPUA SYmErGR5SwjFt7F176jMaLLZg/ZWnmEeXc5McMR0CcG485T6LbMet2cSgz+PfEp+w/2Y ApBcF50OkKyDaW8fJJMwyRM+XYcRj9EKPl9XgiH9Uf4m3g78mLZ5P9ri3cR9f5K4nmF6 QgRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749226733; x=1749831533; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B1D/UwBfO9rb1qi8d6FbfGYivA4b5vNq0TisU6WUgjI=; b=ZBHBkm+/zcpfRhJXXnU5P+d+ZUU1KuDjWrqMoGe8GX0ADqVHQdmVLEIDEx32M11mAb EW4r8dQbP/Yi45k0S5P5UXNflhB45dOkttGGi+dEe0BI4RxWOhUgApTEJhzpPJARF0U4 9+rJdZF8QyxJS4ePJIqF+NEeUKvwMO1bd5+Tdi3+qdf3dZTa7D0dA6GKyA7c/KnhAV+t FHuBnSpbk/InyqqlUZHZILZnQNO7W4wBuel6NkTIUKoTogXaSSUVze97Re/F+2ZH+MOb k1lnIObrl2ons5iGMd5lL4nnEZiIUXs65kdcRI94rgP+SSfoxgMpwFmas1auqlPPk4BK ad3A== X-Forwarded-Encrypted: i=1; AJvYcCVl+3JrhTz3SOB4XJrdoI0rYvPpTex0HTE1WVLmXu3zBia7eiWOBWPWWeWdBPakRNDy0AbTUHEa@vger.kernel.org, AJvYcCX/s9Ae70YhE+OxO6qcybbLPwMvW179772Hy/CY29AMO5+yavJfUMvKeJBl/DleXg2WdPCk90dXdi8/Sp9C@vger.kernel.org X-Gm-Message-State: AOJu0Yz5ZxWKFfbsuLfCv0+mHyh68P4npQWdJCa2+trm5UryOXdOy8EL S8INQJKmVYuq9NZ2ntneDHXyJnfEay8fGytj3KeUa4WPQACWxoJPBJBkuk4hJpnwGuE= X-Gm-Gg: ASbGnctHUhDbTdy+H4sP7oLDyXmRG1g42dLAiOywH0wxbMul5FWtYGioXjis1XCsbrd HgClQ7bLAeY9JjP8P6/PvWqFRW6u52E/2k0GGfsgRyBW41c1zWE4e2dfvw3vBgu3RtUDC8lognL TYFg4bXi1hpkEEJu69dPRpizLwq4HYvdjl1DcKc89Gu6i1CrMW8QNo83vIZpYvGXmh8R1EFuJWD qQeyS+D4JYA1UQ0ahZQgPAWr7JOuJS6xkZKcK3Aue9ICSaG7zooQr44Vj6Ln5dasLa1P5tWMvXL sG0oInFl3AUMR8tLND+xRzxISipMq0Dd4VNG05UpNQajTQUUSrw= X-Google-Smtp-Source: AGHT+IFAIDJwYl1tg2vjnywkbJPdZQ9Xx5bwB7Cih6rVhVZKDzc36c6HegpZFRbjInSIIK5CEsY+FA== X-Received: by 2002:a05:6a20:db0c:b0:215:e818:9fe5 with SMTP id adf61e73a8af0-21ee2500408mr5198758637.18.1749226733054; Fri, 06 Jun 2025 09:18:53 -0700 (PDT) Received: from jemmy.. ([180.172.46.169]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7482af38353sm1518675b3a.10.2025.06.06.09.18.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 09:18:52 -0700 (PDT) From: Jemmy Wong To: =?UTF-8?q?Michal=20Koutn=C3=BD?= Cc: Tejun Heo , Johannes Weiner , Waiman Long , cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 1/3] cgroup: add lock guard support for cgroup_muetx Date: Sat, 7 Jun 2025 00:18:39 +0800 Message-ID: <20250606161841.44354-2-jemmywong512@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606161841.44354-1-jemmywong512@gmail.com> References: <20250606161841.44354-1-jemmywong512@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: Jemmy Wong --- include/linux/cgroup.h | 7 ++++ kernel/cgroup/cgroup-v1.c | 16 ++++---- kernel/cgroup/cgroup.c | 81 +++++++++++++++++++-------------------- 3 files changed, 54 insertions(+), 50 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index b18fb5fcb38e..979f827452ad 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -382,6 +382,10 @@ static inline void cgroup_put(struct cgroup *cgrp) =20 extern struct mutex cgroup_mutex; =20 +DEFINE_LOCK_GUARD_0(cgroup_mutex, + mutex_lock(&cgroup_mutex), + mutex_unlock(&cgroup_mutex)) + static inline void cgroup_lock(void) { mutex_lock(&cgroup_mutex); @@ -656,6 +660,9 @@ struct cgroup *cgroup_get_from_id(u64 id); struct cgroup_subsys_state; struct cgroup; =20 +extern struct mutex cgroup_mutex; +DEFINE_LOCK_GUARD_0(cgroup_mutex, , ,) + static inline u64 cgroup_id(const struct cgroup *cgrp) { return 1; } static inline void css_get(struct cgroup_subsys_state *css) {} static inline void css_put(struct cgroup_subsys_state *css) {} diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c index fa24c032ed6f..f4658eda4445 100644 --- a/kernel/cgroup/cgroup-v1.c +++ b/kernel/cgroup/cgroup-v1.c @@ -64,7 +64,8 @@ int cgroup_attach_task_all(struct task_struct *from, stru= ct task_struct *tsk) struct cgroup_root *root; int retval =3D 0; =20 - cgroup_lock(); + guard(cgroup_mutex)(); + cgroup_attach_lock(true); for_each_root(root) { struct cgroup *from_cgrp; @@ -78,7 +79,6 @@ int cgroup_attach_task_all(struct task_struct *from, stru= ct task_struct *tsk) break; } cgroup_attach_unlock(true); - cgroup_unlock(); =20 return retval; } @@ -862,13 +862,11 @@ static int cgroup1_rename(struct kernfs_node *kn, str= uct kernfs_node *new_parent kernfs_break_active_protection(new_parent); kernfs_break_active_protection(kn); =20 - cgroup_lock(); - - ret =3D kernfs_rename(kn, new_parent, new_name_str); - if (!ret) - TRACE_CGROUP_PATH(rename, cgrp); - - cgroup_unlock(); + scoped_guard(cgroup_mutex) { + ret =3D kernfs_rename(kn, new_parent, new_name_str); + if (!ret) + TRACE_CGROUP_PATH(rename, cgrp); + } =20 kernfs_unbreak_active_protection(kn); kernfs_unbreak_active_protection(new_parent); diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index a723b7dc6e4e..54f80afe4f65 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -2224,13 +2224,13 @@ int cgroup_do_get_tree(struct fs_context *fc) struct super_block *sb =3D fc->root->d_sb; struct cgroup *cgrp; =20 - cgroup_lock(); - spin_lock_irq(&css_set_lock); + scoped_guard(cgroup_mutex) { + spin_lock_irq(&css_set_lock); =20 - cgrp =3D cset_cgroup_from_root(ctx->ns->root_cset, ctx->root); + cgrp =3D cset_cgroup_from_root(ctx->ns->root_cset, ctx->root); =20 - spin_unlock_irq(&css_set_lock); - cgroup_unlock(); + spin_unlock_irq(&css_set_lock); + } =20 nsdentry =3D kernfs_node_dentry(cgrp->kn, sb); dput(fc->root); @@ -2440,13 +2440,12 @@ int cgroup_path_ns(struct cgroup *cgrp, char *buf, = size_t buflen, { int ret; =20 - cgroup_lock(); + guard(cgroup_mutex)(); spin_lock_irq(&css_set_lock); =20 ret =3D cgroup_path_ns_locked(cgrp, buf, buflen, ns); =20 spin_unlock_irq(&css_set_lock); - cgroup_unlock(); =20 return ret; } @@ -4472,9 +4471,10 @@ int cgroup_rm_cftypes(struct cftype *cfts) if (!(cfts[0].flags & __CFTYPE_ADDED)) return -ENOENT; =20 - cgroup_lock(); - cgroup_rm_cftypes_locked(cfts); - cgroup_unlock(); + scoped_guard(cgroup_mutex) { + cgroup_rm_cftypes_locked(cfts); + } + return 0; } =20 @@ -4506,14 +4506,13 @@ int cgroup_add_cftypes(struct cgroup_subsys *ss, st= ruct cftype *cfts) if (ret) return ret; =20 - cgroup_lock(); - - list_add_tail(&cfts->node, &ss->cfts); - ret =3D cgroup_apply_cftypes(cfts, true); - if (ret) - cgroup_rm_cftypes_locked(cfts); + scoped_guard(cgroup_mutex) { + list_add_tail(&cfts->node, &ss->cfts); + ret =3D cgroup_apply_cftypes(cfts, true); + if (ret) + cgroup_rm_cftypes_locked(cfts); + } =20 - cgroup_unlock(); return ret; } =20 @@ -5489,14 +5488,14 @@ static void css_free_rwork_fn(struct work_struct *w= ork) } } =20 -static void css_release_work_fn(struct work_struct *work) +static inline void css_release_work_fn_locked(struct work_struct *work) { struct cgroup_subsys_state *css =3D container_of(work, struct cgroup_subsys_state, destroy_work); struct cgroup_subsys *ss =3D css->ss; struct cgroup *cgrp =3D css->cgroup; =20 - cgroup_lock(); + guard(cgroup_mutex)(); =20 css->flags |=3D CSS_RELEASED; list_del_rcu(&css->sibling); @@ -5550,8 +5549,14 @@ static void css_release_work_fn(struct work_struct *= work) NULL); } =20 - cgroup_unlock(); +} + +static void css_release_work_fn(struct work_struct *work) +{ + struct cgroup_subsys_state *css =3D + container_of(work, struct cgroup_subsys_state, destroy_work); =20 + css_release_work_fn_locked(work); INIT_RCU_WORK(&css->destroy_rwork, css_free_rwork_fn); queue_rcu_work(cgroup_destroy_wq, &css->destroy_rwork); } @@ -5914,7 +5919,7 @@ static void css_killed_work_fn(struct work_struct *wo= rk) struct cgroup_subsys_state *css =3D container_of(work, struct cgroup_subsys_state, destroy_work); =20 - cgroup_lock(); + guard(cgroup_mutex)(); =20 do { offline_css(css); @@ -5922,8 +5927,6 @@ static void css_killed_work_fn(struct work_struct *wo= rk) /* @css can't go away while we're holding cgroup_mutex */ css =3D css->parent; } while (css && atomic_dec_and_test(&css->online_cnt)); - - cgroup_unlock(); } =20 /* css kill confirmation processing requires process context, bounce */ @@ -6115,7 +6118,7 @@ static void __init cgroup_init_subsys(struct cgroup_s= ubsys *ss, bool early) =20 pr_debug("Initializing cgroup subsys %s\n", ss->name); =20 - cgroup_lock(); + guard(cgroup_mutex)(); =20 idr_init(&ss->css_idr); INIT_LIST_HEAD(&ss->cfts); @@ -6161,8 +6164,6 @@ static void __init cgroup_init_subsys(struct cgroup_s= ubsys *ss, bool early) BUG_ON(!list_empty(&init_task.tasks)); =20 BUG_ON(online_css(css)); - - cgroup_unlock(); } =20 /** @@ -6224,20 +6225,18 @@ int __init cgroup_init(void) =20 get_user_ns(init_cgroup_ns.user_ns); =20 - cgroup_lock(); - - /* - * Add init_css_set to the hash table so that dfl_root can link to - * it during init. - */ - hash_add(css_set_table, &init_css_set.hlist, - css_set_hash(init_css_set.subsys)); - - cgroup_bpf_lifetime_notifier_init(); + scoped_guard(cgroup_mutex) { + /* + * Add init_css_set to the hash table so that dfl_root can link to + * it during init. + */ + hash_add(css_set_table, &init_css_set.hlist, + css_set_hash(init_css_set.subsys)); =20 - BUG_ON(cgroup_setup_root(&cgrp_dfl_root, 0)); + cgroup_bpf_lifetime_notifier_init(); =20 - cgroup_unlock(); + BUG_ON(cgroup_setup_root(&cgrp_dfl_root, 0)); + } =20 for_each_subsys(ss, ssid) { if (ss->early_init) { @@ -6289,9 +6288,9 @@ int __init cgroup_init(void) if (ss->bind) ss->bind(init_css_set.subsys[ssid]); =20 - cgroup_lock(); - css_populate_dir(init_css_set.subsys[ssid]); - cgroup_unlock(); + scoped_guard(cgroup_mutex) { + css_populate_dir(init_css_set.subsys[ssid]); + } } =20 /* init_css_set.subsys[] has been updated, re-hash */ --=20 2.43.0 From nobody Fri Dec 19 00:05:22 2025 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65CFC289E1E; Fri, 6 Jun 2025 16:18:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749226740; cv=none; b=TZPC7wivowUtTaNerXPLkKy4MmTzVynduCjuRFDTbKqPMw6e6eoG2hoX5JofwTfEAJIKJB7xJc414yy6Ua7BCNajV2+apb23DjTiMus8oW3QtTEPygChHpDyWBB3ZHqc8EcTdsDLZJY8+DHWpefNpoP+GTO907J6BkP62qItqF8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749226740; c=relaxed/simple; bh=66uE4ADvwMO+JAwbLytri/SGyTvQWdDnR/8WUrZcnu0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c8VjlXAVpkVcgGBHuEE3atcLSV5oX2vNFceDTncxYi72jkoDVsWk0OepHIuB4TJXKbCT3Vd+oE53OwnabSpBISmNP8gXrzAObARpRYOg5248tQUWL2clby/NU/5TCfodVVBVqHBgnvthEcwIv5kEtf2RO9/qPIelaijUQN8hli0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=VCdCwFXC; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VCdCwFXC" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-73972a54919so2126966b3a.3; Fri, 06 Jun 2025 09:18:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1749226737; x=1749831537; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qOGlw03uEXEglmTbHf49Wqe0F7+bcmF9jdey3sgb7do=; b=VCdCwFXCqdYkcY4Xq72a5XGaAaTHxnmGlUPl8rw9aBbrluZMJyGl8ATazOgK3G1mmJ Z2ggp1yyhxy0YqfXK5VvdJjay309dzIHpxcV2zxkBVnzAVX2ucT2gh8Hps+6//c2hbbC 1ZbhaKQmQiZceN4DExLSjVanRwrXKmB4Hg4qpRQgUMD5fZP2p4ywcbKSMskyRhky92Q6 le1n8xkkl0WHp9L31nDFSjD/P1XWOqDAvZjGixcE0PukAJ92BnIuOQtsiR/aMTDr0Wp1 Zc4VUBU16V9KXG2wiSBmOLORHdVZMHQNmca9mjypJ5qEzorb50lQ6hEdJguyzgqwGRM1 EXeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749226737; x=1749831537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qOGlw03uEXEglmTbHf49Wqe0F7+bcmF9jdey3sgb7do=; b=KirRaNWPOlUKlLwuPbR9vjsbQHW0dU30GCuCNuttXJum1Rw0wqKBCH//4UpSm9Uoe4 Z2H+uFD2Fkvb7Nhm7hIfY9YjnhukrEbr5uNKRztRjG1jckiG6BM2LZpiOhxqCFXVD700 7R5EdlivtupSBIuuff0eMcbG3EkEnkfhx0pMLsjshW1qpLjVI4G4vhgz9E5t8FJzLsQh 36GFD6kZBqvlDs1bCX8k/7CMwRrkAflffr8TuZmjX4+VwSlh4hKKCLg8D9hTj+k49DpV 5Y67Kr03567yOUr4dRWwAUcbL5zYO4NilMnx9R4I9gNWAAjMeJ/HuNlL4H2aVEEVPzaV WRew== X-Forwarded-Encrypted: i=1; AJvYcCVDez6guwNmq/Bc6PaRC81gVeRGox90DUkSkSftw4cniqrH3hx9Fx0YNi5N0JI5Fj0Re3YvNWFvcHFbm6ii@vger.kernel.org, AJvYcCXV+ubV0XfNYzwXAB5BZDHxYboDdsBNj0Pq3Ze4F9jV4fttUFWOfLvlHEptUB/o/LBfcwezCXGp@vger.kernel.org X-Gm-Message-State: AOJu0YxG8zqeZkbyMlEPY7JCXUZsOUxeZHY4x4+wqOmbT3HC9uygPAGm fjjcguSfaB99bnWo+Zb8hHGF4iT7q5QINmrd8f+ZmCDqiqHr8/cWpKLP X-Gm-Gg: ASbGncs0tY+NUbaMgXdyHWP3H3WYnpqtDNF07iPyDNMbF8DeHqlZkkGn4Jsga6tZY5Q hNkhd6R3vIwSwr4VuzhX3TkdfYi/XUcah6gUw/4vTimdcCopjgVJM61d5AGwZI0BtmRysdqUAci LYdx+FoyVtcMKS4TeYkf61ooCP2oX2QOJF2gsW6eyl/GBuwBzbWyzJJvfe0bQxNHvKY7LsLp1Qx 3nLvEY15V5ox9/GJIFLVNjzYB0DmQIeSsRg/fsmHqjeCc39PrSdubIpd7VMKQzaHefWwXVc3Kwu GKjKFcfSVI80Cg0AVP2dVg2o2M+6TM4kJCJe9lfLCN4QSbOO8mcHwi2b53uaiw== X-Google-Smtp-Source: AGHT+IGe3M+8qdP4zCxVxsd9uyxwdFMpt1ozi68ZkJe0eF5GPp6N9qZ+dH1oZ8S4SSjZyF69rqE6WA== X-Received: by 2002:a05:6a00:993:b0:73e:23be:11fc with SMTP id d2e1a72fcca58-74827f379b0mr5443465b3a.22.1749226736294; Fri, 06 Jun 2025 09:18:56 -0700 (PDT) Received: from jemmy.. ([180.172.46.169]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7482af38353sm1518675b3a.10.2025.06.06.09.18.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 09:18:55 -0700 (PDT) From: Jemmy Wong To: =?UTF-8?q?Michal=20Koutn=C3=BD?= Cc: Tejun Heo , Johannes Weiner , Waiman Long , cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 2/3] cgroup: add lock guard support for css_set_lock and rcu Date: Sat, 7 Jun 2025 00:18:40 +0800 Message-ID: <20250606161841.44354-3-jemmywong512@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606161841.44354-1-jemmywong512@gmail.com> References: <20250606161841.44354-1-jemmywong512@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: Jemmy Wong --- kernel/cgroup/cgroup-v1.c | 29 +- kernel/cgroup/cgroup.c | 580 ++++++++++++++++++-------------------- kernel/cgroup/cpuset-v1.c | 16 +- kernel/cgroup/debug.c | 185 ++++++------ kernel/cgroup/freezer.c | 28 +- kernel/cgroup/namespace.c | 8 +- 6 files changed, 401 insertions(+), 445 deletions(-) diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c index f4658eda4445..fcc2d474b470 100644 --- a/kernel/cgroup/cgroup-v1.c +++ b/kernel/cgroup/cgroup-v1.c @@ -70,9 +70,9 @@ int cgroup_attach_task_all(struct task_struct *from, stru= ct task_struct *tsk) for_each_root(root) { struct cgroup *from_cgrp; =20 - spin_lock_irq(&css_set_lock); - from_cgrp =3D task_cgroup_from_root(from, root); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + from_cgrp =3D task_cgroup_from_root(from, root); + } =20 retval =3D cgroup_attach_task(from_cgrp, tsk, false); if (retval) @@ -117,10 +117,10 @@ int cgroup_transfer_tasks(struct cgroup *to, struct c= group *from) cgroup_attach_lock(true); =20 /* all tasks in @from are being moved, all csets are source */ - spin_lock_irq(&css_set_lock); - list_for_each_entry(link, &from->cset_links, cset_link) - cgroup_migrate_add_src(link->cset, to, &mgctx); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + list_for_each_entry(link, &from->cset_links, cset_link) + cgroup_migrate_add_src(link->cset, to, &mgctx); + } =20 ret =3D cgroup_migrate_prepare_dst(&mgctx); if (ret) @@ -728,13 +728,12 @@ int cgroupstats_build(struct cgroupstats *stats, stru= ct dentry *dentry) * @kn->priv's validity. For this and css_tryget_online_from_dir(), * @kn->priv is RCU safe. Let's do the RCU dancing. */ - rcu_read_lock(); - cgrp =3D rcu_dereference(*(void __rcu __force **)&kn->priv); - if (!cgrp || !cgroup_tryget(cgrp)) { - rcu_read_unlock(); - return -ENOENT; + scoped_guard(rcu) { + cgrp =3D rcu_dereference(*(void __rcu __force **)&kn->priv); + if (!cgrp || !cgroup_tryget(cgrp)) { + return -ENOENT; + } } - rcu_read_unlock(); =20 css_task_iter_start(&cgrp->self, 0, &it); while ((tsk =3D css_task_iter_next(&it))) { @@ -1294,7 +1293,7 @@ struct cgroup *task_get_cgroup1(struct task_struct *t= sk, int hierarchy_id) struct cgroup_root *root; unsigned long flags; =20 - rcu_read_lock(); + guard(rcu)(); for_each_root(root) { /* cgroup1 only*/ if (root =3D=3D &cgrp_dfl_root) @@ -1308,7 +1307,7 @@ struct cgroup *task_get_cgroup1(struct task_struct *t= sk, int hierarchy_id) spin_unlock_irqrestore(&css_set_lock, flags); break; } - rcu_read_unlock(); + return cgrp; } =20 diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 54f80afe4f65..46b677a066d1 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -631,13 +631,8 @@ int __cgroup_task_count(const struct cgroup *cgrp) */ int cgroup_task_count(const struct cgroup *cgrp) { - int count; - - spin_lock_irq(&css_set_lock); - count =3D __cgroup_task_count(cgrp); - spin_unlock_irq(&css_set_lock); - - return count; + guard(spinlock_irq)(&css_set_lock); + return __cgroup_task_count(cgrp); } =20 static struct cgroup *kn_priv(struct kernfs_node *kn) @@ -1202,11 +1197,11 @@ static struct css_set *find_css_set(struct css_set = *old_cset, =20 /* First see if we already have a cgroup group that matches * the desired set */ - spin_lock_irq(&css_set_lock); - cset =3D find_existing_css_set(old_cset, cgrp, template); - if (cset) - get_css_set(cset); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + cset =3D find_existing_css_set(old_cset, cgrp, template); + if (cset) + get_css_set(cset); + } =20 if (cset) return cset; @@ -1238,34 +1233,33 @@ static struct css_set *find_css_set(struct css_set = *old_cset, * find_existing_css_set() */ memcpy(cset->subsys, template, sizeof(cset->subsys)); =20 - spin_lock_irq(&css_set_lock); - /* Add reference counts and links from the new css_set. */ - list_for_each_entry(link, &old_cset->cgrp_links, cgrp_link) { - struct cgroup *c =3D link->cgrp; + scoped_guard(spinlock_irq, &css_set_lock) { + /* Add reference counts and links from the new css_set. */ + list_for_each_entry(link, &old_cset->cgrp_links, cgrp_link) { + struct cgroup *c =3D link->cgrp; =20 - if (c->root =3D=3D cgrp->root) - c =3D cgrp; - link_css_set(&tmp_links, cset, c); - } + if (c->root =3D=3D cgrp->root) + c =3D cgrp; + link_css_set(&tmp_links, cset, c); + } =20 - BUG_ON(!list_empty(&tmp_links)); + BUG_ON(!list_empty(&tmp_links)); =20 - css_set_count++; + css_set_count++; =20 - /* Add @cset to the hash table */ - key =3D css_set_hash(cset->subsys); - hash_add(css_set_table, &cset->hlist, key); + /* Add @cset to the hash table */ + key =3D css_set_hash(cset->subsys); + hash_add(css_set_table, &cset->hlist, key); =20 - for_each_subsys(ss, ssid) { - struct cgroup_subsys_state *css =3D cset->subsys[ssid]; + for_each_subsys(ss, ssid) { + struct cgroup_subsys_state *css =3D cset->subsys[ssid]; =20 - list_add_tail(&cset->e_cset_node[ssid], - &css->cgroup->e_csets[ssid]); - css_get(css); + list_add_tail(&cset->e_cset_node[ssid], + &css->cgroup->e_csets[ssid]); + css_get(css); + } } =20 - spin_unlock_irq(&css_set_lock); - /* * If @cset should be threaded, look up the matching dom_cset and * link them up. We first fully initialize @cset then look for the @@ -1281,11 +1275,11 @@ static struct css_set *find_css_set(struct css_set = *old_cset, return NULL; } =20 - spin_lock_irq(&css_set_lock); - cset->dom_cset =3D dcset; - list_add_tail(&cset->threaded_csets_node, - &dcset->threaded_csets); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + cset->dom_cset =3D dcset; + list_add_tail(&cset->threaded_csets_node, + &dcset->threaded_csets); + } } =20 return cset; @@ -1362,16 +1356,14 @@ static void cgroup_destroy_root(struct cgroup_root = *root) * Release all the links from cset_links to this hierarchy's * root cgroup */ - spin_lock_irq(&css_set_lock); - - list_for_each_entry_safe(link, tmp_link, &cgrp->cset_links, cset_link) { - list_del(&link->cset_link); - list_del(&link->cgrp_link); - kfree(link); + scoped_guard(spinlock_irq, &css_set_lock) { + list_for_each_entry_safe(link, tmp_link, &cgrp->cset_links, cset_link) { + list_del(&link->cset_link); + list_del(&link->cgrp_link); + kfree(link); + } } =20 - spin_unlock_irq(&css_set_lock); - WARN_ON_ONCE(list_empty(&root->root_list)); list_del_rcu(&root->root_list); cgroup_root_count--; @@ -1437,13 +1429,10 @@ current_cgns_cgroup_from_root(struct cgroup_root *r= oot) =20 lockdep_assert_held(&css_set_lock); =20 - rcu_read_lock(); - - cset =3D current->nsproxy->cgroup_ns->root_cset; - res =3D __cset_cgroup_from_root(cset, root); - - rcu_read_unlock(); - + scoped_guard(rcu) { + cset =3D current->nsproxy->cgroup_ns->root_cset; + res =3D __cset_cgroup_from_root(cset, root); + } /* * The namespace_sem is held by current, so the root cgroup can't * be umounted. Therefore, we can ensure that the res is non-NULL. @@ -1867,25 +1856,25 @@ int rebind_subsystems(struct cgroup_root *dst_root,= u16 ss_mask) rcu_assign_pointer(dcgrp->subsys[ssid], css); ss->root =3D dst_root; =20 - spin_lock_irq(&css_set_lock); - css->cgroup =3D dcgrp; - WARN_ON(!list_empty(&dcgrp->e_csets[ss->id])); - list_for_each_entry_safe(cset, cset_pos, &scgrp->e_csets[ss->id], - e_cset_node[ss->id]) { - list_move_tail(&cset->e_cset_node[ss->id], - &dcgrp->e_csets[ss->id]); - /* - * all css_sets of scgrp together in same order to dcgrp, - * patch in-flight iterators to preserve correct iteration. - * since the iterator is always advanced right away and - * finished when it->cset_pos meets it->cset_head, so only - * update it->cset_head is enough here. - */ - list_for_each_entry(it, &cset->task_iters, iters_node) - if (it->cset_head =3D=3D &scgrp->e_csets[ss->id]) - it->cset_head =3D &dcgrp->e_csets[ss->id]; + scoped_guard(spinlock_irq, &css_set_lock) { + css->cgroup =3D dcgrp; + WARN_ON(!list_empty(&dcgrp->e_csets[ss->id])); + list_for_each_entry_safe(cset, cset_pos, &scgrp->e_csets[ss->id], + e_cset_node[ss->id]) { + list_move_tail(&cset->e_cset_node[ss->id], + &dcgrp->e_csets[ss->id]); + /* + * all css_sets of scgrp together in same order to dcgrp, + * patch in-flight iterators to preserve correct iteration. + * since the iterator is always advanced right away and + * finished when it->cset_pos meets it->cset_head, so only + * update it->cset_head is enough here. + */ + list_for_each_entry(it, &cset->task_iters, iters_node) + if (it->cset_head =3D=3D &scgrp->e_csets[ss->id]) + it->cset_head =3D &dcgrp->e_csets[ss->id]; + } } - spin_unlock_irq(&css_set_lock); =20 /* default hierarchy doesn't enable controllers by default */ dst_root->subsys_mask |=3D 1 << ssid; @@ -1921,10 +1910,10 @@ int cgroup_show_path(struct seq_file *sf, struct ke= rnfs_node *kf_node, if (!buf) return -ENOMEM; =20 - spin_lock_irq(&css_set_lock); - ns_cgroup =3D current_cgns_cgroup_from_root(kf_cgroot); - len =3D kernfs_path_from_node(kf_node, ns_cgroup->kn, buf, PATH_MAX); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + ns_cgroup =3D current_cgns_cgroup_from_root(kf_cgroot); + len =3D kernfs_path_from_node(kf_node, ns_cgroup->kn, buf, PATH_MAX); + } =20 if (len =3D=3D -E2BIG) len =3D -ERANGE; @@ -2175,13 +2164,13 @@ int cgroup_setup_root(struct cgroup_root *root, u16= ss_mask) * Link the root cgroup in this hierarchy into all the css_set * objects. */ - spin_lock_irq(&css_set_lock); - hash_for_each(css_set_table, i, cset, hlist) { - link_css_set(&tmp_links, cset, root_cgrp); - if (css_set_populated(cset)) - cgroup_update_populated(root_cgrp, true); + scoped_guard(spinlock_irq, &css_set_lock) { + hash_for_each(css_set_table, i, cset, hlist) { + link_css_set(&tmp_links, cset, root_cgrp); + if (css_set_populated(cset)) + cgroup_update_populated(root_cgrp, true); + } } - spin_unlock_irq(&css_set_lock); =20 BUG_ON(!list_empty(&root_cgrp->self.children)); BUG_ON(atomic_read(&root->nr_cgrps) !=3D 1); @@ -2225,11 +2214,8 @@ int cgroup_do_get_tree(struct fs_context *fc) struct cgroup *cgrp; =20 scoped_guard(cgroup_mutex) { - spin_lock_irq(&css_set_lock); - + guard(spinlock_irq)(&css_set_lock); cgrp =3D cset_cgroup_from_root(ctx->ns->root_cset, ctx->root); - - spin_unlock_irq(&css_set_lock); } =20 nsdentry =3D kernfs_node_dentry(cgrp->kn, sb); @@ -2438,16 +2424,10 @@ int cgroup_path_ns_locked(struct cgroup *cgrp, char= *buf, size_t buflen, int cgroup_path_ns(struct cgroup *cgrp, char *buf, size_t buflen, struct cgroup_namespace *ns) { - int ret; - guard(cgroup_mutex)(); - spin_lock_irq(&css_set_lock); - - ret =3D cgroup_path_ns_locked(cgrp, buf, buflen, ns); - - spin_unlock_irq(&css_set_lock); + guard(spinlock_irq)(&css_set_lock); =20 - return ret; + return cgroup_path_ns_locked(cgrp, buf, buflen, ns); } EXPORT_SYMBOL_GPL(cgroup_path_ns); =20 @@ -2629,27 +2609,27 @@ static int cgroup_migrate_execute(struct cgroup_mgc= tx *mgctx) * the new cgroup. There are no failure cases after here, so this * is the commit point. */ - spin_lock_irq(&css_set_lock); - list_for_each_entry(cset, &tset->src_csets, mg_node) { - list_for_each_entry_safe(task, tmp_task, &cset->mg_tasks, cg_list) { - struct css_set *from_cset =3D task_css_set(task); - struct css_set *to_cset =3D cset->mg_dst_cset; - - get_css_set(to_cset); - to_cset->nr_tasks++; - css_set_move_task(task, from_cset, to_cset, true); - from_cset->nr_tasks--; - /* - * If the source or destination cgroup is frozen, - * the task might require to change its state. - */ - cgroup_freezer_migrate_task(task, from_cset->dfl_cgrp, - to_cset->dfl_cgrp); - put_css_set_locked(from_cset); + scoped_guard(spinlock_irq, &css_set_lock) { + list_for_each_entry(cset, &tset->src_csets, mg_node) { + list_for_each_entry_safe(task, tmp_task, &cset->mg_tasks, cg_list) { + struct css_set *from_cset =3D task_css_set(task); + struct css_set *to_cset =3D cset->mg_dst_cset; + + get_css_set(to_cset); + to_cset->nr_tasks++; + css_set_move_task(task, from_cset, to_cset, true); + from_cset->nr_tasks--; + /* + * If the source or destination cgroup is frozen, + * the task might require to change its state. + */ + cgroup_freezer_migrate_task(task, from_cset->dfl_cgrp, + to_cset->dfl_cgrp); + put_css_set_locked(from_cset); =20 + } } } - spin_unlock_irq(&css_set_lock); =20 /* * Migration is committed, all target tasks are now on dst_csets. @@ -2682,13 +2662,13 @@ static int cgroup_migrate_execute(struct cgroup_mgc= tx *mgctx) } while_each_subsys_mask(); } out_release_tset: - spin_lock_irq(&css_set_lock); - list_splice_init(&tset->dst_csets, &tset->src_csets); - list_for_each_entry_safe(cset, tmp_cset, &tset->src_csets, mg_node) { - list_splice_tail_init(&cset->mg_tasks, &cset->tasks); - list_del_init(&cset->mg_node); + scoped_guard(spinlock_irq, &css_set_lock) { + list_splice_init(&tset->dst_csets, &tset->src_csets); + list_for_each_entry_safe(cset, tmp_cset, &tset->src_csets, mg_node) { + list_splice_tail_init(&cset->mg_tasks, &cset->tasks); + list_del_init(&cset->mg_node); + } } - spin_unlock_irq(&css_set_lock); =20 /* * Re-initialize the cgroup_taskset structure in case it is reused @@ -2746,7 +2726,7 @@ void cgroup_migrate_finish(struct cgroup_mgctx *mgctx) =20 lockdep_assert_held(&cgroup_mutex); =20 - spin_lock_irq(&css_set_lock); + guard(spinlock_irq)(&css_set_lock); =20 list_for_each_entry_safe(cset, tmp_cset, &mgctx->preloaded_src_csets, mg_src_preload_node) { @@ -2765,8 +2745,6 @@ void cgroup_migrate_finish(struct cgroup_mgctx *mgctx) list_del_init(&cset->mg_dst_preload_node); put_css_set_locked(cset); } - - spin_unlock_irq(&css_set_lock); } =20 /** @@ -2909,14 +2887,14 @@ int cgroup_migrate(struct task_struct *leader, bool= threadgroup, * section to prevent tasks from being freed while taking the snapshot. * spin_lock_irq() implies RCU critical section here. */ - spin_lock_irq(&css_set_lock); - task =3D leader; - do { - cgroup_migrate_add_task(task, mgctx); - if (!threadgroup) - break; - } while_each_thread(leader, task); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + task =3D leader; + do { + cgroup_migrate_add_task(task, mgctx); + if (!threadgroup) + break; + } while_each_thread(leader, task); + } =20 return cgroup_migrate_execute(mgctx); } @@ -2937,16 +2915,15 @@ int cgroup_attach_task(struct cgroup *dst_cgrp, str= uct task_struct *leader, int ret =3D 0; =20 /* look up all src csets */ - spin_lock_irq(&css_set_lock); - rcu_read_lock(); - task =3D leader; - do { - cgroup_migrate_add_src(task_css_set(task), dst_cgrp, &mgctx); - if (!threadgroup) - break; - } while_each_thread(leader, task); - rcu_read_unlock(); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + guard(rcu)(); + task =3D leader; + do { + cgroup_migrate_add_src(task_css_set(task), dst_cgrp, &mgctx); + if (!threadgroup) + break; + } while_each_thread(leader, task); + } =20 /* prepare dst csets and commit */ ret =3D cgroup_migrate_prepare_dst(&mgctx); @@ -3088,23 +3065,23 @@ static int cgroup_update_dfl_csses(struct cgroup *c= grp) lockdep_assert_held(&cgroup_mutex); =20 /* look up all csses currently attached to @cgrp's subtree */ - spin_lock_irq(&css_set_lock); - cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) { - struct cgrp_cset_link *link; + scoped_guard(spinlock_irq, &css_set_lock) { + cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) { + struct cgrp_cset_link *link; =20 - /* - * As cgroup_update_dfl_csses() is only called by - * cgroup_apply_control(). The csses associated with the - * given cgrp will not be affected by changes made to - * its subtree_control file. We can skip them. - */ - if (dsct =3D=3D cgrp) - continue; + /* + * As cgroup_update_dfl_csses() is only called by + * cgroup_apply_control(). The csses associated with the + * given cgrp will not be affected by changes made to + * its subtree_control file. We can skip them. + */ + if (dsct =3D=3D cgrp) + continue; =20 - list_for_each_entry(link, &dsct->cset_links, cset_link) - cgroup_migrate_add_src(link->cset, dsct, &mgctx); + list_for_each_entry(link, &dsct->cset_links, cset_link) + cgroup_migrate_add_src(link->cset, dsct, &mgctx); + } } - spin_unlock_irq(&css_set_lock); =20 /* * We need to write-lock threadgroup_rwsem while migrating tasks. @@ -3120,16 +3097,16 @@ static int cgroup_update_dfl_csses(struct cgroup *c= grp) if (ret) goto out_finish; =20 - spin_lock_irq(&css_set_lock); - list_for_each_entry(src_cset, &mgctx.preloaded_src_csets, - mg_src_preload_node) { - struct task_struct *task, *ntask; + scoped_guard(spinlock_irq, &css_set_lock) { + list_for_each_entry(src_cset, &mgctx.preloaded_src_csets, + mg_src_preload_node) { + struct task_struct *task, *ntask; =20 - /* all tasks in src_csets need to be migrated */ - list_for_each_entry_safe(task, ntask, &src_cset->tasks, cg_list) - cgroup_migrate_add_task(task, &mgctx); + /* all tasks in src_csets need to be migrated */ + list_for_each_entry_safe(task, ntask, &src_cset->tasks, cg_list) + cgroup_migrate_add_task(task, &mgctx); + } } - spin_unlock_irq(&css_set_lock); =20 ret =3D cgroup_migrate_execute(&mgctx); out_finish: @@ -3734,7 +3711,8 @@ static int cgroup_stat_show(struct seq_file *seq, voi= d *v) * Without proper lock protection, racing is possible. So the * numbers may not be consistent when that happens. */ - rcu_read_lock(); + guard(rcu)(); + for (ssid =3D 0; ssid < CGROUP_SUBSYS_COUNT; ssid++) { dying_cnt[ssid] =3D -1; if ((BIT(ssid) & cgrp_dfl_inhibit_ss_mask) || @@ -3753,7 +3731,6 @@ static int cgroup_stat_show(struct seq_file *seq, voi= d *v) seq_printf(seq, "nr_dying_subsys_%s %d\n", cgroup_subsys[ssid]->name, dying_cnt[ssid]); } - rcu_read_unlock(); return 0; } =20 @@ -3771,11 +3748,10 @@ static struct cgroup_subsys_state *cgroup_tryget_cs= s(struct cgroup *cgrp, { struct cgroup_subsys_state *css; =20 - rcu_read_lock(); + guard(rcu)(); css =3D cgroup_css(cgrp, ss); if (css && !css_tryget_online(css)) css =3D NULL; - rcu_read_unlock(); =20 return css; } @@ -4056,9 +4032,9 @@ static void __cgroup_kill(struct cgroup *cgrp) =20 lockdep_assert_held(&cgroup_mutex); =20 - spin_lock_irq(&css_set_lock); - cgrp->kill_seq++; - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + cgrp->kill_seq++; + } =20 css_task_iter_start(&cgrp->self, CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THRE= ADED, &it); while ((task =3D css_task_iter_next(&it))) { @@ -4187,9 +4163,9 @@ static ssize_t cgroup_file_write(struct kernfs_open_f= ile *of, char *buf, * doesn't need to be pinned. The RCU locking is not necessary * either. It's just for the convenience of using cgroup_css(). */ - rcu_read_lock(); - css =3D cgroup_css(cgrp, cft->ss); - rcu_read_unlock(); + scoped_guard(rcu) { + css =3D cgroup_css(cgrp, cft->ss); + } =20 if (cft->write_u64) { unsigned long long v; @@ -4815,14 +4791,14 @@ bool css_has_online_children(struct cgroup_subsys_s= tate *css) struct cgroup_subsys_state *child; bool ret =3D false; =20 - rcu_read_lock(); + guard(rcu)(); css_for_each_child(child, css) { if (child->flags & CSS_ONLINE) { ret =3D true; break; } } - rcu_read_unlock(); + return ret; } =20 @@ -5247,9 +5223,9 @@ static ssize_t __cgroup_procs_write(struct kernfs_ope= n_file *of, char *buf, goto out_unlock; =20 /* find the source cgroup */ - spin_lock_irq(&css_set_lock); - src_cgrp =3D task_cgroup_from_root(task, &cgrp_dfl_root); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + src_cgrp =3D task_cgroup_from_root(task, &cgrp_dfl_root); + } =20 /* * Process and thread migrations follow same delegation rule. Check @@ -5531,11 +5507,11 @@ static inline void css_release_work_fn_locked(struc= t work_struct *work) =20 css_rstat_flush(&cgrp->self); =20 - spin_lock_irq(&css_set_lock); - for (tcgrp =3D cgroup_parent(cgrp); tcgrp; - tcgrp =3D cgroup_parent(tcgrp)) - tcgrp->nr_dying_descendants--; - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + for (tcgrp =3D cgroup_parent(cgrp); tcgrp; + tcgrp =3D cgroup_parent(tcgrp)) + tcgrp->nr_dying_descendants--; + } =20 /* * There are two control paths which try to determine @@ -5790,20 +5766,20 @@ static struct cgroup *cgroup_create(struct cgroup *= parent, const char *name, goto out_psi_free; =20 /* allocation complete, commit to creation */ - spin_lock_irq(&css_set_lock); - for (i =3D 0; i < level; i++) { - tcgrp =3D cgrp->ancestors[i]; - tcgrp->nr_descendants++; + scoped_guard(spinlock_irq, &css_set_lock) { + for (i =3D 0; i < level; i++) { + tcgrp =3D cgrp->ancestors[i]; + tcgrp->nr_descendants++; =20 - /* - * If the new cgroup is frozen, all ancestor cgroups get a new - * frozen descendant, but their state can't change because of - * this. - */ - if (cgrp->freezer.e_freeze) - tcgrp->freezer.nr_frozen_descendants++; + /* + * If the new cgroup is frozen, all ancestor cgroups get a new + * frozen descendant, but their state can't change because of + * this. + */ + if (cgrp->freezer.e_freeze) + tcgrp->freezer.nr_frozen_descendants++; + } } - spin_unlock_irq(&css_set_lock); =20 list_add_tail_rcu(&cgrp->self.sibling, &cgroup_parent(cgrp)->self.childre= n); atomic_inc(&root->nr_cgrps); @@ -6047,10 +6023,10 @@ static int cgroup_destroy_locked(struct cgroup *cgr= p) */ cgrp->self.flags &=3D ~CSS_ONLINE; =20 - spin_lock_irq(&css_set_lock); - list_for_each_entry(link, &cgrp->cset_links, cset_link) - link->cset->dead =3D true; - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + list_for_each_entry(link, &cgrp->cset_links, cset_link) + link->cset->dead =3D true; + } =20 /* initiate massacre of all css's */ for_each_css(css, ssid, cgrp) @@ -6063,18 +6039,18 @@ static int cgroup_destroy_locked(struct cgroup *cgr= p) if (cgroup_is_threaded(cgrp)) parent->nr_threaded_children--; =20 - spin_lock_irq(&css_set_lock); - for (tcgrp =3D parent; tcgrp; tcgrp =3D cgroup_parent(tcgrp)) { - tcgrp->nr_descendants--; - tcgrp->nr_dying_descendants++; - /* - * If the dying cgroup is frozen, decrease frozen descendants - * counters of ancestor cgroups. - */ - if (test_bit(CGRP_FROZEN, &cgrp->flags)) - tcgrp->freezer.nr_frozen_descendants--; + scoped_guard(spinlock_irq, &css_set_lock) { + for (tcgrp =3D parent; tcgrp; tcgrp =3D cgroup_parent(tcgrp)) { + tcgrp->nr_descendants--; + tcgrp->nr_dying_descendants++; + /* + * If the dying cgroup is frozen, decrease frozen descendants + * counters of ancestor cgroups. + */ + if (test_bit(CGRP_FROZEN, &cgrp->flags)) + tcgrp->freezer.nr_frozen_descendants--; + } } - spin_unlock_irq(&css_set_lock); =20 cgroup1_check_for_release(parent); =20 @@ -6356,13 +6332,12 @@ struct cgroup *cgroup_get_from_id(u64 id) return ERR_PTR(-ENOENT); } =20 - rcu_read_lock(); - - cgrp =3D rcu_dereference(*(void __rcu __force **)&kn->priv); - if (cgrp && !cgroup_tryget(cgrp)) - cgrp =3D NULL; + scoped_guard(rcu) { + cgrp =3D rcu_dereference(*(void __rcu __force **)&kn->priv); + if (cgrp && !cgroup_tryget(cgrp)) + cgrp =3D NULL; + } =20 - rcu_read_unlock(); kernfs_put(kn); =20 if (!cgrp) @@ -6539,14 +6514,14 @@ static int cgroup_css_set_fork(struct kernel_clone_= args *kargs) =20 cgroup_threadgroup_change_begin(current); =20 - spin_lock_irq(&css_set_lock); - cset =3D task_css_set(current); - get_css_set(cset); - if (kargs->cgrp) - kargs->kill_seq =3D kargs->cgrp->kill_seq; - else - kargs->kill_seq =3D cset->dfl_cgrp->kill_seq; - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + cset =3D task_css_set(current); + get_css_set(cset); + if (kargs->cgrp) + kargs->kill_seq =3D kargs->cgrp->kill_seq; + else + kargs->kill_seq =3D cset->dfl_cgrp->kill_seq; + } =20 if (!(kargs->flags & CLONE_INTO_CGROUP)) { kargs->cset =3D cset; @@ -6736,56 +6711,53 @@ void cgroup_post_fork(struct task_struct *child, cset =3D kargs->cset; kargs->cset =3D NULL; =20 - spin_lock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + /* init tasks are special, only link regular threads */ + if (likely(child->pid)) { + if (kargs->cgrp) { + cgrp_flags =3D kargs->cgrp->flags; + cgrp_kill_seq =3D kargs->cgrp->kill_seq; + } else { + cgrp_flags =3D cset->dfl_cgrp->flags; + cgrp_kill_seq =3D cset->dfl_cgrp->kill_seq; + } =20 - /* init tasks are special, only link regular threads */ - if (likely(child->pid)) { - if (kargs->cgrp) { - cgrp_flags =3D kargs->cgrp->flags; - cgrp_kill_seq =3D kargs->cgrp->kill_seq; + WARN_ON_ONCE(!list_empty(&child->cg_list)); + cset->nr_tasks++; + css_set_move_task(child, NULL, cset, false); } else { - cgrp_flags =3D cset->dfl_cgrp->flags; - cgrp_kill_seq =3D cset->dfl_cgrp->kill_seq; + put_css_set(cset); + cset =3D NULL; } =20 - WARN_ON_ONCE(!list_empty(&child->cg_list)); - cset->nr_tasks++; - css_set_move_task(child, NULL, cset, false); - } else { - put_css_set(cset); - cset =3D NULL; - } - - if (!(child->flags & PF_KTHREAD)) { - if (unlikely(test_bit(CGRP_FREEZE, &cgrp_flags))) { - /* - * If the cgroup has to be frozen, the new task has - * too. Let's set the JOBCTL_TRAP_FREEZE jobctl bit to - * get the task into the frozen state. - */ - spin_lock(&child->sighand->siglock); - WARN_ON_ONCE(child->frozen); - child->jobctl |=3D JOBCTL_TRAP_FREEZE; - spin_unlock(&child->sighand->siglock); + if (!(child->flags & PF_KTHREAD)) { + if (unlikely(test_bit(CGRP_FREEZE, &cgrp_flags))) { + /* + * If the cgroup has to be frozen, the new task has + * too. Let's set the JOBCTL_TRAP_FREEZE jobctl bit to + * get the task into the frozen state. + */ + spin_lock(&child->sighand->siglock); + WARN_ON_ONCE(child->frozen); + child->jobctl |=3D JOBCTL_TRAP_FREEZE; + spin_unlock(&child->sighand->siglock); + + /* + * Calling cgroup_update_frozen() isn't required here, + * because it will be called anyway a bit later from + * do_freezer_trap(). So we avoid cgroup's transient + * switch from the frozen state and back. + */ + } =20 /* - * Calling cgroup_update_frozen() isn't required here, - * because it will be called anyway a bit later from - * do_freezer_trap(). So we avoid cgroup's transient - * switch from the frozen state and back. + * If the cgroup is to be killed notice it now and take the + * child down right after we finished preparing it for + * userspace. */ + kill =3D kargs->kill_seq !=3D cgrp_kill_seq; } - - /* - * If the cgroup is to be killed notice it now and take the - * child down right after we finished preparing it for - * userspace. - */ - kill =3D kargs->kill_seq !=3D cgrp_kill_seq; } - - spin_unlock_irq(&css_set_lock); - /* * Call ss->fork(). This must happen after @child is linked on * css_set; otherwise, @child might change state between ->fork() @@ -6824,25 +6796,23 @@ void cgroup_exit(struct task_struct *tsk) struct css_set *cset; int i; =20 - spin_lock_irq(&css_set_lock); - - WARN_ON_ONCE(list_empty(&tsk->cg_list)); - cset =3D task_css_set(tsk); - css_set_move_task(tsk, cset, NULL, false); - cset->nr_tasks--; - /* matches the signal->live check in css_task_iter_advance() */ - if (thread_group_leader(tsk) && atomic_read(&tsk->signal->live)) - list_add_tail(&tsk->cg_list, &cset->dying_tasks); - - if (dl_task(tsk)) - dec_dl_tasks_cs(tsk); + scoped_guard(spinlock_irq, &css_set_lock) { + WARN_ON_ONCE(list_empty(&tsk->cg_list)); + cset =3D task_css_set(tsk); + css_set_move_task(tsk, cset, NULL, false); + cset->nr_tasks--; + /* matches the signal->live check in css_task_iter_advance() */ + if (thread_group_leader(tsk) && atomic_read(&tsk->signal->live)) + list_add_tail(&tsk->cg_list, &cset->dying_tasks); =20 - WARN_ON_ONCE(cgroup_task_frozen(tsk)); - if (unlikely(!(tsk->flags & PF_KTHREAD) && - test_bit(CGRP_FREEZE, &task_dfl_cgroup(tsk)->flags))) - cgroup_update_frozen(task_dfl_cgroup(tsk)); + if (dl_task(tsk)) + dec_dl_tasks_cs(tsk); =20 - spin_unlock_irq(&css_set_lock); + WARN_ON_ONCE(cgroup_task_frozen(tsk)); + if (unlikely(!(tsk->flags & PF_KTHREAD) && + test_bit(CGRP_FREEZE, &task_dfl_cgroup(tsk)->flags))) + cgroup_update_frozen(task_dfl_cgroup(tsk)); + } =20 /* see cgroup_post_fork() for details */ do_each_subsys_mask(ss, i, have_exit_callback) { @@ -6860,10 +6830,9 @@ void cgroup_release(struct task_struct *task) } while_each_subsys_mask(); =20 if (!list_empty(&task->cg_list)) { - spin_lock_irq(&css_set_lock); + guard(spinlock_irq)(&css_set_lock); css_set_skip_task_iters(task_css_set(task), task); list_del_init(&task->cg_list); - spin_unlock_irq(&css_set_lock); } } =20 @@ -6944,7 +6913,7 @@ struct cgroup_subsys_state *css_tryget_online_from_di= r(struct dentry *dentry, !kn || kernfs_type(kn) !=3D KERNFS_DIR) return ERR_PTR(-EBADF); =20 - rcu_read_lock(); + guard(rcu)(); =20 /* * This path doesn't originate from kernfs and @kn could already @@ -6958,7 +6927,6 @@ struct cgroup_subsys_state *css_tryget_online_from_di= r(struct dentry *dentry, if (!css || !css_tryget_online(css)) css =3D ERR_PTR(-ENOENT); =20 - rcu_read_unlock(); return css; } =20 @@ -7001,13 +6969,11 @@ struct cgroup *cgroup_get_from_path(const char *pat= h) goto out_kernfs; } =20 - rcu_read_lock(); - - cgrp =3D rcu_dereference(*(void __rcu __force **)&kn->priv); - if (!cgrp || !cgroup_tryget(cgrp)) - cgrp =3D ERR_PTR(-ENOENT); - - rcu_read_unlock(); + scoped_guard(rcu) { + cgrp =3D rcu_dereference(*(void __rcu __force **)&kn->priv); + if (!cgrp || !cgroup_tryget(cgrp)) + cgrp =3D ERR_PTR(-ENOENT); + } =20 out_kernfs: kernfs_put(kn); @@ -7106,28 +7072,28 @@ void cgroup_sk_alloc(struct sock_cgroup_data *skcd) { struct cgroup *cgroup; =20 - rcu_read_lock(); - /* Don't associate the sock with unrelated interrupted task's cgroup. */ - if (in_interrupt()) { - cgroup =3D &cgrp_dfl_root.cgrp; - cgroup_get(cgroup); - goto out; - } + scoped_guard(rcu) { + /* Don't associate the sock with unrelated interrupted task's cgroup. */ + if (in_interrupt()) { + cgroup =3D &cgrp_dfl_root.cgrp; + cgroup_get(cgroup); + break; + } =20 - while (true) { - struct css_set *cset; + while (true) { + struct css_set *cset; =20 - cset =3D task_css_set(current); - if (likely(cgroup_tryget(cset->dfl_cgrp))) { - cgroup =3D cset->dfl_cgrp; - break; + cset =3D task_css_set(current); + if (likely(cgroup_tryget(cset->dfl_cgrp))) { + cgroup =3D cset->dfl_cgrp; + break; + } + cpu_relax(); } - cpu_relax(); + + skcd->cgroup =3D cgroup; + cgroup_bpf_get(cgroup); } -out: - skcd->cgroup =3D cgroup; - cgroup_bpf_get(cgroup); - rcu_read_unlock(); } =20 void cgroup_sk_clone(struct sock_cgroup_data *skcd) diff --git a/kernel/cgroup/cpuset-v1.c b/kernel/cgroup/cpuset-v1.c index b69a7db67090..114a63432d23 100644 --- a/kernel/cgroup/cpuset-v1.c +++ b/kernel/cgroup/cpuset-v1.c @@ -140,9 +140,8 @@ int cpuset_memory_pressure_enabled __read_mostly; =20 void __cpuset_memory_pressure_bump(void) { - rcu_read_lock(); + guard(rcu)(); fmeter_markevent(&task_cs(current)->fmeter); - rcu_read_unlock(); } =20 static int update_relax_domain_level(struct cpuset *cs, s64 val) @@ -393,13 +392,12 @@ int proc_cpuset_show(struct seq_file *m, struct pid_n= amespace *ns, if (!buf) goto out; =20 - rcu_read_lock(); - spin_lock_irq(&css_set_lock); - css =3D task_css(tsk, cpuset_cgrp_id); - retval =3D cgroup_path_ns_locked(css->cgroup, buf, PATH_MAX, - current->nsproxy->cgroup_ns); - spin_unlock_irq(&css_set_lock); - rcu_read_unlock(); + scoped_guard(rcu) { + guard(spinlock_irq)(&css_set_lock); + css =3D task_css(tsk, cpuset_cgrp_id); + retval =3D cgroup_path_ns_locked(css->cgroup, buf, PATH_MAX, + current->nsproxy->cgroup_ns); + } =20 if (retval =3D=3D -E2BIG) retval =3D -ENAMETOOLONG; diff --git a/kernel/cgroup/debug.c b/kernel/cgroup/debug.c index 80aa3f027ac3..2f04db56c8ac 100644 --- a/kernel/cgroup/debug.c +++ b/kernel/cgroup/debug.c @@ -48,27 +48,26 @@ static int current_css_set_read(struct seq_file *seq, v= oid *v) if (!cgroup_kn_lock_live(of->kn, false)) return -ENODEV; =20 - spin_lock_irq(&css_set_lock); - rcu_read_lock(); - cset =3D task_css_set(current); - refcnt =3D refcount_read(&cset->refcount); - seq_printf(seq, "css_set %pK %d", cset, refcnt); - if (refcnt > cset->nr_tasks) - seq_printf(seq, " +%d", refcnt - cset->nr_tasks); - seq_puts(seq, "\n"); + scoped_guard(spinlock_irq, &css_set_lock) { + guard(rcu)(); + cset =3D task_css_set(current); + refcnt =3D refcount_read(&cset->refcount); + seq_printf(seq, "css_set %pK %d", cset, refcnt); + if (refcnt > cset->nr_tasks) + seq_printf(seq, " +%d", refcnt - cset->nr_tasks); + seq_puts(seq, "\n"); =20 - /* - * Print the css'es stored in the current css_set. - */ - for_each_subsys(ss, i) { - css =3D cset->subsys[ss->id]; - if (!css) - continue; - seq_printf(seq, "%2d: %-4s\t- %p[%d]\n", ss->id, ss->name, - css, css->id); + /* + * Print the css'es stored in the current css_set. + */ + for_each_subsys(ss, i) { + css =3D cset->subsys[ss->id]; + if (!css) + continue; + seq_printf(seq, "%2d: %-4s\t- %p[%d]\n", ss->id, ss->name, + css, css->id); + } } - rcu_read_unlock(); - spin_unlock_irq(&css_set_lock); cgroup_kn_unlock(of->kn); return 0; } @@ -76,12 +75,8 @@ static int current_css_set_read(struct seq_file *seq, vo= id *v) static u64 current_css_set_refcount_read(struct cgroup_subsys_state *css, struct cftype *cft) { - u64 count; - - rcu_read_lock(); - count =3D refcount_read(&task_css_set(current)->refcount); - rcu_read_unlock(); - return count; + guard(rcu)(); + return refcount_read(&task_css_set(current)->refcount); } =20 static int current_css_set_cg_links_read(struct seq_file *seq, void *v) @@ -94,18 +89,17 @@ static int current_css_set_cg_links_read(struct seq_fil= e *seq, void *v) if (!name_buf) return -ENOMEM; =20 - spin_lock_irq(&css_set_lock); - rcu_read_lock(); - cset =3D task_css_set(current); - list_for_each_entry(link, &cset->cgrp_links, cgrp_link) { - struct cgroup *c =3D link->cgrp; + scoped_guard(spinlock_irq, &css_set_lock) { + guard(rcu)(); + cset =3D task_css_set(current); + list_for_each_entry(link, &cset->cgrp_links, cgrp_link) { + struct cgroup *c =3D link->cgrp; =20 - cgroup_name(c, name_buf, NAME_MAX + 1); - seq_printf(seq, "Root %d group %s\n", - c->root->hierarchy_id, name_buf); + cgroup_name(c, name_buf, NAME_MAX + 1); + seq_printf(seq, "Root %d group %s\n", + c->root->hierarchy_id, name_buf); + } } - rcu_read_unlock(); - spin_unlock_irq(&css_set_lock); kfree(name_buf); return 0; } @@ -117,74 +111,73 @@ static int cgroup_css_links_read(struct seq_file *seq= , void *v) struct cgrp_cset_link *link; int dead_cnt =3D 0, extra_refs =3D 0, threaded_csets =3D 0; =20 - spin_lock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + list_for_each_entry(link, &css->cgroup->cset_links, cset_link) { + struct css_set *cset =3D link->cset; + struct task_struct *task; + int count =3D 0; + int refcnt =3D refcount_read(&cset->refcount); + + /* + * Print out the proc_cset and threaded_cset relationship + * and highlight difference between refcount and task_count. + */ + seq_printf(seq, "css_set %pK", cset); + if (rcu_dereference_protected(cset->dom_cset, 1) !=3D cset) { + threaded_csets++; + seq_printf(seq, "=3D>%pK", cset->dom_cset); + } + if (!list_empty(&cset->threaded_csets)) { + struct css_set *tcset; + int idx =3D 0; + + list_for_each_entry(tcset, &cset->threaded_csets, + threaded_csets_node) { + seq_puts(seq, idx ? "," : "<=3D"); + seq_printf(seq, "%pK", tcset); + idx++; + } + } else { + seq_printf(seq, " %d", refcnt); + if (refcnt - cset->nr_tasks > 0) { + int extra =3D refcnt - cset->nr_tasks; + + seq_printf(seq, " +%d", extra); + /* + * Take out the one additional reference in + * init_css_set. + */ + if (cset =3D=3D &init_css_set) + extra--; + extra_refs +=3D extra; + } + } + seq_puts(seq, "\n"); =20 - list_for_each_entry(link, &css->cgroup->cset_links, cset_link) { - struct css_set *cset =3D link->cset; - struct task_struct *task; - int count =3D 0; - int refcnt =3D refcount_read(&cset->refcount); + list_for_each_entry(task, &cset->tasks, cg_list) { + if (count++ <=3D MAX_TASKS_SHOWN_PER_CSS) + seq_printf(seq, " task %d\n", + task_pid_vnr(task)); + } =20 - /* - * Print out the proc_cset and threaded_cset relationship - * and highlight difference between refcount and task_count. - */ - seq_printf(seq, "css_set %pK", cset); - if (rcu_dereference_protected(cset->dom_cset, 1) !=3D cset) { - threaded_csets++; - seq_printf(seq, "=3D>%pK", cset->dom_cset); - } - if (!list_empty(&cset->threaded_csets)) { - struct css_set *tcset; - int idx =3D 0; - - list_for_each_entry(tcset, &cset->threaded_csets, - threaded_csets_node) { - seq_puts(seq, idx ? "," : "<=3D"); - seq_printf(seq, "%pK", tcset); - idx++; + list_for_each_entry(task, &cset->mg_tasks, cg_list) { + if (count++ <=3D MAX_TASKS_SHOWN_PER_CSS) + seq_printf(seq, " task %d\n", + task_pid_vnr(task)); } - } else { - seq_printf(seq, " %d", refcnt); - if (refcnt - cset->nr_tasks > 0) { - int extra =3D refcnt - cset->nr_tasks; - - seq_printf(seq, " +%d", extra); - /* - * Take out the one additional reference in - * init_css_set. - */ - if (cset =3D=3D &init_css_set) - extra--; - extra_refs +=3D extra; + /* show # of overflowed tasks */ + if (count > MAX_TASKS_SHOWN_PER_CSS) + seq_printf(seq, " ... (%d)\n", + count - MAX_TASKS_SHOWN_PER_CSS); + + if (cset->dead) { + seq_puts(seq, " [dead]\n"); + dead_cnt++; } - } - seq_puts(seq, "\n"); - - list_for_each_entry(task, &cset->tasks, cg_list) { - if (count++ <=3D MAX_TASKS_SHOWN_PER_CSS) - seq_printf(seq, " task %d\n", - task_pid_vnr(task)); - } =20 - list_for_each_entry(task, &cset->mg_tasks, cg_list) { - if (count++ <=3D MAX_TASKS_SHOWN_PER_CSS) - seq_printf(seq, " task %d\n", - task_pid_vnr(task)); + WARN_ON(count !=3D cset->nr_tasks); } - /* show # of overflowed tasks */ - if (count > MAX_TASKS_SHOWN_PER_CSS) - seq_printf(seq, " ... (%d)\n", - count - MAX_TASKS_SHOWN_PER_CSS); - - if (cset->dead) { - seq_puts(seq, " [dead]\n"); - dead_cnt++; - } - - WARN_ON(count !=3D cset->nr_tasks); } - spin_unlock_irq(&css_set_lock); =20 if (!dead_cnt && !extra_refs && !threaded_csets) return 0; diff --git a/kernel/cgroup/freezer.c b/kernel/cgroup/freezer.c index bf1690a167dd..01134b3af176 100644 --- a/kernel/cgroup/freezer.c +++ b/kernel/cgroup/freezer.c @@ -108,12 +108,12 @@ void cgroup_enter_frozen(void) if (current->frozen) return; =20 - spin_lock_irq(&css_set_lock); + guard(spinlock_irq)(&css_set_lock); + current->frozen =3D true; cgrp =3D task_dfl_cgroup(current); cgroup_inc_frozen_cnt(cgrp); cgroup_update_frozen(cgrp); - spin_unlock_irq(&css_set_lock); } =20 /* @@ -129,7 +129,8 @@ void cgroup_leave_frozen(bool always_leave) { struct cgroup *cgrp; =20 - spin_lock_irq(&css_set_lock); + guard(spinlock_irq)(&css_set_lock); + cgrp =3D task_dfl_cgroup(current); if (always_leave || !test_bit(CGRP_FREEZE, &cgrp->flags)) { cgroup_dec_frozen_cnt(cgrp); @@ -142,7 +143,6 @@ void cgroup_leave_frozen(bool always_leave) set_thread_flag(TIF_SIGPENDING); spin_unlock(¤t->sighand->siglock); } - spin_unlock_irq(&css_set_lock); } =20 /* @@ -178,12 +178,12 @@ static void cgroup_do_freeze(struct cgroup *cgrp, boo= l freeze) =20 lockdep_assert_held(&cgroup_mutex); =20 - spin_lock_irq(&css_set_lock); - if (freeze) - set_bit(CGRP_FREEZE, &cgrp->flags); - else - clear_bit(CGRP_FREEZE, &cgrp->flags); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + if (freeze) + set_bit(CGRP_FREEZE, &cgrp->flags); + else + clear_bit(CGRP_FREEZE, &cgrp->flags); + } =20 if (freeze) TRACE_CGROUP_PATH(freeze, cgrp); @@ -206,10 +206,10 @@ static void cgroup_do_freeze(struct cgroup *cgrp, boo= l freeze) * Cgroup state should be revisited here to cover empty leaf cgroups * and cgroups which descendants are already in the desired state. */ - spin_lock_irq(&css_set_lock); - if (cgrp->nr_descendants =3D=3D cgrp->freezer.nr_frozen_descendants) - cgroup_update_frozen(cgrp); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + if (cgrp->nr_descendants =3D=3D cgrp->freezer.nr_frozen_descendants) + cgroup_update_frozen(cgrp); + } } =20 /* diff --git a/kernel/cgroup/namespace.c b/kernel/cgroup/namespace.c index 144a464e45c6..3e2454a106aa 100644 --- a/kernel/cgroup/namespace.c +++ b/kernel/cgroup/namespace.c @@ -71,10 +71,10 @@ struct cgroup_namespace *copy_cgroup_ns(unsigned long f= lags, return ERR_PTR(-ENOSPC); =20 /* It is not safe to take cgroup_mutex here */ - spin_lock_irq(&css_set_lock); - cset =3D task_css_set(current); - get_css_set(cset); - spin_unlock_irq(&css_set_lock); + scoped_guard(spinlock_irq, &css_set_lock) { + cset =3D task_css_set(current); + get_css_set(cset); + } =20 new_ns =3D alloc_cgroup_ns(); if (IS_ERR(new_ns)) { --=20 2.43.0 From nobody Fri Dec 19 00:05:22 2025 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8287628A1CE; Fri, 6 Jun 2025 16:19:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749226742; cv=none; b=Q02eMB5WQN/efE4OvnIR6SUSknxpogN25O/LTGyJRF2VpzShogqHe4jgg+dWTGTPBoTVZWJMaWlzh/VMlfTtSHl0Uqq585T3ct75dNKNss/ZizMd/qghHzhRI8g1FFMDyezgnU+AAzA8KKdfQpW374yMYQX9vQgvYWcl1w+3OOU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749226742; c=relaxed/simple; bh=lLb+jNOrtfcqwgNscb056qIBxvga8xPQUWy2hvwYULk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kv3YVW3lgYZJoyRLRMTuhllY5n85asDFThUerTbmrK3XJi4l+pSx4jnhC3LEgUnLBOrVwjXpf4Ed/frHCD9IxXlGvHNXLbp3chCswOsrE27Vx9KmgX832ohhJfKxV0xDEZ7T+8xZdg7gvg9CsTgwOgdgAu1RTjU1S6ZazkmGXmw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JvfOMurp; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JvfOMurp" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-72d3b48d2ffso1912247b3a.2; Fri, 06 Jun 2025 09:19:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1749226740; x=1749831540; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WmddC+fvugcNh6NPL5Z6V5D8fMLub+35xnJ4f4NuuCY=; b=JvfOMurpwNZkdo5/SkBpxrhg7n1NReDOujg2lnxmifhJ3zThO6fnSOiZv2L1KmTO13 4OLlmAWhz9GmC7T/RRMRQetl5UsWT8G0AGzYDEUAbN5NIqOprard2nTtVVAwvGzU4lFI 8q43M5m1jpioPfu0AxgmFmi6ieFiz5aIcBdkiDcbnww7/CO7bA6SPR5gMhO9oaX2DYDf /UXxFwiVSELirSkpMD2oKWdvJDJTZrpOYS3zxbua0mj/SgHKLdkOZhpzU8XmjKd4+7He aVDdo2nHURy7kVVJJpOO4q5hyO/y/oo3Kcfx51jhm5a/CXoi/r6uVfKyFMDxGUtfkyHZ sNgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749226740; x=1749831540; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WmddC+fvugcNh6NPL5Z6V5D8fMLub+35xnJ4f4NuuCY=; b=Rz4/X62ozB8MM0JudsOZETocJ9I3QgwpMvqS2RBHtUQcozIXaKnark066tZeAoAS/E YLBcQMcw3qfz6Hq4rvki1U5dOQqkanR5Lpc+mSXgkLzbAImLiXR3ydtY00VAORcXhf/L yyxnSVYxSXTbqOfvTni5kvLPe2K5uZrDX0wiJBg1yyh/+o6HwAiNExaYkf7e6G4/Hs0H yWpvgzj38R1IzasSLHe+yTJQfkRd9gfP/C/A167aGwcWS5d3OGqYNJYahyWCeQ5cnIT6 gejxIACPKUwpVEXPsQvpbn8/jwoXNWvtSD0xER3d45e3h7bYk1v3+JtaFeu2wcUS2h// oDsg== X-Forwarded-Encrypted: i=1; AJvYcCV1Gr4k06cbg1Etx70if3399gHw4xwo/W5xdvvGy8dv+6qBIKMfX6RC/qJjfUW2Fma27Di0dtlfXAbBRzUu@vger.kernel.org, AJvYcCWQklR7TqNtjESjFp3v13ZpXkv+EbFOMdLm0izv2L8ihr2fn6CjgqkbkaFZB4DKdtWVgGyZM+s3@vger.kernel.org X-Gm-Message-State: AOJu0Ywg1LqKXP38TQtVk7RJXjurUEc++RVE+tpVi9ZSmYJ/Iby3Ny0x 9lKtwMaYAkpNd5UhvunBwrHVM+Ir97bct4c24tqYHcVMEX+bJTe9wRKd X-Gm-Gg: ASbGncvpRSHkXfqav0egCunitZ8nsfOmeCw7T18/CN5/pwe0zkc5a29b9TojNGDTurF 3AXj/8TBnV1/aimswrA8NfvhiY4EP6fYB/FtEcgl5YDje6l8Ez3UYh50ekYuwhxN8Ni2CK4Dbi8 DTYBKCfuMqwZ6Ud2Hq2eOKXYtrHhP886hhIupbcaQST3yGMxR/zMKW9lrpKq9jPn2m3P4hvXjVM wIaRzvk1zvfBa3BaArhGp7dXAdsACLEPnMDmrqNJ1tAt4aEz+r4/1RoCCoR2/UHibWJbb1c3kug /LDADIhjsmCfkUYfWFXUbeY76gxmDeYTqzWeP6Ae/Fz+bwGYvXK023lTph22cA== X-Google-Smtp-Source: AGHT+IFcXtPfFGYXvkgSpvJ7vtSOpxnnhRLikBp1U3gJWE6ad36suuVrBjAV85Ov9GEOGtdfNwEpHg== X-Received: by 2002:a05:6a00:4646:b0:73e:1566:5960 with SMTP id d2e1a72fcca58-748280ab71bmr6362559b3a.19.1749226739523; Fri, 06 Jun 2025 09:18:59 -0700 (PDT) Received: from jemmy.. ([180.172.46.169]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7482af38353sm1518675b3a.10.2025.06.06.09.18.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jun 2025 09:18:59 -0700 (PDT) From: Jemmy Wong To: =?UTF-8?q?Michal=20Koutn=C3=BD?= Cc: Tejun Heo , Johannes Weiner , Waiman Long , cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 3/3] cgroup: add lock guard support for others Date: Sat, 7 Jun 2025 00:18:41 +0800 Message-ID: <20250606161841.44354-4-jemmywong512@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250606161841.44354-1-jemmywong512@gmail.com> References: <20250606161841.44354-1-jemmywong512@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: Jemmy Wong --- kernel/cgroup/cgroup-internal.h | 8 ++-- kernel/cgroup/cgroup-v1.c | 11 +++-- kernel/cgroup/cgroup.c | 73 ++++++++++++--------------------- 3 files changed, 35 insertions(+), 57 deletions(-) diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-interna= l.h index b14e61c64a34..5430454d38ca 100644 --- a/kernel/cgroup/cgroup-internal.h +++ b/kernel/cgroup/cgroup-internal.h @@ -198,8 +198,6 @@ void put_css_set_locked(struct css_set *cset); =20 static inline void put_css_set(struct css_set *cset) { - unsigned long flags; - /* * Ensure that the refcount doesn't hit zero while any readers * can see it. Similar to atomic_dec_and_lock(), but for an @@ -208,9 +206,9 @@ static inline void put_css_set(struct css_set *cset) if (refcount_dec_not_one(&cset->refcount)) return; =20 - spin_lock_irqsave(&css_set_lock, flags); - put_css_set_locked(cset); - spin_unlock_irqrestore(&css_set_lock, flags); + scoped_guard(spinlock_irqsave, &css_set_lock) { + put_css_set_locked(cset); + } } =20 /* diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c index fcc2d474b470..91c6ba4e441d 100644 --- a/kernel/cgroup/cgroup-v1.c +++ b/kernel/cgroup/cgroup-v1.c @@ -1291,7 +1291,6 @@ struct cgroup *task_get_cgroup1(struct task_struct *t= sk, int hierarchy_id) { struct cgroup *cgrp =3D ERR_PTR(-ENOENT); struct cgroup_root *root; - unsigned long flags; =20 guard(rcu)(); for_each_root(root) { @@ -1300,11 +1299,11 @@ struct cgroup *task_get_cgroup1(struct task_struct = *tsk, int hierarchy_id) continue; if (root->hierarchy_id !=3D hierarchy_id) continue; - spin_lock_irqsave(&css_set_lock, flags); - cgrp =3D task_cgroup_from_root(tsk, root); - if (!cgrp || !cgroup_tryget(cgrp)) - cgrp =3D ERR_PTR(-ENOENT); - spin_unlock_irqrestore(&css_set_lock, flags); + scoped_guard(spinlock_irqsave, &css_set_lock) { + cgrp =3D task_cgroup_from_root(tsk, root); + if (!cgrp || !cgroup_tryget(cgrp)) + cgrp =3D ERR_PTR(-ENOENT); + } break; } =20 diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 46b677a066d1..ea98679b01e1 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -335,28 +335,23 @@ static int cgroup_idr_alloc(struct idr *idr, void *pt= r, int start, int end, int ret; =20 idr_preload(gfp_mask); - spin_lock_bh(&cgroup_idr_lock); - ret =3D idr_alloc(idr, ptr, start, end, gfp_mask & ~__GFP_DIRECT_RECLAIM); - spin_unlock_bh(&cgroup_idr_lock); + scoped_guard(spinlock_bh, &cgroup_idr_lock) { + ret =3D idr_alloc(idr, ptr, start, end, gfp_mask & ~__GFP_DIRECT_RECLAIM= ); + } idr_preload_end(); return ret; } =20 static void *cgroup_idr_replace(struct idr *idr, void *ptr, int id) { - void *ret; - - spin_lock_bh(&cgroup_idr_lock); - ret =3D idr_replace(idr, ptr, id); - spin_unlock_bh(&cgroup_idr_lock); - return ret; + guard(spinlock_bh)(&cgroup_idr_lock); + return idr_replace(idr, ptr, id); } =20 static void cgroup_idr_remove(struct idr *idr, int id) { - spin_lock_bh(&cgroup_idr_lock); + guard(spinlock_bh)(&cgroup_idr_lock); idr_remove(idr, id); - spin_unlock_bh(&cgroup_idr_lock); } =20 static bool cgroup_has_tasks(struct cgroup *cgrp) @@ -583,20 +578,19 @@ struct cgroup_subsys_state *cgroup_get_e_css(struct c= group *cgrp, if (!CGROUP_HAS_SUBSYS_CONFIG) return NULL; =20 - rcu_read_lock(); + guard(rcu)(); =20 do { css =3D cgroup_css(cgrp, ss); =20 if (css && css_tryget_online(css)) - goto out_unlock; + return css; cgrp =3D cgroup_parent(cgrp); } while (cgrp); =20 css =3D init_css_set.subsys[ss->id]; css_get(css); -out_unlock: - rcu_read_unlock(); + return css; } EXPORT_SYMBOL_GPL(cgroup_get_e_css); @@ -1691,9 +1685,9 @@ static void cgroup_rm_file(struct cgroup *cgrp, const= struct cftype *cft) struct cgroup_subsys_state *css =3D cgroup_css(cgrp, cft->ss); struct cgroup_file *cfile =3D (void *)css + cft->file_offset; =20 - spin_lock_irq(&cgroup_file_kn_lock); - cfile->kn =3D NULL; - spin_unlock_irq(&cgroup_file_kn_lock); + scoped_guard(spinlock_irq, &cgroup_file_kn_lock) { + cfile->kn =3D NULL; + } =20 timer_delete_sync(&cfile->notify_timer); } @@ -4277,9 +4271,9 @@ static int cgroup_add_file(struct cgroup_subsys_state= *css, struct cgroup *cgrp, =20 timer_setup(&cfile->notify_timer, cgroup_file_notify_timer, 0); =20 - spin_lock_irq(&cgroup_file_kn_lock); - cfile->kn =3D kn; - spin_unlock_irq(&cgroup_file_kn_lock); + scoped_guard(spinlock_irq, &cgroup_file_kn_lock) { + cfile->kn =3D kn; + } } =20 return 0; @@ -4534,9 +4528,7 @@ int cgroup_add_legacy_cftypes(struct cgroup_subsys *s= s, struct cftype *cfts) */ void cgroup_file_notify(struct cgroup_file *cfile) { - unsigned long flags; - - spin_lock_irqsave(&cgroup_file_kn_lock, flags); + guard(spinlock_irqsave)(&cgroup_file_kn_lock); if (cfile->kn) { unsigned long last =3D cfile->notified_at; unsigned long next =3D last + CGROUP_FILE_NOTIFY_MIN_INTV; @@ -4548,7 +4540,6 @@ void cgroup_file_notify(struct cgroup_file *cfile) cfile->notified_at =3D jiffies; } } - spin_unlock_irqrestore(&cgroup_file_kn_lock, flags); } =20 /** @@ -4560,10 +4551,10 @@ void cgroup_file_show(struct cgroup_file *cfile, bo= ol show) { struct kernfs_node *kn; =20 - spin_lock_irq(&cgroup_file_kn_lock); - kn =3D cfile->kn; - kernfs_get(kn); - spin_unlock_irq(&cgroup_file_kn_lock); + scoped_guard(spinlock_irq, &cgroup_file_kn_lock) { + kn =3D cfile->kn; + kernfs_get(kn); + } =20 if (kn) kernfs_show(kn, show); @@ -4987,11 +4978,10 @@ static void css_task_iter_advance(struct css_task_i= ter *it) void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int fla= gs, struct css_task_iter *it) { - unsigned long irqflags; =20 memset(it, 0, sizeof(*it)); =20 - spin_lock_irqsave(&css_set_lock, irqflags); + guard(spinlock_irqsave)(&css_set_lock); =20 it->ss =3D css->ss; it->flags =3D flags; @@ -5004,8 +4994,6 @@ void css_task_iter_start(struct cgroup_subsys_state *= css, unsigned int flags, it->cset_head =3D it->cset_pos; =20 css_task_iter_advance(it); - - spin_unlock_irqrestore(&css_set_lock, irqflags); } =20 /** @@ -5018,14 +5006,12 @@ void css_task_iter_start(struct cgroup_subsys_state= *css, unsigned int flags, */ struct task_struct *css_task_iter_next(struct css_task_iter *it) { - unsigned long irqflags; - if (it->cur_task) { put_task_struct(it->cur_task); it->cur_task =3D NULL; } =20 - spin_lock_irqsave(&css_set_lock, irqflags); + guard(spinlock_irqsave)(&css_set_lock); =20 /* @it may be half-advanced by skips, finish advancing */ if (it->flags & CSS_TASK_ITER_SKIPPED) @@ -5038,8 +5024,6 @@ struct task_struct *css_task_iter_next(struct css_tas= k_iter *it) css_task_iter_advance(it); } =20 - spin_unlock_irqrestore(&css_set_lock, irqflags); - return it->cur_task; } =20 @@ -5051,13 +5035,10 @@ struct task_struct *css_task_iter_next(struct css_t= ask_iter *it) */ void css_task_iter_end(struct css_task_iter *it) { - unsigned long irqflags; - if (it->cur_cset) { - spin_lock_irqsave(&css_set_lock, irqflags); + guard(spinlock_irqsave)(&css_set_lock); list_del(&it->iters_node); put_css_set_locked(it->cur_cset); - spin_unlock_irqrestore(&css_set_lock, irqflags); } =20 if (it->cur_dcset) @@ -6737,10 +6718,10 @@ void cgroup_post_fork(struct task_struct *child, * too. Let's set the JOBCTL_TRAP_FREEZE jobctl bit to * get the task into the frozen state. */ - spin_lock(&child->sighand->siglock); - WARN_ON_ONCE(child->frozen); - child->jobctl |=3D JOBCTL_TRAP_FREEZE; - spin_unlock(&child->sighand->siglock); + scoped_guard(spinlock, &child->sighand->siglock) { + WARN_ON_ONCE(child->frozen); + child->jobctl |=3D JOBCTL_TRAP_FREEZE; + } =20 /* * Calling cgroup_update_frozen() isn't required here, --=20 2.43.0