From nobody Tue Feb 10 06:05:06 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D191AC77B6F for ; Tue, 11 Apr 2023 13:38:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230253AbjDKNiA (ORCPT ); Tue, 11 Apr 2023 09:38:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230025AbjDKNhx (ORCPT ); Tue, 11 Apr 2023 09:37:53 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3042B3AAA for ; Tue, 11 Apr 2023 06:36:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1681220184; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8knbqgpwDwW2izpddcqO89y+h297sf846uyalSPG3Do=; b=E/zF09R6cplpbjMAXqkui4bKg9+vh3oyNPnVqovrDK+doOBKPdbHGA3v1smYGSQlXYh62Y ltojLl6jbUMOKgP3OXXpSNq2Uq9kk1yLj2VqmxvnbhCI/qMWhJ3TOe6EvM3b4opkWEINMX 5bSuLFJIEpKcvoRnqEUzNJN1zaLeZc8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-650-Gfi479X1MjWH0AwHmlI-Qg-1; Tue, 11 Apr 2023 09:36:19 -0400 X-MC-Unique: Gfi479X1MjWH0AwHmlI-Qg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 770F71C08786; Tue, 11 Apr 2023 13:36:18 +0000 (UTC) Received: from llong.com (unknown [10.22.33.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B22C47CD0; Tue, 11 Apr 2023 13:36:18 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Juri Lelli , Dietmar Eggemann , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Giuseppe Scrivano , Waiman Long Subject: [PATCH v4 1/5] cgroup/cpuset: Wake up cpuset_attach_wq tasks in cpuset_cancel_attach() Date: Tue, 11 Apr 2023 09:35:57 -0400 Message-Id: <20230411133601.2969636-2-longman@redhat.com> In-Reply-To: <20230411133601.2969636-1-longman@redhat.com> References: <20230411133601.2969636-1-longman@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org After a successful cpuset_can_attach() call which increments the attach_in_progress flag, either cpuset_cancel_attach() or cpuset_attach() will be called later. In cpuset_attach(), tasks in cpuset_attach_wq, if present, will be woken up at the end. That is not the case in cpuset_cancel_attach(). So missed wakeup is possible if the attach operation is somehow cancelled. Fix that by doing the wakeup in cpuset_cancel_attach() as well. Fixes: e44193d39e8d ("cpuset: let hotplug propagation work wait for task at= taching") Signed-off-by: Waiman Long Reviewed-by: Michal Koutn=C3=BD --- kernel/cgroup/cpuset.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index bc4dcfd7bee5..066689a7dcc3 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -2507,11 +2507,15 @@ static int cpuset_can_attach(struct cgroup_taskset = *tset) static void cpuset_cancel_attach(struct cgroup_taskset *tset) { struct cgroup_subsys_state *css; + struct cpuset *cs; =20 cgroup_taskset_first(tset, &css); + cs =3D css_cs(css); =20 percpu_down_write(&cpuset_rwsem); - css_cs(css)->attach_in_progress--; + cs->attach_in_progress--; + if (!cs->attach_in_progress) + wake_up(&cpuset_attach_wq); percpu_up_write(&cpuset_rwsem); } =20 --=20 2.31.1 From nobody Tue Feb 10 06:05:06 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BABC3C76196 for ; Tue, 11 Apr 2023 13:37:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229894AbjDKNh5 (ORCPT ); Tue, 11 Apr 2023 09:37:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230070AbjDKNhw (ORCPT ); Tue, 11 Apr 2023 09:37:52 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F43D3C25 for ; Tue, 11 Apr 2023 06:36:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1681220185; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uYcLtg+5b5UC5zlkP1+hLNis1rBXu0TWvq3MyhigVbA=; b=B30YUB+07Ko331WVvVCCrXwdD0sNsx5j6azWdUEsOqOT086PF7ckWsZroS914mEMmvAMzV /itPO9KuDbB8ATyAtm6mVIt1zw5r/hnay751AvhxZrWb3NAnWQhj0uGQ3ZxdtUZPiT4l+U Ro5tJq7kWJFlvKbqc3WUd/fxAf9xkZA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-622-AstwV-79MBeNhDWsoe8GHg-1; Tue, 11 Apr 2023 09:36:20 -0400 X-MC-Unique: AstwV-79MBeNhDWsoe8GHg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EFEBC855429; Tue, 11 Apr 2023 13:36:18 +0000 (UTC) Received: from llong.com (unknown [10.22.33.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 83B7F47CD0; Tue, 11 Apr 2023 13:36:18 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Juri Lelli , Dietmar Eggemann , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Giuseppe Scrivano , Waiman Long Subject: [PATCH v4 2/5] cgroup/cpuset: Make cpuset_fork() handle CLONE_INTO_CGROUP properly Date: Tue, 11 Apr 2023 09:35:58 -0400 Message-Id: <20230411133601.2969636-3-longman@redhat.com> In-Reply-To: <20230411133601.2969636-1-longman@redhat.com> References: <20230411133601.2969636-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" By default, the clone(2) syscall spawn a child process into the same cgroup as its parent. With the use of the CLONE_INTO_CGROUP flag introduced by commit ef2c41cf38a7 ("clone3: allow spawning processes into cgroups"), the child will be spawned into a different cgroup which is somewhat similar to writing the child's tid into "cgroup.threads". The current cpuset_fork() method does not properly handle the CLONE_INTO_CGROUP case where the cpuset of the child may be different from that of its parent. Update the cpuset_fork() method to treat the CLONE_INTO_CGROUP case similar to cpuset_attach(). Since the newly cloned task has not been running yet, its actual memory usage isn't known. So it is not necessary to make change to mm in cpuset_fork(). Fixes: ef2c41cf38a7 ("clone3: allow spawning processes into cgroups") Reported-by: Giuseppe Scrivano Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 62 ++++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 20 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 066689a7dcc3..e954d5abb784 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -2520,16 +2520,33 @@ static void cpuset_cancel_attach(struct cgroup_task= set *tset) } =20 /* - * Protected by cpuset_rwsem. cpus_attach is used only by cpuset_attach() + * Protected by cpuset_rwsem. cpus_attach is used only by cpuset_attach_ta= sk() * but we can't allocate it dynamically there. Define it global and * allocate from cpuset_init(). */ static cpumask_var_t cpus_attach; +static nodemask_t cpuset_attach_nodemask_to; + +static void cpuset_attach_task(struct cpuset *cs, struct task_struct *task) +{ + percpu_rwsem_assert_held(&cpuset_rwsem); + + if (cs !=3D &top_cpuset) + guarantee_online_cpus(task, cpus_attach); + else + cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); + /* + * can_attach beforehand should guarantee that this doesn't + * fail. TODO: have a better way to handle failure here + */ + WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach)); + + cpuset_change_task_nodemask(task, &cpuset_attach_nodemask_to); + cpuset_update_task_spread_flags(cs, task); +} =20 static void cpuset_attach(struct cgroup_taskset *tset) { - /* static buf protected by cpuset_rwsem */ - static nodemask_t cpuset_attach_nodemask_to; struct task_struct *task; struct task_struct *leader; struct cgroup_subsys_state *css; @@ -2560,20 +2577,8 @@ static void cpuset_attach(struct cgroup_taskset *tse= t) =20 guarantee_online_mems(cs, &cpuset_attach_nodemask_to); =20 - cgroup_taskset_for_each(task, css, tset) { - if (cs !=3D &top_cpuset) - guarantee_online_cpus(task, cpus_attach); - else - cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); - /* - * can_attach beforehand should guarantee that this doesn't - * fail. TODO: have a better way to handle failure here - */ - WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach)); - - cpuset_change_task_nodemask(task, &cpuset_attach_nodemask_to); - cpuset_update_task_spread_flags(cs, task); - } + cgroup_taskset_for_each(task, css, tset) + cpuset_attach_task(cs, task); =20 /* * Change mm for all threadgroup leaders. This is expensive and may @@ -3271,11 +3276,28 @@ static void cpuset_bind(struct cgroup_subsys_state = *root_css) */ static void cpuset_fork(struct task_struct *task) { - if (task_css_is_root(task, cpuset_cgrp_id)) + struct cpuset *cs; + bool same_cs; + + rcu_read_lock(); + cs =3D task_cs(task); + same_cs =3D (cs =3D=3D task_cs(current)); + rcu_read_unlock(); + + if (same_cs) { + if (cs =3D=3D &top_cpuset) + return; + + set_cpus_allowed_ptr(task, current->cpus_ptr); + task->mems_allowed =3D current->mems_allowed; return; + } =20 - set_cpus_allowed_ptr(task, current->cpus_ptr); - task->mems_allowed =3D current->mems_allowed; + /* CLONE_INTO_CGROUP */ + percpu_down_write(&cpuset_rwsem); + guarantee_online_mems(cs, &cpuset_attach_nodemask_to); + cpuset_attach_task(cs, task); + percpu_up_write(&cpuset_rwsem); } =20 struct cgroup_subsys cpuset_cgrp_subsys =3D { --=20 2.31.1 From nobody Tue Feb 10 06:05:06 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE432C77B6F for ; Tue, 11 Apr 2023 13:37:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229571AbjDKNhK (ORCPT ); Tue, 11 Apr 2023 09:37:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229713AbjDKNhG (ORCPT ); Tue, 11 Apr 2023 09:37:06 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2805835B3 for ; Tue, 11 Apr 2023 06:36:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1681220183; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DGStpea+VS32btfmnxUOKPtTpHs1gG9V6kkk3RmSq8M=; b=AFwFw33umj77hlJegvUS0oatyLZ+4rpKxNG9NAzTFRV+aPEprOIqWajwNf4FbDXjSkof8e TOxJCLUtusO0bcfXj5XI798S4N9FsEpnsv8s/BQe2GWEUYMPwxzoWufqvCzR34u4ejer47 BCBFf2T90rVcrPHUaDWoUC0ovsPOoJc= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-627-O4LmuVFQMla6WtKOu0eN7Q-1; Tue, 11 Apr 2023 09:36:20 -0400 X-MC-Unique: O4LmuVFQMla6WtKOu0eN7Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 76029280D587; Tue, 11 Apr 2023 13:36:19 +0000 (UTC) Received: from llong.com (unknown [10.22.33.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0904C47CDC; Tue, 11 Apr 2023 13:36:19 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Juri Lelli , Dietmar Eggemann , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Giuseppe Scrivano , Waiman Long Subject: [PATCH v4 3/5] cgroup/cpuset: Add cpuset_can_fork() and cpuset_cancel_fork() methods Date: Tue, 11 Apr 2023 09:35:59 -0400 Message-Id: <20230411133601.2969636-4-longman@redhat.com> In-Reply-To: <20230411133601.2969636-1-longman@redhat.com> References: <20230411133601.2969636-1-longman@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In the case of CLONE_INTO_CGROUP, not all cpusets are ready to accept new tasks. It is too late to check that in cpuset_fork(). So we need to add the cpuset_can_fork() and cpuset_cancel_fork() methods to pre-check it before we can allow attachment to a different cpuset. We also need to set the attach_in_progress flag to alert other code that a new task is going to be added to the cpuset. Fixes: ef2c41cf38a7 ("clone3: allow spawning processes into cgroups") Suggested-by: Michal Koutn=C3=BD Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 97 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 86 insertions(+), 11 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index e954d5abb784..132cdae4f03c 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -2458,6 +2458,20 @@ static int fmeter_getrate(struct fmeter *fmp) =20 static struct cpuset *cpuset_attach_old_cs; =20 +/* + * Check to see if a cpuset can accept a new task + * For v1, cpus_allowed and mems_allowed can't be empty. + * For v2, effective_cpus can't be empty. + * Note that in v1, effective_cpus =3D cpus_allowed. + */ +static int cpuset_can_attach_check(struct cpuset *cs) +{ + if (cpumask_empty(cs->effective_cpus) || + (!is_in_v2_mode() && nodes_empty(cs->mems_allowed))) + return -ENOSPC; + return 0; +} + /* Called by cgroups to determine if a cpuset is usable; cpuset_rwsem held= */ static int cpuset_can_attach(struct cgroup_taskset *tset) { @@ -2472,16 +2486,9 @@ static int cpuset_can_attach(struct cgroup_taskset *= tset) =20 percpu_down_write(&cpuset_rwsem); =20 - /* allow moving tasks into an empty cpuset if on default hierarchy */ - ret =3D -ENOSPC; - if (!is_in_v2_mode() && - (cpumask_empty(cs->cpus_allowed) || nodes_empty(cs->mems_allowed))) - goto out_unlock; - - /* - * Task cannot be moved to a cpuset with empty effective cpus. - */ - if (cpumask_empty(cs->effective_cpus)) + /* Check to see if task is allowed in the cpuset */ + ret =3D cpuset_can_attach_check(cs); + if (ret) goto out_unlock; =20 cgroup_taskset_for_each(task, css, tset) { @@ -2498,7 +2505,6 @@ static int cpuset_can_attach(struct cgroup_taskset *t= set) * changes which zero cpus/mems_allowed. */ cs->attach_in_progress++; - ret =3D 0; out_unlock: percpu_up_write(&cpuset_rwsem); return ret; @@ -3269,6 +3275,68 @@ static void cpuset_bind(struct cgroup_subsys_state *= root_css) percpu_up_write(&cpuset_rwsem); } =20 +/* + * In case the child is cloned into a cpuset different from its parent, + * additional checks are done to see if the move is allowed. + */ +static int cpuset_can_fork(struct task_struct *task, struct css_set *cset) +{ + struct cpuset *cs =3D css_cs(cset->subsys[cpuset_cgrp_id]); + bool same_cs; + int ret; + + rcu_read_lock(); + same_cs =3D (cs =3D=3D task_cs(current)); + rcu_read_unlock(); + + if (same_cs) + return 0; + + lockdep_assert_held(&cgroup_mutex); + percpu_down_write(&cpuset_rwsem); + + /* Check to see if task is allowed in the cpuset */ + ret =3D cpuset_can_attach_check(cs); + if (ret) + goto out_unlock; + + ret =3D task_can_attach(task, cs->effective_cpus); + if (ret) + goto out_unlock; + + ret =3D security_task_setscheduler(task); + if (ret) + goto out_unlock; + + /* + * Mark attach is in progress. This makes validate_change() fail + * changes which zero cpus/mems_allowed. + */ + cs->attach_in_progress++; +out_unlock: + percpu_up_write(&cpuset_rwsem); + return ret; +} + +static void cpuset_cancel_fork(struct task_struct *task, struct css_set *c= set) +{ + struct cpuset *cs =3D css_cs(cset->subsys[cpuset_cgrp_id]); + bool same_cs; + + rcu_read_lock(); + same_cs =3D (cs =3D=3D task_cs(current)); + rcu_read_unlock(); + + if (same_cs) + return; + + percpu_down_write(&cpuset_rwsem); + cs->attach_in_progress--; + if (!cs->attach_in_progress) + wake_up(&cpuset_attach_wq); + percpu_up_write(&cpuset_rwsem); +} + /* * Make sure the new task conform to the current state of its parent, * which could have been changed by cpuset just after it inherits the @@ -3297,6 +3365,11 @@ static void cpuset_fork(struct task_struct *task) percpu_down_write(&cpuset_rwsem); guarantee_online_mems(cs, &cpuset_attach_nodemask_to); cpuset_attach_task(cs, task); + + cs->attach_in_progress--; + if (!cs->attach_in_progress) + wake_up(&cpuset_attach_wq); + percpu_up_write(&cpuset_rwsem); } =20 @@ -3310,6 +3383,8 @@ struct cgroup_subsys cpuset_cgrp_subsys =3D { .attach =3D cpuset_attach, .post_attach =3D cpuset_post_attach, .bind =3D cpuset_bind, + .can_fork =3D cpuset_can_fork, + .cancel_fork =3D cpuset_cancel_fork, .fork =3D cpuset_fork, .legacy_cftypes =3D legacy_files, .dfl_cftypes =3D dfl_files, --=20 2.31.1 From nobody Tue Feb 10 06:05:06 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17787C76196 for ; Tue, 11 Apr 2023 13:38:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229742AbjDKNiD (ORCPT ); Tue, 11 Apr 2023 09:38:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230052AbjDKNhx (ORCPT ); Tue, 11 Apr 2023 09:37:53 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55F853AB6 for ; Tue, 11 Apr 2023 06:36:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1681220184; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kzazreGEyW5lPkWO+0aC7Q/lAqe7xC+WA1Ec8aj/SsU=; b=G3SKqpzRdGVqpUObO2GXOzUZQkYckK2bjZlt0fOz/Xbh7F1gLD0VGXaeqPh16U1VX/VSPB EyqKYFU6QTi4HL5EINo5UCK2TN3qvJVw5lK6Ec0saJFsURiBdUC76WXzYvEAL0F5ajaGTr BikaHc875+XSsEHr6DmrF3dNnO5Ugnc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-631-O11OGfWNMfio1MX3RF7cGg-1; Tue, 11 Apr 2023 09:36:21 -0400 X-MC-Unique: O11OGfWNMfio1MX3RF7cGg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F048F855432; Tue, 11 Apr 2023 13:36:19 +0000 (UTC) Received: from llong.com (unknown [10.22.33.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8256947CDC; Tue, 11 Apr 2023 13:36:19 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Juri Lelli , Dietmar Eggemann , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Giuseppe Scrivano , Waiman Long Subject: [PATCH v4 4/5] cgroup/cpuset: Make cpuset_attach_task() skip subpartitions CPUs for top_cpuset Date: Tue, 11 Apr 2023 09:36:00 -0400 Message-Id: <20230411133601.2969636-5-longman@redhat.com> In-Reply-To: <20230411133601.2969636-1-longman@redhat.com> References: <20230411133601.2969636-1-longman@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org It is found that attaching a task to the top_cpuset does not currently ignore CPUs allocated to subpartitions in cpuset_attach_task(). So the code is changed to fix that. Signed-off-by: Waiman Long Reviewed-by: Michal Koutn=C3=BD --- kernel/cgroup/cpuset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 132cdae4f03c..e4ca2dd2b764 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -2540,7 +2540,8 @@ static void cpuset_attach_task(struct cpuset *cs, str= uct task_struct *task) if (cs !=3D &top_cpuset) guarantee_online_cpus(task, cpus_attach); else - cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); + cpumask_andnot(cpus_attach, task_cpu_possible_mask(task), + cs->subparts_cpus); /* * can_attach beforehand should guarantee that this doesn't * fail. TODO: have a better way to handle failure here --=20 2.31.1 From nobody Tue Feb 10 06:05:06 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A7A8C76196 for ; Tue, 11 Apr 2023 13:37:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230104AbjDKNhU (ORCPT ); Tue, 11 Apr 2023 09:37:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229661AbjDKNhK (ORCPT ); Tue, 11 Apr 2023 09:37:10 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E31013A91 for ; Tue, 11 Apr 2023 06:36:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1681220184; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xK1NCjqBm+Noko8JAoUQYHl2QqweOko54B+7fb/aUQk=; b=Qpxux3t/jAPWs4Z4I5FiIbMArm5L0WPre2d47TWcyiXHkFnejqb5LqvnKltjcbhs0uwdMD rkC6z7U/OvaiBaJtPLoCx+wmwKCjk2G6wmSXhMosESarQ7LtqFLh9W1099VBSlV9CaLGeR 3TbgpsAG2qfqvpBZxwakK9GGt9NqOxs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-613-DwDKLdOJOFO6fjaq4KMiHg-1; Tue, 11 Apr 2023 09:36:21 -0400 X-MC-Unique: DwDKLdOJOFO6fjaq4KMiHg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 78B611C0878A; Tue, 11 Apr 2023 13:36:20 +0000 (UTC) Received: from llong.com (unknown [10.22.33.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 09FBA47CD0; Tue, 11 Apr 2023 13:36:20 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Juri Lelli , Dietmar Eggemann , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Giuseppe Scrivano , Waiman Long Subject: [PATCH v4 5/5] cgroup/cpuset: Optimize out unneeded cpuset_can_fork/cpuset_cancel_fork calls Date: Tue, 11 Apr 2023 09:36:01 -0400 Message-Id: <20230411133601.2969636-6-longman@redhat.com> In-Reply-To: <20230411133601.2969636-1-longman@redhat.com> References: <20230411133601.2969636-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The newly introduced cpuset_can_fork() and cpuset_cancel_fork() calls are only needed when the CLONE_INTO_CGROUP flag is set which is not likely. Adding an extra cpuset_can_fork() call does introduce a bit of performance overhead in the fork/clone fastpath. To reduce this performance overhead, introduce a new clone_into_cgroup_can_fork flag into the cgroup_subsys structure. This flag, when set, will call the can_fork and cancel_fork methods only if the CLONE_INTO_CGROUP flag is set. The cpuset code is now modified to set this flag. The same cpuset checking code in cpuset_can_fork() and cpuset_cancel_fork() will have to stay as the cgroups can be different, but the cpusets may still be the same. So the same check must be present in both cpuset_fork() and cpuset_can_fork() to make sure that attach_in_progress is correctly set. Signed-off-by: Waiman Long --- include/linux/cgroup-defs.h | 6 ++++++ kernel/cgroup/cgroup.c | 23 ++++++++++++++++++----- kernel/cgroup/cpuset.c | 1 + 3 files changed, 25 insertions(+), 5 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 8a0d5466c7be..0087a47d80a2 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -701,6 +701,12 @@ struct cgroup_subsys { */ bool threaded:1; =20 + /* + * If %true, the controller will call can_fork and cancel_fork + * methods only if CLONE_INTO_CGROUP flag is set. + */ + bool clone_into_cgroup_can_fork:1; + /* the following two fields are initialized automatically during boot */ int id; const char *name; diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 83ea13f2ccb1..23701e959ef5 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -6517,6 +6517,10 @@ int cgroup_can_fork(struct task_struct *child, struc= t kernel_clone_args *kargs) return ret; =20 do_each_subsys_mask(ss, i, have_canfork_callback) { + if (ss->clone_into_cgroup_can_fork && + !(kargs->flags & CLONE_INTO_CGROUP)) + continue; + ret =3D ss->can_fork(child, kargs->cset); if (ret) goto out_revert; @@ -6528,8 +6532,12 @@ int cgroup_can_fork(struct task_struct *child, struc= t kernel_clone_args *kargs) for_each_subsys(ss, j) { if (j >=3D i) break; - if (ss->cancel_fork) - ss->cancel_fork(child, kargs->cset); + if (!ss->cancel_fork || + (ss->clone_into_cgroup_can_fork && + !(kargs->flags & CLONE_INTO_CGROUP))) + continue; + + ss->cancel_fork(child, kargs->cset); } =20 cgroup_css_set_put_fork(kargs); @@ -6552,9 +6560,14 @@ void cgroup_cancel_fork(struct task_struct *child, struct cgroup_subsys *ss; int i; =20 - for_each_subsys(ss, i) - if (ss->cancel_fork) - ss->cancel_fork(child, kargs->cset); + for_each_subsys(ss, i) { + if (!ss->cancel_fork || + (ss->clone_into_cgroup_can_fork && + !(kargs->flags & CLONE_INTO_CGROUP))) + continue; + + ss->cancel_fork(child, kargs->cset); + } =20 cgroup_css_set_put_fork(kargs); } diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index e4ca2dd2b764..937ef4d60cd4 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -3391,6 +3391,7 @@ struct cgroup_subsys cpuset_cgrp_subsys =3D { .dfl_cftypes =3D dfl_files, .early_init =3D true, .threaded =3D true, + .clone_into_cgroup_can_fork =3D true, }; =20 /** --=20 2.31.1