From nobody Tue Feb 10 06:06:00 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7526AC76196 for ; Fri, 31 Mar 2023 14:52:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232964AbjCaOwy (ORCPT ); Fri, 31 Mar 2023 10:52:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232556AbjCaOwt (ORCPT ); Fri, 31 Mar 2023 10:52:49 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B43B20C0B for ; Fri, 31 Mar 2023 07:51:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680274278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=304LdT+/7RZOGHsAhGWSWVEPTe0KatK6bcaF4lkQOkY=; b=jSnLghUqtG+VF9/a6OY6l0rl+lyXIJFIlic5MKoFVIC5jUuzhv/DTBQGM/XJMNDvChACyg hrtdlyuzK3qJJtOtzIbOIntNa5ZiJ/5tkBTMcN0OkWO1Bdptw02B56C3qeZPBtclDGaSh/ WZ6MwLQCKoBJVJn67yThpzu4hGl0k/U= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-74-BwyEO9JYNeiCQkclWjnU5Q-1; Fri, 31 Mar 2023 10:51:17 -0400 X-MC-Unique: BwyEO9JYNeiCQkclWjnU5Q-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 44A6B3823A1B; Fri, 31 Mar 2023 14:51:16 +0000 (UTC) Received: from llong.com (unknown [10.22.17.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB063492B00; Fri, 31 Mar 2023 14:51:15 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Juri Lelli , Dietmar Eggemann , gscrivan@redhat.com, Waiman Long Subject: [PATCH 1/3] cgroup/cpuset: Make cpuset_fork() handle CLONE_INTO_CGROUP properly Date: Fri, 31 Mar 2023 10:50:43 -0400 Message-Id: <20230331145045.2251683-2-longman@redhat.com> In-Reply-To: <20230331145045.2251683-1-longman@redhat.com> References: <20230331145045.2251683-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" By default, the clone(2) syscall spawn a child process into the same cgroup as its parent. With the use of the CLONE_INTO_CGROUP flag introduced by commit ef2c41cf38a7 ("clone3: allow spawning processes into cgroups"), the child will be spawned into a different cgroup which is somewhat similar to writing the child's tid into "cgroup.threads". The current cpuset_fork() method does not properly handle the CLONE_INTO_CGROUP case where the cpuset of the child may be different from that of its parent. Update the cpuset_fork() method to treat the CLONE_INTO_CGROUP case similar to cpuset_attach(). Since the newly cloned task has not been running yet, its actual memory usage isn't known. So it is not necessary to make change to mm in cpuset_fork(). Fixes: ef2c41cf38a7 ("clone3: allow spawning processes into cgroups") Signed-off-by: Waiman Long Reported-by: Giuseppe Scrivano --- kernel/cgroup/cpuset.c | 56 +++++++++++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 20 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index bc4dcfd7bee5..f6d5614982d7 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -2516,16 +2516,33 @@ static void cpuset_cancel_attach(struct cgroup_task= set *tset) } =20 /* - * Protected by cpuset_rwsem. cpus_attach is used only by cpuset_attach() + * Protected by cpuset_rwsem. cpus_attach is used only by cpuset_attach_ta= sk() * but we can't allocate it dynamically there. Define it global and * allocate from cpuset_init(). */ static cpumask_var_t cpus_attach; +static nodemask_t cpuset_attach_nodemask_to; + +static void cpuset_attach_task(struct cpuset *cs, struct task_struct *task) +{ + percpu_rwsem_assert_held(&cpuset_rwsem); + + if (cs !=3D &top_cpuset) + guarantee_online_cpus(task, cpus_attach); + else + cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); + /* + * can_attach beforehand should guarantee that this doesn't + * fail. TODO: have a better way to handle failure here + */ + WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach)); + + cpuset_change_task_nodemask(task, &cpuset_attach_nodemask_to); + cpuset_update_task_spread_flags(cs, task); +} =20 static void cpuset_attach(struct cgroup_taskset *tset) { - /* static buf protected by cpuset_rwsem */ - static nodemask_t cpuset_attach_nodemask_to; struct task_struct *task; struct task_struct *leader; struct cgroup_subsys_state *css; @@ -2556,20 +2573,8 @@ static void cpuset_attach(struct cgroup_taskset *tse= t) =20 guarantee_online_mems(cs, &cpuset_attach_nodemask_to); =20 - cgroup_taskset_for_each(task, css, tset) { - if (cs !=3D &top_cpuset) - guarantee_online_cpus(task, cpus_attach); - else - cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); - /* - * can_attach beforehand should guarantee that this doesn't - * fail. TODO: have a better way to handle failure here - */ - WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach)); - - cpuset_change_task_nodemask(task, &cpuset_attach_nodemask_to); - cpuset_update_task_spread_flags(cs, task); - } + cgroup_taskset_for_each(task, css, tset) + cpuset_attach_task(cs, task); =20 /* * Change mm for all threadgroup leaders. This is expensive and may @@ -3267,11 +3272,22 @@ static void cpuset_bind(struct cgroup_subsys_state = *root_css) */ static void cpuset_fork(struct task_struct *task) { - if (task_css_is_root(task, cpuset_cgrp_id)) + struct cpuset *cs =3D task_cs(task); + + if (cs =3D=3D task_cs(current)) { + if (cs =3D=3D &top_cpuset) + return; + + set_cpus_allowed_ptr(task, current->cpus_ptr); + task->mems_allowed =3D current->mems_allowed; return; + } =20 - set_cpus_allowed_ptr(task, current->cpus_ptr); - task->mems_allowed =3D current->mems_allowed; + /* CLONE_INTO_CGROUP */ + percpu_down_write(&cpuset_rwsem); + guarantee_online_mems(cs, &cpuset_attach_nodemask_to); + cpuset_attach_task(cs, task); + percpu_up_write(&cpuset_rwsem); } =20 struct cgroup_subsys cpuset_cgrp_subsys =3D { --=20 2.31.1 From nobody Tue Feb 10 06:06:00 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27E59C761A6 for ; Fri, 31 Mar 2023 14:53:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233016AbjCaOxB (ORCPT ); Fri, 31 Mar 2023 10:53:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232650AbjCaOwv (ORCPT ); Fri, 31 Mar 2023 10:52:51 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCB2F1E71E for ; Fri, 31 Mar 2023 07:51:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680274283; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+9i7sLun1jgv9svlyB9jcs4mFG0+IKoU4iPpMJ0u9ZQ=; b=LvrIXz6WGo4R2W2uI9IDoagN0uhNYZ+lsQGhOM1ptyhgMmSYRSzD/Qwq23rkkRRd27wkUn FgK3h8NE1D97TuTJ6TR6KdXJ/sJNHTHUnm5mSJbgEl/S1/Zm1oy+G16G4kXf5GaaE6KCAQ QSn11VCRyL9daNrOeW7ZR5jsSeVdQlg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-655-MV6FSdOcPKuZsRbCle0GAg-1; Fri, 31 Mar 2023 10:51:17 -0400 X-MC-Unique: MV6FSdOcPKuZsRbCle0GAg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C9FF8884ED0; Fri, 31 Mar 2023 14:51:16 +0000 (UTC) Received: from llong.com (unknown [10.22.17.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id 53611492B02; Fri, 31 Mar 2023 14:51:16 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Juri Lelli , Dietmar Eggemann , gscrivan@redhat.com, Waiman Long Subject: [PATCH 2/3] cgroup/cpuset: Make cpuset_attach_task() skip subpartitions CPUs for top_cpuset Date: Fri, 31 Mar 2023 10:50:44 -0400 Message-Id: <20230331145045.2251683-3-longman@redhat.com> In-Reply-To: <20230331145045.2251683-1-longman@redhat.com> References: <20230331145045.2251683-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" It is found that attaching a task to the top_cpuset does not currently ignore CPUs allocated to subpartitions in cpuset_attach_task(). So the code is changed to fix that. Signed-off-by: Waiman Long Reviewed-by: Michal Koutn=C3=BD --- kernel/cgroup/cpuset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index f6d5614982d7..2367de611c42 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -2530,7 +2530,8 @@ static void cpuset_attach_task(struct cpuset *cs, str= uct task_struct *task) if (cs !=3D &top_cpuset) guarantee_online_cpus(task, cpus_attach); else - cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); + cpumask_andnot(cpus_attach, task_cpu_possible_mask(task), + cs->subparts_cpus); /* * can_attach beforehand should guarantee that this doesn't * fail. TODO: have a better way to handle failure here --=20 2.31.1 From nobody Tue Feb 10 06:06:00 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD274C76196 for ; Fri, 31 Mar 2023 14:52:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232991AbjCaOw5 (ORCPT ); Fri, 31 Mar 2023 10:52:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232865AbjCaOwt (ORCPT ); Fri, 31 Mar 2023 10:52:49 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC69F1E724 for ; Fri, 31 Mar 2023 07:51:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680274279; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=spcnenCmiq2qvQdcV7Gzytx8zR6D02wtS5KoOT6JRrs=; b=TmyIha7SbcMZwtlKksOLE6lHICiktsHNdlYM/hRvXxGAfdUdWeZhhYDqfC+IcztR7g4oQg xWSzxgHGdrLbeiavHUhLPf/sxK2RN0ytKUMzfA5hafNH7qoyW5xfvOT8VsuXbgyGPD1Qax NizYx1SU1MiU6gAqniujagjBfC2t738= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-619-HcamogozNVGQqZtQgZ91Hw-1; Fri, 31 Mar 2023 10:51:17 -0400 X-MC-Unique: HcamogozNVGQqZtQgZ91Hw-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 68BA41C0878B; Fri, 31 Mar 2023 14:51:17 +0000 (UTC) Received: from llong.com (unknown [10.22.17.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id D8BB3492B00; Fri, 31 Mar 2023 14:51:16 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Juri Lelli , Dietmar Eggemann , gscrivan@redhat.com, Waiman Long Subject: [PATCH 3/3] cgroup/cpuset: Allow only one active attach operation per cpuset Date: Fri, 31 Mar 2023 10:50:45 -0400 Message-Id: <20230331145045.2251683-4-longman@redhat.com> In-Reply-To: <20230331145045.2251683-1-longman@redhat.com> References: <20230331145045.2251683-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The current cpuset code uses the global cpuset_attach_old_cs variable to store the old cpuset value between consecutive cpuset_can_attach() and cpuset_attach() calls. Since a caller of cpuset_can_attach() may not need to hold the global cgroup_threadgroup_rwsem, parallel cpuset attach operations are possible. When there are concurrent cpuset attach operations in progress, cpuset_attach() may fetch the wrong value from cpuset_attach_old_cs causing incorrect result. To avoid this problem while still allowing certain level of parallelism, drop cpuset_attach_old_cs and use a per-cpuset attach_old_cs value. Also restrict to at most one active attach operation per cpuset to avoid corrupting the value of the per-cpuset attach_old_cs value. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 2367de611c42..3f925c261513 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -198,6 +198,8 @@ struct cpuset { =20 /* Handle for cpuset.cpus.partition */ struct cgroup_file partition_file; + + struct cpuset *attach_old_cs; }; =20 /* @@ -2456,22 +2458,27 @@ static int fmeter_getrate(struct fmeter *fmp) return val; } =20 -static struct cpuset *cpuset_attach_old_cs; - /* Called by cgroups to determine if a cpuset is usable; cpuset_rwsem held= */ static int cpuset_can_attach(struct cgroup_taskset *tset) { struct cgroup_subsys_state *css; - struct cpuset *cs; + struct cpuset *cs, *oldcs; struct task_struct *task; int ret; =20 /* used later by cpuset_attach() */ - cpuset_attach_old_cs =3D task_cs(cgroup_taskset_first(tset, &css)); + oldcs =3D task_cs(cgroup_taskset_first(tset, &css)); cs =3D css_cs(css); =20 percpu_down_write(&cpuset_rwsem); =20 + /* + * Only one cpuset attach operation is allowed for each cpuset. + */ + ret =3D -EBUSY; + if (cs->attach_in_progress) + goto out_unlock; + /* allow moving tasks into an empty cpuset if on default hierarchy */ ret =3D -ENOSPC; if (!is_in_v2_mode() && @@ -2498,6 +2505,7 @@ static int cpuset_can_attach(struct cgroup_taskset *t= set) * changes which zero cpus/mems_allowed. */ cs->attach_in_progress++; + cs->attach_old_cs =3D oldcs; ret =3D 0; out_unlock: percpu_up_write(&cpuset_rwsem); @@ -2548,7 +2556,7 @@ static void cpuset_attach(struct cgroup_taskset *tset) struct task_struct *leader; struct cgroup_subsys_state *css; struct cpuset *cs; - struct cpuset *oldcs =3D cpuset_attach_old_cs; + struct cpuset *oldcs; bool cpus_updated, mems_updated; =20 cgroup_taskset_first(tset, &css); @@ -2556,6 +2564,7 @@ static void cpuset_attach(struct cgroup_taskset *tset) =20 lockdep_assert_cpus_held(); /* see cgroup_attach_lock() */ percpu_down_write(&cpuset_rwsem); + oldcs =3D cs->attach_old_cs; cpus_updated =3D !cpumask_equal(cs->effective_cpus, oldcs->effective_cpus); mems_updated =3D !nodes_equal(cs->effective_mems, oldcs->effective_mems); --=20 2.31.1