From nobody Thu Dec 18 04:29:25 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 323D2CDB483 for ; Fri, 13 Oct 2023 18:12:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231489AbjJMSMq (ORCPT ); Fri, 13 Oct 2023 14:12:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229704AbjJMSMm (ORCPT ); Fri, 13 Oct 2023 14:12:42 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5C3CC2 for ; Fri, 13 Oct 2023 11:11:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697220713; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nolWi3aqBikqVy187I9lW2xbSl+2rIWALGYPILniGBo=; b=cOHryA/eIb1Nf5W7UK9/hlmAu4lqIxaHUytSoEHDoV14rwfKVNefdbqMB+1GgPEI5rZuFR F2VRlxDwCzOUxtUDe7l9Fa8wVzuHn39Ltp8D0auFr+oteCh8sMzeiIEEZ3/SfaVlQgqqHR NUY5WE5qUNrZxXZHRf7GoiNDHotzqC0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-88-Ar3Y8fhFM3iF4CHeOZYpwQ-1; Fri, 13 Oct 2023 14:11:49 -0400 X-MC-Unique: Ar3Y8fhFM3iF4CHeOZYpwQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C94FC862DFD; Fri, 13 Oct 2023 18:11:48 +0000 (UTC) Received: from llong.com (unknown [10.22.17.138]) by smtp.corp.redhat.com (Postfix) with ESMTP id 46A3020296DB; Fri, 13 Oct 2023 18:11:48 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Lai Jiangshan , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Waiman Long Subject: [PATCH-cgroup 1/4] workqueue: Add workqueue_unbound_exclude_cpumask() to exclude CPUs from wq_unbound_cpumask Date: Fri, 13 Oct 2023 14:11:19 -0400 Message-Id: <20231013181122.3518610-2-longman@redhat.com> In-Reply-To: <20231013181122.3518610-1-longman@redhat.com> References: <20231013181122.3518610-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When the "isolcpus" boot command line option is used to add a set of isolated CPUs, those CPUs will be excluded automatically from wq_unbound_cpumask to avoid running work functions from unbound workqueues. Recently cpuset has been extended to allow the creation of partitions of isolated CPUs dynamically. To make it closer to the "isolcpus" in functionality, the CPUs in those isolated cpuset partitions should be excluded from wq_unbound_cpumask as well. This can be done currently by explicitly writing to the workqueue's cpumask sysfs file after creating the isolated partitions. However, this process can be error prone. Ideally, the cpuset code should be allowed to request the workqueue code to exclude those isolated CPUs from wq_unbound_cpumask so that this operation can be done automatically and the isolated CPUs will be returned back to wq_unbound_cpumask after the destructions of the isolated cpuset partitions. This patch adds a new workqueue_unbound_exclude_cpumask() to enable that. This new function will exclude the specified isolated CPUs from wq_unbound_cpumask. To be able to restore those isolated CPUs back after the destruction of isolated cpuset partitions, a new wq_user_unbound_cpumask is added to store the user provided unbound cpumask either from the boot command line options or from writing to the cpumask sysfs file. This new cpumask provides the basis for CPU exclusion. Signed-off-by: Waiman Long --- include/linux/workqueue.h | 2 +- kernel/workqueue.c | 42 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index 1c1d06804d45..a936460ccc7e 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -483,7 +483,7 @@ struct workqueue_attrs *alloc_workqueue_attrs(void); void free_workqueue_attrs(struct workqueue_attrs *attrs); int apply_workqueue_attrs(struct workqueue_struct *wq, const struct workqueue_attrs *attrs); -int workqueue_set_unbound_cpumask(cpumask_var_t cpumask); +extern int workqueue_unbound_exclude_cpumask(cpumask_var_t cpumask); =20 extern bool queue_work_on(int cpu, struct workqueue_struct *wq, struct work_struct *work); diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 010b674b02a7..19d403aa41b0 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -381,6 +381,9 @@ static bool workqueue_freezing; /* PL: have wqs starte= d freezing? */ /* PL&A: allowable cpus for unbound wqs and work items */ static cpumask_var_t wq_unbound_cpumask; =20 +/* PL: user-provided unbound cpumask via sysfs */ +static cpumask_var_t wq_user_unbound_cpumask; + /* for further constrain wq_unbound_cpumask by cmdline parameter*/ static struct cpumask wq_cmdline_cpumask __initdata; =20 @@ -5825,7 +5828,7 @@ static int workqueue_apply_unbound_cpumask(const cpum= ask_var_t unbound_cpumask) * -EINVAL - Invalid @cpumask * -ENOMEM - Failed to allocate memory for attrs or pwqs. */ -int workqueue_set_unbound_cpumask(cpumask_var_t cpumask) +static int workqueue_set_unbound_cpumask(cpumask_var_t cpumask) { int ret =3D -EINVAL; =20 @@ -5836,6 +5839,7 @@ int workqueue_set_unbound_cpumask(cpumask_var_t cpuma= sk) cpumask_and(cpumask, cpumask, cpu_possible_mask); if (!cpumask_empty(cpumask)) { apply_wqattrs_lock(); + cpumask_copy(wq_user_unbound_cpumask, cpumask); if (cpumask_equal(cpumask, wq_unbound_cpumask)) { ret =3D 0; goto out_unlock; @@ -5850,6 +5854,40 @@ int workqueue_set_unbound_cpumask(cpumask_var_t cpum= ask) return ret; } =20 +/** + * workqueue_unbound_exclude_cpumask - Exclude given CPUs from unbound cpu= mask + * @exclude_cpumask: the cpumask to be excluded from wq_unbound_cpumask + * + * This function can be called from cpuset code to provide a set of isolat= ed + * CPUs that should be excluded from wq_unbound_cpumask. + */ +int workqueue_unbound_exclude_cpumask(cpumask_var_t exclude_cpumask) +{ + cpumask_var_t cpumask; + int ret =3D 0; + + if (!zalloc_cpumask_var(&cpumask, GFP_KERNEL)) + return -ENOMEM; + + /* + * The caller of this function may have called cpus_read_lock(), + * use cpus_read_trylock() to avoid potential deadlock. + */ + if (!cpus_read_trylock()) + return -EBUSY; + mutex_lock(&wq_pool_mutex); + + if (!cpumask_andnot(cpumask, wq_user_unbound_cpumask, exclude_cpumask)) + ret =3D -EINVAL; /* The new cpumask can't be empty */ + else if (!cpumask_equal(cpumask, wq_unbound_cpumask)) + ret =3D workqueue_apply_unbound_cpumask(cpumask); + + mutex_unlock(&wq_pool_mutex); + cpus_read_unlock(); + free_cpumask_var(cpumask); + return ret; +} + static int parse_affn_scope(const char *val) { int i; @@ -6520,11 +6558,13 @@ void __init workqueue_init_early(void) BUILD_BUG_ON(__alignof__(struct pool_workqueue) < __alignof__(long long)); =20 BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL)); + BUG_ON(!alloc_cpumask_var(&wq_user_unbound_cpumask, GFP_KERNEL)); cpumask_copy(wq_unbound_cpumask, housekeeping_cpumask(HK_TYPE_WQ)); cpumask_and(wq_unbound_cpumask, wq_unbound_cpumask, housekeeping_cpumask(= HK_TYPE_DOMAIN)); =20 if (!cpumask_empty(&wq_cmdline_cpumask)) cpumask_and(wq_unbound_cpumask, wq_unbound_cpumask, &wq_cmdline_cpumask); + cpumask_copy(wq_user_unbound_cpumask, wq_unbound_cpumask); =20 pwq_cache =3D KMEM_CACHE(pool_workqueue, SLAB_PANIC); =20 --=20 2.39.3 From nobody Thu Dec 18 04:29:25 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 881F7CDB47E for ; Fri, 13 Oct 2023 18:12:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231621AbjJMSMx (ORCPT ); Fri, 13 Oct 2023 14:12:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231421AbjJMSMm (ORCPT ); Fri, 13 Oct 2023 14:12:42 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9048DE0 for ; Fri, 13 Oct 2023 11:11:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697220715; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nT8mn+CYHK0/rYux6OcfYs1wPw2RH0li5fDfUDzstNM=; b=S4XM0rGW7b59pGi1QgD+vTK7Dm/ei7BbIxNgWYfcEbFHbISnOX5YVPOzCD505F/lg8+OaN YWrQ53reSVf+6oewVHtt2g77CCCSYZN7aOk4Xil95upOo6Kzar6Pf1nlIQgMeq/L3KsF/n K9j1CR6fqO7RInR+3aVI60FpKsaA5O8= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-126-vaEAS-0aOmyl2sINyVIXPw-1; Fri, 13 Oct 2023 14:11:50 -0400 X-MC-Unique: vaEAS-0aOmyl2sINyVIXPw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 66AE81C0650E; Fri, 13 Oct 2023 18:11:49 +0000 (UTC) Received: from llong.com (unknown [10.22.17.138]) by smtp.corp.redhat.com (Postfix) with ESMTP id D81BE2026831; Fri, 13 Oct 2023 18:11:48 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Lai Jiangshan , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Waiman Long Subject: [PATCH-cgroup 2/4] selftests/cgroup: Minor code cleanup and reorganization of test_cpuset_prs.sh Date: Fri, 13 Oct 2023 14:11:20 -0400 Message-Id: <20231013181122.3518610-3-longman@redhat.com> In-Reply-To: <20231013181122.3518610-1-longman@redhat.com> References: <20231013181122.3518610-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Minor cleanup of test matrix and relocation of test_isolated() function to prepare for the next patch. There is no functional change. Signed-off-by: Waiman Long --- .../selftests/cgroup/test_cpuset_prs.sh | 142 +++++++++--------- 1 file changed, 71 insertions(+), 71 deletions(-) diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/test= ing/selftests/cgroup/test_cpuset_prs.sh index a6e9848189d6..2b825019f806 100755 --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh @@ -146,71 +146,6 @@ test_add_proc() echo $$ > $CGROUP2/cgroup.procs # Move out the task } =20 -# -# Testing the new "isolated" partition root type -# -test_isolated() -{ - cd $CGROUP2/test - echo 2-3 > cpuset.cpus - TYPE=3D$(cat cpuset.cpus.partition) - [[ $TYPE =3D member ]] || echo member > cpuset.cpus.partition - - console_msg "Change from member to root" - test_partition root - - console_msg "Change from root to isolated" - test_partition isolated - - console_msg "Change from isolated to member" - test_partition member - - console_msg "Change from member to isolated" - test_partition isolated - - console_msg "Change from isolated to root" - test_partition root - - console_msg "Change from root to member" - test_partition member - - # - # Testing partition root with no cpu - # - console_msg "Distribute all cpus to child partition" - echo +cpuset > cgroup.subtree_control - test_partition root - - mkdir A1 - cd A1 - echo 2-3 > cpuset.cpus - test_partition root - test_effective_cpus 2-3 - cd .. - test_effective_cpus "" - - console_msg "Moving task to partition test" - test_add_proc "No space left" - cd A1 - test_add_proc "" - cd .. - - console_msg "Shrink and expand child partition" - cd A1 - echo 2 > cpuset.cpus - cd .. - test_effective_cpus 3 - cd A1 - echo 2-3 > cpuset.cpus - cd .. - test_effective_cpus "" - - # Cleaning up - console_msg "Cleaning up" - echo $$ > $CGROUP2/cgroup.procs - [[ -d A1 ]] && rmdir A1 -} - # # Cpuset controller state transition test matrix. # @@ -304,7 +239,7 @@ TEST_MATRIX=3D( A1:P0,A2:P2,A3:P1 2-4" " C0-4:X2-4:S+ C1-4:X2-4:S+:P2 C2-4:X4:P1 \ . . X5 . . 0 A1:0-4,A2:1-4,A3:2-4 \ - A1:P0,A2:P-2,A3:P-1 ." + A1:P0,A2:P-2,A3:P-1" " C0-4:X2-4:S+ C1-4:X2-4:S+:P2 C2-4:X4:P1 \ . . . X1 . 0 A1:0-1,A2:2-4,A3:2-4 \ A1:P0,A2:P2,A3:P-1 2-4" @@ -347,10 +282,10 @@ TEST_MATRIX=3D( # cpus_allowed/exclusive_cpus update tests " C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3 \ . C4 . P2 . 0 A1:4,A2:4,XA2:,XA3:,A3:4 \ - A1:P0,A3:P-2 ." + A1:P0,A3:P-2" " C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3 \ . X1 . P2 . 0 A1:0-3,A2:1-3,XA1:1,XA2:,XA3:,A= 3:2-3 \ - A1:P0,A3:P-2 ." + A1:P0,A3:P-2" " C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3 \ . . C3 P2 . 0 A1:0-2,A2:0-2,XA2:3,XA3:3,A3:3 \ A1:P0,A3:P2 3" @@ -359,13 +294,13 @@ TEST_MATRIX=3D( A1:P0,A3:P2 3" " C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3:P2 \ . . X3 . . 0 A1:0-3,A2:1-3,XA2:3,XA3:3,A3:2-= 3 \ - A1:P0,A3:P-2 ." + A1:P0,A3:P-2" " C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3:P2 \ . . C3 . . 0 A1:0-3,A2:3,XA2:3,XA3:3,A3:3 \ - A1:P0,A3:P-2 ." + A1:P0,A3:P-2" " C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3:P2 \ . C4 . . . 0 A1:4,A2:4,A3:4,XA1:,XA2:,XA3 \ - A1:P0,A3:P-2 ." + A1:P0,A3:P-2" =20 # old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pst= ate ISOLCPUS # ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ---= --- -------- @@ -804,6 +739,71 @@ run_state_test() echo "All $I tests of $TEST PASSED." } =20 +# +# Testing the new "isolated" partition root type +# +test_isolated() +{ + cd $CGROUP2/test + echo 2-3 > cpuset.cpus + TYPE=3D$(cat cpuset.cpus.partition) + [[ $TYPE =3D member ]] || echo member > cpuset.cpus.partition + + console_msg "Change from member to root" + test_partition root + + console_msg "Change from root to isolated" + test_partition isolated + + console_msg "Change from isolated to member" + test_partition member + + console_msg "Change from member to isolated" + test_partition isolated + + console_msg "Change from isolated to root" + test_partition root + + console_msg "Change from root to member" + test_partition member + + # + # Testing partition root with no cpu + # + console_msg "Distribute all cpus to child partition" + echo +cpuset > cgroup.subtree_control + test_partition root + + mkdir A1 + cd A1 + echo 2-3 > cpuset.cpus + test_partition root + test_effective_cpus 2-3 + cd .. + test_effective_cpus "" + + console_msg "Moving task to partition test" + test_add_proc "No space left" + cd A1 + test_add_proc "" + cd .. + + console_msg "Shrink and expand child partition" + cd A1 + echo 2 > cpuset.cpus + cd .. + test_effective_cpus 3 + cd A1 + echo 2-3 > cpuset.cpus + cd .. + test_effective_cpus "" + + # Cleaning up + console_msg "Cleaning up" + echo $$ > $CGROUP2/cgroup.procs + [[ -d A1 ]] && rmdir A1 +} + # # Wait for inotify event for the given file and read it # $1: cgroup file to wait for --=20 2.39.3 From nobody Thu Dec 18 04:29:25 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CB69CDB47E for ; Fri, 13 Oct 2023 18:13:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231599AbjJMSNE (ORCPT ); Fri, 13 Oct 2023 14:13:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231458AbjJMSM5 (ORCPT ); Fri, 13 Oct 2023 14:12:57 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C03B9E1 for ; Fri, 13 Oct 2023 11:12:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697220724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2+Udgd9t+oJaMjSuWvjsOR91eGe6O+i8JtL7jWCC2do=; b=N6QYRIL+H0w8DRFHMNVEnBSGdUfNGEApuEkedSU1lISGnx+rQ7xPX+lxfTcXXMD+HPmf02 DDRaCtqbJdr+7rv2aYf3C9FrVhZYgQAyKRHF7gue2x9SzNCufqA9/krEuyiYNN3N1tnbsm HoPHdiestNBKRnCDLGJldhTFZaX7mX8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-381-82XO3FSgP_yZC25qtn8fYA-1; Fri, 13 Oct 2023 14:11:50 -0400 X-MC-Unique: 82XO3FSgP_yZC25qtn8fYA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 092B4946DC5; Fri, 13 Oct 2023 18:11:50 +0000 (UTC) Received: from llong.com (unknown [10.22.17.138]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7B5E320296DB; Fri, 13 Oct 2023 18:11:49 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Lai Jiangshan , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Waiman Long Subject: [PATCH-cgroup 3/4] cgroup/cpuset: Keep track of CPUs in isolated partitions Date: Fri, 13 Oct 2023 14:11:21 -0400 Message-Id: <20231013181122.3518610-4-longman@redhat.com> In-Reply-To: <20231013181122.3518610-1-longman@redhat.com> References: <20231013181122.3518610-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a new internal isolated_cpus mask to keep track of the CPUs that are in isolated partitions. Expose that new cpumask as a new root-only control file ".__DEBUG__.cpuset.cpus.isolated" when cgroup_debug command line option is specified. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 190 +++++++++++++++++++++++++++-------------- 1 file changed, 127 insertions(+), 63 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 615daaf87f1f..19c8779798fd 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -204,6 +204,11 @@ struct cpuset { */ static cpumask_var_t subpartitions_cpus; =20 +/* + * Exclusive CPUs in isolated partitions + */ +static cpumask_var_t isolated_cpus; + /* List of remote partition root children */ static struct list_head remote_children; =20 @@ -1317,6 +1322,7 @@ static void compute_effective_cpumask(struct cpumask = *new_cpus, */ enum partition_cmd { partcmd_enable, /* Enable partition root */ + partcmd_enablei, /* Enable isolated partition root */ partcmd_disable, /* Disable partition root */ partcmd_update, /* Update parent's effective_cpus */ partcmd_invalidate, /* Make partition invalid */ @@ -1418,6 +1424,74 @@ static void reset_partition_data(struct cpuset *cs) } } =20 +/* + * partition_xcpus_newstate - Exclusive CPUs state change + * @old_prs: old partition_root_state + * @new_prs: new partition_root_state + * @xcpus: exclusive CPUs with state change + */ +static void partition_xcpus_newstate(int old_prs, int new_prs, struct cpum= ask *xcpus) +{ + WARN_ON_ONCE(old_prs =3D=3D new_prs); + if (new_prs =3D=3D PRS_ISOLATED) + cpumask_or(isolated_cpus, isolated_cpus, xcpus); + else + cpumask_andnot(isolated_cpus, isolated_cpus, xcpus); +} + +/* + * partition_xcpus_add - Add new exclusive CPUs to partition + * @new_prs: new partition_root_state + * @parent: parent cpuset + * @xcpus: exclusive CPUs to be added + * + * Remote partition if parent =3D=3D NULL + */ +static void partition_xcpus_add(int new_prs, struct cpuset *parent, + struct cpumask *xcpus) +{ + WARN_ON_ONCE(new_prs < 0); + lockdep_assert_held(&callback_lock); + if (!parent) + parent =3D &top_cpuset; + + if (parent =3D=3D &top_cpuset) + cpumask_or(subpartitions_cpus, subpartitions_cpus, xcpus); + + if (new_prs !=3D parent->partition_root_state) + partition_xcpus_newstate(parent->partition_root_state, new_prs, + xcpus); + + cpumask_andnot(parent->effective_cpus, parent->effective_cpus, xcpus); +} + +/* + * partition_xcpus_del - Remove exclusive CPUs from partition + * @old_prs: old partition_root_state + * @parent: parent cpuset + * @xcpus: exclusive CPUs to be removed + * + * Remote partition if parent =3D=3D NULL + */ +static void partition_xcpus_del(int old_prs, struct cpuset *parent, + struct cpumask *xcpus) +{ + WARN_ON_ONCE(old_prs < 0); + lockdep_assert_held(&callback_lock); + if (!parent) + parent =3D &top_cpuset; + + if (parent =3D=3D &top_cpuset) + cpumask_andnot(subpartitions_cpus, subpartitions_cpus, xcpus); + + if (old_prs !=3D parent->partition_root_state) + partition_xcpus_newstate(old_prs, parent->partition_root_state, + xcpus); + + cpumask_and(xcpus, xcpus, cpu_active_mask); + cpumask_or(parent->effective_cpus, parent->effective_cpus, xcpus); +} + /* * compute_effective_exclusive_cpumask - compute effective exclusive CPUs * @cs: cpuset @@ -1456,13 +1530,15 @@ static inline bool is_local_partition(struct cpuset= *cs) /* * remote_partition_enable - Enable current cpuset as a remote partition r= oot * @cs: the cpuset to update + * @new_prs: new partition_root_state * @tmp: temparary masks * Return: 1 if successful, 0 if error * * Enable the current cpuset to become a remote partition root taking CPUs * directly from the top cpuset. cpuset_mutex must be held by the caller. */ -static int remote_partition_enable(struct cpuset *cs, struct tmpmasks *tmp) +static int remote_partition_enable(struct cpuset *cs, int new_prs, + struct tmpmasks *tmp) { /* * The user must have sysadmin privilege. @@ -1485,18 +1561,14 @@ static int remote_partition_enable(struct cpuset *c= s, struct tmpmasks *tmp) return 0; =20 spin_lock_irq(&callback_lock); - cpumask_andnot(top_cpuset.effective_cpus, - top_cpuset.effective_cpus, tmp->new_cpus); - cpumask_or(subpartitions_cpus, - subpartitions_cpus, tmp->new_cpus); - + partition_xcpus_add(new_prs, NULL, tmp->new_cpus); + list_add(&cs->remote_sibling, &remote_children); if (cs->use_parent_ecpus) { struct cpuset *parent =3D parent_cs(cs); =20 cs->use_parent_ecpus =3D false; parent->child_ecpus_count--; } - list_add(&cs->remote_sibling, &remote_children); spin_unlock_irq(&callback_lock); =20 /* @@ -1524,13 +1596,8 @@ static void remote_partition_disable(struct cpuset *= cs, struct tmpmasks *tmp) WARN_ON_ONCE(!cpumask_subset(tmp->new_cpus, subpartitions_cpus)); =20 spin_lock_irq(&callback_lock); - cpumask_andnot(subpartitions_cpus, - subpartitions_cpus, tmp->new_cpus); - cpumask_and(tmp->new_cpus, - tmp->new_cpus, cpu_active_mask); - cpumask_or(top_cpuset.effective_cpus, - top_cpuset.effective_cpus, tmp->new_cpus); list_del_init(&cs->remote_sibling); + partition_xcpus_del(cs->partition_root_state, NULL, tmp->new_cpus); cs->partition_root_state =3D -cs->partition_root_state; if (!cs->prs_err) cs->prs_err =3D PERR_INVCPUS; @@ -1557,6 +1624,7 @@ static void remote_cpus_update(struct cpuset *cs, str= uct cpumask *newmask, struct tmpmasks *tmp) { bool adding, deleting; + int prs =3D cs->partition_root_state; =20 if (WARN_ON_ONCE(!is_remote_partition(cs))) return; @@ -1580,20 +1648,10 @@ static void remote_cpus_update(struct cpuset *cs, s= truct cpumask *newmask, goto invalidate; =20 spin_lock_irq(&callback_lock); - if (adding) { - cpumask_or(subpartitions_cpus, - subpartitions_cpus, tmp->addmask); - cpumask_andnot(top_cpuset.effective_cpus, - top_cpuset.effective_cpus, tmp->addmask); - } - if (deleting) { - cpumask_andnot(subpartitions_cpus, - subpartitions_cpus, tmp->delmask); - cpumask_and(tmp->delmask, - tmp->delmask, cpu_active_mask); - cpumask_or(top_cpuset.effective_cpus, - top_cpuset.effective_cpus, tmp->delmask); - } + if (adding) + partition_xcpus_add(prs, NULL, tmp->addmask); + if (deleting) + partition_xcpus_del(prs, NULL, tmp->delmask); spin_unlock_irq(&callback_lock); =20 /* @@ -1676,11 +1734,11 @@ static bool prstate_housekeeping_conflict(int prsta= te, struct cpumask *new_cpus) * @tmp: Temporary addmask and delmask * Return: 0 or a partition root state error code * - * For partcmd_enable, the cpuset is being transformed from a non-partition - * root to a partition root. The effective_xcpus (cpus_allowed if effectiv= e_xcpus - * not set) mask of the given cpuset will be taken away from parent's - * effective_cpus. The function will return 0 if all the CPUs listed in - * effective_xcpus can be granted or an error code will be returned. + * For partcmd_enable*, the cpuset is being transformed from a non-partiti= on + * root to a partition root. The effective_xcpus (cpus_allowed if + * effective_xcpus not set) mask of the given cpuset will be taken away fr= om + * parent's effective_cpus. The function will return 0 if all the CPUs lis= ted + * in effective_xcpus can be granted or an error code will be returned. * * For partcmd_disable, the cpuset is being transformed from a partition * root back to a non-partition root. Any CPUs in effective_xcpus will be @@ -1695,7 +1753,7 @@ static bool prstate_housekeeping_conflict(int prstate= , struct cpumask *new_cpus) * * For partcmd_invalidate, the current partition will be made invalid. * - * The partcmd_enable and partcmd_disable commands are used by + * The partcmd_enable* and partcmd_disable commands are used by * update_prstate(). An error code may be returned and the caller will che= ck * for error. * @@ -1760,7 +1818,7 @@ static int update_parent_effective_cpumask(struct cpu= set *cs, int cmd, =20 nocpu =3D tasks_nocpu_error(parent, cs, xcpus); =20 - if (cmd =3D=3D partcmd_enable) { + if ((cmd =3D=3D partcmd_enable) || (cmd =3D=3D partcmd_enablei)) { /* * Enabling partition root is not allowed if its * effective_xcpus is empty or doesn't overlap with @@ -1783,6 +1841,7 @@ static int update_parent_effective_cpumask(struct cpu= set *cs, int cmd, cpumask_copy(tmp->delmask, xcpus); deleting =3D true; subparts_delta++; + new_prs =3D (cmd =3D=3D partcmd_enable) ? PRS_ROOT : PRS_ISOLATED; } else if (cmd =3D=3D partcmd_disable) { /* * May need to add cpus to parent's effective_cpus for @@ -1792,6 +1851,7 @@ static int update_parent_effective_cpumask(struct cpu= set *cs, int cmd, cpumask_and(tmp->addmask, xcpus, parent->effective_xcpus); if (adding) subparts_delta--; + new_prs =3D PRS_MEMBER; } else if (newmask) { /* * Empty cpumask is not allowed @@ -1940,37 +2000,24 @@ static int update_parent_effective_cpumask(struct c= puset *cs, int cmd, * newly deleted ones will be added back to effective_cpus. */ spin_lock_irq(&callback_lock); - if (adding) { - if (parent =3D=3D &top_cpuset) - cpumask_andnot(subpartitions_cpus, - subpartitions_cpus, tmp->addmask); - /* - * Some of the CPUs in effective_xcpus might have been offlined. - */ - cpumask_or(parent->effective_cpus, - parent->effective_cpus, tmp->addmask); - cpumask_and(parent->effective_cpus, - parent->effective_cpus, cpu_active_mask); - } - if (deleting) { - if (parent =3D=3D &top_cpuset) - cpumask_or(subpartitions_cpus, - subpartitions_cpus, tmp->delmask); - cpumask_andnot(parent->effective_cpus, - parent->effective_cpus, tmp->delmask); - } - - if (is_partition_valid(parent)) { - parent->nr_subparts +=3D subparts_delta; - WARN_ON_ONCE(parent->nr_subparts < 0); - } - if (old_prs !=3D new_prs) { cs->partition_root_state =3D new_prs; if (new_prs <=3D 0) cs->nr_subparts =3D 0; } + /* + * Adding to parent's effective_cpus means deletion CPUs from cs + * and vice versa. + */ + if (adding) + partition_xcpus_del(old_prs, parent, tmp->addmask); + if (deleting) + partition_xcpus_add(new_prs, parent, tmp->delmask); =20 + if (is_partition_valid(parent)) { + parent->nr_subparts +=3D subparts_delta; + WARN_ON_ONCE(parent->nr_subparts < 0); + } spin_unlock_irq(&callback_lock); =20 if ((old_prs !=3D new_prs) && (cmd =3D=3D partcmd_update)) @@ -2948,6 +2995,7 @@ static int update_prstate(struct cpuset *cs, int new_= prs) int err =3D PERR_NONE, old_prs =3D cs->partition_root_state; struct cpuset *parent =3D parent_cs(cs); struct tmpmasks tmpmask; + bool new_xcpus_state =3D false; =20 if (old_prs =3D=3D new_prs) return 0; @@ -2977,6 +3025,9 @@ static int update_prstate(struct cpuset *cs, int new_= prs) goto out; =20 if (!old_prs) { + enum partition_cmd cmd =3D (new_prs =3D=3D PRS_ROOT) + ? partcmd_enable : partcmd_enablei; + /* * cpus_allowed cannot be empty. */ @@ -2985,19 +3036,18 @@ static int update_prstate(struct cpuset *cs, int ne= w_prs) goto out; } =20 - err =3D update_parent_effective_cpumask(cs, partcmd_enable, - NULL, &tmpmask); + err =3D update_parent_effective_cpumask(cs, cmd, NULL, &tmpmask); /* * If an attempt to become local partition root fails, * try to become a remote partition root instead. */ - if (err && remote_partition_enable(cs, &tmpmask)) + if (err && remote_partition_enable(cs, new_prs, &tmpmask)) err =3D 0; } else if (old_prs && new_prs) { /* * A change in load balance state only, no change in cpumasks. */ - ; + new_xcpus_state =3D true; } else { /* * Switching back to member is always allowed even if it @@ -3029,6 +3079,8 @@ static int update_prstate(struct cpuset *cs, int new_= prs) WRITE_ONCE(cs->prs_err, err); if (!is_partition_valid(cs)) reset_partition_data(cs); + else if (new_xcpus_state) + partition_xcpus_newstate(old_prs, new_prs, cs->effective_xcpus); spin_unlock_irq(&callback_lock); =20 /* Force update if switching back to member */ @@ -3386,6 +3438,7 @@ typedef enum { FILE_SUBPARTS_CPULIST, FILE_EXCLUSIVE_CPULIST, FILE_EFFECTIVE_XCPULIST, + FILE_ISOLATED_CPULIST, FILE_CPU_EXCLUSIVE, FILE_MEM_EXCLUSIVE, FILE_MEM_HARDWALL, @@ -3582,6 +3635,9 @@ static int cpuset_common_seq_show(struct seq_file *sf= , void *v) case FILE_SUBPARTS_CPULIST: seq_printf(sf, "%*pbl\n", cpumask_pr_args(subpartitions_cpus)); break; + case FILE_ISOLATED_CPULIST: + seq_printf(sf, "%*pbl\n", cpumask_pr_args(isolated_cpus)); + break; default: ret =3D -EINVAL; } @@ -3875,6 +3931,13 @@ static struct cftype dfl_files[] =3D { .flags =3D CFTYPE_ONLY_ON_ROOT | CFTYPE_DEBUG, }, =20 + { + .name =3D "cpus.isolated", + .seq_show =3D cpuset_common_seq_show, + .private =3D FILE_ISOLATED_CPULIST, + .flags =3D CFTYPE_ONLY_ON_ROOT | CFTYPE_DEBUG, + }, + { } /* terminate */ }; =20 @@ -4194,6 +4257,7 @@ int __init cpuset_init(void) BUG_ON(!alloc_cpumask_var(&top_cpuset.effective_xcpus, GFP_KERNEL)); BUG_ON(!alloc_cpumask_var(&top_cpuset.exclusive_cpus, GFP_KERNEL)); BUG_ON(!zalloc_cpumask_var(&subpartitions_cpus, GFP_KERNEL)); + BUG_ON(!zalloc_cpumask_var(&isolated_cpus, GFP_KERNEL)); =20 cpumask_setall(top_cpuset.cpus_allowed); nodes_setall(top_cpuset.mems_allowed); --=20 2.39.3 From nobody Thu Dec 18 04:29:25 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AF1BCDB482 for ; Fri, 13 Oct 2023 18:12:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231492AbjJMSMs (ORCPT ); Fri, 13 Oct 2023 14:12:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231400AbjJMSMm (ORCPT ); Fri, 13 Oct 2023 14:12:42 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5508CC for ; Fri, 13 Oct 2023 11:11:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697220715; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=006b2BOLjbBbZ4FbAl+3x7+rZlipZKC0l5bf6ZYnHfE=; b=HjJ2p3p5B4Wqi9QoYsUlzf6Msx/8mGy6zlQWgWMUwjob0rKVPOeI3MAZwZMm5lXENMNR0E ThysUQtTJzIfoD1Ib7gW2mfi8PocYErlyy8M7J9RLrZ5ZQWa4hVd/FPReBjcP/KcJRqG82 jQQ0neOOsdOqhzU3KJaAEK4kvDqmg3s= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-306-hZICA_IWNiGAnrq0VT935g-1; Fri, 13 Oct 2023 14:11:51 -0400 X-MC-Unique: hZICA_IWNiGAnrq0VT935g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CC14E1C0651A; Fri, 13 Oct 2023 18:11:50 +0000 (UTC) Received: from llong.com (unknown [10.22.17.138]) by smtp.corp.redhat.com (Postfix) with ESMTP id 18F2220296DB; Fri, 13 Oct 2023 18:11:50 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Lai Jiangshan , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Waiman Long Subject: [PATCH-cgroup 4/4] cgroup/cpuset: Take isolated CPUs out of workqueue unbound cpumask Date: Fri, 13 Oct 2023 14:11:22 -0400 Message-Id: <20231013181122.3518610-5-longman@redhat.com> In-Reply-To: <20231013181122.3518610-1-longman@redhat.com> References: <20231013181122.3518610-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To make CPUs in isolated cpuset partition closer in isolation to the boot time isolated CPUs specified in the "isolcpus" boot command line option, we need to take those CPUs out of the workqueue unbound cpumask so that work functions from the unbound workqueues won't run on those CPUs. Otherwise, they will interfere the user tasks running on those isolated CPUs. With the introduction of the workqueue_unbound_exclude_cpumask() helper function in a previous commit, those isolated CPUs can now be taken out from the workqueue unbound cpumask. This patch also updates cgroup-v2.rst to mention that isolated CPUs will be excluded from unbound workqueue cpumask as well as updating test_cpuset_prs.sh to verify the correctness of the new *cpuset.cpus.isolated file, if available via cgroup_debug option. Signed-off-by: Waiman Long --- Documentation/admin-guide/cgroup-v2.rst | 10 +-- kernel/cgroup/cpuset.c | 67 ++++++++++++++++--- .../selftests/cgroup/test_cpuset_prs.sh | 67 ++++++++++++++++--- 3 files changed, 120 insertions(+), 24 deletions(-) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-= guide/cgroup-v2.rst index e40b8560e002..d91ec638403b 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2311,11 +2311,11 @@ Cpuset Interface Files partition or scheduling domain. The set of exclusive CPUs is determined by the value of its "cpuset.cpus.exclusive.effective". =20 - When set to "isolated", the CPUs in that partition will - be in an isolated state without any load balancing from the - scheduler. Tasks placed in such a partition with multiple - CPUs should be carefully distributed and bound to each of the - individual CPUs for optimal performance. + When set to "isolated", the CPUs in that partition will be in + an isolated state without any load balancing from the scheduler + and excluded from the unbound workqueues. Tasks placed in such + a partition with multiple CPUs should be carefully distributed + and bound to each of the individual CPUs for optimal performance. =20 A partition root ("root" or "isolated") can be in one of the two possible states - valid or invalid. An invalid partition diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 19c8779798fd..da251764611f 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -43,6 +43,7 @@ #include #include #include +#include =20 DEFINE_STATIC_KEY_FALSE(cpusets_pre_enable_key); DEFINE_STATIC_KEY_FALSE(cpusets_enabled_key); @@ -1444,25 +1445,31 @@ static void partition_xcpus_newstate(int old_prs, i= nt new_prs, struct cpumask *x * @new_prs: new partition_root_state * @parent: parent cpuset * @xcpus: exclusive CPUs to be added + * Return: true if isolated_cpus modified, false otherwise * * Remote partition if parent =3D=3D NULL */ -static void partition_xcpus_add(int new_prs, struct cpuset *parent, +static bool partition_xcpus_add(int new_prs, struct cpuset *parent, struct cpumask *xcpus) { + bool isolcpus_updated; + WARN_ON_ONCE(new_prs < 0); lockdep_assert_held(&callback_lock); if (!parent) parent =3D &top_cpuset; =20 + if (parent =3D=3D &top_cpuset) cpumask_or(subpartitions_cpus, subpartitions_cpus, xcpus); =20 - if (new_prs !=3D parent->partition_root_state) + isolcpus_updated =3D (new_prs !=3D parent->partition_root_state); + if (isolcpus_updated) partition_xcpus_newstate(parent->partition_root_state, new_prs, xcpus); =20 cpumask_andnot(parent->effective_cpus, parent->effective_cpus, xcpus); + return isolcpus_updated; } =20 /* @@ -1470,12 +1477,15 @@ static void partition_xcpus_add(int new_prs, struct= cpuset *parent, * @old_prs: old partition_root_state * @parent: parent cpuset * @xcpus: exclusive CPUs to be removed + * Return: true if isolated_cpus modified, false otherwise * * Remote partition if parent =3D=3D NULL */ -static void partition_xcpus_del(int old_prs, struct cpuset *parent, +static bool partition_xcpus_del(int old_prs, struct cpuset *parent, struct cpumask *xcpus) { + bool isolcpus_updated; + WARN_ON_ONCE(old_prs < 0); lockdep_assert_held(&callback_lock); if (!parent) @@ -1484,12 +1494,34 @@ static void partition_xcpus_del(int old_prs, struct= cpuset *parent, if (parent =3D=3D &top_cpuset) cpumask_andnot(subpartitions_cpus, subpartitions_cpus, xcpus); =20 - if (old_prs !=3D parent->partition_root_state) + isolcpus_updated =3D (old_prs !=3D parent->partition_root_state); + if (isolcpus_updated) partition_xcpus_newstate(old_prs, parent->partition_root_state, xcpus); =20 cpumask_and(xcpus, xcpus, cpu_active_mask); cpumask_or(parent->effective_cpus, parent->effective_cpus, xcpus); + return isolcpus_updated; +} + +static void update_unbound_workqueue_cpumask(bool isolcpus_updated) +{ + int ret; + + if (!isolcpus_updated) + return; + + ret =3D workqueue_unbound_exclude_cpumask(isolated_cpus); + + /* + * An error of -EBUSY will be returned if there is a hotplug operation + * in progress. Skip the update and hopefully the unbound workqueue + * cpumask will be correctly set next time. + */ + if (ret =3D=3D -EBUSY) + return; + + WARN_ON_ONCE(ret < 0); } =20 /* @@ -1540,6 +1572,8 @@ static inline bool is_local_partition(struct cpuset *= cs) static int remote_partition_enable(struct cpuset *cs, int new_prs, struct tmpmasks *tmp) { + bool isolcpus_updated; + /* * The user must have sysadmin privilege. */ @@ -1561,7 +1595,7 @@ static int remote_partition_enable(struct cpuset *cs,= int new_prs, return 0; =20 spin_lock_irq(&callback_lock); - partition_xcpus_add(new_prs, NULL, tmp->new_cpus); + isolcpus_updated =3D partition_xcpus_add(new_prs, NULL, tmp->new_cpus); list_add(&cs->remote_sibling, &remote_children); if (cs->use_parent_ecpus) { struct cpuset *parent =3D parent_cs(cs); @@ -1570,13 +1604,13 @@ static int remote_partition_enable(struct cpuset *c= s, int new_prs, parent->child_ecpus_count--; } spin_unlock_irq(&callback_lock); + update_unbound_workqueue_cpumask(isolcpus_updated); =20 /* * Proprogate changes in top_cpuset's effective_cpus down the hierarchy. */ update_tasks_cpumask(&top_cpuset, tmp->new_cpus); update_sibling_cpumasks(&top_cpuset, NULL, tmp); - return 1; } =20 @@ -1591,18 +1625,22 @@ static int remote_partition_enable(struct cpuset *c= s, int new_prs, */ static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *t= mp) { + bool isolcpus_updated; + compute_effective_exclusive_cpumask(cs, tmp->new_cpus); WARN_ON_ONCE(!is_remote_partition(cs)); WARN_ON_ONCE(!cpumask_subset(tmp->new_cpus, subpartitions_cpus)); =20 spin_lock_irq(&callback_lock); list_del_init(&cs->remote_sibling); - partition_xcpus_del(cs->partition_root_state, NULL, tmp->new_cpus); + isolcpus_updated =3D partition_xcpus_del(cs->partition_root_state, + NULL, tmp->new_cpus); cs->partition_root_state =3D -cs->partition_root_state; if (!cs->prs_err) cs->prs_err =3D PERR_INVCPUS; reset_partition_data(cs); spin_unlock_irq(&callback_lock); + update_unbound_workqueue_cpumask(isolcpus_updated); =20 /* * Proprogate changes in top_cpuset's effective_cpus down the hierarchy. @@ -1625,6 +1663,7 @@ static void remote_cpus_update(struct cpuset *cs, str= uct cpumask *newmask, { bool adding, deleting; int prs =3D cs->partition_root_state; + int isolcpus_updated =3D 0; =20 if (WARN_ON_ONCE(!is_remote_partition(cs))) return; @@ -1649,10 +1688,11 @@ static void remote_cpus_update(struct cpuset *cs, s= truct cpumask *newmask, =20 spin_lock_irq(&callback_lock); if (adding) - partition_xcpus_add(prs, NULL, tmp->addmask); + isolcpus_updated +=3D partition_xcpus_add(prs, NULL, tmp->addmask); if (deleting) - partition_xcpus_del(prs, NULL, tmp->delmask); + isolcpus_updated +=3D partition_xcpus_del(prs, NULL, tmp->delmask); spin_unlock_irq(&callback_lock); + update_unbound_workqueue_cpumask(isolcpus_updated); =20 /* * Proprogate changes in top_cpuset's effective_cpus down the hierarchy. @@ -1774,6 +1814,7 @@ static int update_parent_effective_cpumask(struct cpu= set *cs, int cmd, int part_error =3D PERR_NONE; /* Partition error? */ int subparts_delta =3D 0; struct cpumask *xcpus; /* cs effective_xcpus */ + int isolcpus_updated =3D 0; bool nocpu; =20 lockdep_assert_held(&cpuset_mutex); @@ -2010,15 +2051,18 @@ static int update_parent_effective_cpumask(struct c= puset *cs, int cmd, * and vice versa. */ if (adding) - partition_xcpus_del(old_prs, parent, tmp->addmask); + isolcpus_updated +=3D partition_xcpus_del(old_prs, parent, + tmp->addmask); if (deleting) - partition_xcpus_add(new_prs, parent, tmp->delmask); + isolcpus_updated +=3D partition_xcpus_add(new_prs, parent, + tmp->delmask); =20 if (is_partition_valid(parent)) { parent->nr_subparts +=3D subparts_delta; WARN_ON_ONCE(parent->nr_subparts < 0); } spin_unlock_irq(&callback_lock); + update_unbound_workqueue_cpumask(isolcpus_updated); =20 if ((old_prs !=3D new_prs) && (cmd =3D=3D partcmd_update)) update_partition_exclusive(cs, new_prs); @@ -3082,6 +3126,7 @@ static int update_prstate(struct cpuset *cs, int new_= prs) else if (new_xcpus_state) partition_xcpus_newstate(old_prs, new_prs, cs->effective_xcpus); spin_unlock_irq(&callback_lock); + update_unbound_workqueue_cpumask(new_xcpus_state); =20 /* Force update if switching back to member */ update_cpumasks_hier(cs, &tmpmask, !new_prs ? HIER_CHECKALL : 0); diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/test= ing/selftests/cgroup/test_cpuset_prs.sh index 2b825019f806..f117f704e675 100755 --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh @@ -232,11 +232,11 @@ TEST_MATRIX=3D( " C0-3:S+ C1-3:S+ C2-3 C4-5 X2-3 X2-3:P1 P2 P1 0 A1:0-1,A2:= ,A3:2-3,B1:4-5 \ A1:P0,A2:P1,A3:P2,B1:P1 2-3" " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3:P1 P2 P1 0 A1:0-1,A2:= ,A3:2-3,B1:4 \ - A1:P0,A2:P1,A3:P2,B1:P1 2-4" + A1:P0,A2:P1,A3:P2,B1:P1 2-4,2-3" " C0-3:S+ C1-3:S+ C3 C4 X2-3 X2-3:P1 P2 P1 0 A1:0-1,A2:= 2,A3:3,B1:4 \ - A1:P0,A2:P1,A3:P2,B1:P1 2-4" + A1:P0,A2:P1,A3:P2,B1:P1 2-4,3" " C0-4:S+ C1-4:S+ C2-4 . X2-4 X2-4:P2 X4:P1 . 0 A1:0-1,A2:= 2-3,A3:4 \ - A1:P0,A2:P2,A3:P1 2-4" + A1:P0,A2:P2,A3:P1 2-4,2-3" " C0-4:X2-4:S+ C1-4:X2-4:S+:P2 C2-4:X4:P1 \ . . X5 . . 0 A1:0-4,A2:1-4,A3:2-4 \ A1:P0,A2:P-2,A3:P-1" @@ -248,7 +248,7 @@ TEST_MATRIX=3D( " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2:O2=3D0 . 0 A1:0-1,A= 2:1,A3:3 A1:P0,A3:P2 2-3" " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2:O2=3D0 O2=3D1 0 A1:0-= 1,A2:1,A3:2-3 A1:P0,A3:P2 2-3" " C0-3:S+ C1-3:S+ C3 . X2-3 X2-3 P2:O3=3D0 . 0 A1:0-2,A= 2:1-2,A3: A1:P0,A3:P2 3" - " C0-3:S+ C1-3:S+ C3 . X2-3 X2-3 T:P2:O3=3D0 . 0 A1:0-2,A= 2:1-2,A3:1-2 A1:P0,A3:P-2 3" + " C0-3:S+ C1-3:S+ C3 . X2-3 X2-3 T:P2:O3=3D0 . 0 A1:0-2,A= 2:1-2,A3:1-2 A1:P0,A3:P-2 3," =20 # An invalidated remote partition cannot self-recover from hotplug " C0-3:S+ C1-3:S+ C2 . X2-3 X2-3 T:P2:O2=3D0 O2=3D1 0 A1:0-3= ,A2:1-3,A3:2 A1:P0,A3:P-2" @@ -508,12 +508,14 @@ dump_states() XECPUS=3D$DIR/cpuset.cpus.exclusive.effective PRS=3D$DIR/cpuset.cpus.partition PCPUS=3D$DIR/.__DEBUG__.cpuset.cpus.subpartitions + ISCPUS=3D$DIR/.__DEBUG__.cpuset.cpus.isolated [[ -e $CPUS ]] && echo "$CPUS: $(cat $CPUS)" [[ -e $XCPUS ]] && echo "$XCPUS: $(cat $XCPUS)" [[ -e $ECPUS ]] && echo "$ECPUS: $(cat $ECPUS)" [[ -e $XECPUS ]] && echo "$XECPUS: $(cat $XECPUS)" [[ -e $PRS ]] && echo "$PRS: $(cat $PRS)" [[ -e $PCPUS ]] && echo "$PCPUS: $(cat $PCPUS)" + [[ -e $ISCPUS ]] && echo "$ISCPUS: $(cat $ISCPUS)" done } =20 @@ -591,11 +593,17 @@ check_cgroup_states() =20 # # Get isolated (including offline) CPUs by looking at -# /sys/kernel/debug/sched/domains and compare that with the expected value. +# /sys/kernel/debug/sched/domains and *cpuset.cpus.isolated control file, +# if available, and compare that with the expected value. # -# Note that a sched domain of just 1 CPU will be considered isolated. +# Note that isolated CPUs from the sched/domains context include offline +# CPUs as well as CPUs in non-isolated 1-CPU partition. Those CPUs may +# not be included in the *cpuset.cpus.isolated control file which contains +# only CPUs in isolated partitions. # -# $1 - expected isolated cpu list +# $1 - expected isolated cpu list(s) {,} +# - expected sched/domains value +# - *cpuset.cpus.isolated value =3D if not defined # check_isolcpus() { @@ -603,8 +611,33 @@ check_isolcpus() ISOLCPUS=3D LASTISOLCPU=3D SCHED_DOMAINS=3D/sys/kernel/debug/sched/domains + ISCPUS=3D${CGROUP2}/.__DEBUG__.cpuset.cpus.isolated + if [[ $EXPECT_VAL =3D . ]] + then + EXPECT_VAL=3D + EXPECT_VAL2=3D + elif [[ $(expr $EXPECT_VAL : ".*,.*") > 0 ]] + then + set -- $(echo $EXPECT_VAL | sed -e "s/,/ /g") + EXPECT_VAL=3D$1 + EXPECT_VAL2=3D$2 + else + EXPECT_VAL2=3D$EXPECT_VAL + fi + + # + # Check the debug isolated cpumask, if present + # + [[ -f $ISCPUS ]] && { + ISOLCPUS=3D$(cat $ISCPUS) + [[ "$EXPECT_VAL2" !=3D "$ISOLCPUS" ]] && return 1 + ISOLCPUS=3D + } + + # + # Use the sched domain in debugfs to check isolated CPUs, if available + # [[ -d $SCHED_DOMAINS ]] || return 0 - [[ $EXPECT_VAL =3D . ]] && EXPECT_VAL=3D =20 for ((CPU=3D0; CPU < $NR_CPUS; CPU++)) do @@ -648,6 +681,22 @@ test_fail() exit 1 } =20 +# +# Check to see if there are unexpected isolated CPUs left +# +null_isolcpus_check() +{ + [[ $VERBOSE -gt 0 ]] || return 0 + pause 0.01 + check_isolcpus "." + if [[ $? -ne 0 ]] + then + echo "Unexpected isolated CPUs: $ISOLCPUS" + dump_states + exit 1 + fi +} + # # Run cpuset state transition test # $1 - test matrix name @@ -733,6 +782,7 @@ run_state_test() echo "Effective cpus changed to $NEWLIST after test $I!" exit 1 } + null_isolcpus_check [[ $VERBOSE -gt 0 ]] && echo "Test $I done." ((I++)) done @@ -802,6 +852,7 @@ test_isolated() console_msg "Cleaning up" echo $$ > $CGROUP2/cgroup.procs [[ -d A1 ]] && rmdir A1 + null_isolcpus_check } =20 # --=20 2.39.3