From nobody Wed Jul 1 01:53:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90676C433EF for ; Tue, 4 Jan 2022 14:50:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234415AbiADOt7 (ORCPT ); Tue, 4 Jan 2022 09:49:59 -0500 Received: from dfw.source.kernel.org ([139.178.84.217]:41676 "EHLO dfw.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234396AbiADOtz (ORCPT ); Tue, 4 Jan 2022 09:49:55 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2E606612E7 for ; Tue, 4 Jan 2022 14:49:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8A340C36AF4; Tue, 4 Jan 2022 14:49:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641307794; bh=cJkpsMsk8EI3sJ8PqluUGLW+fhH0tNSXs7iR4dXdCtA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ch/lBCHLSyq8gCEdNHCkdKkmiQk6No64C4YwAuEk/f+DdhQ+RSUZnIB5Sh8nJs5qm wNkzNghyO3YRgoTA9o8MDWCsnDB++oW4Iab2HxsacZN31CLahucsOfV8RiadgFFYhb mjCtSIxU/risED/X4UWqa+wnabGuZWYZrZyPhYe0cohAkTdNuTEX5OMxpcsFvQduvn npSl2Lvh62t6QdmJkUPpIVtilgPmDjtzr6sJbMOnlxd5x8gUcU3ohRO2tgHZf4/AAs ubMpsdq0eke/1UWCF8KX0KrGcwfnlkJKW9hIMvaFCAN4r1ed3rEui8PQY+nE1WeQua Kwk9SaBuVxuPw== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , Tejun Heo , Juri Lelli , Alex Belits , Nitesh Lal , Thomas Gleixner , Nicolas Saenz , Christoph Lameter , Marcelo Tosatti , Zefan Li , Paul Gortmaker , "Paul E . McKenney" Subject: [PATCH 1/8] pci: Decouple HK_FLAG_WQ and HK_FLAG_DOMAIN cpumask fetch Date: Tue, 4 Jan 2022 15:49:37 +0100 Message-Id: <20220104144944.1278663-2-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220104144944.1278663-1-frederic@kernel.org> References: <20220104144944.1278663-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To prepare for supporting each feature of the housekeeping cpumask toward cpuset, prepare each of the HK_FLAG_* entries to move to their own cpumask with enforcing to fetch them individually. The new constraint is that multiple HK_FLAG_* entries can't be mixed together anymore in a single call to housekeeping cpumask(). This will later allow, for example, to runtime modify the cpulist passed through "isolcpus=3D", "nohz_full=3D" and "rcu_nocbs=3D" kernel boot parameters. Signed-off-by: Frederic Weisbecker Cc: Thomas Gleixner Cc: Juri Lelli Cc: Marcelo Tosatti Cc: Nitesh Lal Cc: Nicolas Saenz Cc: Peter Zijlstra Cc: Christoph Lameter Cc: Tejun Heo Cc: Zefan Li Cc: Alex Belits Reviewed-by: Marcelo Tosatti Reviewed-by: Phil Auld --- drivers/pci/pci-driver.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 588588cfda48..4a5792c82d08 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -350,7 +350,6 @@ static int pci_call_probe(struct pci_driver *drv, struc= t pci_dev *dev, const struct pci_device_id *id) { int error, node, cpu; - int hk_flags =3D HK_FLAG_DOMAIN | HK_FLAG_WQ; struct drv_dev_and_id ddi =3D { drv, dev, id }; =20 /* @@ -368,17 +367,29 @@ static int pci_call_probe(struct pci_driver *drv, str= uct pci_dev *dev, * device is probed from work_on_cpu() of the Physical device. */ if (node < 0 || node >=3D MAX_NUMNODES || !node_online(node) || - pci_physfn_is_probed(dev)) + pci_physfn_is_probed(dev)) { cpu =3D nr_cpu_ids; - else + } else { + cpumask_var_t wq_domain_mask; + + if (!zalloc_cpumask_var(&wq_domain_mask, GFP_KERNEL)) { + error =3D -ENOMEM; + goto out; + } + cpumask_and(wq_domain_mask, + housekeeping_cpumask(HK_FLAG_WQ), + housekeeping_cpumask(HK_FLAG_DOMAIN)); + cpu =3D cpumask_any_and(cpumask_of_node(node), - housekeeping_cpumask(hk_flags)); + wq_domain_mask); + free_cpumask_var(wq_domain_mask); + } =20 if (cpu < nr_cpu_ids) error =3D work_on_cpu(cpu, local_pci_probe, &ddi); else error =3D local_pci_probe(&ddi); - +out: dev->is_probed =3D 0; cpu_hotplug_enable(); return error; --=20 2.25.1 From nobody Wed Jul 1 01:53:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C708DC433F5 for ; Tue, 4 Jan 2022 14:50:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234420AbiADOuH (ORCPT ); Tue, 4 Jan 2022 09:50:07 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:35086 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234396AbiADOuA (ORCPT ); Tue, 4 Jan 2022 09:50:00 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6F307B81125 for ; Tue, 4 Jan 2022 14:49:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1FD80C36AF3; Tue, 4 Jan 2022 14:49:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641307798; bh=WrWKpVgJLQAdQeNhZs1tCFJTul6BO40+uMOAm3lLUB4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cNNurJN/fidbldLKUBiTyVYObezMR2edKh58mg4Zz4I0cVqVPbebvnMnNI5YQc3Lo 87Gg9SqQ4zdsrs0k87YvIj0IhC/YK6kBSm+mA/epzMcPtAX2IIVwffA1W0AL2HMXPQ iTcZYEC2LXZcY4ZmgJedi+sdpk+FzKPT5wrLXUq+uX0NX3R9DHiCFRhdYPnk0fehNq SuCjgkKv7qQLJhj2/oV4RZyi6xnmcCRfFgqGiGOWnBLhBnWYpUxUNY60c8DoErl+h0 WvbTQCNoy/LfZ4cePzlhnQnYvKL1z2D0wFBcf5zQlHwtkSIQ9A0rih/eNfsoOpRW6x yM2AOTij/dpFw== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , Tejun Heo , Juri Lelli , Alex Belits , Nitesh Lal , Thomas Gleixner , Nicolas Saenz , Christoph Lameter , Marcelo Tosatti , Zefan Li , Paul Gortmaker , "Paul E . McKenney" Subject: [PATCH 2/8] workqueue: Decouple HK_FLAG_WQ and HK_FLAG_DOMAIN cpumask fetch Date: Tue, 4 Jan 2022 15:49:38 +0100 Message-Id: <20220104144944.1278663-3-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220104144944.1278663-1-frederic@kernel.org> References: <20220104144944.1278663-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To prepare for supporting each feature of the housekeeping cpumask toward cpuset, prepare each of the HK_FLAG_* entries to move to their own cpumask with enforcing to fetch them individually. The new constraint is that multiple HK_FLAG_* entries can't be mixed together anymore in a single call to housekeeping cpumask(). This will later allow, for example, to runtime modify the cpulist passed through "isolcpus=3D", "nohz_full=3D" and "rcu_nocbs=3D" kernel boot parameters. Signed-off-by: Frederic Weisbecker Cc: Thomas Gleixner Cc: Juri Lelli Cc: Marcelo Tosatti Cc: Nitesh Lal Cc: Nicolas Saenz Cc: Peter Zijlstra Cc: Christoph Lameter Cc: Tejun Heo Cc: Zefan Li Cc: Alex Belits Reviewed-by: Marcelo Tosatti Reviewed-by: Phil Auld --- kernel/workqueue.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 613917bbc4e7..dbd9ad4873fc 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -6017,13 +6017,13 @@ static void __init wq_numa_init(void) void __init workqueue_init_early(void) { int std_nice[NR_STD_WORKER_POOLS] =3D { 0, HIGHPRI_NICE_LEVEL }; - int hk_flags =3D HK_FLAG_DOMAIN | HK_FLAG_WQ; int i, cpu; =20 BUILD_BUG_ON(__alignof__(struct pool_workqueue) < __alignof__(long long)); =20 BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL)); - cpumask_copy(wq_unbound_cpumask, housekeeping_cpumask(hk_flags)); + cpumask_copy(wq_unbound_cpumask, housekeeping_cpumask(HK_FLAG_WQ)); + cpumask_and(wq_unbound_cpumask, wq_unbound_cpumask, housekeeping_cpumask(= HK_FLAG_DOMAIN)); =20 pwq_cache =3D KMEM_CACHE(pool_workqueue, SLAB_PANIC); =20 --=20 2.25.1 From nobody Wed Jul 1 01:53:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74113C433EF for ; Tue, 4 Jan 2022 14:50:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234436AbiADOuJ (ORCPT ); Tue, 4 Jan 2022 09:50:09 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:35142 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234435AbiADOuE (ORCPT ); Tue, 4 Jan 2022 09:50:04 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 15532B81646 for ; Tue, 4 Jan 2022 14:50:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7925C36AE9; Tue, 4 Jan 2022 14:49:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641307801; bh=5Wg5M9I3hKWSbc7MWsUiph0hQbWiXIBYvfHiAxqg/tg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EmucEgPMFjGrZvjt8jSq9mirg0iMoD6i6fSiSIPVlmI2yUcKM1XADHOZxv9ikKkr/ S/GC9o0Vd0ESzb+k2c3h4X/fceu65IWNNVZzeBfLh5VJQfEQtgHxiPuvbGc6dhd+SV FVRwx71UvAFXUrBa05SUDo0PgSEidsmKFnFGvxVcseUYiBHFfQFw6LixQr6u/XHC3E 875vaJ6qhDePDtba6f+tOH04Z4oYpkB3ttTv4ECutJgUEnt0mQBw9psVnPRq+nlQTP OmvG/t0+8PpWGh1iQSuPhYd/ZI27JVW5VWw0q+p/+d3D45dDc0pO1HR6Z+cXluctZW UurQTJpWAblww== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , Tejun Heo , Juri Lelli , Alex Belits , Nitesh Lal , Thomas Gleixner , Nicolas Saenz , Christoph Lameter , Marcelo Tosatti , Zefan Li , Paul Gortmaker , "Paul E . McKenney" Subject: [PATCH 3/8] net: Decouple HK_FLAG_WQ and HK_FLAG_DOMAIN cpumask fetch Date: Tue, 4 Jan 2022 15:49:39 +0100 Message-Id: <20220104144944.1278663-4-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220104144944.1278663-1-frederic@kernel.org> References: <20220104144944.1278663-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To prepare for supporting each feature of the housekeeping cpumask toward cpuset, prepare each of the HK_FLAG_* entries to move to their own cpumask with enforcing to fetch them individually. The new constraint is that multiple HK_FLAG_* entries can't be mixed together anymore in a single call to housekeeping cpumask(). This will later allow, for example, to runtime modify the cpulist passed through "isolcpus=3D", "nohz_full=3D" and "rcu_nocbs=3D" kernel boot parameters. Signed-off-by: Frederic Weisbecker Cc: Thomas Gleixner Cc: Juri Lelli Cc: Marcelo Tosatti Cc: Nitesh Lal Cc: Nicolas Saenz Cc: Peter Zijlstra Cc: Christoph Lameter Cc: Tejun Heo Cc: Zefan Li Cc: Alex Belits Reviewed-by: Marcelo Tosatti Reviewed-by: Phil Auld --- net/core/net-sysfs.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 9c01c642cf9e..57181577f273 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -831,7 +831,7 @@ static ssize_t store_rps_map(struct netdev_rx_queue *qu= eue, { struct rps_map *old_map, *map; cpumask_var_t mask; - int err, cpu, i, hk_flags; + int err, cpu, i; static DEFINE_MUTEX(rps_map_mutex); =20 if (!capable(CAP_NET_ADMIN)) @@ -847,8 +847,8 @@ static ssize_t store_rps_map(struct netdev_rx_queue *qu= eue, } =20 if (!cpumask_empty(mask)) { - hk_flags =3D HK_FLAG_DOMAIN | HK_FLAG_WQ; - cpumask_and(mask, mask, housekeeping_cpumask(hk_flags)); + cpumask_and(mask, mask, housekeeping_cpumask(HK_FLAG_DOMAIN)); + cpumask_and(mask, mask, housekeeping_cpumask(HK_FLAG_WQ)); if (cpumask_empty(mask)) { free_cpumask_var(mask); return -EINVAL; --=20 2.25.1 From nobody Wed Jul 1 01:53:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEF47C433F5 for ; Tue, 4 Jan 2022 14:50:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234427AbiADOuL (ORCPT ); Tue, 4 Jan 2022 09:50:11 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:35184 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234422AbiADOuI (ORCPT ); Tue, 4 Jan 2022 09:50:08 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id E0C4DB81642 for ; Tue, 4 Jan 2022 14:50:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4BD0AC36AEF; Tue, 4 Jan 2022 14:50:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641307805; bh=184CwW5a9P17Pvlbko0si4bWmB5A4LExtqPbQxMwFGU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ehr/YOESQnq/a6rjrdNp4GQ9gogbWBA7JgFG6z+wVXCuCVYn11mFgMfgDddSNpW41 eahH+dGvLjbV86GaNAnwz3yOlxfNZUHKMLmsGBGig9cs4Zazppb64e6PorGmW8ZGaV XGfQk0Soj8LPp+OTt+xbiDAJzVeqoaH/6P3GN+wGfsS7yl+Wh1R7HfIoetnGBEl6WY zBSsyGn3RRQ/SPW/ijwrg/24jx/YAsd8Qoid8v7+mQUgcZpmJTGWsw/4qhGZgfwzPD 1lvqM5gjrTJQaNou5PLFWJCs5ZSJl9Xm2YtD2yeBDQCAIBZ3YKkaytMuPYAdmFykNL 56h/3BQgpHDvA== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , Tejun Heo , Juri Lelli , Alex Belits , Nitesh Lal , Thomas Gleixner , Nicolas Saenz , Christoph Lameter , Marcelo Tosatti , Zefan Li , Paul Gortmaker , "Paul E . McKenney" Subject: [PATCH 4/8] sched/isolation: Use single feature type while referring to housekeeping cpumask Date: Tue, 4 Jan 2022 15:49:40 +0100 Message-Id: <20220104144944.1278663-5-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220104144944.1278663-1-frederic@kernel.org> References: <20220104144944.1278663-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Refer to housekeeping APIs using single feature types instead of flags. This prevents from passing multiple isolation features at once to housekeeping interfaces, which soon won't be possible anymore as each isolation features will have their own cpumask. Signed-off-by: Frederic Weisbecker Cc: Thomas Gleixner Cc: Juri Lelli Cc: Marcelo Tosatti Cc: Nitesh Lal Cc: Nicolas Saenz Cc: Peter Zijlstra Cc: Christoph Lameter Cc: Tejun Heo Cc: Zefan Li Cc: Alex Belits Reviewed-by: Marcelo Tosatti Reviewed-by: Phil Auld --- arch/x86/kernel/cpu/aperfmperf.c | 6 ++--- arch/x86/kvm/x86.c | 2 +- drivers/base/cpu.c | 2 +- drivers/pci/pci-driver.c | 4 +-- include/linux/sched/isolation.h | 43 ++++++++++++++++---------------- kernel/cgroup/cpuset.c | 6 ++--- kernel/cpu.c | 4 +-- kernel/irq/cpuhotplug.c | 4 +-- kernel/irq/manage.c | 4 +-- kernel/kthread.c | 4 +-- kernel/rcu/tasks.h | 2 +- kernel/rcu/tree_plugin.h | 2 +- kernel/sched/core.c | 12 ++++----- kernel/sched/fair.c | 10 ++++---- kernel/sched/isolation.c | 32 ++++++++++++++++-------- kernel/sched/topology.c | 8 +++--- kernel/watchdog.c | 2 +- kernel/workqueue.c | 4 +-- net/core/net-sysfs.c | 4 +-- 19 files changed, 84 insertions(+), 71 deletions(-) diff --git a/arch/x86/kernel/cpu/aperfmperf.c b/arch/x86/kernel/cpu/aperfmp= erf.c index 22911deacb6e..9ca008f9e9b1 100644 --- a/arch/x86/kernel/cpu/aperfmperf.c +++ b/arch/x86/kernel/cpu/aperfmperf.c @@ -91,7 +91,7 @@ unsigned int aperfmperf_get_khz(int cpu) if (!boot_cpu_has(X86_FEATURE_APERFMPERF)) return 0; =20 - if (!housekeeping_cpu(cpu, HK_FLAG_MISC)) + if (!housekeeping_cpu(cpu, HK_TYPE_MISC)) return 0; =20 if (rcu_is_idle_cpu(cpu)) @@ -114,7 +114,7 @@ void arch_freq_prepare_all(void) return; =20 for_each_online_cpu(cpu) { - if (!housekeeping_cpu(cpu, HK_FLAG_MISC)) + if (!housekeeping_cpu(cpu, HK_TYPE_MISC)) continue; if (rcu_is_idle_cpu(cpu)) continue; /* Idle CPUs are completely uninteresting. */ @@ -136,7 +136,7 @@ unsigned int arch_freq_get_on_cpu(int cpu) if (!boot_cpu_has(X86_FEATURE_APERFMPERF)) return 0; =20 - if (!housekeeping_cpu(cpu, HK_FLAG_MISC)) + if (!housekeeping_cpu(cpu, HK_TYPE_MISC)) return 0; =20 if (aperfmperf_snapshot_cpu(cpu, ktime_get(), true)) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e50e97ac4408..e6c90087c700 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8684,7 +8684,7 @@ int kvm_arch_init(void *opaque) } =20 if (pi_inject_timer =3D=3D -1) - pi_inject_timer =3D housekeeping_enabled(HK_FLAG_TIMER); + pi_inject_timer =3D housekeeping_enabled(HK_TYPE_TIMER); #ifdef CONFIG_X86_64 pvclock_gtod_register_notifier(&pvclock_gtod_notifier); =20 diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c index 5fc258073bc7..2ef23fce0860 100644 --- a/drivers/base/cpu.c +++ b/drivers/base/cpu.c @@ -275,7 +275,7 @@ static ssize_t print_cpus_isolated(struct device *dev, return -ENOMEM; =20 cpumask_andnot(isolated, cpu_possible_mask, - housekeeping_cpumask(HK_FLAG_DOMAIN)); + housekeeping_cpumask(HK_TYPE_DOMAIN)); len =3D sysfs_emit(buf, "%*pbl\n", cpumask_pr_args(isolated)); =20 free_cpumask_var(isolated); diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 4a5792c82d08..f61c40a47891 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -377,8 +377,8 @@ static int pci_call_probe(struct pci_driver *drv, struc= t pci_dev *dev, goto out; } cpumask_and(wq_domain_mask, - housekeeping_cpumask(HK_FLAG_WQ), - housekeeping_cpumask(HK_FLAG_DOMAIN)); + housekeeping_cpumask(HK_TYPE_WQ), + housekeeping_cpumask(HK_TYPE_DOMAIN)); =20 cpu =3D cpumask_any_and(cpumask_of_node(node), wq_domain_mask); diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolatio= n.h index cc9f393e2a70..8c15abd67aed 100644 --- a/include/linux/sched/isolation.h +++ b/include/linux/sched/isolation.h @@ -5,54 +5,55 @@ #include #include =20 -enum hk_flags { - HK_FLAG_TIMER =3D 1, - HK_FLAG_RCU =3D (1 << 1), - HK_FLAG_MISC =3D (1 << 2), - HK_FLAG_SCHED =3D (1 << 3), - HK_FLAG_TICK =3D (1 << 4), - HK_FLAG_DOMAIN =3D (1 << 5), - HK_FLAG_WQ =3D (1 << 6), - HK_FLAG_MANAGED_IRQ =3D (1 << 7), - HK_FLAG_KTHREAD =3D (1 << 8), +enum hk_type { + HK_TYPE_TIMER, + HK_TYPE_RCU, + HK_TYPE_MISC, + HK_TYPE_SCHED, + HK_TYPE_TICK, + HK_TYPE_DOMAIN, + HK_TYPE_WQ, + HK_TYPE_MANAGED_IRQ, + HK_TYPE_KTHREAD, + HK_TYPE_MAX }; =20 #ifdef CONFIG_CPU_ISOLATION DECLARE_STATIC_KEY_FALSE(housekeeping_overridden); -extern int housekeeping_any_cpu(enum hk_flags flags); -extern const struct cpumask *housekeeping_cpumask(enum hk_flags flags); -extern bool housekeeping_enabled(enum hk_flags flags); -extern void housekeeping_affine(struct task_struct *t, enum hk_flags flags= ); -extern bool housekeeping_test_cpu(int cpu, enum hk_flags flags); +extern int housekeeping_any_cpu(enum hk_type type); +extern const struct cpumask *housekeeping_cpumask(enum hk_type type); +extern bool housekeeping_enabled(enum hk_type type); +extern void housekeeping_affine(struct task_struct *t, enum hk_type type); +extern bool housekeeping_test_cpu(int cpu, enum hk_type type); extern void __init housekeeping_init(void); =20 #else =20 -static inline int housekeeping_any_cpu(enum hk_flags flags) +static inline int housekeeping_any_cpu(enum hk_type type) { return smp_processor_id(); } =20 -static inline const struct cpumask *housekeeping_cpumask(enum hk_flags fla= gs) +static inline const struct cpumask *housekeeping_cpumask(enum hk_type type) { return cpu_possible_mask; } =20 -static inline bool housekeeping_enabled(enum hk_flags flags) +static inline bool housekeeping_enabled(enum hk_type type) { return false; } =20 static inline void housekeeping_affine(struct task_struct *t, - enum hk_flags flags) { } + enum hk_type type) { } static inline void housekeeping_init(void) { } #endif /* CONFIG_CPU_ISOLATION */ =20 -static inline bool housekeeping_cpu(int cpu, enum hk_flags flags) +static inline bool housekeeping_cpu(int cpu, enum hk_type type) { #ifdef CONFIG_CPU_ISOLATION if (static_branch_unlikely(&housekeeping_overridden)) - return housekeeping_test_cpu(cpu, flags); + return housekeeping_test_cpu(cpu, type); #endif return true; } diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index d0e163a02099..d8c20bf2ff38 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -811,7 +811,7 @@ static int generate_sched_domains(cpumask_var_t **domai= ns, update_domain_attr_tree(dattr, &top_cpuset); } cpumask_and(doms[0], top_cpuset.effective_cpus, - housekeeping_cpumask(HK_FLAG_DOMAIN)); + housekeeping_cpumask(HK_TYPE_DOMAIN)); =20 goto done; } @@ -841,7 +841,7 @@ static int generate_sched_domains(cpumask_var_t **domai= ns, if (!cpumask_empty(cp->cpus_allowed) && !(is_sched_load_balance(cp) && cpumask_intersects(cp->cpus_allowed, - housekeeping_cpumask(HK_FLAG_DOMAIN)))) + housekeeping_cpumask(HK_TYPE_DOMAIN)))) continue; =20 if (root_load_balance && @@ -930,7 +930,7 @@ static int generate_sched_domains(cpumask_var_t **domai= ns, =20 if (apn =3D=3D b->pn) { cpumask_or(dp, dp, b->effective_cpus); - cpumask_and(dp, dp, housekeeping_cpumask(HK_FLAG_DOMAIN)); + cpumask_and(dp, dp, housekeeping_cpumask(HK_TYPE_DOMAIN)); if (dattr) update_domain_attr_tree(dattr + nslot, b); =20 diff --git a/kernel/cpu.c b/kernel/cpu.c index 407a2568f35e..f39eb0b52dfe 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1488,8 +1488,8 @@ int freeze_secondary_cpus(int primary) cpu_maps_update_begin(); if (primary =3D=3D -1) { primary =3D cpumask_first(cpu_online_mask); - if (!housekeeping_cpu(primary, HK_FLAG_TIMER)) - primary =3D housekeeping_any_cpu(HK_FLAG_TIMER); + if (!housekeeping_cpu(primary, HK_TYPE_TIMER)) + primary =3D housekeeping_any_cpu(HK_TYPE_TIMER); } else { if (!cpu_online(primary)) primary =3D cpumask_first(cpu_online_mask); diff --git a/kernel/irq/cpuhotplug.c b/kernel/irq/cpuhotplug.c index 39a41c56ad4f..1ed2b1739363 100644 --- a/kernel/irq/cpuhotplug.c +++ b/kernel/irq/cpuhotplug.c @@ -176,10 +176,10 @@ static bool hk_should_isolate(struct irq_data *data, = unsigned int cpu) { const struct cpumask *hk_mask; =20 - if (!housekeeping_enabled(HK_FLAG_MANAGED_IRQ)) + if (!housekeeping_enabled(HK_TYPE_MANAGED_IRQ)) return false; =20 - hk_mask =3D housekeeping_cpumask(HK_FLAG_MANAGED_IRQ); + hk_mask =3D housekeeping_cpumask(HK_TYPE_MANAGED_IRQ); if (cpumask_subset(irq_data_get_effective_affinity_mask(data), hk_mask)) return false; =20 diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index 7405e384e5ed..f5bdcbf70dda 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -247,13 +247,13 @@ int irq_do_set_affinity(struct irq_data *data, const = struct cpumask *mask, * online. */ if (irqd_affinity_is_managed(data) && - housekeeping_enabled(HK_FLAG_MANAGED_IRQ)) { + housekeeping_enabled(HK_TYPE_MANAGED_IRQ)) { const struct cpumask *hk_mask, *prog_mask; =20 static DEFINE_RAW_SPINLOCK(tmp_mask_lock); static struct cpumask tmp_mask; =20 - hk_mask =3D housekeeping_cpumask(HK_FLAG_MANAGED_IRQ); + hk_mask =3D housekeeping_cpumask(HK_TYPE_MANAGED_IRQ); =20 raw_spin_lock(&tmp_mask_lock); cpumask_and(&tmp_mask, mask, hk_mask); diff --git a/kernel/kthread.c b/kernel/kthread.c index 7113003fab63..1b52ba68fdee 100644 --- a/kernel/kthread.c +++ b/kernel/kthread.c @@ -306,7 +306,7 @@ static int kthread(void *_create) * back to default in case they have been changed. */ sched_setscheduler_nocheck(current, SCHED_NORMAL, ¶m); - set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_FLAG_KTHREAD)); + set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_TYPE_KTHREAD)); =20 /* OK, tell user we're spawned, wait for stop or wakeup */ __set_current_state(TASK_UNINTERRUPTIBLE); @@ -661,7 +661,7 @@ int kthreadd(void *unused) /* Setup a clean context for our children to inherit. */ set_task_comm(tsk, "kthreadd"); ignore_signals(tsk); - set_cpus_allowed_ptr(tsk, housekeeping_cpumask(HK_FLAG_KTHREAD)); + set_cpus_allowed_ptr(tsk, housekeeping_cpumask(HK_TYPE_KTHREAD)); set_mems_allowed(node_states[N_MEMORY]); =20 current->flags |=3D PF_NOFREEZE; diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 7da3c81c3f59..4ff54d7632cf 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -187,7 +187,7 @@ static int __noreturn rcu_tasks_kthread(void *arg) struct rcu_tasks *rtp =3D arg; =20 /* Run on housekeeping CPUs by default. Sysadm can move if desired. */ - housekeeping_affine(current, HK_FLAG_RCU); + housekeeping_affine(current, HK_TYPE_RCU); WRITE_ONCE(rtp->kthread_ptr, current); // Let GPs start! =20 /* diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 5199559fbbf0..438956e4e912 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -1475,7 +1475,7 @@ static void rcu_bind_gp_kthread(void) { if (!tick_nohz_full_enabled()) return; - housekeeping_affine(current, HK_FLAG_RCU); + housekeeping_affine(current, HK_TYPE_RCU); } =20 /* Record the current task on dyntick-idle entry. */ diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 77563109c0ea..b62d9c0a936a 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1013,13 +1013,13 @@ int get_nohz_timer_target(void) struct sched_domain *sd; const struct cpumask *hk_mask; =20 - if (housekeeping_cpu(cpu, HK_FLAG_TIMER)) { + if (housekeeping_cpu(cpu, HK_TYPE_TIMER)) { if (!idle_cpu(cpu)) return cpu; default_cpu =3D cpu; } =20 - hk_mask =3D housekeeping_cpumask(HK_FLAG_TIMER); + hk_mask =3D housekeeping_cpumask(HK_TYPE_TIMER); =20 rcu_read_lock(); for_each_domain(cpu, sd) { @@ -1035,7 +1035,7 @@ int get_nohz_timer_target(void) } =20 if (default_cpu =3D=3D -1) - default_cpu =3D housekeeping_any_cpu(HK_FLAG_TIMER); + default_cpu =3D housekeeping_any_cpu(HK_TYPE_TIMER); cpu =3D default_cpu; unlock: rcu_read_unlock(); @@ -5355,7 +5355,7 @@ static void sched_tick_start(int cpu) int os; struct tick_work *twork; =20 - if (housekeeping_cpu(cpu, HK_FLAG_TICK)) + if (housekeeping_cpu(cpu, HK_TYPE_TICK)) return; =20 WARN_ON_ONCE(!tick_work_cpu); @@ -5376,7 +5376,7 @@ static void sched_tick_stop(int cpu) struct tick_work *twork; int os; =20 - if (housekeeping_cpu(cpu, HK_FLAG_TICK)) + if (housekeeping_cpu(cpu, HK_TYPE_TICK)) return; =20 WARN_ON_ONCE(!tick_work_cpu); @@ -9212,7 +9212,7 @@ void __init sched_init_smp(void) mutex_unlock(&sched_domains_mutex); =20 /* Move init over to a non-isolated CPU */ - if (set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_FLAG_DOMAIN)) <= 0) + if (set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_TYPE_DOMAIN)) <= 0) BUG(); current->flags &=3D ~PF_NO_SETAFFINITY; sched_init_granularity(); diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6e476f6d9435..7be90d2b0f58 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10362,7 +10362,7 @@ static inline int on_null_domain(struct rq *rq) * - When one of the busy CPUs notice that there may be an idle rebalancing * needed, they will kick the idle load balancer, which then does idle * load balancing for all the idle CPUs. - * - HK_FLAG_MISC CPUs are used for this task, because HK_FLAG_SCHED not s= et + * - HK_TYPE_MISC CPUs are used for this task, because HK_TYPE_SCHED not s= et * anywhere yet. */ =20 @@ -10371,7 +10371,7 @@ static inline int find_new_ilb(void) int ilb; const struct cpumask *hk_mask; =20 - hk_mask =3D housekeeping_cpumask(HK_FLAG_MISC); + hk_mask =3D housekeeping_cpumask(HK_TYPE_MISC); =20 for_each_cpu_and(ilb, nohz.idle_cpus_mask, hk_mask) { =20 @@ -10387,7 +10387,7 @@ static inline int find_new_ilb(void) =20 /* * Kick a CPU to do the nohz balancing, if it is time for it. We pick any - * idle CPU in the HK_FLAG_MISC housekeeping set (if there is one). + * idle CPU in the HK_TYPE_MISC housekeeping set (if there is one). */ static void kick_ilb(unsigned int flags) { @@ -10600,7 +10600,7 @@ void nohz_balance_enter_idle(int cpu) return; =20 /* Spare idle load balancing on CPUs that don't want to be disturbed: */ - if (!housekeeping_cpu(cpu, HK_FLAG_SCHED)) + if (!housekeeping_cpu(cpu, HK_TYPE_SCHED)) return; =20 /* @@ -10816,7 +10816,7 @@ static void nohz_newidle_balance(struct rq *this_rq) * This CPU doesn't want to be disturbed by scheduler * housekeeping */ - if (!housekeeping_cpu(this_cpu, HK_FLAG_SCHED)) + if (!housekeeping_cpu(this_cpu, HK_TYPE_SCHED)) return; =20 /* Will wake up very soon. No time for doing anything else*/ diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 7f06eaf12818..a735d9e229dd 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -9,23 +9,35 @@ */ #include "sched.h" =20 +enum hk_flags { + HK_FLAG_TIMER =3D BIT(HK_TYPE_TIMER), + HK_FLAG_RCU =3D BIT(HK_TYPE_RCU), + HK_FLAG_MISC =3D BIT(HK_TYPE_MISC), + HK_FLAG_SCHED =3D BIT(HK_TYPE_SCHED), + HK_FLAG_TICK =3D BIT(HK_TYPE_TICK), + HK_FLAG_DOMAIN =3D BIT(HK_TYPE_DOMAIN), + HK_FLAG_WQ =3D BIT(HK_TYPE_WQ), + HK_FLAG_MANAGED_IRQ =3D BIT(HK_TYPE_MANAGED_IRQ), + HK_FLAG_KTHREAD =3D BIT(HK_TYPE_KTHREAD), +}; + DEFINE_STATIC_KEY_FALSE(housekeeping_overridden); EXPORT_SYMBOL_GPL(housekeeping_overridden); static cpumask_var_t housekeeping_mask; static unsigned int housekeeping_flags; =20 -bool housekeeping_enabled(enum hk_flags flags) +bool housekeeping_enabled(enum hk_type type) { - return !!(housekeeping_flags & flags); + return !!(housekeeping_flags & BIT(type)); } EXPORT_SYMBOL_GPL(housekeeping_enabled); =20 -int housekeeping_any_cpu(enum hk_flags flags) +int housekeeping_any_cpu(enum hk_type type) { int cpu; =20 if (static_branch_unlikely(&housekeeping_overridden)) { - if (housekeeping_flags & flags) { + if (housekeeping_flags & BIT(type)) { cpu =3D sched_numa_find_closest(housekeeping_mask, smp_processor_id()); if (cpu < nr_cpu_ids) return cpu; @@ -37,27 +49,27 @@ int housekeeping_any_cpu(enum hk_flags flags) } EXPORT_SYMBOL_GPL(housekeeping_any_cpu); =20 -const struct cpumask *housekeeping_cpumask(enum hk_flags flags) +const struct cpumask *housekeeping_cpumask(enum hk_type type) { if (static_branch_unlikely(&housekeeping_overridden)) - if (housekeeping_flags & flags) + if (housekeeping_flags & BIT(type)) return housekeeping_mask; return cpu_possible_mask; } EXPORT_SYMBOL_GPL(housekeeping_cpumask); =20 -void housekeeping_affine(struct task_struct *t, enum hk_flags flags) +void housekeeping_affine(struct task_struct *t, enum hk_type type) { if (static_branch_unlikely(&housekeeping_overridden)) - if (housekeeping_flags & flags) + if (housekeeping_flags & BIT(type)) set_cpus_allowed_ptr(t, housekeeping_mask); } EXPORT_SYMBOL_GPL(housekeeping_affine); =20 -bool housekeeping_test_cpu(int cpu, enum hk_flags flags) +bool housekeeping_test_cpu(int cpu, enum hk_type type) { if (static_branch_unlikely(&housekeeping_overridden)) - if (housekeeping_flags & flags) + if (housekeeping_flags & BIT(type)) return cpumask_test_cpu(cpu, housekeeping_mask); return true; } diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index d201a7052a29..5318940b0b76 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1366,7 +1366,7 @@ static void asym_cpu_capacity_scan(void) list_for_each_entry(entry, &asym_cap_list, link) cpumask_clear(cpu_capacity_span(entry)); =20 - for_each_cpu_and(cpu, cpu_possible_mask, housekeeping_cpumask(HK_FLAG_DOM= AIN)) + for_each_cpu_and(cpu, cpu_possible_mask, housekeeping_cpumask(HK_TYPE_DOM= AIN)) asym_cpu_capacity_update_data(cpu); =20 list_for_each_entry_safe(entry, next, &asym_cap_list, link) { @@ -2351,7 +2351,7 @@ int sched_init_domains(const struct cpumask *cpu_map) doms_cur =3D alloc_sched_domains(ndoms_cur); if (!doms_cur) doms_cur =3D &fallback_doms; - cpumask_and(doms_cur[0], cpu_map, housekeeping_cpumask(HK_FLAG_DOMAIN)); + cpumask_and(doms_cur[0], cpu_map, housekeeping_cpumask(HK_TYPE_DOMAIN)); err =3D build_sched_domains(doms_cur[0], NULL); =20 return err; @@ -2440,7 +2440,7 @@ void partition_sched_domains_locked(int ndoms_new, cp= umask_var_t doms_new[], if (doms_new) { n =3D 1; cpumask_and(doms_new[0], cpu_active_mask, - housekeeping_cpumask(HK_FLAG_DOMAIN)); + housekeeping_cpumask(HK_TYPE_DOMAIN)); } } else { n =3D ndoms_new; @@ -2475,7 +2475,7 @@ void partition_sched_domains_locked(int ndoms_new, cp= umask_var_t doms_new[], n =3D 0; doms_new =3D &fallback_doms; cpumask_and(doms_new[0], cpu_active_mask, - housekeeping_cpumask(HK_FLAG_DOMAIN)); + housekeeping_cpumask(HK_TYPE_DOMAIN)); } =20 /* Build new domains: */ diff --git a/kernel/watchdog.c b/kernel/watchdog.c index ad912511a0c0..c177926ebba3 100644 --- a/kernel/watchdog.c +++ b/kernel/watchdog.c @@ -748,7 +748,7 @@ void __init lockup_detector_init(void) pr_info("Disabling watchdog on nohz_full cores by default\n"); =20 cpumask_copy(&watchdog_cpumask, - housekeeping_cpumask(HK_FLAG_TIMER)); + housekeeping_cpumask(HK_TYPE_TIMER)); =20 if (!watchdog_nmi_probe()) nmi_watchdog_available =3D true; diff --git a/kernel/workqueue.c b/kernel/workqueue.c index dbd9ad4873fc..d6736d00db1f 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -6022,8 +6022,8 @@ void __init workqueue_init_early(void) BUILD_BUG_ON(__alignof__(struct pool_workqueue) < __alignof__(long long)); =20 BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL)); - cpumask_copy(wq_unbound_cpumask, housekeeping_cpumask(HK_FLAG_WQ)); - cpumask_and(wq_unbound_cpumask, wq_unbound_cpumask, housekeeping_cpumask(= HK_FLAG_DOMAIN)); + cpumask_copy(wq_unbound_cpumask, housekeeping_cpumask(HK_TYPE_WQ)); + cpumask_and(wq_unbound_cpumask, wq_unbound_cpumask, housekeeping_cpumask(= HK_TYPE_DOMAIN)); =20 pwq_cache =3D KMEM_CACHE(pool_workqueue, SLAB_PANIC); =20 diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 57181577f273..6f3d3d0b19a0 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -847,8 +847,8 @@ static ssize_t store_rps_map(struct netdev_rx_queue *qu= eue, } =20 if (!cpumask_empty(mask)) { - cpumask_and(mask, mask, housekeeping_cpumask(HK_FLAG_DOMAIN)); - cpumask_and(mask, mask, housekeeping_cpumask(HK_FLAG_WQ)); + cpumask_and(mask, mask, housekeeping_cpumask(HK_TYPE_DOMAIN)); + cpumask_and(mask, mask, housekeeping_cpumask(HK_TYPE_WQ)); if (cpumask_empty(mask)) { free_cpumask_var(mask); return -EINVAL; --=20 2.25.1 From nobody Wed Jul 1 01:53:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2667C433F5 for ; Tue, 4 Jan 2022 14:50:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233087AbiADOuT (ORCPT ); Tue, 4 Jan 2022 09:50:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234394AbiADOuM (ORCPT ); Tue, 4 Jan 2022 09:50:12 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9CBD3C061785 for ; Tue, 4 Jan 2022 06:50:11 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5CD5CB81644 for ; Tue, 4 Jan 2022 14:50:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 174D5C36AF4; Tue, 4 Jan 2022 14:50:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641307809; bh=OED46LT67AFvjgQn5KGOgsAY64a3OfX3DiIjD6Yfka8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fIicaQQ2jSzEuYvWOBK7GqBS4CsECaWymbEXH4/OSAl0kL9uvlHrUFNH5l90pkBUh yP3cMEjtXRAT5xqMcbmlr562lH6sWn9JGUcgoXVqYjKhUKnqCgNZ58JogQXyUZIiC4 VB+9ygcJjqxHxBGlfkBiIQjExAa29SmlUqmv2AhMb/ogn8lVlKqygmNHygIMEjaK54 Bc/Ef49MQdwK3UZOd7qf0KcFw83vrfwM3hDkiXbtWu4xhAZEYh9wEFZmHkQT4sA8Cu /oeEfkHmCDHzcl04fCGi/X5bGVduIdIUXR+wRvMUSLkDM+8XPC/fxkCsILSQXbpB/8 B6+uQj8DuS7wQ== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , Tejun Heo , Juri Lelli , Alex Belits , Nitesh Lal , Thomas Gleixner , Nicolas Saenz , Christoph Lameter , Marcelo Tosatti , Zefan Li , Paul Gortmaker , "Paul E . McKenney" Subject: [PATCH 5/8] sched/isolation: Consolidate check for housekeeping minimum service Date: Tue, 4 Jan 2022 15:49:41 +0100 Message-Id: <20220104144944.1278663-6-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220104144944.1278663-1-frederic@kernel.org> References: <20220104144944.1278663-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" There can be two subsequent calls to housekeeping_setup() due to "nohz_full=3D" and "isolcpus=3D" that can mix up. The two passes each have their own way to deal with an empty housekeeping set of CPUs. Consolidate this part and remove the awful "tmp" based naming. Signed-off-by: Frederic Weisbecker Cc: Thomas Gleixner Cc: Juri Lelli Cc: Marcelo Tosatti Cc: Nitesh Lal Cc: Nicolas Saenz Cc: Peter Zijlstra Cc: Christoph Lameter Cc: Tejun Heo Cc: Zefan Li Cc: Alex Belits Reviewed-by: Marcelo Tosatti Reviewed-by: Phil Auld --- kernel/sched/isolation.c | 35 +++++++++++++++++------------------ 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index a735d9e229dd..23085e665faa 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -91,8 +91,7 @@ void __init housekeeping_init(void) =20 static int __init housekeeping_setup(char *str, enum hk_flags flags) { - cpumask_var_t non_housekeeping_mask; - cpumask_var_t tmp; + cpumask_var_t non_housekeeping_mask, housekeeping_staging; =20 alloc_bootmem_cpumask_var(&non_housekeeping_mask); if (cpulist_parse(str, non_housekeeping_mask) < 0) { @@ -101,32 +100,32 @@ static int __init housekeeping_setup(char *str, enum = hk_flags flags) return 0; } =20 - alloc_bootmem_cpumask_var(&tmp); - if (!housekeeping_flags) { - alloc_bootmem_cpumask_var(&housekeeping_mask); - cpumask_andnot(housekeeping_mask, - cpu_possible_mask, non_housekeeping_mask); + alloc_bootmem_cpumask_var(&housekeeping_staging); + cpumask_andnot(housekeeping_staging, + cpu_possible_mask, non_housekeeping_mask); =20 - cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask); - if (cpumask_empty(tmp)) { + if (!cpumask_intersects(cpu_present_mask, housekeeping_staging)) { + __cpumask_set_cpu(smp_processor_id(), housekeeping_staging); + __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); + if (!housekeeping_flags) { pr_warn("Housekeeping: must include one present CPU, " "using boot CPU:%d\n", smp_processor_id()); - __cpumask_set_cpu(smp_processor_id(), housekeeping_mask); - __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); } + } + + if (!housekeeping_flags) { + alloc_bootmem_cpumask_var(&housekeeping_mask); + cpumask_copy(housekeeping_mask, housekeeping_staging); } else { - cpumask_andnot(tmp, cpu_present_mask, non_housekeeping_mask); - if (cpumask_empty(tmp)) - __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); - cpumask_andnot(tmp, cpu_possible_mask, non_housekeeping_mask); - if (!cpumask_equal(tmp, housekeeping_mask)) { + if (!cpumask_equal(housekeeping_staging, housekeeping_mask)) { pr_warn("Housekeeping: nohz_full=3D must match isolcpus=3D\n"); - free_bootmem_cpumask_var(tmp); + free_bootmem_cpumask_var(housekeeping_staging); free_bootmem_cpumask_var(non_housekeeping_mask); return 0; } } - free_bootmem_cpumask_var(tmp); + + free_bootmem_cpumask_var(housekeeping_staging); =20 if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) { if (IS_ENABLED(CONFIG_NO_HZ_FULL)) { --=20 2.25.1 From nobody Wed Jul 1 01:53:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C105C433EF for ; Tue, 4 Jan 2022 14:50:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234428AbiADOuV (ORCPT ); Tue, 4 Jan 2022 09:50:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234466AbiADOuS (ORCPT ); Tue, 4 Jan 2022 09:50:18 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B38CC06179C for ; Tue, 4 Jan 2022 06:50:15 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 17C40B81125 for ; Tue, 4 Jan 2022 14:50:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A3B5DC36AEF; Tue, 4 Jan 2022 14:50:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641307812; bh=ltlg5jlfdPZvcV9QPEZ3osCQGIe/VIdkjKzyZPSKghU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ImV2ZppcLJ6/4BTYYOxGlDKPxS20UcwhHQJRKAB/ev4JduitpfzPos6Y3+OjizkZi h2BbypM5+TmYhMYzZeAyeg+sAbzh/VIvCOl5Tvcj/PRe4Gyq9/yW7JCl6yIZC6qMZK z3QfP55HiTXI6E5AjqZVI8y8Je/BK8nh39QZb+S6T1L1U/5jUM+WNW1n1I8V0z0Thv I7OZXxtzB2w/ccEn0HsLUGKWLjCWEMlwZdNcUztd3SacG2UU6CL3XEdgYEDOkdKgDE +pNxKa6+uLMvYpa/sgcEFcv41BrLlF6K8xsBTFEHhpyDaHQWOoDBeDo9HW99aoVlWC khmKsi6shEpJA== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , Tejun Heo , Juri Lelli , Alex Belits , Nitesh Lal , Thomas Gleixner , Nicolas Saenz , Christoph Lameter , Marcelo Tosatti , Zefan Li , Paul Gortmaker , "Paul E . McKenney" Subject: [PATCH 6/8] sched/isolation: Consolidate error handling Date: Tue, 4 Jan 2022 15:49:42 +0100 Message-Id: <20220104144944.1278663-7-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220104144944.1278663-1-frederic@kernel.org> References: <20220104144944.1278663-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Centralize the mask freeing and return value for the error path. This makes potential leaks more visible. Signed-off-by: Frederic Weisbecker Cc: Thomas Gleixner Cc: Juri Lelli Cc: Marcelo Tosatti Cc: Nitesh Lal Cc: Nicolas Saenz Cc: Peter Zijlstra Cc: Christoph Lameter Cc: Tejun Heo Cc: Zefan Li Cc: Alex Belits Reviewed-by: Marcelo Tosatti Reviewed-by: Phil Auld --- kernel/sched/isolation.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 23085e665faa..828dacec483e 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -92,12 +92,12 @@ void __init housekeeping_init(void) static int __init housekeeping_setup(char *str, enum hk_flags flags) { cpumask_var_t non_housekeeping_mask, housekeeping_staging; + int err =3D 0; =20 alloc_bootmem_cpumask_var(&non_housekeeping_mask); if (cpulist_parse(str, non_housekeeping_mask) < 0) { pr_warn("Housekeeping: nohz_full=3D or isolcpus=3D incorrect CPU range\n= "); - free_bootmem_cpumask_var(non_housekeeping_mask); - return 0; + goto free_non_housekeeping_mask; } =20 alloc_bootmem_cpumask_var(&housekeeping_staging); @@ -119,30 +119,29 @@ static int __init housekeeping_setup(char *str, enum = hk_flags flags) } else { if (!cpumask_equal(housekeeping_staging, housekeeping_mask)) { pr_warn("Housekeeping: nohz_full=3D must match isolcpus=3D\n"); - free_bootmem_cpumask_var(housekeeping_staging); - free_bootmem_cpumask_var(non_housekeeping_mask); - return 0; + goto free_housekeeping_staging; } } =20 - free_bootmem_cpumask_var(housekeeping_staging); - if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) { if (IS_ENABLED(CONFIG_NO_HZ_FULL)) { tick_nohz_full_setup(non_housekeeping_mask); } else { pr_warn("Housekeeping: nohz unsupported." " Build with CONFIG_NO_HZ_FULL\n"); - free_bootmem_cpumask_var(non_housekeeping_mask); - return 0; + goto free_housekeeping_staging; } } =20 housekeeping_flags |=3D flags; + err =3D 1; =20 +free_housekeeping_staging: + free_bootmem_cpumask_var(housekeeping_staging); +free_non_housekeeping_mask: free_bootmem_cpumask_var(non_housekeeping_mask); =20 - return 1; + return err; } =20 static int __init housekeeping_nohz_full_setup(char *str) --=20 2.25.1 From nobody Wed Jul 1 01:53:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E6CAC433EF for ; Tue, 4 Jan 2022 14:50:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234439AbiADOuX (ORCPT ); Tue, 4 Jan 2022 09:50:23 -0500 Received: from dfw.source.kernel.org ([139.178.84.217]:42064 "EHLO dfw.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234461AbiADOuR (ORCPT ); Tue, 4 Jan 2022 09:50:17 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D7F0761488 for ; Tue, 4 Jan 2022 14:50:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3E561C36AED; Tue, 4 Jan 2022 14:50:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641307816; bh=jPTZQrqO8CoTMJAUHYLOdsNJt5UYV2szJE+ZXdnYbJw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sA/iL8kDN433Yv7ZY33MaiDyGA3Sj739XEXsQFOnRFZFZo3PGdovVSQ1lcV+oAll8 688ir/ey47It/8ZL8u4ALNWC91RAMCOgsBTQ2qO3TEChUSsVc2ssIgI79iD+Mmzfof v58sk8z9RXPDT4LlBXXq9gQxOlTyVRBkv9eqxs8Il1k+crr8urg98LCVvf+R0f3xny cBeyl8/DsVgYZl2+viqNPVa9E+EunVzUjLOhgbQfb8CiqWC2CDX4N9fmnvNtemidtT WAEVc7EDukoSSUAcTNPhO5hVv5A5DO3Pu9Lt2N00njqv1NBXNufut40q5bz23WdGsv clUz+zOrfG/qg== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , Tejun Heo , Juri Lelli , Alex Belits , Nitesh Lal , Thomas Gleixner , Nicolas Saenz , Christoph Lameter , Marcelo Tosatti , Zefan Li , Paul Gortmaker , "Paul E . McKenney" Subject: [PATCH 7/8] sched/isolation: Fix housekeeping_mask memory leak Date: Tue, 4 Jan 2022 15:49:43 +0100 Message-Id: <20220104144944.1278663-8-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220104144944.1278663-1-frederic@kernel.org> References: <20220104144944.1278663-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" If "nohz_full=3D" or "isolcpus=3Dnohz" are called with CONFIG_NO_HZ_FULL=3D= n, housekeeping_mask doesn't get freed despite it being unused if housekeeping_setup() is called for the first time. Check this scenario first to fix this, so that no useless allocation is performed. Signed-off-by: Frederic Weisbecker Cc: Thomas Gleixner Cc: Juri Lelli Cc: Marcelo Tosatti Cc: Nitesh Lal Cc: Nicolas Saenz Cc: Peter Zijlstra Cc: Christoph Lameter Cc: Tejun Heo Cc: Zefan Li Cc: Alex Belits Reviewed-by: Marcelo Tosatti Reviewed-by: Phil Auld --- kernel/sched/isolation.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 828dacec483e..883eee9fae22 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -94,6 +94,14 @@ static int __init housekeeping_setup(char *str, enum hk_= flags flags) cpumask_var_t non_housekeeping_mask, housekeeping_staging; int err =3D 0; =20 + if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) { + if (!IS_ENABLED(CONFIG_NO_HZ_FULL)) { + pr_warn("Housekeeping: nohz unsupported." + " Build with CONFIG_NO_HZ_FULL\n"); + return 0; + } + } + alloc_bootmem_cpumask_var(&non_housekeeping_mask); if (cpulist_parse(str, non_housekeeping_mask) < 0) { pr_warn("Housekeeping: nohz_full=3D or isolcpus=3D incorrect CPU range\n= "); @@ -123,15 +131,8 @@ static int __init housekeeping_setup(char *str, enum h= k_flags flags) } } =20 - if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) { - if (IS_ENABLED(CONFIG_NO_HZ_FULL)) { - tick_nohz_full_setup(non_housekeeping_mask); - } else { - pr_warn("Housekeeping: nohz unsupported." - " Build with CONFIG_NO_HZ_FULL\n"); - goto free_housekeeping_staging; - } - } + if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) + tick_nohz_full_setup(non_housekeeping_mask); =20 housekeeping_flags |=3D flags; err =3D 1; --=20 2.25.1 From nobody Wed Jul 1 01:53:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 509B3C433F5 for ; Tue, 4 Jan 2022 14:50:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234462AbiADOu0 (ORCPT ); Tue, 4 Jan 2022 09:50:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234449AbiADOuV (ORCPT ); Tue, 4 Jan 2022 09:50:21 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BECAFC061792 for ; Tue, 4 Jan 2022 06:50:20 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 68DC461342 for ; Tue, 4 Jan 2022 14:50:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C5BD9C36AE9; Tue, 4 Jan 2022 14:50:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1641307819; bh=RFPi4gfxnFhvm7pPVFQqXf6bt1XxB3cVmIeHqAdGB+U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TYOZrg7gWqePynUqIY14NfPNJrE/MqUtWQvmYGwCJNrqGd+q5FM3mwtmjIDZI51pQ YLsy7+jK1cc2nBy0btOy6BkUi0NPgC7WGt+eHdWlB6i9v6Am6SXGwpsXLEGDp7kJhq ZImwi2Rdf4tSc9T6NCu4RNAe9j9oUoWlV+S1AqXrgwE/Qt3c0h3HT4iLAm7OmwrjkO Nwf48GDasJztWR3ZXMMijZQt4z13FMwFXG19wAcNUu5s1LMC24V4rK7gvqE89f/Zq+ wrP8zcG8xPYmBdZgCYFjeq4em9GuHcqRCPadZbXUHGty8a0lKhCJolIcB5uUsBuu0H vHgXr4Fr5kSng== From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Frederic Weisbecker , Tejun Heo , Juri Lelli , Alex Belits , Nitesh Lal , Thomas Gleixner , Nicolas Saenz , Christoph Lameter , Marcelo Tosatti , Zefan Li , Paul Gortmaker , "Paul E . McKenney" Subject: [PATCH 8/8] sched/isolation: Split housekeeping cpumask per isolation features Date: Tue, 4 Jan 2022 15:49:44 +0100 Message-Id: <20220104144944.1278663-9-frederic@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220104144944.1278663-1-frederic@kernel.org> References: <20220104144944.1278663-1-frederic@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To prepare for supporting each housekeeping feature toward cpuset, split the global housekeeping cpumask per HK_TYPE_* entry. This will later allow, for example, to runtime modify the cpulist passed through "isolcpus=3D", "nohz_full=3D" and "rcu_nocbs=3D" kernel boot parameters. Signed-off-by: Frederic Weisbecker Cc: Thomas Gleixner Cc: Juri Lelli Cc: Marcelo Tosatti Cc: Nitesh Lal Cc: Nicolas Saenz Cc: Peter Zijlstra Cc: Christoph Lameter Cc: Tejun Heo Cc: Zefan Li Cc: Alex Belits Reviewed-by: Marcelo Tosatti Reviewed-by: Phil Auld --- kernel/sched/isolation.c | 91 +++++++++++++++++++++++++++------------- 1 file changed, 62 insertions(+), 29 deletions(-) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 883eee9fae22..b4d10815c45a 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -23,12 +23,17 @@ enum hk_flags { =20 DEFINE_STATIC_KEY_FALSE(housekeeping_overridden); EXPORT_SYMBOL_GPL(housekeeping_overridden); -static cpumask_var_t housekeeping_mask; -static unsigned int housekeeping_flags; + +struct housekeeping { + cpumask_var_t cpumasks[HK_TYPE_MAX]; + unsigned long flags; +}; + +static struct housekeeping housekeeping; =20 bool housekeeping_enabled(enum hk_type type) { - return !!(housekeeping_flags & BIT(type)); + return !!(housekeeping.flags & BIT(type)); } EXPORT_SYMBOL_GPL(housekeeping_enabled); =20 @@ -37,12 +42,12 @@ int housekeeping_any_cpu(enum hk_type type) int cpu; =20 if (static_branch_unlikely(&housekeeping_overridden)) { - if (housekeeping_flags & BIT(type)) { - cpu =3D sched_numa_find_closest(housekeeping_mask, smp_processor_id()); + if (housekeeping.flags & BIT(type)) { + cpu =3D sched_numa_find_closest(housekeeping.cpumasks[type], smp_proces= sor_id()); if (cpu < nr_cpu_ids) return cpu; =20 - return cpumask_any_and(housekeeping_mask, cpu_online_mask); + return cpumask_any_and(housekeeping.cpumasks[type], cpu_online_mask); } } return smp_processor_id(); @@ -52,8 +57,8 @@ EXPORT_SYMBOL_GPL(housekeeping_any_cpu); const struct cpumask *housekeeping_cpumask(enum hk_type type) { if (static_branch_unlikely(&housekeeping_overridden)) - if (housekeeping_flags & BIT(type)) - return housekeeping_mask; + if (housekeeping.flags & BIT(type)) + return housekeeping.cpumasks[type]; return cpu_possible_mask; } EXPORT_SYMBOL_GPL(housekeeping_cpumask); @@ -61,40 +66,53 @@ EXPORT_SYMBOL_GPL(housekeeping_cpumask); void housekeeping_affine(struct task_struct *t, enum hk_type type) { if (static_branch_unlikely(&housekeeping_overridden)) - if (housekeeping_flags & BIT(type)) - set_cpus_allowed_ptr(t, housekeeping_mask); + if (housekeeping.flags & BIT(type)) + set_cpus_allowed_ptr(t, housekeeping.cpumasks[type]); } EXPORT_SYMBOL_GPL(housekeeping_affine); =20 bool housekeeping_test_cpu(int cpu, enum hk_type type) { if (static_branch_unlikely(&housekeeping_overridden)) - if (housekeeping_flags & BIT(type)) - return cpumask_test_cpu(cpu, housekeeping_mask); + if (housekeeping.flags & BIT(type)) + return cpumask_test_cpu(cpu, housekeeping.cpumasks[type]); return true; } EXPORT_SYMBOL_GPL(housekeeping_test_cpu); =20 void __init housekeeping_init(void) { - if (!housekeeping_flags) + enum hk_type type; + + if (!housekeeping.flags) return; =20 static_branch_enable(&housekeeping_overridden); =20 - if (housekeeping_flags & HK_FLAG_TICK) + if (housekeeping.flags & HK_FLAG_TICK) sched_tick_offload_init(); =20 - /* We need at least one CPU to handle housekeeping work */ - WARN_ON_ONCE(cpumask_empty(housekeeping_mask)); + for_each_set_bit(type, &housekeeping.flags, HK_TYPE_MAX) { + /* We need at least one CPU to handle housekeeping work */ + WARN_ON_ONCE(cpumask_empty(housekeeping.cpumasks[type])); + } } =20 -static int __init housekeeping_setup(char *str, enum hk_flags flags) +static void __init housekeeping_setup_type(enum hk_type type, + cpumask_var_t housekeeping_staging) +{ + + alloc_bootmem_cpumask_var(&housekeeping.cpumasks[type]); + cpumask_copy(housekeeping.cpumasks[type], + housekeeping_staging); +} + +static int __init housekeeping_setup(char *str, unsigned long flags) { cpumask_var_t non_housekeeping_mask, housekeeping_staging; int err =3D 0; =20 - if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) { + if ((flags & HK_FLAG_TICK) && !(housekeeping.flags & HK_FLAG_TICK)) { if (!IS_ENABLED(CONFIG_NO_HZ_FULL)) { pr_warn("Housekeeping: nohz unsupported." " Build with CONFIG_NO_HZ_FULL\n"); @@ -115,26 +133,41 @@ static int __init housekeeping_setup(char *str, enum = hk_flags flags) if (!cpumask_intersects(cpu_present_mask, housekeeping_staging)) { __cpumask_set_cpu(smp_processor_id(), housekeeping_staging); __cpumask_clear_cpu(smp_processor_id(), non_housekeeping_mask); - if (!housekeeping_flags) { + if (!housekeeping.flags) { pr_warn("Housekeeping: must include one present CPU, " "using boot CPU:%d\n", smp_processor_id()); } } =20 - if (!housekeeping_flags) { - alloc_bootmem_cpumask_var(&housekeeping_mask); - cpumask_copy(housekeeping_mask, housekeeping_staging); + if (!housekeeping.flags) { + /* First setup call ("nohz_full=3D" or "isolcpus=3D") */ + enum hk_type type; + + for_each_set_bit(type, &flags, HK_TYPE_MAX) + housekeeping_setup_type(type, housekeeping_staging); } else { - if (!cpumask_equal(housekeeping_staging, housekeeping_mask)) { - pr_warn("Housekeeping: nohz_full=3D must match isolcpus=3D\n"); - goto free_housekeeping_staging; + /* Second setup call ("nohz_full=3D" after "isolcpus=3D" or the reverse)= */ + enum hk_type type; + unsigned long iter_flags =3D flags & housekeeping.flags; + + for_each_set_bit(type, &iter_flags, HK_TYPE_MAX) { + if (!cpumask_equal(housekeeping_staging, + housekeeping.cpumasks[type])) { + pr_warn("Housekeeping: nohz_full=3D must match isolcpus=3D\n"); + goto free_housekeeping_staging; + } } + + iter_flags =3D flags & ~housekeeping.flags; + + for_each_set_bit(type, &iter_flags, HK_TYPE_MAX) + housekeeping_setup_type(type, housekeeping_staging); } =20 - if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) + if ((flags & HK_FLAG_TICK) && !(housekeeping.flags & HK_FLAG_TICK)) tick_nohz_full_setup(non_housekeeping_mask); =20 - housekeeping_flags |=3D flags; + housekeeping.flags |=3D flags; err =3D 1; =20 free_housekeeping_staging: @@ -147,7 +180,7 @@ static int __init housekeeping_setup(char *str, enum hk= _flags flags) =20 static int __init housekeeping_nohz_full_setup(char *str) { - unsigned int flags; + unsigned long flags; =20 flags =3D HK_FLAG_TICK | HK_FLAG_WQ | HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC | HK_FLAG_KTHREAD; @@ -158,7 +191,7 @@ __setup("nohz_full=3D", housekeeping_nohz_full_setup); =20 static int __init housekeeping_isolcpus_setup(char *str) { - unsigned int flags =3D 0; + unsigned long flags =3D 0; bool illegal =3D false; char *par; int len; --=20 2.25.1