From nobody Fri Jul 24 05:21:41 2026 Received: from canpmsgout09.his.huawei.com (canpmsgout09.his.huawei.com [113.46.200.224]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5804033B951 for ; Thu, 23 Jul 2026 03:38:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.224 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784777932; cv=none; b=WZB68YRjDEQOSYknRXng2HEIPJa54ArVRi4BLZ+sLNtf84BEOm/piqgG+6Fc3JFxFPIjhrE0Vkrnlb8kVASPVesOzvrsF1sodowcnEANWQ9eaQhzpxhFSdWFHBY5V0q6XjLTb5m8xrki02VlZqDVwLO5HysN+Mkot4bqSx+lmAI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784777932; c=relaxed/simple; bh=g7CG0qgn34cB1SzLMmpf1wFvAUfqbcLsH+tUnIHpW/c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=COd3/Ywi7Dam12iXVDrMluwGpScC/H9B0cty67TNu11m2292W1b+gi0cZBaz59BaEsiiDZcK1UfWp9zB6Mal7ifN6pRnx0YICrJCBKf327BgcduRHx/XIw6wwccMnlJEmV5PZP0ItHJw6N1CMNyufbSNZFwnFjKMLCKkOBNQTW8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=egca8RjQ; arc=none smtp.client-ip=113.46.200.224 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="egca8RjQ" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=8ZTvDlqDqb670Xw6qM8212HKAhq4LAsQETOIRaVrGEE=; b=egca8RjQa6qZ4Kgw0XWA3VMWyr+YNUO5f8fBy6NJEIwyxThgD465LBnj6dwsY0NbRHMprn+Ss NWpkRmK65j6V+juTTySWuxHTNSVwpLTCpEEpdB9pzBGzokLn7LpbCqdses91Sxozlq1ttn746hG LC+WZ4FNbhCIbVLxNAOu4UU= Received: from mail.maildlp.com (unknown [172.19.163.127]) by canpmsgout09.his.huawei.com (SkyGuard) with ESMTPS id 4h5GnS53Svz1cyQQ; Thu, 23 Jul 2026 11:29:16 +0800 (CST) Received: from kwepemj100017.china.huawei.com (unknown [7.202.194.11]) by mail.maildlp.com (Postfix) with ESMTPS id 62A3B40572; Thu, 23 Jul 2026 11:38:38 +0800 (CST) Received: from huawei.com (10.67.174.193) by kwepemj100017.china.huawei.com (7.202.194.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.36; Thu, 23 Jul 2026 11:38:37 +0800 From: Luo Gengkun To: , , , , , CC: , , , , , , Subject: [PATCH v8 1/2] sched/cache: Reduce the overhead of task_cache_work by only scan the visisted cpus Date: Thu, 23 Jul 2026 04:04:28 +0000 Message-ID: <20260723040429.630176-2-luogengkun2@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260723040429.630176-1-luogengkun2@huawei.com> References: <20260723040429.630176-1-luogengkun2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems200001.china.huawei.com (7.221.188.67) To kwepemj100017.china.huawei.com (7.202.194.11) Content-Type: text/plain; charset="utf-8" The overhead of task_cache_work() is high, especially in multi-NUMA systems. Currently, task_cache_work() tries to find the pref_llc by scanning all CPUs in the system. However, most of these scans are meaningless, such as those for CPUs that have never been visited or were accessed a long time ago. To address this problem, introduce visited_cpus to track the visited CPUs and evict them once they have not been accessed for a duration exceeding llc_epoch_affinity_timeout. Now that we know exactly which CPUs to scan from visited_cpus, we can remove get_scan_cpumasks(). Signed-off-by: Luo Gengkun --- include/linux/mm_types.h | 6 +++ include/linux/sched.h | 2 + kernel/sched/fair.c | 94 ++++++++++++++++------------------------ 3 files changed, 46 insertions(+), 56 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index b18c2b2e7d2c..35559079e4d4 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -1620,6 +1620,11 @@ static inline int mm_alloc_sched_noprof(struct mm_st= ruct *mm) if (!pcpu_sched) return -ENOMEM; =20 + if (!zalloc_cpumask_var(&mm->sc_stat.visited_cpus, GFP_KERNEL)) { + free_percpu(pcpu_sched); + return -ENOMEM; + } + mm_init_sched(mm, pcpu_sched); return 0; } @@ -1630,6 +1635,7 @@ static inline void mm_destroy_sched(struct mm_struct = *mm) { free_percpu(mm->sc_stat.pcpu_sched); mm->sc_stat.pcpu_sched =3D NULL; + free_cpumask_var(mm->sc_stat.visited_cpus); } #else /* !CONFIG_SCHED_CACHE */ =20 diff --git a/include/linux/sched.h b/include/linux/sched.h index 373bcc0598d1..b461a71a65da 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2388,6 +2388,7 @@ static __always_inline int task_mm_cid(struct task_st= ruct *t) struct sched_cache_time { u64 runtime; unsigned long epoch; + unsigned long epoch_last_visit; }; =20 struct sched_cache_stat { @@ -2398,6 +2399,7 @@ struct sched_cache_stat { unsigned long next_scan; unsigned long footprint; int cpu; + cpumask_var_t visited_cpus; } ____cacheline_aligned_in_smp; =20 #else diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d78467ec6ee1..10d442074f21 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1585,6 +1585,7 @@ void mm_init_sched(struct mm_struct *mm, pcpu_sched->runtime =3D 0; /* a slightly stale cpu epoch is acceptible */ pcpu_sched->epoch =3D rq->cpu_epoch; + pcpu_sched->epoch_last_visit =3D rq->cpu_epoch; epoch =3D rq->cpu_epoch; } =20 @@ -1635,13 +1636,23 @@ static inline void __update_mm_sched(struct rq *rq, } } =20 -static unsigned long fraction_mm_sched(struct rq *rq, - struct sched_cache_time *pcpu_sched) +static unsigned long fraction_mm_sched(int cpu, + struct mm_struct *mm) { + struct sched_cache_time *pcpu_sched =3D + per_cpu_ptr(mm->sc_stat.pcpu_sched, cpu); + struct rq *rq =3D cpu_rq(cpu); + guard(raw_spinlock_irqsave)(&rq->cpu_epoch_lock); =20 __update_mm_sched(rq, pcpu_sched); =20 + /* Skip the rq that has not been hit for a long time */ + if ((rq->cpu_epoch - pcpu_sched->epoch_last_visit) > llc_epoch_affinity_t= imeout) { + cpumask_clear_cpu(cpu, mm->sc_stat.visited_cpus); + return 0; + } + /* * Runtime is a geometric series (r=3D0.5) and as such will sum to twice * the accumulation period, this means the multiplcation here should @@ -1711,6 +1722,9 @@ void account_mm_sched(struct rq *rq, struct task_stru= ct *p, s64 delta_exec) pcpu_sched->runtime +=3D delta_exec; rq->cpu_runtime +=3D delta_exec; epoch =3D rq->cpu_epoch; + pcpu_sched->epoch_last_visit =3D epoch; + if (!cpumask_test_cpu(cpu_of(rq), mm->sc_stat.visited_cpus)) + cpumask_set_cpu(cpu_of(rq), mm->sc_stat.visited_cpus); } =20 /* @@ -1761,51 +1775,6 @@ static void task_tick_cache(struct rq *rq, struct ta= sk_struct *p) } } =20 -static void get_scan_cpumasks(cpumask_var_t cpus, struct task_struct *p) -{ -#ifdef CONFIG_NUMA_BALANCING - int cpu, curr_cpu, nid, pref_nid; - - if (!static_branch_likely(&sched_numa_balancing)) - goto out; - - cpu =3D READ_ONCE(p->mm->sc_stat.cpu); - if (cpu !=3D -1) - nid =3D cpu_to_node(cpu); - curr_cpu =3D task_cpu(p); - - /* - * Scanning in the preferred NUMA node is ideal. However, the NUMA - * preferred node is per-task rather than per-process. It is possible - * for different threads of the process to have distinct preferred - * nodes; consequently, the process-wide preferred LLC may bounce - * between different nodes. As a workaround, maintain the scan - * CPU mask to also cover the process's current preferred LLC and the - * current running node to mitigate the bouncing risk. - * TBD: numa_group should be considered during task aggregation. - */ - pref_nid =3D p->numa_preferred_nid; - /* honor the task's preferred node */ - if (pref_nid =3D=3D NUMA_NO_NODE) - goto out; - - cpumask_or(cpus, cpus, cpumask_of_node(pref_nid)); - - /* honor the task's preferred LLC CPU */ - if (cpu !=3D -1 && !cpumask_test_cpu(cpu, cpus) && nid !=3D NUMA_NO_NODE) - cpumask_or(cpus, cpus, cpumask_of_node(nid)); - - /* make sure the task's current running node is included */ - if (!cpumask_test_cpu(curr_cpu, cpus)) - cpumask_or(cpus, cpus, cpumask_of_node(cpu_to_node(curr_cpu))); - - return; - -out: -#endif - cpumask_copy(cpus, cpu_online_mask); -} - static inline void update_avg_scale(u64 *avg, u64 sample) { int factor =3D per_cpu(sd_llc_size, raw_smp_processor_id()); @@ -1866,7 +1835,18 @@ static void task_cache_work(struct callback_head *wo= rk) scoped_guard (cpus_read_lock) { guard(rcu)(); =20 - get_scan_cpumasks(cpus, p); + /* + * Data race: While evaluating the visited_cpus without + * a lock, a CPU could be concurrently set by + * account_mm_sched(), meaning the scan might skip the newly + * visited CPU if the bit changes during the scan. This is + * a deliberate trade-off between accuracy and efficiency: + * locking would prevent this race but incur extra overhead. + * The missed runtime contribution is negligible because it + * implies this process hasn't run on that CPU for a long + * time, and will be captured in the next cycle. + */ + cpumask_and(cpus, cpu_online_mask, mm->sc_stat.visited_cpus); =20 for_each_cpu(cpu, cpus) { /* XXX sched_cluster_active */ @@ -1877,19 +1857,21 @@ static void task_cache_work(struct callback_head *w= ork) if (!sd) continue; =20 - for_each_cpu(i, sched_domain_span(sd)) { - occ =3D fraction_mm_sched(cpu_rq(i), - per_cpu_ptr(mm->sc_stat.pcpu_sched, i)); + for_each_cpu_and(i, sched_domain_span(sd), mm->sc_stat.visited_cpus) { + cur =3D rcu_dereference_all(cpu_rq(i)->curr); + if (cur && !(cur->flags & (PF_EXITING | PF_KTHREAD)) && + cur->mm =3D=3D mm) + nr_running++; + + occ =3D fraction_mm_sched(i, mm); + if (occ =3D=3D 0) + continue; + a_occ +=3D occ; if (occ > m_occ) { m_occ =3D occ; m_cpu =3D i; } - - cur =3D rcu_dereference_all(cpu_rq(i)->curr); - if (cur && !(cur->flags & (PF_EXITING | PF_KTHREAD)) && - cur->mm =3D=3D mm) - nr_running++; } =20 /* --=20 2.34.1 From nobody Fri Jul 24 05:21:41 2026 Received: from canpmsgout06.his.huawei.com (canpmsgout06.his.huawei.com [113.46.200.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 580C934B1AD for ; Thu, 23 Jul 2026 03:38:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.221 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784777931; cv=none; b=tda0n3tjVtvbt5thaDT9F+o5XaGWbVZpW3lbcenPDPrqntlu2tPNDFH8Q6oEacEOUG65tG2XdkT2cs7+F4qEoRnVvM5AFedgHi1+Ks4quUgnYGdN2k+M/tFk/KzSKL9peG2gr7FQclBfKrcppIGZ868HTWsDeKq6DaSsnRQIUcI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784777931; c=relaxed/simple; bh=3KJsCkKkFb+jjg06dsO1QDmAIitEjg/w6tU9mdetR64=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dy37nB/s5STwilKCUtp6NjtUrN29ZL6hUcYBqueVch9VkgjXoaksoWb2Cl20wOcbVcGhDsn2E1/6K0Ltrg2DghPiAa+SBrtYx7YjiWAaJ5/I1OXxtRrLmMZXTTZdJ1QKafsQ9JWt4c9pwSVHdzFEYAcm14LTbiB15SpSNE6/VEw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=4ALF70GI; arc=none smtp.client-ip=113.46.200.221 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="4ALF70GI" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=aWDXnKFOQE/lkHq3zmfyrCm4u60xUHEMRnW5BS/TpBw=; b=4ALF70GIscGLd7WTn5cG5wfotZMk5bTkLHpQl1MKI76kjvTJalhVsQhlsnk2R1DYO0HH58Q/t arC4kJb0aZFmPvrvkvywscPVvMKwFeR6b8bhhLH0HmB3x+UuDWH6ZSExA5Qs8WEHBqqBfj/oITu /kE0kB/rmO2lhu86xefxfUY= Received: from mail.maildlp.com (unknown [172.19.162.223]) by canpmsgout06.his.huawei.com (SkyGuard) with ESMTPS id 4h5GnZ41wgzRhQR; Thu, 23 Jul 2026 11:29:22 +0800 (CST) Received: from kwepemj100017.china.huawei.com (unknown [7.202.194.11]) by mail.maildlp.com (Postfix) with ESMTPS id 76F2140561; Thu, 23 Jul 2026 11:38:44 +0800 (CST) Received: from huawei.com (10.67.174.193) by kwepemj100017.china.huawei.com (7.202.194.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.36; Thu, 23 Jul 2026 11:38:43 +0800 From: Luo Gengkun To: , , , , , CC: , , , , , , Subject: [PATCH v8 2/2] -- DO NOT APPLY!!! -- sched/cache/debug: Add trace event and sched feature to track scan cost Date: Thu, 23 Jul 2026 04:04:29 +0000 Message-ID: <20260723040429.630176-3-luogengkun2@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260723040429.630176-1-luogengkun2@huawei.com> References: <20260723040429.630176-1-luogengkun2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: kwepems200001.china.huawei.com (7.221.188.67) To kwepemj100017.china.huawei.com (7.202.194.11) Content-Type: text/plain; charset="utf-8" This commit re-adds get_scan_cpumasks() and introduces two sched features as well as trace events to track the scanning cost for testing purposes. Signed-off-by: Luo Gengkun --- include/trace/events/sched.h | 21 +++++++++++ kernel/sched/fair.c | 73 +++++++++++++++++++++++++++++++++--- kernel/sched/features.h | 2 + 3 files changed, 91 insertions(+), 5 deletions(-) diff --git a/include/trace/events/sched.h b/include/trace/events/sched.h index 535860581f15..aced624f198d 100644 --- a/include/trace/events/sched.h +++ b/include/trace/events/sched.h @@ -10,6 +10,27 @@ #include #include =20 +TRACE_EVENT(sched_cache_scan, + + TP_PROTO(struct task_struct *t, int scan), + + TP_ARGS(t, scan), + + TP_STRUCT__entry( + __string( comm, t->comm ) + __field( pid_t, pid ) + __field( int, scan ) + ), + + TP_fast_assign( + __assign_str(comm); + __entry->pid =3D t->pid; + __entry->scan =3D scan; + ), + + TP_printk("comm=3D%s pid=3D%d scan=3D%d", __get_str(comm), __entry->pid, + __entry->scan) +); /* * Tracepoint for calling kthread_stop, performed to end a kthread: */ diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 10d442074f21..2d4e543ee158 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1648,7 +1648,8 @@ static unsigned long fraction_mm_sched(int cpu, __update_mm_sched(rq, pcpu_sched); =20 /* Skip the rq that has not been hit for a long time */ - if ((rq->cpu_epoch - pcpu_sched->epoch_last_visit) > llc_epoch_affinity_t= imeout) { + if (sched_feat(SC_VISIT) && + (rq->cpu_epoch - pcpu_sched->epoch_last_visit) > llc_epoch_affinity_t= imeout) { cpumask_clear_cpu(cpu, mm->sc_stat.visited_cpus); return 0; } @@ -1723,7 +1724,8 @@ void account_mm_sched(struct rq *rq, struct task_stru= ct *p, s64 delta_exec) rq->cpu_runtime +=3D delta_exec; epoch =3D rq->cpu_epoch; pcpu_sched->epoch_last_visit =3D epoch; - if (!cpumask_test_cpu(cpu_of(rq), mm->sc_stat.visited_cpus)) + if (sched_feat(SC_VISIT) && + !cpumask_test_cpu(cpu_of(rq), mm->sc_stat.visited_cpus)) cpumask_set_cpu(cpu_of(rq), mm->sc_stat.visited_cpus); } =20 @@ -1775,6 +1777,51 @@ static void task_tick_cache(struct rq *rq, struct ta= sk_struct *p) } } =20 +static void get_scan_cpumasks(cpumask_var_t cpus, struct task_struct *p) +{ +#ifdef CONFIG_NUMA_BALANCING + int cpu, curr_cpu, nid, pref_nid; + + if (!static_branch_likely(&sched_numa_balancing)) + goto out; + + cpu =3D READ_ONCE(p->mm->sc_stat.cpu); + if (cpu !=3D -1) + nid =3D cpu_to_node(cpu); + curr_cpu =3D task_cpu(p); + + /* + * Scanning in the preferred NUMA node is ideal. However, the NUMA + * preferred node is per-task rather than per-process. It is possible + * for different threads of the process to have distinct preferred + * nodes; consequently, the process-wide preferred LLC may bounce + * between different nodes. As a workaround, maintain the scan + * CPU mask to also cover the process's current preferred LLC and the + * current running node to mitigate the bouncing risk. + * TBD: numa_group should be considered during task aggregation. + */ + pref_nid =3D p->numa_preferred_nid; + /* honor the task's preferred node */ + if (pref_nid =3D=3D NUMA_NO_NODE) + goto out; + + cpumask_or(cpus, cpus, cpumask_of_node(pref_nid)); + + /* honor the task's preferred LLC CPU */ + if (cpu !=3D -1 && !cpumask_test_cpu(cpu, cpus) && nid !=3D NUMA_NO_NODE) + cpumask_or(cpus, cpus, cpumask_of_node(nid)); + + /* make sure the task's current running node is included */ + if (!cpumask_test_cpu(curr_cpu, cpus)) + cpumask_or(cpus, cpus, cpumask_of_node(cpu_to_node(curr_cpu))); + + return; + +out: +#endif + cpumask_copy(cpus, cpu_online_mask); +} + static inline void update_avg_scale(u64 *avg, u64 sample) { int factor =3D per_cpu(sd_llc_size, raw_smp_processor_id()); @@ -1801,7 +1848,8 @@ static void task_cache_work(struct callback_head *wor= k) unsigned long curr_m_a_occ =3D 0; struct mm_struct *mm =3D p->mm; unsigned long m_a_occ =3D 0; - cpumask_var_t cpus; + cpumask_var_t cpus, scan_cpus; + int scanned =3D 0; =20 WARN_ON_ONCE(work !=3D &p->cache_work); =20 @@ -1832,6 +1880,11 @@ static void task_cache_work(struct callback_head *wo= rk) if (!zalloc_cpumask_var(&cpus, GFP_KERNEL)) return; =20 + if (!zalloc_cpumask_var(&scan_cpus, GFP_KERNEL)) { + free_cpumask_var(cpus); + return; + } + scoped_guard (cpus_read_lock) { guard(rcu)(); =20 @@ -1846,7 +1899,10 @@ static void task_cache_work(struct callback_head *wo= rk) * implies this process hasn't run on that CPU for a long * time, and will be captured in the next cycle. */ - cpumask_and(cpus, cpu_online_mask, mm->sc_stat.visited_cpus); + if (sched_feat(SC_NODE)) + get_scan_cpumasks(cpus, p); + else if (sched_feat(SC_VISIT)) + cpumask_and(cpus, cpu_online_mask, mm->sc_stat.visited_cpus); =20 for_each_cpu(cpu, cpus) { /* XXX sched_cluster_active */ @@ -1857,12 +1913,17 @@ static void task_cache_work(struct callback_head *w= ork) if (!sd) continue; =20 - for_each_cpu_and(i, sched_domain_span(sd), mm->sc_stat.visited_cpus) { + cpumask_copy(scan_cpus, sched_domain_span(sd)); + if (sched_feat(SC_VISIT)) + cpumask_and(scan_cpus, scan_cpus, mm->sc_stat.visited_cpus); + + for_each_cpu(i, scan_cpus) { cur =3D rcu_dereference_all(cpu_rq(i)->curr); if (cur && !(cur->flags & (PF_EXITING | PF_KTHREAD)) && cur->mm =3D=3D mm) nr_running++; =20 + scanned++; occ =3D fraction_mm_sched(i, mm); if (occ =3D=3D 0) continue; @@ -1917,6 +1978,8 @@ static void task_cache_work(struct callback_head *wor= k) =20 update_avg_scale(&mm->sc_stat.nr_running_avg, nr_running); free_cpumask_var(cpus); + free_cpumask_var(scan_cpus); + trace_sched_cache_scan(p, scanned); } =20 void init_sched_mm(struct task_struct *p) diff --git a/kernel/sched/features.h b/kernel/sched/features.h index 8f0dee8fc475..98701c52b919 100644 --- a/kernel/sched/features.h +++ b/kernel/sched/features.h @@ -142,3 +142,5 @@ SCHED_FEAT(LATENCY_WARN, false) */ SCHED_FEAT(NI_RANDOM, true) SCHED_FEAT(NI_RATE, true) +SCHED_FEAT(SC_VISIT, false) +SCHED_FEAT(SC_NODE, false) --=20 2.34.1