From nobody Mon May 25 00:09:08 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 382C63AA187; Wed, 20 May 2026 08:34:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779266097; cv=none; b=AvtyOexwPeUk9vV7oHCpe2sPyS+2Ng8r6cH9jfyJlWCfVZfhE4yimfvKmsWk4LtgRKw8stqlBs8/IFhW42cHlckCfzBnpRkNLX989uxT3SeX0/Gqg8NFYlcB614FRvxwCEMLN3fgjkksG5S8k84fYJh1Fu7ojU4DeldUG15wHMs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779266097; c=relaxed/simple; bh=P0A6nzAX4enTid3j8r+qmqbXATP44ANR+ocAsKMsegA=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=j/P6GWK595lbwrUAIocfYdeQHtTGJgb4gDo+L8un1qufBXGaMynFRrrQ1uUzzy116Jx4u9JWu/+rPfAV4+DxK05mZ0xyFEEjLiSTGB2gvrGT9k42fv0Kh5z7PONY7uIWB+nj3pgO9jtJFepTFjh5BF6D+s/BqBKluBgU58U55NQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=BtIGYUwH; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=1uGwrdi0; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="BtIGYUwH"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="1uGwrdi0" Date: Wed, 20 May 2026 08:34:53 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1779266094; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0yLC01BWAuewfMvNOiaHAdc44KHwqFHjohI6Rgf+3/k=; b=BtIGYUwHT5UftfZ1iE855qCqasgD0qdpSwG85p8zmQWfn9JC/PhlQdOXnenEI4Ophdpe7+ t9f9Mya7UKLCjpgVOEfiDDCa1cHFPo38DCIGOIpRlXKcUOZ1E0U6cO0tTbZRW/1ICuc1Kd dZNq3yfsOkgTA9ZtvmE6KRw0AcaHgW49LR9v8iVct188R1a3B/BMGoo3LM/MtDlpgRHwFO LwIsE34uumR1u0N12p8UtJLCgztbidnwE+s+R/MPZFybf/MikTZQaK9Xs3vjrn8XRCv9+5 9dslB+rR534sgDUgi1b4+QAdOo0aZD4cE7nX9Z3+2Nn67hCdNtYX8x3rLzvnPg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1779266094; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0yLC01BWAuewfMvNOiaHAdc44KHwqFHjohI6Rgf+3/k=; b=1uGwrdi0Tlu6+oRzCUZJTDROwxpFtJ+xJHMkh+6zDyUbdUT8smeNkjrO3ed7WZRRETK5KN 4Kc8TiQJeVsHClAQ== From: "tip-bot2 for Tim Chen" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/core] sched/cache: Introduce per CPU's tasks LLC preference counter Cc: "Peter Zijlstra (Intel)" , K Prateek Nayak , Chen Yu , Tim Chen , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: =?utf-8?q?=3C42e79eceb8cd6be8a032401d481d101913bc5703=2E1775065?= =?utf-8?q?312=2Egit=2Etim=2Ec=2Echen=40linux=2Eintel=2Ecom=3E?= References: =?utf-8?q?=3C42e79eceb8cd6be8a032401d481d101913bc5703=2E17750653?= =?utf-8?q?12=2Egit=2Etim=2Ec=2Echen=40linux=2Eintel=2Ecom=3E?= Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <177926609340.711.15819761101007845706.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the sched/core branch of tip: Commit-ID: a8d0ca0b7f2f7b53565d1e30e509d3d74d1f5460 Gitweb: https://git.kernel.org/tip/a8d0ca0b7f2f7b53565d1e30e509d3d74= d1f5460 Author: Tim Chen AuthorDate: Wed, 01 Apr 2026 14:52:20 -07:00 Committer: Peter Zijlstra CommitterDate: Thu, 09 Apr 2026 15:49:49 +02:00 sched/cache: Introduce per CPU's tasks LLC preference counter The lowest level of sched domain for each CPU is assigned an array where each element tracks the number of tasks preferring a given LLC, indexed from 0 to max_lid. Since each CPU has its dedicated sd, this implies that each CPU will have a dedicated task LLC preference counter. For example, sd->llc_counts[3] =3D 2 signifies that there are 2 tasks on this runqueue which prefer to run within LLC3. The load balancer can use this information to identify busy runqueues and migrate tasks to their preferred LLC domains. This array will be reallocated at runtime during sched domain rebuild. Introduce the buffer allocation mechanism, and the statistics will be calculated in the subsequent patch. Note: the LLC preference statistics of each CPU are reset on sched domain rebuild and may under count temporarily, until the CPU becomes idle and the count is cleared. This is a trade off to avoid complex data synchronization across sched domain builds. Suggested-by: Peter Zijlstra (Intel) Suggested-by: K Prateek Nayak Co-developed-by: Chen Yu Signed-off-by: Chen Yu Signed-off-by: Tim Chen Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/42e79eceb8cd6be8a032401d481d101913bc5703.177= 5065312.git.tim.c.chen@linux.intel.com --- include/linux/sched/topology.h | 5 +++- kernel/sched/topology.c | 62 ++++++++++++++++++++++++++++++++- 2 files changed, 66 insertions(+), 1 deletion(-) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index 159716f..0036d6b 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -103,6 +103,11 @@ struct sched_domain { u64 max_newidle_lb_cost; unsigned long last_decay_max_lb_cost; =20 +#ifdef CONFIG_SCHED_CACHE + unsigned int llc_max; + unsigned int *llc_counts __counted_by_ptr(llc_max); +#endif + #ifdef CONFIG_SCHEDSTATS /* sched_balance_rq() stats */ unsigned int lb_count[CPU_MAX_IDLE_TYPES]; diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 1200670..8954bf7 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -634,6 +634,11 @@ static void destroy_sched_domain(struct sched_domain *= sd) =20 if (sd->shared && atomic_dec_and_test(&sd->shared->ref)) kfree(sd->shared); + +#ifdef CONFIG_SCHED_CACHE + /* only the bottom sd has llc_counts array */ + kfree(sd->llc_counts); +#endif kfree(sd); } =20 @@ -763,10 +768,18 @@ cpu_attach_domain(struct sched_domain *sd, struct roo= t_domain *rd, int cpu) if (sd && sd_degenerate(sd)) { tmp =3D sd; sd =3D sd->parent; - destroy_sched_domain(tmp); + if (sd) { struct sched_group *sg =3D sd->groups; =20 +#ifdef CONFIG_SCHED_CACHE + /* move buffer to parent as child is being destroyed */ + sd->llc_counts =3D tmp->llc_counts; + sd->llc_max =3D tmp->llc_max; + /* make sure destroy_sched_domain() does not free it */ + tmp->llc_counts =3D NULL; + tmp->llc_max =3D 0; +#endif /* * sched groups hold the flags of the child sched * domain for convenience. Clear such flags since @@ -778,6 +791,8 @@ cpu_attach_domain(struct sched_domain *sd, struct root_= domain *rd, int cpu) =20 sd->child =3D NULL; } + + destroy_sched_domain(tmp); } =20 sched_domain_debug(sd, cpu); @@ -805,6 +820,49 @@ enum s_alloc { sa_none, }; =20 +#ifdef CONFIG_SCHED_CACHE +static bool alloc_sd_llc(const struct cpumask *cpu_map, + struct s_data *d) +{ + struct sched_domain *sd; + unsigned int *p; + int i; + + for_each_cpu(i, cpu_map) { + sd =3D *per_cpu_ptr(d->sd, i); + if (!sd) + goto err; + + p =3D kcalloc_node(max_lid + 1, sizeof(unsigned int), + GFP_KERNEL, cpu_to_node(i)); + if (!p) + goto err; + + sd->llc_max =3D max_lid + 1; + sd->llc_counts =3D p; + } + + return true; +err: + for_each_cpu(i, cpu_map) { + sd =3D *per_cpu_ptr(d->sd, i); + if (sd) { + kfree(sd->llc_counts); + sd->llc_counts =3D NULL; + sd->llc_max =3D 0; + } + } + + return false; +} +#else +static bool alloc_sd_llc(const struct cpumask *cpu_map, + struct s_data *d) +{ + return false; +} +#endif + /* * Return the canonical balance CPU for this group, this is the first CPU * of this group that's also in the balance mask. @@ -2828,6 +2886,8 @@ build_sched_domains(const struct cpumask *cpu_map, st= ruct sched_domain_attr *att init_sched_groups_capacity(i, sd); } =20 + alloc_sd_llc(cpu_map, &d); + /* Attach the domains */ rcu_read_lock(); for_each_cpu(i, cpu_map) {