From nobody Mon May 25 00:08:59 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68AC93EA960; Wed, 20 May 2026 08:34:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779266096; cv=none; b=C6JsGg/ECIWa5FXoCYIJGawns51b5SOfzZXA999h1Lz6F4LnYPvkE1WUAqPs2zH4dG+klk+Cq/K7tGJzaqzkwh71ZDT1THWM/yrWhMltpoQEWrkcd7LBGZoWpp+q7WKM7xOFGzjd6/AqlywrLvhp52adhOcJkxfKU0IDCOH4pn4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779266096; c=relaxed/simple; bh=xiFA2wKH/yXa7o77/fmvJ9J6GSX/W1yKND7L0gLD7qM=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=QVP3i5vnryMxb0M3mbqmFkO/D7aji5s9i9oJcyfZCvqII1gaa1XkGknr2b0X/mqBVI8e1dE3HNAIgeiRwVtLRZzjTuLMc7nZW79N+nr1GCbj/bDU5pXZwckkPzrVkvD9mIP9uUJDTlxJ5Y+ADILGhu+GpW/NwrdZ+RX7r4Rzgko= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=y6QwLaB4; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=ow7+VJpn; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="y6QwLaB4"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="ow7+VJpn" Date: Wed, 20 May 2026 08:34:52 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1779266093; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MO33i0gPaB2VkYAuQ/cG7habUOdd2lXLDcMuSZelVqM=; b=y6QwLaB4mQbKqeeZEyPgXVLy6NEiBoL130/+TI7oeWPFJb6kCVbqpHASSexDO2kGwNpxnm bHIGqIhziwK8QWwHs0juKPQyUuJlprwRtaYxF6iT9FpvYKipgirB6COXS07MYZdEjGlSsH jTRLAAzAz4/obA5j9sTXyAitNgLGY21oZLSZ7XaRWNPjI6e6/WFIxnVwXZHr4QpyKQiXzL bhTpgc5i/zR/cBayUGnER8Z2LAg5AdC3UGESRyH13LQAEF3iOKykRlW1C6ZiDoTdlGbIfL LS+ci1Tg43ykt31W9SA2Se6GweykVaPbdls3nNwU0Gd4/HB+xZaKxooDARcA+Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1779266093; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MO33i0gPaB2VkYAuQ/cG7habUOdd2lXLDcMuSZelVqM=; b=ow7+VJpnc5aejhOd+SMSC6SsWca7GwkiKkZ3sWStj4YPGAk+7M1w0EsOItr6KjH4HXOdXv icojzgTWpY5ebrCw== From: "tip-bot2 for Tim Chen" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/core] sched/cache: Calculate the percpu sd task LLC preference Cc: Chen Yu , Tim Chen , "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: =?utf-8?q?=3Cd15a64436d3acd19c5c53344c5e9d3d0b79b3233=2E1775065?= =?utf-8?q?312=2Egit=2Etim=2Ec=2Echen=40linux=2Eintel=2Ecom=3E?= References: =?utf-8?q?=3Cd15a64436d3acd19c5c53344c5e9d3d0b79b3233=2E17750653?= =?utf-8?q?12=2Egit=2Etim=2Ec=2Echen=40linux=2Eintel=2Ecom=3E?= Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <177926609210.711.10997264530242968700.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the sched/core branch of tip: Commit-ID: 82c960aee304bf286552046b66d5b0b3933b2418 Gitweb: https://git.kernel.org/tip/82c960aee304bf286552046b66d5b0b39= 33b2418 Author: Tim Chen AuthorDate: Wed, 01 Apr 2026 14:52:21 -07:00 Committer: Peter Zijlstra CommitterDate: Thu, 09 Apr 2026 15:49:49 +02:00 sched/cache: Calculate the percpu sd task LLC preference Calculate the number of tasks' LLC preferences for each runqueue. This statistic is computed during task enqueue and dequeue operations, and is used by the cache-aware load balancing. Co-developed-by: Chen Yu Signed-off-by: Chen Yu Signed-off-by: Tim Chen Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/d15a64436d3acd19c5c53344c5e9d3d0b79b3233.177= 5065312.git.tim.c.chen@linux.intel.com --- kernel/sched/fair.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e66da7a..7d52cf0 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1386,6 +1386,7 @@ static int llc_id(int cpu) =20 static void account_llc_enqueue(struct rq *rq, struct task_struct *p) { + struct sched_domain *sd; int pref_llc; =20 pref_llc =3D p->preferred_llc; @@ -1394,10 +1395,15 @@ static void account_llc_enqueue(struct rq *rq, stru= ct task_struct *p) =20 rq->nr_llc_running++; rq->nr_pref_llc_running +=3D (pref_llc =3D=3D task_llc(p)); + + sd =3D rcu_dereference_all(rq->sd); + if (sd && (unsigned int)pref_llc < sd->llc_max) + sd->llc_counts[pref_llc]++; } =20 static void account_llc_dequeue(struct rq *rq, struct task_struct *p) { + struct sched_domain *sd; int pref_llc; =20 pref_llc =3D p->preferred_llc; @@ -1406,6 +1412,24 @@ static void account_llc_dequeue(struct rq *rq, struc= t task_struct *p) =20 rq->nr_llc_running--; rq->nr_pref_llc_running -=3D (pref_llc =3D=3D task_llc(p)); + + sd =3D rcu_dereference_all(rq->sd); + if (sd && (unsigned int)pref_llc < sd->llc_max) { + /* + * There is a race condition between dequeue + * and CPU hotplug. After a task has been enqueued + * on CPUx, a CPU hotplug event occurs, and all online + * CPUs (including CPUx) rebuild their sched_domains + * and reset statistics to zero(including sd->llc_counts). + * This can cause temporary undercount and we have to + * check for such underflow in sd->llc_counts. + * + * This undercount is temporary and accurate accounting + * will resume once the rq has a chance to be idle. + */ + if (sd->llc_counts[pref_llc]) + sd->llc_counts[pref_llc]--; + } } =20 void mm_init_sched(struct mm_struct *mm,