From nobody Mon May 25 00:08:59 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 986693D7D6E; Wed, 20 May 2026 08:34:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779266073; cv=none; b=WT3vk6NnPMY3bCkV5/yDp+DT2E56DK1Hsm9Rcngyhfc/zJJpy9sDg/FhVtS223YOC9KquSY/o7wFHTPvVyniXWwU451rrM/WBxvmruLjXDeiLVtoGuEeokQCsL3L80nUNjPg4RJLdf9RerSnnDtgWImU/HEucWEh80rzJYK7RHU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779266073; c=relaxed/simple; bh=53VG2pIKqqKz/pj2lwgKNAQpXCpe1ybE8RoJLcAqP+E=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=am5OWV0D9cwFPpnxVbToBGk3Pti5c/8uZggAnBu6J5/FtAU67TRFrWGD+rnVNq71aO8H+Bdm56irxn9Ra1Lo5rt4gDbJS76tkDYtmqH0WUFrrfG0mYOVsLMrXEdFYDXhMBfrtQ36BDQ48EoKRyZe3P2veEX1bElA3ggGMA03Azw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=g1WTy5NL; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=ZigWgkN8; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="g1WTy5NL"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="ZigWgkN8" Date: Wed, 20 May 2026 08:34:28 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1779266070; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZSw/x3Ag0SFQIjhXQxHr4+DZSRfWZ429j8/ecNzoTaI=; b=g1WTy5NLRTbH9N82b7zXMnQkmpFrV+FRPPmNQyxRc7QRTsYCWJQovmNi8n71CsVxiNo861 HV4/qbFfVve188bnuGXMOTzt494UPtU/+p7Fj/hmU69VxL7cTrQjnrPi5gE7BixdsgG1Yu qyam2ypqvjtJRoEi9/9vfsGTGR3DK+9tlVfpsmMsEJlwazgNs10KWnBjgmlJAYZ7vKc6Lb NbbSeF1g43Y87wG2A69buhQK6BLwLICpTDQkKXJUGxCELTy011glzU5Fm1L1Mpi1Tdq1fX kW5mbd+UhebXi/FhrUIrYcZnQuq4OIt/b0ZXKL4A0OZTWf6UJBsVB532pKd1ig== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1779266070; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZSw/x3Ag0SFQIjhXQxHr4+DZSRfWZ429j8/ecNzoTaI=; b=ZigWgkN8ZJ6uui+jSzQQ8jHGF1p59/IoEzmV3VbJZ1aPW8tD30CDleY2u6cjJEVkbSv6l0 JSrpzgwwP6lafoAg== From: "tip-bot2 for Chen Yu" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/core] sched/cache: Fix unpaired account_llc_enqueue/dequeue Cc: K Prateek Nayak , Chen Yu , Tim Chen , "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: =?utf-8?q?=3C0c8c6a1571d66792a4d2ff0103ba3cc13e059046=2E1778703?= =?utf-8?q?694=2Egit=2Etim=2Ec=2Echen=40linux=2Eintel=2Ecom=3E?= References: =?utf-8?q?=3C0c8c6a1571d66792a4d2ff0103ba3cc13e059046=2E17787036?= =?utf-8?q?94=2Egit=2Etim=2Ec=2Echen=40linux=2Eintel=2Ecom=3E?= Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <177926606871.711.7939301409479648499.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the sched/core branch of tip: Commit-ID: 03755348b8e74421f92ffed9da159175a698290b Gitweb: https://git.kernel.org/tip/03755348b8e74421f92ffed9da159175a= 698290b Author: Chen Yu AuthorDate: Wed, 13 May 2026 13:39:21 -07:00 Committer: Peter Zijlstra CommitterDate: Mon, 18 May 2026 21:33:17 +02:00 sched/cache: Fix unpaired account_llc_enqueue/dequeue There is a race condition that, after a task is enqueued on a runqueue, task_llc(p) may change due to CPU hotplug, because the llc_id is dynamically allocated and adjusted at runtime. Therefore, checking task_llc(p) to determine whether the task is being dequeued from its preferred LLC is unreliable and can cause inconsistent values. To fix this problem, record whether p is enqueued on its preferred LLC, in order to pair with account_llc_dequeue() to maintain a consistent nr_pref_llc_running per runqueue. This bug was reported by sashiko, and the solution was once suggested by Prateek. Fixes: 46afe3af7ead ("sched/cache: Track LLC-preferred tasks per runqueue") Suggested-by: K Prateek Nayak Signed-off-by: Chen Yu Co-developed-by: Tim Chen Signed-off-by: Tim Chen Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/0c8c6a1571d66792a4d2ff0103ba3cc13e059046.177= 8703694.git.tim.c.chen@linux.intel.com --- include/linux/sched.h | 2 ++ init/init_task.c | 1 + kernel/sched/fair.c | 31 ++++++++++++++++++++++++++++--- 3 files changed, 31 insertions(+), 3 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 9572967..2c9e8e2 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1410,6 +1410,8 @@ struct task_struct { #ifdef CONFIG_SCHED_CACHE struct callback_head cache_work; int preferred_llc; + /* 1: task was enqueued to its preferred LLC, 0 otherwise */ + int pref_llc_queued; #endif =20 struct rseq_data rseq; diff --git a/init/init_task.c b/init/init_task.c index 5d90db4..3ecd66f 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -217,6 +217,7 @@ struct task_struct init_task __aligned(L1_CACHE_BYTES) = =3D { #endif #ifdef CONFIG_SCHED_CACHE .preferred_llc =3D -1, + .pref_llc_queued =3D 0, #endif #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) .kasan_depth =3D 1, diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 087445e..96c61ce 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1472,15 +1472,32 @@ static bool invalid_llc_nr(struct mm_struct *mm, st= ruct task_struct *p, =20 static void account_llc_enqueue(struct rq *rq, struct task_struct *p) { + int pref_llc, pref_llc_queued; struct sched_domain *sd; - int pref_llc; =20 pref_llc =3D p->preferred_llc; if (pref_llc < 0) return; =20 + pref_llc_queued =3D (pref_llc =3D=3D task_llc(p)); rq->nr_llc_running++; - rq->nr_pref_llc_running +=3D (pref_llc =3D=3D task_llc(p)); + rq->nr_pref_llc_running +=3D pref_llc_queued; + + /* + * Record whether p is enqueued on its preferred + * LLC, in order to pair with account_llc_dequeue() + * to maintain a consistent nr_pref_llc_running per + * runqueue. + * This is necessary because a race condition exists: + * after a task is enqueued on a runqueue, task_llc(p) + * may change due to CPU hotplug. Therefore, checking + * task_llc(p) to determine whether the task is being + * dequeued from its preferred LLC is unreliable and + * can cause inconsistent values - checking the + * p->pref_llc_queued in account_llc_dequeue() would + * be reliable. + */ + p->pref_llc_queued =3D pref_llc_queued; =20 sd =3D rcu_dereference_all(rq->sd); if (sd && (unsigned int)pref_llc < sd->llc_max) @@ -1497,7 +1514,15 @@ static void account_llc_dequeue(struct rq *rq, struc= t task_struct *p) return; =20 rq->nr_llc_running--; - rq->nr_pref_llc_running -=3D (pref_llc =3D=3D task_llc(p)); + if (p->pref_llc_queued) { + rq->nr_pref_llc_running--; + /* + * Update the status in case + * other logic might query + * this. + */ + p->pref_llc_queued =3D 0; + } =20 sd =3D rcu_dereference_all(rq->sd); if (sd && (unsigned int)pref_llc < sd->llc_max) {