From nobody Tue Sep 29 06:07:42 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3EDBA37E5C1; Wed, 12 Aug 2026 05:41:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513281; cv=none; b=RQXH6f2V0sJgknkFbzdhwPgfoAtU79C6UIBbRIM0UbmY6ldiGl5b0nPy6BulX8XCSaOaf2MFHfqldiT7Vr8CBuaHRA/dq+dCjDH0SV6ARsaOdK+AaD9XWimE6HGhnpY0GPJE5k5C9GK19+GlG2Raz2YXP3A9hxg2EYNeHMdDgnE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513281; c=relaxed/simple; bh=6r9992HIygzYISTW/2+VEWQl2b2N806ylXZtH7gKNJY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y+WWnVnv+LEgAeXamPfYU8CKHGarNR0l/2QCDuWhxIP6oH2b6utAyCK1XO7To3CZUaK5K/K/vlQ+UDxEvrF8VzLTb44pxRGq1+7nQtVQgzuSAQlmlp8g0uHwkswP6y0wg52u4E4kz45ldQMAos25szaEU2YW5ytM/uV2oNGgGDE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=qyu02fi1; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="qyu02fi1" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31lOg2313688; Wed, 12 Aug 2026 05:40:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=NYIKiFngYTGj6eldO JLN6PCfJ/iqmxUsus4gz85gXXw=; b=qyu02fi17ifUVjKHre6vr4l68BNVl2u5U DH5HkMCoj/jxw/7xzjUvGBNEOvABCtlZA7cG2hiwsRScCyngFBPBZT6WZ3m1vWz1 G2lrQ5538yYEz00+oedvmKCcHWo8c3W+iR5iH872VD8ZCruhPUd1N3alPhepB3V6 Dmc9WKDhPKWs8kzp6fcVLSB0RngNWIxtfoZlnmQOVhAUwFZ3LQnx2b9vFnmqm5g2 Q2fmxNgGudG7Vq1F1zSJK6t2yU3i7WDbGzTINXQSqedb7c4OHr1GPhTD2L7mqrrM EJiX/uwIiYJ8vlx8i4MHbB3ybBLlRHGe7NQDWGp4JaW8gMHP9Cblw== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvnw87qp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:40:58 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5QMkQ023465; Wed, 12 Aug 2026 05:40:57 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4fxhfy4fmh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:40:57 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5er9V50135422 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:40:53 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A749A2004B; Wed, 12 Aug 2026 05:40:53 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AE65C20040; Wed, 12 Aug 2026 05:40:44 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:40:44 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 01/12] sched/cputime: Add kcpustat_field_total helper Date: Wed, 12 Aug 2026 11:10:22 +0530 Message-ID: <20260812054033.95658-2-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Authority-Analysis: v=2.4 cv=RsP16imK c=1 sm=1 tr=0 ts=6a7c076b cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=e8TCS894uZmGZegT9CcA:9 X-Proofpoint-GUID: YNhKPf_ONA_FuoRHQd0FqK9bIBBrMiP7 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX9mEHCmAOQ7kv PSmzcan80mNPn3KgvGuXcTgazU+7R2pKkIrsA2aSY0DIpB6Xww0A1dT7CivVBsjnKqJRzQHSuzG jM2QibXO9cnKb86C1HN47EVKfxRTyOoknwVKBlL32DK3IQqy8aqL4Z55Q5Yjzx04Vb0RbtTU5wA VxaRpmiZY4KI73XQAptK8z37o8gynYTRPhJVcTmMDxPkDRxWqqJFKyWO+6Q4e3fnlKoVr4s/nxf 5393IOdZywLtxwCl24JVeiT82vTNozeUht3705T8Fm47p49Uu6RcJksZ3iIccLkTX48H0tqrq34 Q+KN75CTlAnZrXW+ZL/OtnNdnvJbMgN+nLM+lt0B9kbzKiveM86OKotHQ7LmXJkuvu4+l6LU89+ zllcnJ7AAU7GhbHdnuCUn8+JWEJ+oFx1HSJdPBhJI88T7UUw5/Tbxr04GtMTC5H5f82Hm9yfgHN 7j9ncG+bp6YE5JhoTdQ== X-Proofpoint-ORIG-GUID: HiowgnlEHKyexb8VOMNZrBNEIGDntEVu X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfXyHzRZDM3ngPc SMxMQY5MWQkMxcw8/yHDkqQi/840tzzqp8Il0M/ROMIVEQrFU2lPWs0pFphztLgdoGJTOHOa/Kz chhLh2RI2mpEawmQQOIZJ5GFZdaCOTM= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0 suspectscore=0 clxscore=1015 malwarescore=0 phishscore=0 adultscore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" Provide a new helper function which sums up a given type of cpustat over a specified cpumask. This allows the caller's code to be simpler and avoids duplication. For example, subsequent patch in the steal governor use this exact same pattern when calculating steal time. Suggested-by: Yury Norov Signed-off-by: Shrikanth Hegde Reviewed-by: Mete Durlu Reviewed-by: Yury Norov --- arch/s390/kernel/hiperdispatch.c | 8 ++------ fs/proc/uptime.c | 6 +----- include/linux/kernel_stat.h | 11 +++++++++++ 3 files changed, 14 insertions(+), 11 deletions(-) diff --git a/arch/s390/kernel/hiperdispatch.c b/arch/s390/kernel/hiperdispa= tch.c index 217206522266..e5c7c818c178 100644 --- a/arch/s390/kernel/hiperdispatch.c +++ b/arch/s390/kernel/hiperdispatch.c @@ -210,13 +210,9 @@ static unsigned long hd_calculate_steal_percentage(voi= d) int cpus, cpu; ktime_t now; =20 - cpus =3D 0; - steal =3D 0; percentage =3D 0; - for_each_cpu(cpu, &hd_vmvl_cpumask) { - steal +=3D kcpustat_cpu(cpu).cpustat[CPUTIME_STEAL]; - cpus++; - } + steal =3D kcpustat_field_total(CPUTIME_STEAL, &hd_vmvl_cpumask); + cpus =3D cpumask_weight(&hd_vmvl_cpumask); /* * If there is no vertical medium and low CPUs steal time * is 0 as vertical high CPUs shouldn't experience steal time. diff --git a/fs/proc/uptime.c b/fs/proc/uptime.c index 433aa947cd57..53143c66cbe1 100644 --- a/fs/proc/uptime.c +++ b/fs/proc/uptime.c @@ -15,12 +15,8 @@ static int uptime_proc_show(struct seq_file *m, void *v) struct timespec64 idle; u64 idle_nsec; u32 rem; - int i; - - idle_nsec =3D 0; - for_each_possible_cpu(i) - idle_nsec +=3D kcpustat_field(CPUTIME_IDLE, i); =20 + idle_nsec =3D kcpustat_field_total(CPUTIME_IDLE, cpu_possible_mask); ktime_get_boottime_ts64(&uptime); timens_add_boottime(&uptime); =20 diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h index 9ca6c2259dfe..c1e85550bf12 100644 --- a/include/linux/kernel_stat.h +++ b/include/linux/kernel_stat.h @@ -196,6 +196,17 @@ static inline void kcpustat_cpu_fetch(struct kernel_cp= ustat *dst, int cpu) } #endif /* !CONFIG_VIRT_CPU_ACCOUNTING_GEN */ =20 +static inline u64 kcpustat_field_total(enum cpu_usage_stat usage, const st= ruct cpumask *cpus) +{ + u64 total =3D 0; + int cpu; + + for_each_cpu(cpu, cpus) + total +=3D kcpustat_field(usage, cpu); + + return total; +} + extern void account_user_time(struct task_struct *, u64); extern void account_guest_time(struct task_struct *, u64); extern void account_system_time(struct task_struct *, int, u64); --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE17537F01C; Wed, 12 Aug 2026 05:41:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513297; cv=none; b=RejqxMl2PZ3HbR0yZeIAxVo7ngbGUelqUPtiYiDVb40ilRIG0w+cKRKaQrYFkSQaMZB9inuOLIDpZBEwAoo1v2C/PamHr1HsjKPgkOfY5g4HMOyARISzYB8ORbbpCWzw0MtGV/b3pLpTqxINHulpj9ZmM38h5Uj9wv74vBk4nCU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513297; c=relaxed/simple; bh=qrEJ/vOMRn+hJwEzFw50NZd1Piogma6ZrdDkk7k13bA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OBu6mnU8/zcHwwCS/ja/mZ4k7MXcYJPuRBjeWlUdAQmW+63GaSKNS/kF0UsZd6+PB2xmT7UMeH2hotmQkqHHg79I8pjesCBMVd/0NsbYim2WKuOWm+00Ao6CxFJWFkHDQ0p4X2ivRrRJmSaNK5fWSnVAs7ZtipY/WlTz6NSRJz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=OEynovt6; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="OEynovt6" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31Vsp416373; Wed, 12 Aug 2026 05:41:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=9xJKG7CyiFCg/Eo9f ElnKaEIAlsTOizSYleCtLkAlqA=; b=OEynovt6uDylo3Zz6tn5o+SvNr+DMLLFb GMa5lQItHxRR6AATlzITgaINbYw7Gja/R/HAM93WypRnYJPa89WQpMcgr5MkjwBH ylcZfNJJIyocc8vsvGEBPxmr2dfVlPrni1wOkLzu/zk8fEARy7ekr+XKm2MNuvkl ifRWzWmqseL3WmFhkRbKJDEDYJ5e6Ayn9sOXCfADu55YKHXhWdM+XP065BdQQrSz 56KdbDk9oPHJGVD9qVhSi32MSGQdOcfXNOzVhmmL5VN/tjMAYVBTdnzVazrYYGgy uOBKSr4jcZAkZndh4sXwewI6ABcfkMSQ6QZaUAWEIlW9z2xE/mxFA== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvk00yj4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:07 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5QKUu005693; Wed, 12 Aug 2026 05:41:06 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4fxesq4uh2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:06 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5f2Sh37486982 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:41:02 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A0FDB20040; Wed, 12 Aug 2026 05:41:02 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1D71D2004B; Wed, 12 Aug 2026 05:40:54 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:40:53 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 02/12] sched/docs: Document cpu_preferred_mask and Preferred CPU concept Date: Wed, 12 Aug 2026 11:10:23 +0530 Message-ID: <20260812054033.95658-3-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfXxfGfyzu/inb8 fgqaT1TzcddeymYyDEeaES6YlfYd2ZdlU8fpOQi8SdD2y7MivmEQ5zMAz61vBmRMx9NALCBGpfO cFcBKYEzHt6N0p6phwYqVfNoWTcBDTdDLRkcbyuKbR/uAlQd5HaDDL3h0UaHCI0Ku2Mk4K3ua2x 0lo/rrDUqu9ykEiS31HOwjASJNgoB78ary4kyOiKbIK+w3rD5nNa6ITOE+B0GIEI6rIvHOF20EZ 8mIn7ZHljsq1gpDDpv7CJWlFFrqIP9aZbY19MSxw4aRQUR2R0uuHx8G0YLiWhIbh5yrwieiE7G3 fau935e5d49ZgSgTXmszRVeGQ8EEmnBlAzfKlphXZMYaPXbIaiolWhEkhWB5SYwovIBFKotkDXs 6eiI5dUahzupfcB2vXnBPrzXV1k+Q/0E4G0xNLyufTBINkuS19aNhsw7x49n/No9MAS5E+Juqkh rPIF/4JNhd0OKwPVKGg== X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX8TK9SbAxEf+t slhjc/sDketsRLepVpxtnHx88dwAosdHC0IT7ejDXC/Lu3PjrltIemikAk0mFIs0dKoP8ooVr7d q5YQY7WpkQ+hk31LSlkrqf8+DCQFvJU= X-Authority-Analysis: v=2.4 cv=RqD16imK c=1 sm=1 tr=0 ts=6a7c0774 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=2BnLtqgf9UOq1ijo76cA:9 X-Proofpoint-GUID: 3t4MCixTYo0ByHz4NIVWokOIKs_lSnjj X-Proofpoint-ORIG-GUID: c-FCX5a4dEmjxI3JI1yYfp4aCdQN44Uv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 suspectscore=0 lowpriorityscore=0 clxscore=1015 adultscore=0 bulkscore=0 malwarescore=0 impostorscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" Add documentation for new CPU state called preferred CPU state and corresponding cpumask called cpu_preferred_mask. This could help users in understanding what it is and how to use it. Document the role of scheduler and driver for this feature to work. Details regarding the driver documentation will be added in later patches under Documentation/driver-api/steal-governor.rst. Newly added file could be used for other paravirt usecase documentation. Signed-off-by: Shrikanth Hegde --- Documentation/scheduler/index.rst | 1 + Documentation/scheduler/sched-paravirt.rst | 67 ++++++++++++++++++++++ 2 files changed, 68 insertions(+) create mode 100644 Documentation/scheduler/sched-paravirt.rst diff --git a/Documentation/scheduler/index.rst b/Documentation/scheduler/in= dex.rst index 17ce8d76befc..a43647b9706d 100644 --- a/Documentation/scheduler/index.rst +++ b/Documentation/scheduler/index.rst @@ -23,5 +23,6 @@ Scheduler sched-stats sched-ext sched-debug + sched-paravirt =20 text_files diff --git a/Documentation/scheduler/sched-paravirt.rst b/Documentation/sch= eduler/sched-paravirt.rst new file mode 100644 index 000000000000..3f06294714e6 --- /dev/null +++ b/Documentation/scheduler/sched-paravirt.rst @@ -0,0 +1,67 @@ +.. SPDX-License-Identifier: GPL-2.0 +.. _sched-paravirt: + +Preferred CPUs +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +In paravirtualized environments CPU overcommit is a common scenario. +i.e. the sum of virtual CPUs (vCPUs) of all VMs is greater than number of +physical CPUs (pCPUs). Under such conditions when all or many VMs have +high utilization, hypervisor won't be able to satisfy the CPU requirement +and has to context switch within or across VMs. The hypervisor needs to +preempt one vCPU to run another. This is called vCPU preemption. +This is more expensive compared to task context switch within a vCPU, since +hypervisor lacks vCPU context and could preempt a critical section which +slows forward progress. + +In such cases it is better that combined vCPU demand from all VMs is reduc= ed +by not using some of the vCPUs in each VM. vCPUs where workload can be saf= ely +scheduled which won't increase any contention for pCPU are called +"Preferred CPUs". + +One of the main design constructs is that preferred CPUs are always +a subset of active CPUs. In most cases preferred CPUs will be same as +active CPUs. When there is pCPU contention, Preferred CPUs will reduce +based on the steal time. When the pCPU contention goes away as indicated +by steal time, Preferred CPUs could become same as active CPUs again. +The policy decisions are to be taken by driver. +For example, steal_governor. Look at its documentation for more +details. (``drivers/virt/steal_governor.c``) + +Scheduling decisions such as wakeup, pushing the task etc, need this +CPU state info. This is maintained in ``cpu_preferred_mask``. +vCPUs which are not in ``cpu_preferred_mask`` should be treated as vCPUs w= hich +should not be used at this moment provided it doesn't break user affinity. + +This is achieved by: + +1. Selecting a preferred CPU at wakeup using fallback mechanism. +2. Pushing the task away from non-preferred CPU at tick. +3. Selecting only preferred CPUs for load balance. + +``/sys/devices/system/cpu/preferred`` prints the current ``cpu_preferred_m= ask`` +in cpulist format. + +Notes: + +1. This feature is available under ``CONFIG_PREFERRED_CPU``. Driver which + makes decisions should enable it. For example, steal_governor driver + (``CONFIG_STEAL_GOVERNOR``). On enabling the driver, CPU preferred state + can change based on steal time. Without the driver, preferred CPUs is + same as active CPUs. + +2. This feature works for the FAIR class only. + +3. A pinned task, which can't be moved to preferred CPUs will continue + to run based on its affinity. But no load balancing happens if it is af= fined + only on non-preferred CPUs. + +4. Decision to change the preferred CPU state is driven by the kernel. + Hence it shouldn't break user affinities. One of the main reasons why + CPU hotplug or Isolated cpuset partitions was not a solution. + +5. This feature works best only when all the VMs enable the feature as + it is a co-operative scheme. If a specific VM doesn't enable this featu= re + it may end up with more CPUs than others, still should lead to better + performance when seen from system view. + Users who enable this driver must ensure it is enabled in all VMs. --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 026563905E6; Wed, 12 Aug 2026 05:41:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513299; cv=none; b=Tfh3ACez/zeeBS2COOily6dIBaS50qNVty/0bWuOF800C8bZyETgkqsXVxzSq1sjvdnU30RxiSZ7jRSai92s1trOcHdwRsSvhwChdRJQF3krLQZ2Of1cY4vENZkbR30cwn50E9fpgmjGXQehfyi/TTtKrl6gPfA52T/LVRhbQnY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513299; c=relaxed/simple; bh=NHxNTsuQ5kYaOU6wiQ3qsu9Kigs6uHpexu2ofPPXkNM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qswCp3ROoCTtgXMcirnE7xhyoJGBgV8QolTsUCXZjbGUdSYnaLltSsBHuLnJrrV5vjBhQj8bPQc+/yQrCSlevqpUF7RGfp4wtR7U6JybLVJNR3tWNzkGMJKZQ7fnAyEFSe2pSO74qomVs54Etkl9WtUf0nUzn2rVv1HIEG0TdBQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=np+XEFKr; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="np+XEFKr" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31dJD2313519; Wed, 12 Aug 2026 05:41:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=CjMua/ sfQJspq5hiL4ZeydI1nOj4VlT3LwXZv+NrykY=; b=np+XEFKrXs6CI/8Jwq8km3 d5L1g4dFdyRAjD9wq4ycURIjLq1mi9PQ+5Qmspw6nQs/YxXKGZGpCpT339AoKzFt uzRxBG+cLy0Kuc6yJlsxMoWXXYv0KPQKGBJKj4MG7j67UgzIe7a7L4SSf4P+SyEE FIMHYUJt6ERQ/vmN/ij6EacN14oudv2Y5eU73L20L3IEzrPNsW4fzAUHkm0+s+R2 gS7eWA+661CLvt76WKQcLIufXSQVs1qrDccBb6kQKsfKs6Zb7ZZ/lUSOTyGTJvHQ ZWUu9bnhC7SR13quB6xx9iL6Ou7fAcodDx0L94OARA+BAluO4xjp2pVMYJnI450g == Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvnw87rd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:16 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5BPRR023149; Wed, 12 Aug 2026 05:41:15 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4fxf5w4t16-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:15 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5fBCx34668908 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:41:11 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1609720043; Wed, 12 Aug 2026 05:41:11 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0FD8820040; Wed, 12 Aug 2026 05:41:03 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:41:02 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 03/12] cpumask: Introduce cpu_preferred_mask Date: Wed, 12 Aug 2026 11:10:24 +0530 Message-ID: <20260812054033.95658-4-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Authority-Analysis: v=2.4 cv=RsP16imK c=1 sm=1 tr=0 ts=6a7c077c cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=IkcTkHD0fZMA:10 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=NEAV23lmAAAA:8 a=VnNF1IyMAAAA:8 a=9QGdkmv71c8RgJbqbJ4A:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-GUID: 51TYTiDjmkoquLA6_SbG3yuUsJfXh3iQ X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX5Pm3oVXdzO35 qThQqZKKtxXVBnVvdfn/RQwZ9GPvbJt0WzftWVojrerC0N++PttyUJka7dJhWH85nUum8NV3T1S oc9g+rBgT7VB+VEU3oRPmuSaSjHL29xiKYvDdOat2Hifu7PZv5/mTPvwkRUsW8fiIBGhvQa5bJ/ vKahxRh0aXyLK1uI1naub/ZVs1mZRsOS1fqn0qrX6vHNisG0Bjl/FSdHaT4wz/hgmhUtLVh+5Vm VCnyBCvJ6cwZAaYMqi8J4N9HqtWY9IH4LGTqistDP+J6J8XhpMW/nhErWk0h+Zde00yw8yBrPDL Z3X3M+bHnGqrftaL844QwkQQhSna2323YBbRsTyn9VQa6pEbLNyomMbE0ip5W9OLmM3ehieiBA5 7fQcPJ/jS/nWxKgM4A4KRrppkcLH04gZElP+j6o3bR0UKGFMNaVUGgGj6qa/LxrEv95aEDQAbq2 iRR/RV8h1fDnGsrccZQ== X-Proofpoint-ORIG-GUID: jmD1EhkEQykSZwhGL-8AnHxc52pYxi7T X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfXz2VJoHBzbYN1 p0R4C484tTi1STGowbHOhe5tTLnerxItNDgcZEXaWffmaX7oX75xRH/vBVLRE3ysHIQaTbuvmD0 UHZcDK/9+9+K4zJrle8vpUR2VQTAjZ8= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0 suspectscore=0 clxscore=1015 malwarescore=0 phishscore=0 adultscore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Provide the preferred CPU infrastructure. Define get/set macros which could be used to get/set CPU state as preferred. CONFIG_PREFERRED_CPU will be selected by the driver which handles steal time values. It is going to set/clear preferred CPU state. This driver will be called steal_governor and it is introduced in subsequent patches. It periodically computes the steal ratio and decides on preferred CPU state. A CPU is set to preferred when it becomes active. Later it may be marked as non-preferred depending on steal ratio by the steal_governor. Always maintain design construct of preferred is subset of active. i.e. preferred =E2=8A=86 active =E2=8A=86 online =E2=8A=86 present =E2=8A= =86 possible With CONFIG_PREFERRED_CPU=3Dn, ensure set_cpu_preferred is a nop and get method returns the active state in that case. Signed-off-by: Shrikanth Hegde --- include/linux/cpumask.h | 24 ++++++++++++++++++++++++ kernel/Kconfig.preempt | 4 ++++ kernel/cpu.c | 6 ++++++ kernel/sched/core.c | 5 +++++ 4 files changed, 39 insertions(+) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index d3cda0544954..34d08a3d80e1 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -122,12 +122,20 @@ extern struct cpumask __cpu_enabled_mask; extern struct cpumask __cpu_present_mask; extern struct cpumask __cpu_active_mask; extern struct cpumask __cpu_dying_mask; + +#ifdef CONFIG_PREFERRED_CPU +extern struct cpumask __cpu_preferred_mask; +#else +#define __cpu_preferred_mask __cpu_active_mask +#endif + #define cpu_possible_mask ((const struct cpumask *)&__cpu_possible_mask) #define cpu_online_mask ((const struct cpumask *)&__cpu_online_mask) #define cpu_enabled_mask ((const struct cpumask *)&__cpu_enabled_mask) #define cpu_present_mask ((const struct cpumask *)&__cpu_present_mask) #define cpu_active_mask ((const struct cpumask *)&__cpu_active_mask) #define cpu_dying_mask ((const struct cpumask *)&__cpu_dying_mask) +#define cpu_preferred_mask ((const struct cpumask *)&__cpu_preferred_mask) =20 extern atomic_t __num_online_cpus; extern unsigned int __num_possible_cpus; @@ -1164,6 +1172,12 @@ void init_cpu_possible(const struct cpumask *src); #define set_cpu_active(cpu, active) assign_cpu((cpu), &__cpu_active_mask, = (active)) #define set_cpu_dying(cpu, dying) assign_cpu((cpu), &__cpu_dying_mask, (dy= ing)) =20 +#ifdef CONFIG_PREFERRED_CPU +#define set_cpu_preferred(cpu, preferred) assign_cpu((cpu), &__cpu_preferr= ed_mask, (preferred)) +#else +#define set_cpu_preferred(cpu, preferred) do { } while (0) +#endif + void set_cpu_online(unsigned int cpu, bool online); void set_cpu_possible(unsigned int cpu, bool possible); =20 @@ -1258,6 +1272,11 @@ static __always_inline bool cpu_dying(unsigned int c= pu) return cpumask_test_cpu(cpu, cpu_dying_mask); } =20 +static __always_inline bool cpu_preferred(unsigned int cpu) +{ + return cpumask_test_cpu(cpu, cpu_preferred_mask); +} + #else =20 #define num_online_cpus() 1U @@ -1296,6 +1315,11 @@ static __always_inline bool cpu_dying(unsigned int c= pu) return false; } =20 +static __always_inline bool cpu_preferred(unsigned int cpu) +{ + return cpu =3D=3D 0; +} + #endif /* NR_CPUS > 1 */ =20 #define cpu_is_offline(cpu) unlikely(!cpu_online(cpu)) diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt index 88c594c6d7fc..de789b274ba3 100644 --- a/kernel/Kconfig.preempt +++ b/kernel/Kconfig.preempt @@ -192,3 +192,7 @@ config SCHED_CLASS_EXT For more information: Documentation/scheduler/sched-ext.rst https://github.com/sched-ext/scx + +config PREFERRED_CPU + bool + depends on SMP && PARAVIRT diff --git a/kernel/cpu.c b/kernel/cpu.c index b3c8553d7bd6..376d297a6292 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -3103,6 +3103,11 @@ EXPORT_SYMBOL(__cpu_dying_mask); atomic_t __num_online_cpus __read_mostly; EXPORT_SYMBOL(__num_online_cpus); =20 +#ifdef CONFIG_PREFERRED_CPU +struct cpumask __cpu_preferred_mask __read_mostly; +EXPORT_SYMBOL_GPL(__cpu_preferred_mask); +#endif + void init_cpu_present(const struct cpumask *src) { cpumask_copy(&__cpu_present_mask, src); @@ -3160,6 +3165,7 @@ void __init boot_cpu_init(void) /* Mark the boot cpu "present", "online" etc for SMP and UP case */ set_cpu_online(cpu, true); set_cpu_active(cpu, true); + set_cpu_preferred(cpu, true); set_cpu_present(cpu, true); set_cpu_possible(cpu, true); =20 diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2e7cde033a31..a45f7c308329 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8690,6 +8690,9 @@ int sched_cpu_activate(unsigned int cpu) */ sched_set_rq_online(rq, cpu); =20 + /* preferred is subset of active and follows its state */ + set_cpu_preferred(cpu, true); + return 0; } =20 @@ -8703,6 +8706,8 @@ int sched_cpu_deactivate(unsigned int cpu) if (ret) return ret; =20 + set_cpu_preferred(cpu, false); + /* * Remove CPU from nohz.idle_cpus_mask to prevent participating in * load balancing when not active --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 096E93845D5; Wed, 12 Aug 2026 05:41:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513310; cv=none; b=l0sJCTki1Sike7bvE39yDhT6kv5GNS/pyf/AvX7VrbLrdR7jMQx5+V3DK+8r51zDgMfieZP6Y/Zzc39D86dz2XOdhZmDHorDN+eH5wkvVVQSJEUhNn6CoqBOfgy5Q216pbsQ44JIe0GD4L2BYElbAvK0WlhnUY/ZAFlEilpJ/QI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513310; c=relaxed/simple; bh=TDsgIO48g+8UYi8pPB4M66fYIvYKuUVdrZMjt9LJrYw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rtfjZm8kOK3BeBljFXSmY8wYNLVsNYJAWDgIYP2BZdAkMxjfW7/XQMB6/zL0guBhpiAeB6WZ1GEUiJDgsovER7ZDTlFL9Nga718HIaLkv6A12EopvhZgrueTfyi1G7anusrl7aDY/hgEvMbippdIzTGyNmW3Tlq8ERVPHfM8EFs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Aakdkec+; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Aakdkec+" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31qaO2196318; Wed, 12 Aug 2026 05:41:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=IdIQVMSRh1tQgR1Q2 N7Opt2bZYabyV8zrFakwyWbq7c=; b=Aakdkec+6/XV4RR7DK7nAiVKxLxsOfE/y JS+GLg2nvK8Id1u4576x98BAoPQtU4/IZ8jGk3b3mLdwKci/tLeVS4u10fNG/DTb fzZVlZ12VK9kefVhir+XtNnZucyYGzGhnLlPGCQlKOCRGtr1+tcKWglvV3bhMP+p EhJxJiYVTq897w8asuvReUGuxwb4Ar9Osl0lEUhguQICkiZaAWCDfoVqlNpdaP5h 56fLLJPdEI62E4PFXhOUsWfAyuFnQ4g0TK3zXThWXF9UF5NIIrCnoxvXVKDEJ16J B9o4f6QWQq6/CRNdflzi+l2hsADnRIHldQKdhSVs2+pzZb2c7TY4Q== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvp308a4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:24 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5fGRi022468; Wed, 12 Aug 2026 05:41:23 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4fxesq4uj5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:23 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5fKJD38207816 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:41:20 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 033B120043; Wed, 12 Aug 2026 05:41:20 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 820DE20040; Wed, 12 Aug 2026 05:41:11 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:41:11 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 04/12] sysfs: Add preferred CPU file Date: Wed, 12 Aug 2026 11:10:25 +0530 Message-ID: <20260812054033.95658-5-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Authority-Analysis: v=2.4 cv=AMtp2X5w c=1 sm=1 tr=0 ts=6a7c0785 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=VwQbUJbxAAAA:8 a=94fVmjRkuoLjN79_KVgA:9 X-Proofpoint-GUID: BOaivE1vu7VPLZvSOs8ftXtZTlbaeij- X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX2ScjYVITBIGW ePJ0pbPsI7Ku4uaM6t20mVwXzrveq6rNqWo8+5YWRcqGoNMPixP3C3uuvaljPLlZR3KphoN+ieu 157WqguRlGGcXumwstIhcXBANRZwnkvRnIA8b4UdhJFdzA2NoY4IdCuwQemT/LpazX7yTtCsP1j 4XdudSzFqY/F/283f+zrygsb+1N88xsTfovFd9E1AO9G101/GK/HfP3U7nTIgmcCHLmTf/ageok mk7zXCdJQ+IYVaCdVlyS0xoz5SeAod1qTkfWd8bY/S7S75UpYAWYlDL8gDQ3JCzgNcmkKCir2Bb irW/6Kc6SLy0qFScWJtdGBBnBTmofQ0zrYUln2yc1/SHk3Co+NLIjpmUUmfVoGSc/U3OmMnGHaY MSBGaoweW/fi/tTA5mBUoDZ/ou3iSKNb6IXVYUhLVwhEVPhKHRUMvvfODLED8V+Rnw2iFnTW7Uf 6rWGAJyDL516zmQetbA== X-Proofpoint-ORIG-GUID: 3YsrwT2rmgUJ8fPXLjU7JF_GUO1FyY7s X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfXymW7kKc2JnzJ Sb8LhktD45iGwJecingMZEn/HQc0NXS3fjOKyt8w4utDgEtuMtsJlP0z4WFTea0D1IK2s4ds/GJ KfUHITtvjsDyJZYQym25eNcNsoiTFQU= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 priorityscore=1501 impostorscore=0 phishscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" Add a "preferred" file in /sys/devices/system/cpu/ when kernel is built with CONFIG_PREFERRED_CPU=3Dy. This would help - Users to quickly check which CPUs are marked as preferred. - Userspace daemons such as irqbalance to use this mask to route irqs into preferred CPUs. For example: cat /sys/devices/system/cpu/online 0-719 cat /sys/devices/system/cpu/preferred 0-599 <<< Implies 0-599 are preferred for workloads and 600-719 should be avoided at this moment. cat /sys/devices/system/cpu/preferred 0-719 <<< All CPUs are usable. There is no preference. Signed-off-by: Shrikanth Hegde --- Documentation/ABI/testing/sysfs-devices-system-cpu | 14 ++++++++++++++ drivers/base/cpu.c | 12 ++++++++++++ 2 files changed, 26 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documenta= tion/ABI/testing/sysfs-devices-system-cpu index 82d10d556cc8..080c69aca4f0 100644 --- a/Documentation/ABI/testing/sysfs-devices-system-cpu +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu @@ -806,3 +806,17 @@ Date: Nov 2022 Contact: Linux kernel mailing list Description: (RO) the list of CPUs that can be brought online. + +What: /sys/devices/system/cpu/preferred +Date: Aug 2026 +Contact: Linux kernel mailing list +Description: + (RO) the list of preferred CPUs applicable in + paravirtualized environments. + + The steal governor driver dynamically adjusts this mask + based on observed steal time. Scheduling tasks on + CPUs outside of this list may lead to performance + degradations due to underlying physical CPU contention. + + See Documentation/scheduler/sched-paravirt.rst for more details. diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c index 19d288a3c80c..5da2a96fb37e 100644 --- a/drivers/base/cpu.c +++ b/drivers/base/cpu.c @@ -391,6 +391,15 @@ static int cpu_uevent(const struct device *dev, struct= kobj_uevent_env *env) } #endif =20 +#ifdef CONFIG_PREFERRED_CPU +static ssize_t preferred_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%*pbl\n", cpumask_pr_args(cpu_preferred_mask)); +} +static DEVICE_ATTR_RO(preferred); +#endif + const struct bus_type cpu_subsys =3D { .name =3D "cpu", .dev_name =3D "cpu", @@ -531,6 +540,9 @@ static struct attribute *cpu_root_attrs[] =3D { #endif #ifdef CONFIG_GENERIC_CPU_AUTOPROBE &dev_attr_modalias.attr, +#endif +#ifdef CONFIG_PREFERRED_CPU + &dev_attr_preferred.attr, #endif NULL }; --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 152C531F9A8; Wed, 12 Aug 2026 05:41:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513321; cv=none; b=ZAkdAQYcVzAV02LlsXQ0jge/JrE+PFJVDQZ9Xp3ar9JEgb+tuNla8oxe/tK7RpGgqOMSKXHzagcxoB72nw1CAP7Cx6ObYbtNGcmkdhOPb4LtIN2731U8BKIsK/z3nok3q0epu/TLih6wjWIXeEOkWexsI7rqR1AaGXm1d1TTTUQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513321; c=relaxed/simple; bh=kVarbSkXTaCjfYt1E/AFEKjccs8bdUv0Uv4GGJyo+os=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OxJutwi0YgROYUe6CJG4oCMdfxft81SfOSUdT0FBqlnWfkV7btuAKO50WgE121ntUeCRRGNEQ5i9i0wMKoibv2NFdAqu5Y0GaMQGILgAtRG7bsNiVGWBQL78/EZ5tlhX+ZXqs5VgbhOh3iqRjwYZgOdNYBxiZXlWQVqaStTSRCA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Bru99dPX; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Bru99dPX" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31Vsr416373; Wed, 12 Aug 2026 05:41:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=QJ4UHhGufY3DzvJa7 xsdGTzizaUXF5U10Kzs9dh5iU4=; b=Bru99dPXlNUomok69+P36MzQZOn0sa46S uWbRrrsoLDDQIzDCl8hOmN1MIobFh1inqLSObQ3XWCSojtms6KDYcMLOObChVk03 B/NM4KviPQ6Fz/cE3JTPkuVgf1cf8iaJ2Rb64QFB+iBd0w0JmInUdZt5lrxQdOkR jIe3IL/rSTiQ89HnvyFAFaig8h6xHvBzW6hIQUw0GXCRerMU4rfhHMYfA8ZmgzSs Cf2fq1fLdERJSDFYoVxUI8y2gW2ShSwZXxjVRm8//6Qp30M6rwfO26UOoQEkrxiR 5XhP1tuIj9MCUtKs4Ic5rB5FyQqMLU3/F8Dz5RSIm69aZNfP+Wycw== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvk00yk5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:34 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5fJ3e012748; Wed, 12 Aug 2026 05:41:33 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4fxfsjvpat-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:33 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5fTIg35783088 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:41:29 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0F36D2004B; Wed, 12 Aug 2026 05:41:29 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7168820040; Wed, 12 Aug 2026 05:41:20 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:41:20 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 05/12] sched/core: Try to use a preferred CPU in is_cpu_allowed Date: Wed, 12 Aug 2026 11:10:26 +0530 Message-ID: <20260812054033.95658-6-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX3TJOJUgymRxJ U4z0Hp4ruUSaYEF+7pMpnxIyspR3rMDgRBHpmSutDFuVV1Zy+QqDg4gh61fTWK/rguez+MIPwD5 v9Nb33dHWKGzn3OQOB8HbT5Xim9CSb9ZRnr1pWYt9glwy/zYTsTwBj5Jp4M90AdQj6htyRHHtGi kicAlSamw0wtGPkxNR+uHWvGo7I2Z9t0+iZRYrqPteVmf854HwMqitiV91SQkGQ69t7/4/tGf59 BsvRArOubwbwUAbmdwnCKs4VzyjvZjlTToTUDDYGLOda/tVQ2yLoYs7v3fMkStfFs/UQtQHxTC+ 4E2XTMdaRfhpi+Z+9Is421yNepuDVPX/LiOLdk9mYTaUk0h5ASIh57mgKEv5+FNmVxISlkXsn+A GiPp3/okNQnj2+fG1psTnrdivT/Ts0bGJxWrVgCNialXDx39G7E/3LiM6kGQcnNnLXTzcLhW93c UzEtN8+9CoVS2hkHgjQ== X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX2yaZ6tRQHiX3 ttBJkrIZvb9VQRT6ORMOKckhyWxKWVDMbyF8YkaqYI+7zKPonEco+d1SuF/pIwQTcM4ubA1WY9K XfYv3+hs1Ea81ZTkBxKSbkucgtI3Xcg= X-Authority-Analysis: v=2.4 cv=RqD16imK c=1 sm=1 tr=0 ts=6a7c078f cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=TBRyIjXBjLUW-Ng_AUEA:9 X-Proofpoint-GUID: KpscK3vHwCeCHyXvco_HNY3LSCneEIZ0 X-Proofpoint-ORIG-GUID: g-Gl2t0XYBoQIM-mxrMhN1kx24tk9fC6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 suspectscore=0 lowpriorityscore=0 clxscore=1015 adultscore=0 bulkscore=0 malwarescore=0 impostorscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" When possible, try to choose a preferred CPU. This is essential to maintain user affinities when preferred CPUs change. A task pinned on a non-preferred CPU should continue to run there, since this is a non-user triggered event. If a CPU is non-preferred and the task can run on other CPUs which are currently preferred, then choose a preferred CPU instead. This is decided by checking if cpus_ptr and cpu_preferred_mask intersect or not. If yes, then the task has other preferred CPUs. The push task mechanism uses a stopper thread which calls select_fallback_rq() and uses this mechanism to pick a preferred CPU. This takes care of the wakeup path for FAIR tasks too. is_cpu_allowed() is called to ensure wakeups happen on preferred CPUs. With that, additional checks in available_idle_cpu() are not necessary. For the majority of cases, this would still keep select_fallback_rq() as O(N). cpumask_intersects(), which is O(N), is called only if !cpu_preferred. The task running there is expected to move out. Subsequently, it should run on a preferred CPU. This becomes O(N**2) only for tasks pinned solely to non-preferred CPUs. That is a rare case. Overhead is minimal when the CPU is preferred. Signed-off-by: Shrikanth Hegde --- kernel/sched/core.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index a45f7c308329..1c90bcad0a75 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2494,6 +2494,18 @@ static inline bool rq_has_pinned_tasks(struct rq *rq) return rq->nr_pinned; } =20 +static inline bool task_can_sched_on_preferred(int cpu, struct task_struct= *p) +{ + if (cpu_preferred(cpu)) + return false; + + /* Only FAIR tasks honor preferred CPU state */ + if (unlikely(p->sched_class !=3D &fair_sched_class)) + return false; + + return cpumask_intersects(p->cpus_ptr, cpu_preferred_mask); +} + /* * Per-CPU kthreads are allowed to run on !active && online CPUs, see * __set_cpus_allowed_ptr() and select_fallback_rq(). @@ -2509,8 +2521,12 @@ static inline bool is_cpu_allowed(struct task_struct= *p, int cpu) return cpu_online(cpu); =20 /* Non kernel threads are not allowed during either online or offline. */ - if (!(p->flags & PF_KTHREAD)) + if (!(p->flags & PF_KTHREAD)) { + /* Try to use preferred CPU if task's affinity allows */ + if (task_can_sched_on_preferred(cpu, p)) + return false; return cpu_active(cpu); + } =20 /* KTHREAD_IS_PER_CPU is always allowed. */ if (kthread_is_per_cpu(p)) @@ -2520,7 +2536,11 @@ static inline bool is_cpu_allowed(struct task_struct= *p, int cpu) if (cpu_dying(cpu)) return false; =20 - /* But are allowed during online. */ + /* Try to keep unbound kthreads on a preferred CPU if possible. */ + if (task_can_sched_on_preferred(cpu, p)) + return false; + + /* Otherwise, they are allowed to run on online CPU. */ return cpu_online(cpu); } =20 --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A6A937DEBA; Wed, 12 Aug 2026 05:42:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513328; cv=none; b=b23+sBXS+i+dxilG/lBfd1w4lde2N9OaYroNU/IjRzybteatd64dUdUjTOzx3CUmdV52jijKtfREQFtSeFqxMNV26N3yoeD0eyQhoZHUNB8Vr2gXWu5s7ePsrlKv0UD3gSiegXazzzep+Z689lcEE5SjWE/o0eCOf/AMqpDGGbc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513328; c=relaxed/simple; bh=RyklxcPSP7Qfg2QDFZ/nj8HOB60hUtyDriXJP9+ZYJw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TmLDZ4MBULhJiw3JmU6Ts/2H3oEKc0oNEvgjqyUAvGuIk2tlCEAL1Bma9OkyGGyQwUQ1jVsY91XW8l289RusVqExcElbfQXUeVcKUtonbvjiZhhbV9KXXhJFSjJqtNrSEz0j7+0cKjKbl5fuIbD7eQWjjPjz0GajauzF9wAR8+Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=A4/zyx05; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="A4/zyx05" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31co72313501; Wed, 12 Aug 2026 05:41:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=3tg1To8OdOCCr/p31 NSpp5yHMlBFyPnWQDO5YCB6DWk=; b=A4/zyx05bLskf7CitqIpKwXh0YUIWMug5 UHBPgDnp06ToKg8Iaw/polXqUiVHO/wSlXug8KPunXPJWvv2IPJkzY6GbiwnrkKT tMtWJNbNg8PhBPaF1MEaXZuxqK73YxGa8eWJgSZ4EuJ65yvDINBOLq9VUnRzni55 XDkQdJoF05Vux193Sp/RMeYxA5bHzoFPxFNQOAJfdU8u3Ln4W3wHNAFLscwPR3/C uh06Suz6c9Gk+W9nCQpChBPUuvRk2w+/MjnfnxYRq74vIlQfl1eqTihNGlCHVYIy wXAEG93BSy51EqNrF8h1WSmJQxy8lb4HJAIMktrHHbPl8TI2W/Nrg== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvnw87sm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:45 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5fGWa010356; Wed, 12 Aug 2026 05:41:44 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4fxh0gcj24-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:43 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5fcc019595984 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:41:38 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3299B20043; Wed, 12 Aug 2026 05:41:38 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 73BCA20040; Wed, 12 Aug 2026 05:41:29 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:41:29 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 06/12] sched/fair: Load balance only among preferred CPUs Date: Wed, 12 Aug 2026 11:10:27 +0530 Message-ID: <20260812054033.95658-7-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Authority-Analysis: v=2.4 cv=RsP16imK c=1 sm=1 tr=0 ts=6a7c0799 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=f1WSfB00kwmd_FHdG74A:9 X-Proofpoint-GUID: mNiVJSpYnpTkyIIJCVecWnCK9FVlxQ_r X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX77MuemA9o/NU 2dGXoUGyBb6e6FBt3aiR2e87gxUEpYmUlU04W980r4iq9Sr32ENip/G5Pe6zJu9fOaVWR2wmEBx oeUYSxC3ZJIxxP7UsbwvpFkeGIxoZ+zT8CqntwZ/HI//kEkOEjaCEMag5RMDznJwOVF2DNPRJOh 1dw2YOwHRx5AIXMFqv+jaDfYkFLNOTDzIXHMRDWNY7vdtTS91vjoqrUH32mLCY+vjQyEMg9ntjB taZrYjXm5mw7mnDAtqNpkAtKrlcEUZ6W1CSygUhqwUTMiWOAJxLphZ5qIZ0Ssd9hQXEovZRlLRr fCV2UnKLOguENQ8Qc28ZpTToBaJ8kTEZGWKMqy12WfA7sg1/Mkc9tVAQXNMz8k/E8cLdk7LlHA4 z2rZecHUlmWNtbxdN+zb9g7icskU1kzd6LB86nxNvxRI2BJrE8FfBMuO5SZjsrnxiENfr4+WiIx GVXpJnpcdbf63yejaPQ== X-Proofpoint-ORIG-GUID: 9aW2-1L_U2opvLipHrL5XROD2lSmm9YT X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX6MAQbwSScnKx kLzCFq99L2Cgd/+0uf0vkfo3RZJW7qqUt/wnIaABzGA9OaipRespWOHirznO6CWVDvoNxdyjPyq qlhg6SfMSPUeZR7VpirOV+6KkqVu1Z0= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0 suspectscore=0 clxscore=1015 malwarescore=0 phishscore=0 adultscore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" When a CPU is marked as non-preferred, any load pulled towards it is pointless since the task will be pushed out again in the next tick. So, consider only preferred CPUs for load balancing. This ensures load balancing does not fight against the push task mechanism which happens at the tick. Also, this stops active balancing from happening on a non-preferred CPU pulling the load. This also means there is no load balancing if a task is pinned only to non-preferred CPUs. They will continue to run where they were previously running before the CPUs were marked as non-preferred. Bail out early for NEWIDLE balancing, as load balancing is done only on preferred CPUs. Note that idle balancing is allowed to go through, since that naturally updates nohz.next_balance when all the idle CPUs are non-preferred. Also, optimization in find_new_ilb() is skipped. The steal governor driver, which is introduced in later patches, updates the preferred CPUs state in descending order. find_new_ilb() checks for idle CPUs in ascending order. Hence, in most common scenarios, the idle CPU found by find_new_ilb() will already be a preferred CPU. When all idle CPUs are non-preferred, the first idle CPU has to be chosen anyway. All of this is naturally handled in find_new_ilb() currently. Adding additional complexity to it for rare edge cases is not necessary. Signed-off-by: Shrikanth Hegde --- kernel/sched/fair.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index dcf860c59a14..04c2b2e17120 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -13429,7 +13429,7 @@ static int sched_balance_rq(int this_cpu, struct rq= *this_rq, }; bool need_unlock =3D false; =20 - cpumask_and(cpus, sched_domain_span(sd), cpu_active_mask); + cpumask_and(cpus, sched_domain_span(sd), cpu_preferred_mask); =20 schedstat_inc(sd->lb_count[idle]); =20 @@ -14544,10 +14544,8 @@ static int sched_balance_newidle(struct rq *this_r= q, struct rq_flags *rf) */ this_rq->idle_stamp =3D rq_clock(this_rq); =20 - /* - * Do not pull tasks towards !active CPUs... - */ - if (!cpu_active(this_cpu)) + /* Do not pull tasks towards !preferred CPUs */ + if (!cpu_preferred(this_cpu)) return 0; =20 /* --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B5A04B04B7; Wed, 12 Aug 2026 05:42:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513336; cv=none; b=Q/dHnsNVuMrKQmKOyZAKVXP27/O6advEmaChj1rj6xBsiB3IsQNMt/4wC2nLmHgPCMBWYqv6fTgmalq9GR5nSPKoJp1dfRy6ksYtCSbyjukB5gqnMlod6HMTR4L0DRnqEwtegkY+uTOkz+3UBuisjq5QA0JESMSKeS+9A29yQVA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513336; c=relaxed/simple; bh=NGWM9vWEkl8FYFhxzz71q6GQPIX+zMVOKC8a/3xl3HM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tneYAv3ETZBPY44F4nnqsBZ+oUJ5TTb7C2rGU7IVO+sW0Jf24nqo2tgwUfPUVfPbMsJcOpo9/ccR7W118gEsQrunSh2XaS2d7pbNyLxLSBdKgNu8zLky3O3cH+TkdZVssQaNmux2Y2S95wC7yyC/OV44+uh9PRHRZEvJlOP4hx8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=KG9DVudr; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="KG9DVudr" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31XXk2238399; Wed, 12 Aug 2026 05:41:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Y5rmXjg73Xx+HPrgm UoEXLwpNZNmUcVBn2VJOi1Zhtg=; b=KG9DVudrMC8XweJrOj9MbraI9fHFtnmjA XyPdCHYiXa/nMzB5il3M/KJTajebeOb2ibvFiVskiF6OZoSNyeH3NVD9DR7JrFfm a/zkmwI6SKKQaxgYyuP9CnONDhbjc0s37WzVkR3v21DmUdd5smVuMYbVHEzN4rAT JAHHPog1MkIO6jqGyyJVsW+K9Qge/blFHIlK9Po0fLYF5Zpd/wx/oyXZa7egtLUj DSq6hERTerszBGHbUueBYMenF98Cn3UtoD7/Gl5Q9xlEljpwr3OHoE5DKAe2R2gp rnjXLiew9Lnehy25dIFtmqw0xc8CFLUBj4LpqfNmVD1ba+W3F5BJw== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvq9gy45-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:52 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5fHcN012736; Wed, 12 Aug 2026 05:41:51 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4fxfsjvpbm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:41:51 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5flAY33620440 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:41:47 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4DC8120043; Wed, 12 Aug 2026 05:41:47 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A0E9320040; Wed, 12 Aug 2026 05:41:38 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:41:38 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 07/12] sched/core: Push current task from non preferred CPU Date: Wed, 12 Aug 2026 11:10:28 +0530 Message-ID: <20260812054033.95658-8-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-GUID: rF4OcqybNF4WYKlw1759VRk-Dg09BwM8 X-Authority-Analysis: v=2.4 cv=PbDPQChd c=1 sm=1 tr=0 ts=6a7c07a1 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VnNF1IyMAAAA:8 a=rPhwLmmYJdnLsqmuV48A:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfXxUfwXImvFfu0 LrjS2+nKSsvo6v4vBTxntlJz/XQ3lwT1Xg1j/tNsszn5y4sU+1XEDRnR3XSGozJywOMNxphyNdE W5IQN4Ika0FeKE65uiPCDxlffi4tIiEFXvEsARPBKNxh2zulz0zEKgoHAz+Le+2yT/fmDi9TO3o FpZmLHc3Juq3fB9xXjzVEy6do+tJPfITzvoaaXIuGNJU1sRYtQpASHJJU7H9riom4XEzLJ5ROrN tobhZP9UkG06tb0AsCRdWu2HNmOKYOUjPT1itYGSjLYhkG3ih6AZqOD1scznhjKNmmY3DCxfdt5 BkIlY1cL8HQBIMU3hgWqY71O+AmNKNZXScZtbOz1+G3pIF3v4N85CBwvOnt4r+e+OZbdNvun+EV VQKXqXc+vzW/Um/jBPwN2+CSGUy32nJJg9Y9iQM+5wm/S7q0t4YOOP0J0Gu11XHTrM5o4ECKPu5 pla1tXD5ABumMO3EgIw== X-Proofpoint-ORIG-GUID: s3HsnTy3SAQkHNFNqWX74isZPZkIn3ev X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX3baCurEa3Cc4 5tqMxJTj/T3CtJZB71nGtRimkTjKVY8vNMlhZKDIxnagJdrZ46+1bD9jg9YxueVD8341xra3JYd R5pBB5hKDBXP86f5VpW0aflR6mAMVks= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 impostorscore=0 malwarescore=0 adultscore=0 clxscore=1015 priorityscore=1501 suspectscore=0 phishscore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" Actively push out the current running task on a non-preferred CPU. Since the task is currently running, a stopper thread must be queued to push the task out. However, if the task is pinned only to non-preferred CPUs, it will continue running there. This helps to maintain userspace affinities, unlike CPU hotplug or isolated cpusets. Though the code is similar to __balance_push_cpu_stop and quite close to push_cpu_stop, it is kept separate as it provides a cleaner implementation specifically for CONFIG_PREFERRED_CPU. Add the push_task_work_done flag to protect the work buffer. For now, only the currently running task is pushed out. This keeps the code simpler. In the future, an optimization may be added to move all queued tasks on the runqueue. This works only for the FAIR scheduling class. Signed-off-by: Shrikanth Hegde --- kernel/sched/core.c | 78 ++++++++++++++++++++++++++++++++++++++++++++ kernel/sched/sched.h | 8 +++++ 2 files changed, 86 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 1c90bcad0a75..c51ba2979496 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5786,6 +5786,9 @@ void sched_tick(void) unsigned long hw_pressure; u64 resched_latency; =20 + if (!cpu_preferred(cpu)) + sched_push_current_non_preferred_cpu(rq); + if (housekeeping_cpu(cpu, HK_TYPE_KERNEL_NOISE)) arch_scale_freq_tick(); =20 @@ -11304,3 +11307,78 @@ void sched_change_end(struct sched_change_ctx *ctx) p->sched_class->prio_changed(rq, p, ctx->prio); } } + +#ifdef CONFIG_PREFERRED_CPU +static DEFINE_PER_CPU(struct cpu_stop_work, npc_push_task_work); + +static int sched_non_preferred_cpu_push_stop(void *arg) +{ + struct task_struct *p =3D arg; + struct rq *rq =3D this_rq(); + struct rq_flags rf; + int cpu; + + if (cpu_preferred(rq->cpu)) { + scoped_guard(rq_lock, rq) + rq->push_task_work_done =3D false; + put_task_struct(p); + return 0; + } + + raw_spin_lock_irq(&p->pi_lock); + + /* This could take rq lock. So call it before rq lock is taken */ + cpu =3D select_fallback_rq(rq->cpu, p); + rq_lock(rq, &rf); + rq->push_task_work_done =3D false; + update_rq_clock(rq); + + context_unsafe_alias(rq); + + if (task_rq(p) =3D=3D rq && task_on_rq_queued(p) && + !is_migration_disabled(p)) + rq =3D __migrate_task(rq, &rf, p, cpu); + + rq_unlock(rq, &rf); + raw_spin_unlock_irq(&p->pi_lock); + put_task_struct(p); + + return 0; +} + +/* + * Push the current task running on non-preferred CPU(npc). + * Using this non preferred CPU will lead to more contention + * in the host. So it is better not to use this CPU. + * + * Since task is running, call a stopper to push the task out. This is + * similar to how task moves during hotplug. In select_fallback_rq a + * preferred CPU will be chosen and henceforth task shouldn't come back to + * this CPU again. + * + * Works for FAIR class only. + * + * If task is affined only on non-preferred CPUs, no point in moving it ou= t. + */ +void sched_push_current_non_preferred_cpu(struct rq *rq) +{ + struct task_struct *push_task =3D rq->curr; + + scoped_guard(rq_lock, rq) { + /* Push the task if its explicit affinity allows */ + if (!task_can_sched_on_preferred(rq->cpu, push_task)) + return; + + /* There is already a stopper thread. Don't race with it. */ + if (rq->push_task_work_done) + return; + + rq->push_task_work_done =3D true; + } + + /* sched_tick runs with interrupts disabled. */ + get_task_struct(push_task); + stop_one_cpu_nowait(rq->cpu, sched_non_preferred_cpu_push_stop, + push_task, this_cpu_ptr(&npc_push_task_work)); +} +#endif diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 26ae13c86b69..6cee31466087 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1277,6 +1277,8 @@ struct rq { =20 struct list_head cfs_tasks; =20 + bool push_task_work_done; + struct sched_avg avg_rt; struct sched_avg avg_dl; #ifdef CONFIG_HAVE_SCHED_AVG_IRQ @@ -4230,4 +4232,10 @@ DEFINE_CLASS_IS_UNCONDITIONAL(sched_change) =20 #include "ext/ext.h" =20 +#ifdef CONFIG_PREFERRED_CPU +void sched_push_current_non_preferred_cpu(struct rq *rq); +#else /* !CONFIG_PREFERRED_CPU */ +static inline void sched_push_current_non_preferred_cpu(struct rq *rq) { } +#endif + #endif /* _KERNEL_SCHED_SCHED_H */ --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86D882253A1; Wed, 12 Aug 2026 05:42:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513344; cv=none; b=KqgHhhXT3dOAIvgWXOj9twhSkRmuNh5DOxkqxnFOqECTZxZWXV1lnPw+V5V7uLvyxB8lqnrYo/UkaFABE6r3/HM48KW7Dy7MIw0QBRJS/i4z8lQ5S9QpddDQv1O9KQvNsdi0AuaVm8XL1cIbHKXCv1r+PNENfmDEgXAIm7xEaEA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513344; c=relaxed/simple; bh=dMC6uo8OqzZCfT6Bj3/B6PIQvNxpLMj3RhP420uPsCA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UzwEJXblhlELVB+fZ5wfS0zrY3U2W0EVOxgM3m2EnjQ2Fp5jrMTaS/3bk+ScjHilw5d5sk2yZMrEbEQ79b5yK2ZLpfmaXmtGxe0lD2/YOy1e4crXc8zSPsy0Og/yzkUop9GdaotojD2hHsATbM7PnZ7BDJdK7wS7147xGeGkXqI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=sW0AqnxG; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="sW0AqnxG" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31n4E2313788; Wed, 12 Aug 2026 05:42:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=UGCMfmVJfrHqaR3yv 2LCyWHobLH2A6rmfzx3Hj7NYIM=; b=sW0AqnxGzoPAGmwzqOZf1nJdev2Dn5iN9 bH3AaFwbweDljmARNP8H60TbxqupwFB31wLK2VRVJUPdVF/X/jmU7X2g0C76R+CZ 0t8Cs8vD2K4WzVOT7BV9Nj698wnGJX0HlsQEuqh9nh5evZS9YxhhS51P6ebYCQPW 96Rkq5X5AD5cxX6kaAR0SjGm7xmuvsG05SkXRgSy+ua7w+y7n9JveaDxlDx2Uu8S JceoqPdm3gxkF8cxaIwh0m+tLAMlDUXetf9gbAeftwCLtmKfJfYvWjnKdiUAyhw9 fWfQnyvKSQ4+R9qdcBxyLvaDgNVWwCqFptIgebzIgc/q9kh1IQWpA== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvnw87t8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:02 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5fHTN025138; Wed, 12 Aug 2026 05:42:02 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4fxf5w4t3j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:02 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5fwlA51708160 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:41:58 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EBEB920043; Wed, 12 Aug 2026 05:41:57 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E77AD20040; Wed, 12 Aug 2026 05:41:47 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:41:47 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 08/12] sched/debug: Add migration stats due to non preferred CPUs Date: Wed, 12 Aug 2026 11:10:29 +0530 Message-ID: <20260812054033.95658-9-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Authority-Analysis: v=2.4 cv=RsP16imK c=1 sm=1 tr=0 ts=6a7c07ab cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=-i9lZGrASMRMKgkbeRgA:9 X-Proofpoint-GUID: QYQY4Mw9rCvdG_CaUW8tsMSpTsu7lHgD X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX4+hPbOxv6Fge bXcejbUntNTatoAqmEha9NUpHbLiGC9vl5F0UnsypK+k80qSTuRhAFmh81KbeeCKeNoBpe25rfP AIals2s3ForyYMqqnNEmwLBe2uQ78XfhZZtJY1EhSTvhSIkStz5e0ejeZkApLUQDNsHxxxKKF4u OdPfNFfXT436V7ZwhJ4OA0jmHOB8r52MNDFXvhAAQBDf8rNwU0gPMgvsSr6L5N2JRiTsWshSrFn a4AYuEyocFSgEPufjzuF3m1Qube4Vv0fe5qHj/a1dze3wuCcxCFv3BOYalVgdhn0hObDujDEg3T ydhsjaEMAcRyz8EAXFzN50nGECavqvw3A2PPocH4tHe0nsPnL4R8Vtndb6Oqi1VocJKiwTVg3WI ixEXOLDQQp/kLsAeIX7glQPqCrmxUT/GLxryZYiyIQiaDWczST2f6aUkFVM/ykWVgsvbUOAf6Cg 1Bkcf97Wb4J/L1zCw0w== X-Proofpoint-ORIG-GUID: xLELYBjBkbAdnW1zg38ZuYJ6xunjnP8L X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX8QmO+uIFsU2V LMw/RKbuAwT6K5wcE77RY6CGyGra+nyKBcZ1LxI0L+U+6yoYOqRa7GWqGOyC672veAimRafMIMO rYuRAZF1PjVbhKJBz4ksS/q0ENj6Sxo= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0 suspectscore=0 clxscore=1015 malwarescore=0 phishscore=0 adultscore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" Add a new stat, - nr_migrations_cpu_non_preferred: number of migrations happened since a CPU was marked as non preferred due to high steal time. Signed-off-by: Shrikanth Hegde --- include/linux/sched.h | 1 + kernel/sched/core.c | 9 +++++++-- kernel/sched/debug.c | 1 + 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 968b18a7f470..37849d2f1dbd 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -554,6 +554,7 @@ struct sched_statistics { u64 nr_failed_migrations_running; u64 nr_failed_migrations_hot; u64 nr_forced_migrations; + u64 nr_migrations_cpu_non_preferred; =20 u64 nr_wakeups; u64 nr_wakeups_sync; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index c51ba2979496..5fda8234e843 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -11336,8 +11336,13 @@ static int sched_non_preferred_cpu_push_stop(void = *arg) context_unsafe_alias(rq); =20 if (task_rq(p) =3D=3D rq && task_on_rq_queued(p) && - !is_migration_disabled(p)) - rq =3D __migrate_task(rq, &rf, p, cpu); + !is_migration_disabled(p)) { + struct rq *dest_rq =3D __migrate_task(rq, &rf, p, cpu); + + if (rq !=3D dest_rq) + schedstat_inc(p->stats.nr_migrations_cpu_non_preferred); + rq =3D dest_rq; + } =20 rq_unlock(rq, &rf); raw_spin_unlock_irq(&p->pi_lock); diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 72236db67983..5ebb2055e6d5 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -1446,6 +1446,7 @@ void proc_sched_show_task(struct task_struct *p, stru= ct pid_namespace *ns, P_SCHEDSTAT(nr_failed_migrations_running); P_SCHEDSTAT(nr_failed_migrations_hot); P_SCHEDSTAT(nr_forced_migrations); + P_SCHEDSTAT(nr_migrations_cpu_non_preferred); P_SCHEDSTAT(nr_wakeups); P_SCHEDSTAT(nr_wakeups_sync); P_SCHEDSTAT(nr_wakeups_migrate); --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F052537BE81; Wed, 12 Aug 2026 05:42:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513358; cv=none; b=E1fMLgOycL2RGssRHBPXxKl0RwyHrYl747uE0Bk+ctFvQEth99Vm6+ESLhmTfiakOz7ADsauNSbd87cYwWan2HFOt9drB3CEQkPvdRIE4qE0iaggmfybbaFFx/dZCaL7Tt4olVfKKgSwvzploWVaW1Mf5MBS/KoOO/6sNyQiL44= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513358; c=relaxed/simple; bh=+yfHTWgKgLh790EbNhkjVUsRVD0YRzyvN4sLTAprGas=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ghM2XGATnUVZgH2YMsD3Xlp2KAOoGCGVgdWJPhs8SL/n3V3fjRl/1ziQOYXMUqAXtmoJodEAIUOzGLfa5QMdl4LuHffh42MzRQP2HeMTNHkm6XR6hBoN1NeIgOMLGIuV6xdvyBWC3qa1TuTebajzMqwO4hThwV0alEs3MD5qO44= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=mgyoAEZr; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="mgyoAEZr" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31Y882284959; Wed, 12 Aug 2026 05:42:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=isD5y7pJIlIHSlzmA ozVo2wF0pUCCRpOYT3tASXOZsI=; b=mgyoAEZrMUe2G2poM143ikgb9es5YZyfw yZudXVCXDEglaIZhd/ea5vdfxuxp4uhUxlFsZuODMQB4sRttPAeIGtSt9hv3zslU KIoyK/P0sqZ8kyGgNN9ChlOHRxDMQyIbk35rv13zsCyPLUgJSFRJYCLvcWm7Q5c1 HvW6FScumXjIvIV13cDyxlttcESH5WbhXvxdBVTBQHcVmOt4jfDzLbI9cWX15uq0 iNSwpGvxAqPFW0hG56qH53uMWrwNPYjwGgkNcWSn8BIBmTO2/l7R3gg1qJ91RCt0 3Ki5a+hUhJpHRswGcyFNOURxBE12glLR9r9ai7krk6y1DrMFSBLVQ== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvm9s1eh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:13 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5fGWd010356; Wed, 12 Aug 2026 05:42:12 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4fxh0gcj39-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:12 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5g8sM38469986 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:42:08 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B636720043; Wed, 12 Aug 2026 05:42:08 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9CA3120040; Wed, 12 Aug 2026 05:41:58 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:41:58 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 09/12] virt: Introduce steal governor driver Date: Wed, 12 Aug 2026 11:10:30 +0530 Message-ID: <20260812054033.95658-10-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX2q1CLil9G7Dp SX9HBOsChoJUri8a1royNzwX3pW32t39LhkS9xJJyLyxyVuEVx/bLh8wKKH5yXpz04HhgfVSTLG ytVjWibQxPc8pF9vAtrdciDUZUdib7powby/ohvBBH4yHxZwRLluCk75LtfkR68ZlDGpy9UhVCC 6RMOAeZGaIAvRPrskDs4pBFHvB6LuEJBFPsfjZtF76b37OYeU6FrQ2VckX7+GZScitL53z04Rk9 ioFjuqkBKg+0wPdjAl3S460Nt5TPGR2DX1N2i754fj9KcwjPWEh54S7WDRfFU0XL4jEXbw0dhnm JvzX7h50Mi1kHdIQ34rXf3+gd0k+0g9J99BDEBB2EarfAY8eBF4ruCZt2n/mKvgYMbo5wLOhZz+ xVEZmYaA/IR5rARbO2E17vbC/A0lbTxhHJ+PPuDupgugP8ITcvhMtX/zSLI3a/5CW2XDPAMCe7a HX1UOE0KxTB8GIf2eeQ== X-Proofpoint-ORIG-GUID: NnR1Q3IblPpoGQrK8Nc0vQk9WW3Wvokn X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfXzV7Zsn7VdG1A /iGBWJLIzMMqpIqQWKvKedXVmMeILBEYD7bWK4bSySucpDEM4jqlxoJk5QaxD4Kwc3r/tqL6ckP wtrnym1DjUmFfO3SirBqFN2+H6w+MpI= X-Authority-Analysis: v=2.4 cv=IfK3n2qa c=1 sm=1 tr=0 ts=6a7c07b6 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=U7nrCbtTmkRpXpFmAIza:22 a=pGLkceISAAAA:8 a=zd2uoN0lAAAA:8 a=VnNF1IyMAAAA:8 a=VwQbUJbxAAAA:8 a=8b9GpE9nAAAA:8 a=63rvlg9gmhNNS91pnNMA:9 a=T3LWEMljR5ZiDmsYVIUa:22 X-Proofpoint-GUID: 3p1qfRsmXlNtwW5d3MrZUNBuGR7ItCos X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 impostorscore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 adultscore=0 malwarescore=0 spamscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" Introduce a new driver in virt named steal_governor. This driver will compute the steal time and drive the policy decisions regarding the preferred CPU state. More details can be found in Documentation/driver-api/steal-governor.rst. A new kconfig called STEAL_GOVERNOR is introduced in subsequent patches, which enables this driver. This driver will select CONFIG_PREFERRED_CPU. This makes configs driven by user preference/configuration. When the driver is disabled, preferred CPUs remain the same as active CPUs. The file layout of the driver is kept simple for now. The code is in drivers/virt/steal_governor.c, and the configs are part of drivers/virt/Kconfig. The main structure of the steal governor contains: - work, delay: Deferred periodic work function variables. - steal, time: Used to calculate deltas during periodic work. - interval_ms, high_threshold, low_threshold: Tuning knobs for the steal governor. While there, add MAINTAINERS entry for this new driver. Suggested-by: Yury Norov Suggested-by: K Prateek Nayak Signed-off-by: Shrikanth Hegde --- Documentation/driver-api/index.rst | 1 + Documentation/driver-api/steal-governor.rst | 137 ++++++++++++++++++++ MAINTAINERS | 9 ++ drivers/virt/steal_governor.c | 68 ++++++++++ 4 files changed, 215 insertions(+) create mode 100644 Documentation/driver-api/steal-governor.rst create mode 100644 drivers/virt/steal_governor.c diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/= index.rst index eaf7161ff957..0a973b59cba3 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -138,6 +138,7 @@ Subsystem-specific APIs sm501 soundwire/index spi + steal-governor surface_aggregator/index switchtec sync_file diff --git a/Documentation/driver-api/steal-governor.rst b/Documentation/dr= iver-api/steal-governor.rst new file mode 100644 index 000000000000..672eeccabfe8 --- /dev/null +++ b/Documentation/driver-api/steal-governor.rst @@ -0,0 +1,137 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Steal Governor +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +:Author: Shrikanth Hegde + +Introduction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The steal governor is aimed at mitigating the Noisy Neighbour problem +which occurs in paravirtualized environments with CPU overcommit. +The performance of a workload running in one VM gets degraded by +the activity of other VMs on the same host. As a result, all VMs +collectively make slower forward progress. + +In such systems, high utilization in all VMs causes the hypervisor to +frequently preempt vCPUs. This vCPU preemption is expensive. +To mitigate this, the kernel aims to restrict workloads to a subset of +Preferred CPUs to reduce physical CPU contention. +A detailed explanation of Preferred CPUs is available in +``Documentation/scheduler/sched-paravirt.rst``. + +The steal governor selects ``CONFIG_PREFERRED_CPU=3Dy`` which enables the +scheduler core infrastructure to move the tasks to Preferred CPUs where +possible. The driver controls the policy decisions regarding the state of +preferred CPUs. That is, this driver decides which CPUs are preferred +and which CPUs are non-preferred. + +The driver code is available at ``drivers/virt/steal_governor.c``. + +Core idea +=3D=3D=3D=3D=3D=3D=3D=3D=3D + +steal time is an indication available today in Guest which shows contention +for underlying physical CPU. Use it as a hint in the guest to fold the +workload to a reduced set of vCPUs. When there is contention, steal time +will show up in all the guests. When each guest honors the hint and folds +the workload to a smaller set of vCPUs (Preferred CPUs), it reduces the +contention and thereby reduces vCPU preemption. +This is achieved without any cross-guest communication. + +Steal governor driver effectively does: + +1. Periodically computes steal ratio across the possible CPUs. + +2. If steal ratio is greater than high threshold, reduce the number of + preferred CPUs by 1 core. Ensure at least one core is left always. + Skip changing the state of offline CPUs in that core. + +3. If steal ratio is less than or equal to low threshold, increase the + number of preferred CPUs by 1 core. If preferred is same as active, + nothing to be done. Skip changing the state of offline CPUs. + This helps to handle cases where few CPUs are offline in a core and + those offline CPUs will not be marked as preferred. + +4. Ensure preferred CPUs is always subset of active CPUs. + On feature disable it is same as active CPUs. + +This feature works best only when all the VMs enable the feature as +it is a co-operative scheme. If a specific VM doesn't enable this feature +it may end up with more CPUs than others, still should lead to better +performance when seen from system view. +Those who enable this driver must ensure it is enabled in all VMs. + +Module Parameters +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +interval_ms +----------- + +How often steal governor checks for steal time. +Default: 1000 i.e. 1 second. Value should be in between 100ms to 100sec. + +This controls how fast steal governor driver reacts to changes to the +contention of physical CPUs. Since it does a fair amount of work, setting +too low may have overhead. Setting it too high might render it ineffective. + +low_threshold +------------- + +lower threshold value in percentage * 100. +Default: 200, i.e. 2% steal is considered as low threshold. +Can't be higher than high_threshold. + +This determines what values should be considered as nil/no steal values. +When steal governor sees steal ratio is less than or equal to this value, +it will increase the preferred CPUs by 1 core. +Using zero might cause oscillations. + +high_threshold +-------------- + +higher threshold value in percentage * 100 +Default: 500, i.e. 5% steal is considered as high threshold. +Can't be lower than low_threshold. Must be less than 10000. + +This determines what values should be considered as high steal values. +When steal governor sees steal ratio is higher than this value, it will +reduce the preferred CPUs by 1 core. + +Limitations of default values +----------------------------- + +Because of the vast diversity in VM configurations (e.g., highly populated +vs. sparsely populated CPU masks, few offlined CPUs etc), the default +thresholds may not be optimal for all systems. Users may need to tune these +parameters based on the system under test to achieve the best results. + +For example: +Possible CPUs =3D 128 and Active CPUs =3D 8 +Steal on online CPUs =3D 50% +steal ratio: (50% * 8 + 0% * 120) / 128 =3D 3.125% +This would fall in between and default values won't work. +In this example, if one wants effective 2% and 5% limits, then set, +low_threshold =3D (2% * 8 + 0% * 120) / 128 =3D 0.1250% =3D 12 +high_threshold =3D (5% * 8 + 0% * 120) / 128 =3D 0.3125% =3D 31 + +Using possible CPUs helps to handle spikes during CPU hotplug as the steal +time across possible CPUs is a monotonically increasing value. + +Reasons for CONFIG_STEAL_GOVERNOR=3Dm +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Selecting this driver makes CONFIG_PREFERRED_CPU=3Dy. That makes configs +driven by user preference. Though one can have CONFIG_STEAL_GOVERNOR=3Dy, +It is recommended to build CONFIG_STEAL_GOVERNOR=3Dm due to below reasons: + +1. Doing periodic work has additional overheads. Enabling this driver + in systems where steal time cannot happen is of no use. There is no + benefit with additional overheads in such systems. + +2. This works well when all VMs work in co-operative manner. When an + administrative user enables it in one VM, he/she will likely enable + it all VMs. + +3. User can tweak the module parameters by reloading the module. diff --git a/MAINTAINERS b/MAINTAINERS index 15011f5752a9..40d46ba48ecd 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -25914,6 +25914,15 @@ F: rust/helpers/jump_label.c F: rust/kernel/generated_arch_static_branch_asm.rs.S F: rust/kernel/jump_label.rs =20 +STEAL GOVERNOR DRIVER +M: Shrikanth Hegde +R: Yury Norov +L: linux-kernel@vger.kernel.org +S: Maintained +T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched/core +F: Documentation/driver-api/steal-governor.rst +F: drivers/virt/steal_governor.c + STI AUDIO (ASoC) DRIVERS M: Arnaud Pouliquen L: linux-sound@vger.kernel.org diff --git a/drivers/virt/steal_governor.c b/drivers/virt/steal_governor.c new file mode 100644 index 000000000000..d427282966d2 --- /dev/null +++ b/drivers/virt/steal_governor.c @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Steal time governor driver periodically computes steal time. + * Based on the thresholds it either reduce/increase the preferred + * CPUs which can be used by the workload to avoid vCPU preemption + * to an extent possible in paravirtualized environment. + * + * Available with CONFIG_STEAL_GOVERNOR + * + * Copyright (C) 2026 IBM + * Author: Shrikanth Hegde + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#if !IS_ENABLED(CONFIG_PREFERRED_CPU) +#error "Steal Governor requires CONFIG_PREFERRED_CPU" +#endif + +struct steal_governor { + ktime_t time; + u64 steal; + unsigned long delay; + unsigned int interval_ms; + unsigned int high_threshold; + unsigned int low_threshold; + struct delayed_work work; +}; + +static struct steal_governor sg_ctx; + +static void restore_preferred_to_active(void) +{ + int cpu; + + guard(cpus_read_lock)(); + for_each_cpu(cpu, cpu_active_mask) + set_cpu_preferred(cpu, true); +} + +static int __init steal_governor_init(void) +{ + pr_info("enabled\n"); + return 0; +} + +static void __exit steal_governor_exit(void) +{ + restore_preferred_to_active(); + pr_info("disabled\n"); +} + +module_init(steal_governor_init); +module_exit(steal_governor_exit); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("IBM Corporation"); +MODULE_DESCRIPTION("Virtualization Steal Time Governor"); --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 83C89394461; Wed, 12 Aug 2026 05:42:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513364; cv=none; b=qjSOQySfP9mv/Esuy0snjfoQFRQKQLp/CQAEeaNTFjfpmB4YpIr7rFznfVhNqBkrL5pWeGYUaJY/M/3B9zW1kJscwCW3a4RNE1KoCfbLU5UybILMneNtLUbgNSHxylFdf2E6/lOAzsWDr6SX9z6Vz910sy9iAn03AGNolENg9wQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513364; c=relaxed/simple; bh=080FW7/pBVPrAEE6G6RZOuv7FM2p2iJePFSbtKUPn8A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=J3yB9gvD5D36eRt04iJXhpM7n7p0JMSJ9GFrhnOT9iLS4w/1eUPMOVk5KyQwbu4sBRpvcpiJr3ixdDP0hm0aNTySJ3IfM3HHBet+8X+yimW4LwAtntWOooFVdf/9QTZETCxeNGGMMdCswhWYH5UhjwqhOJ2aDxumBwlTFIdPPbY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=agUO8sa4; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="agUO8sa4" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31lxh2196170; Wed, 12 Aug 2026 05:42:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=+KXiQpoD6/H7hE/gH +GmtA+6Hl66Ta6Hh1ZBHVaDiHY=; b=agUO8sa4MLrQkbsGZR+/g5Px4CuMm9oW6 8L7IZ03vgD7XlUPZtzJKV1Jz5YSFBiwExIou0egA51uF0zGFtjW8OFtKiA49+Q/X NiMM12wMNrJO0s1smIM/ciO7D6NiK7mGCtaoeAvnq2YFDEjd8Gfq7DRy4F2yiseR mNPxO84QfTk5G5AWak0akk10zYKyf7mYdlOB9q47IulkD8M95HOY9C1aLHfygfiv Z03fPfuQFhy9Eqarz4l0VnP9fceD98u6fc7LGYYmpheN4um9v8vc2GyuIx25YA1X Jzl5v+X08FKycuZtkZMgYRvVeTdcIxw7HR46R/n9Md3MAifjRpC2A== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvp308cf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:23 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5fVAw012811; Wed, 12 Aug 2026 05:42:22 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4fxfsjvpcw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:22 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5gI0s35783144 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:42:18 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B544D2004B; Wed, 12 Aug 2026 05:42:18 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F5BA20043; Wed, 12 Aug 2026 05:42:09 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:42:08 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 10/12] virt/steal_governor: Add control knobs for handling steal values Date: Wed, 12 Aug 2026 11:10:31 +0530 Message-ID: <20260812054033.95658-11-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Authority-Analysis: v=2.4 cv=AMtp2X5w c=1 sm=1 tr=0 ts=6a7c07c0 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=XC_GqrfF-lTPz34eE_0A:9 X-Proofpoint-GUID: tNGAu6tFwgukEvDzRuqNVexWFQZ2_VvA X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX8TLLC0nP62qt Lst0z1yXp5NhsjiKGtd60SIDw4F4ASa7GsvbrFPcziudWEqjIKqhx+rGOnZxS8PiKG+ZA9VA9MB fCwgMTTBnJs0kpRC9fOoFReCEAe9/1bsHJCtAeiaryvZpmuWTfk/Pv0b8MWcCtE/drvz4CKZS4K PcOGOvZWxzTfIFVdAX/Cb4L8fxVA8sSAKMiB8d7JTwgt5gNoEl8Bkz5KAl0hSzCTUThZOql80ST G9OW9TEFWum5gCnEGHP58dDN3Xt+OW1tkknaAECZG0jaKS5Ig8S+L1UMUpV3Fp82MfD+DJ0ieGp 10Zhauw6/7t/SjFZeyz/mG0bJxmr5WDDOrQJaF3yx3mcC1HX7Qh9AmzsS8f3QF4Af3q0qFySpib gGOTKzfo5wVofBcIgkYNYKvl+EF81j4f+ea8VSBFfvQcMHFz1KysRHdggXYASFHN7iKBGqxzI2y 3Uyb16lhl2r7alfQEBg== X-Proofpoint-ORIG-GUID: Z4jRKlYBcGtx6ZVKpenT5zLHNzRLpe6k X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfXxfQHv5FbnnVS QBi/Nr2e78HzrZZv1DiqA9lNJyeLCgiJzUZjAS+KsNkf5HNdp7tuMyYudVHATo2LgUbQBZKRGQ1 jdGLBCMhM+whnWHQqbE1FOFnKbgd3Rw= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 priorityscore=1501 impostorscore=0 phishscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" These are the knobs to control the steal_governor. interval_ms: How often steal governor checks for steal time. (Default: 1000 i.e 1 second) This controls how fast steal governor driver reacts to changes to the contention of physical CPUs. Can be set between 100 to 100000. i.e. 100ms to 100seconds. 100ms is kept as minimum to ensure few meaningful steal values accumulate even with HZ=3D100. low_threshold: lower threshold value in percentage * 100. (Default: 200, i.e 2% steal is considered as low threshold) This determines what values should be considered as nil/no steal values. When steal governor see steal ratio is below or equal to this value, it will increase the preferred CPUs by 1 core. Having value as zero might cause oscillations high_threshold: higher threshold value in percentage * 100 (Default: 500, i.e 5% steal is considered as high threshold) This determines what values should be considered as high steal values. When steal governor sees steal ratio is higher than this value, it will reduce the preferred CPUs by 1 core. module_param_cb methods are used to do the validation checks. This helps to ensure one configures sane values. Since low and high are dependent, that check is done at module init. Notes: - Parameters values can't be changed at runtime. One has to unload the module and change it. Hence recommended to build it as module. - Default values cannot cater to all configuration such as sparse CPUs, selectively offline CPUs etc. Compute the values based on system under test. Documentation provides an example. Documentation is available at: Documentation/driver-api/steal-governor.rst Suggested-by: Yury Norov Signed-off-by: Shrikanth Hegde --- drivers/virt/steal_governor.c | 73 ++++++++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 2 deletions(-) diff --git a/drivers/virt/steal_governor.c b/drivers/virt/steal_governor.c index d427282966d2..fda86777d6f0 100644 --- a/drivers/virt/steal_governor.c +++ b/drivers/virt/steal_governor.c @@ -37,7 +37,11 @@ struct steal_governor { struct delayed_work work; }; =20 -static struct steal_governor sg_ctx; +static struct steal_governor sg_ctx =3D { + .interval_ms =3D 1000, /* 1 second */ + .high_threshold =3D 500, /* 5% */ + .low_threshold =3D 200, /* 2% */ +}; =20 static void restore_preferred_to_active(void) { @@ -48,9 +52,74 @@ static void restore_preferred_to_active(void) set_cpu_preferred(cpu, true); } =20 +static int param_set_interval_ms(const char *val, const struct kernel_para= m *kp) +{ + unsigned int interval; + int ret; + + ret =3D kstrtouint(val, 0, &interval); + if (ret) + return ret; + + if (interval < 100 || interval > 100000) { + pr_err("interval_ms must be between 100 and 100000\n"); + return -EINVAL; + } + + return param_set_uint(val, kp); +} + +static const struct kernel_param_ops interval_ms_ops =3D { + .set =3D param_set_interval_ms, + .get =3D param_get_uint, +}; + +module_param_cb(interval_ms, &interval_ms_ops, &sg_ctx.interval_ms, 0444); +MODULE_PARM_DESC(interval_ms, + "Sampling frequency in milliseconds. default: 1000"); + +static int param_set_high_threshold(const char *val, const struct kernel_p= aram *kp) +{ + unsigned int threshold; + int ret; + + ret =3D kstrtouint(val, 0, &threshold); + if (ret) + return ret; + + if (threshold >=3D 100 * 100) { + pr_err("high_threshold (%u) can't be more than 99.99%%\n", threshold); + return -EINVAL; + } + + return param_set_uint(val, kp); +} + +static const struct kernel_param_ops high_threshold_ops =3D { + .set =3D param_set_high_threshold, + .get =3D param_get_uint, +}; + +module_param_cb(high_threshold, &high_threshold_ops, &sg_ctx.high_threshol= d, 0444); +MODULE_PARM_DESC(high_threshold, + "High steal threshold. default: 500 i.e 5%. Must be > low_threshold"); + +module_param_named(low_threshold, sg_ctx.low_threshold, uint, 0444); +MODULE_PARM_DESC(low_threshold, + "Low steal threshold. default: 200 i.e 2%. Must be < high_threshold"); + static int __init steal_governor_init(void) { - pr_info("enabled\n"); + if (sg_ctx.low_threshold >=3D sg_ctx.high_threshold) { + pr_err("low_threshold (%u) must be less than high_threshold (%u)\n", + sg_ctx.low_threshold, sg_ctx.high_threshold); + return -EINVAL; + } + + sg_ctx.delay =3D msecs_to_jiffies(sg_ctx.interval_ms); + pr_info("enabled. interval: %ums, high_threshold: %u, low_threshold: %u\n= ", + sg_ctx.interval_ms, sg_ctx.high_threshold, sg_ctx.low_threshold); + return 0; } =20 --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C60893845D5; Wed, 12 Aug 2026 05:43:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513385; cv=none; b=hBV1e04PM099ftE7Nd9j1SeOS3kyYbO0HbNwv/HB3eachcqk3pz+0KIHZb6wvqVu54QEO8I0V1mKfBr1bScQEqGeYiu5YinZCz5G38/mIXoNtiW5dT1PV4d+nxI3mO8nLdd6Uorjpw5hgKX5CKdG3BcLoR1w/L8ZW0h7cYcMngA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513385; c=relaxed/simple; bh=/JUclBsxd26vHYlzOGz7pVWNvsdbBdr6QWISFFG8TLk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OMvy/v4MOhkIh0bYRAcIoed3CtDGlpWRCCnrbf9oWQucMLRo6uQAunoTN7Rvqrc5O2eGBE/s9hqpvSvgyZgvtrcqW8w3T5cgsqbC9zovGnE1z5QRrkvryTHq7RwerA7nJqPzYTX+uHy9oeD6xP+NCM0BFAY+2NEsKYhCgW9121w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=hygnijBY; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="hygnijBY" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31co92313501; Wed, 12 Aug 2026 05:42:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=9Vfp8/AU2oodjXbpB oSH0a9oP9qBxIBXvt15firfRTg=; b=hygnijBYw5lm/KZJnzCzcH/+DVQXoNzrf QXOonbFvmAV1I3PmiDDV1F4Z8l30LI4rlldjUaHWTsq9LZz5S1SgGQWCLKTEE+tO RW6xRTnEis/vCX48M1Y3RBqj6+rBiLAZQ8YJO9wW8xkZ3hQRHDbX7v1vWaIOzyET kc8PzwVUKX5u3+Ioh+84Fk1hoOrb8fWi9Bs30gH+GuD3EUTax7ufaZ8S1PzT8SI4 9BzlKMxF+9pfXZ1iXp4Q7UHC1iEnFhezLq6GmFT2qw75C7EJRtMYXzQHIwx1wZ2t iRR/LUEZcQdKyLJJr+k5/tU83GZz3eSYUSjo7ylT25rDaKcPv/qSA== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvnw87uq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:33 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5fVB1012811; Wed, 12 Aug 2026 05:42:32 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4fxfsjvpdf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:32 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5gTw033620466 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:42:29 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B4E420043; Wed, 12 Aug 2026 05:42:29 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2AEFA20040; Wed, 12 Aug 2026 05:42:19 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:42:18 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 11/12] virt/steal_governor: Implement steal_governor policy loop Date: Wed, 12 Aug 2026 11:10:32 +0530 Message-ID: <20260812054033.95658-12-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Authority-Analysis: v=2.4 cv=RsP16imK c=1 sm=1 tr=0 ts=6a7c07ca cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=zdfznts9tQKWGqFkqZ0A:9 X-Proofpoint-GUID: 0y9fYgof41JZ_nWwrT4ooV62pbpRCo5J X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX6KXBi/Jb9mZC mkXyxZ+GjnCvp3qn94yUa/Bo7cOI5D1+GuCHWi/+7bGOTisZFkyL57TXiPVGulj6PcQiJYHQRL4 /4d5IPx0+nGaZGI1qaYMH8WpXndmq3CvjkTiDp852qA5Ypo5zfnCf8Yr5EmGEdO3FUTZ4XlvJEM hDzz4Faz1BaqVBMG7RYQuMNQrUE6YPSzooVLeUi8zEFd2uY39knyqp7nJ+40kEu7GBsDE5sT8MS 1UU4HTytnKINXnpn83+nCiIGry/hG+C+MSysWEFRJqGaZA1VScrG2sdHGAhgz+ZAhV67SY+afOB ZUwRZRmTj4u0MnZ4QwDdnEyWRDANd+fzLhuV7RlPTcea38z8XRAzV/3NXKPEYYZ2Cgl6gsN2IZi C8qSIWLFc2jeWMAFDr03/qef8W/1W0aFEiYm+aL8/20iO/dAd5xWhgCDHwDS7rcRgvKklJRUNsm srqZUHAg1zVxC6YypCw== X-Proofpoint-ORIG-GUID: qXN_wrF7OFi-ioj8wNavSSN7AFpg615M X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX7hSg716b6511 yHr1gICvfEsXzcxJZrDLsZXY3jJ/F3pT1NK5NSuMqZ3CBrUNM0K2rewLY6eihgdv6PBYD+Zqb7y n5DUO8RWCWmSht9fnUBYCKtje0zP9us= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0 suspectscore=0 clxscore=1015 malwarescore=0 phishscore=0 adultscore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" Schedule work at regular intervals to implement the steal_governor policy loop, which monitors steal time and takes action on the state of preferred CPUs. The interval is determined by the interval_ms parameter. schedule_delayed_work() is used since interval_ms is on the order of milliseconds and the work does not need to happen instantly. Periodic policy loop essentially does: - Gets the total/delta steal values and cpus to use steal_ratio. - Calculate the steal_ratio as below. steal_ratio =3D (delta_steal * 100*100)/(delta_ns * num_cpus()) It is calculated this way to consider the fractional values of steal time. I.e 10 means 0.1% steal time. A few tricks such as divide by 10,000 are used to avoid possible overflow. - If steal ratio is higher than high threshold, call the method to reduce the preferred CPUs. - If steal ratio is lower or equal to low threshold, call the method to increase the preferred CPUs. - If the steal ratio falls in between, no action is taken. - Ensures design constraints always met. 1. At least one core/CPU must be there in preferred mask. 2. preferred CPUs is subset of active CPUs. If not met, then restore preferred CPUs to active and stop requeue of the work. Driver is effectively non-functional after that. Note that design checks are always performed. This helps avoid placing driver-specific design constraints inside the core CPU hotplug mechanism. User may offline specific set of CPUs that could leave the preferred mask as empty. With the design check performed always, driver gracefully shuts down upon detecting that edge case. In order to help the above loop, a few helper functions have been added. 1. get_system_steal_time() - steal governor takes global view of steal time instead of individual vCPU. Collect the steal values across the vCPUs of interest. - Sum up steal time values across possible CPUs. This helps to keep it a monotonically increasing number and avoids spikes due to CPU hotplug. 2. decrease_preferred_cpus() - Called when there is high steal time. It needs to decide which CPUs to mark as non-preferred. - Get first housekeeping CPU and its core mask. Mark it as protected core. This helps to keep at least one core as preferred. (kernel ensures at least one housekeeping CPU stays active.) - Find the last CPU outside of this protected core mask. i.e target CPU - Based on that target CPU, get its sibling and mark them as non-preferred. 3. increase_preferred_cpus() - Called when there is low steal time. It needs to decide which CPUs to mark as preferred and set that state. - Get the first active non-preferred CPUs. This likely is the last set of CPUs being marked as non-preferred. - get the siblings of that CPU and mark them as preferred. 4. get_system_cpus() - informs how many CPUs needs to be considered for steal_ratio calculations. - Return number of possible CPUs as get_system_steal_time computes steal values across possible CPUs. Notes: 1. Using core instead of individual CPUs performs better as SMT is quite common and some hypervisor such as powerVM does core scheduling. 2. This doesn't do any NUMA splicing to keep the code simpler and minimal overhead. Current code expects CPUs spread uniformly across NUMA nodes. Signed-off-by: Shrikanth Hegde --- drivers/virt/steal_governor.c | 149 ++++++++++++++++++++++++++++++++++ 1 file changed, 149 insertions(+) diff --git a/drivers/virt/steal_governor.c b/drivers/virt/steal_governor.c index fda86777d6f0..ee92152e64cc 100644 --- a/drivers/virt/steal_governor.c +++ b/drivers/virt/steal_governor.c @@ -13,13 +13,18 @@ =20 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt =20 +#include #include #include #include #include +#include #include #include +#include #include +#include +#include #include #include =20 @@ -108,6 +113,145 @@ module_param_named(low_threshold, sg_ctx.low_threshol= d, uint, 0444); MODULE_PARM_DESC(low_threshold, "Low steal threshold. default: 200 i.e 2%. Must be < high_threshold"); =20 +/* Return collective steal time across system. */ +static u64 get_system_steal_time(void) +{ + return kcpustat_field_total(CPUTIME_STEAL, cpu_possible_mask); +} + +/* Return number of CPUs to consider for steal ratio. */ +static unsigned int get_system_cpus(void) +{ + return num_possible_cpus(); +} + +/* + * Called when the steal governor detects high physical CPU contention. + * It finds the last active core in the preferred mask and mark those + * CPUs as non-preferred. + * + * Must ensure: + * - at least one core is always kept as preferred + * - preferred is always subset of active. + */ +static void decrease_preferred_cpus(void) +{ + const struct cpumask *first_hk_core; + int target_cpu =3D nr_cpu_ids; + int cpu; + + guard(cpus_read_lock)(); + cpu =3D cpumask_first_and(housekeeping_cpumask(HK_TYPE_KERNEL_NOISE), + cpu_preferred_mask); + if (cpu >=3D nr_cpu_ids) + return; + + /* Always leave first housekeeping core as preferred. */ + first_hk_core =3D topology_sibling_cpumask(cpu); + cpu =3D cpumask_last(cpu_preferred_mask); + if (cpu >=3D nr_cpu_ids) + return; + + /* Find the last CPU which doesn't belong to that first hk_core. */ + if (!cpumask_test_cpu(cpu, first_hk_core)) { + target_cpu =3D cpu; + } else { + for_each_cpu_andnot(cpu, cpu_preferred_mask, first_hk_core) + target_cpu =3D cpu; + } + + /* Only the first housekeeping core remains */ + if (target_cpu >=3D nr_cpu_ids) + return; + + for_each_cpu_and(cpu, topology_sibling_cpumask(target_cpu), + cpu_preferred_mask) + set_cpu_preferred(cpu, false); +} + +/* + * Called when the steal governor detects no/low physical CPU contention. + * It finds the first active core outside of preferred mask and mark + * those CPUs as preferred. + * + * Must ensure preferred is subset of active. + */ +static void increase_preferred_cpus(void) +{ + int first_cpu, cpu; + + guard(cpus_read_lock)(); + first_cpu =3D cpumask_first_andnot(cpu_active_mask, cpu_preferred_mask); + + /* All CPUs are preferred. Nothing to increase further */ + if (first_cpu >=3D nr_cpu_ids) + return; + + for_each_cpu_and(cpu, topology_sibling_cpumask(first_cpu), + cpu_active_mask) + set_cpu_preferred(cpu, true); +} + +static bool preferred_cpus_valid(void) +{ + if (cpumask_empty(cpu_preferred_mask)) { + pr_err("empty preferred mask. stopping\n"); + return false; + } + + if (!cpumask_subset(cpu_preferred_mask, cpu_active_mask)) { + pr_err("preferred: %*pbl is not subset of active: %*pbl, stopping\n", + cpumask_pr_args(cpu_preferred_mask), + cpumask_pr_args(cpu_active_mask)); + return false; + } + + return true; +} + +static void steal_governor_loop(struct work_struct *work) +{ + u64 curr_steal, delta_steal, delta_ns, steal_ratio; + ktime_t now; + + now =3D ktime_get(); + delta_ns =3D ktime_to_ns(ktime_sub(now, sg_ctx.time)); + + if (unlikely(delta_ns < NSEC_PER_MSEC)) { + pr_err_ratelimited("work scheduled too soon delta_ns: %llu\n", delta_ns); + goto requeue_work; + } + + curr_steal =3D get_system_steal_time(); + delta_steal =3D curr_steal > sg_ctx.steal ? curr_steal - sg_ctx.steal : 0; + sg_ctx.steal =3D curr_steal; + sg_ctx.time =3D now; + + /* + * steal_ratio =3D (delta_steal * 100*100)/(delta_ns * num_cpus()) + * To avoid possible overflow, divide the denominator early. + * Note minimum interval is 100ms. + */ + delta_ns =3D max_t(u64, div_u64(delta_ns * get_system_cpus(), 10000), 1); + steal_ratio =3D div64_u64(delta_steal, delta_ns); + + if (steal_ratio > sg_ctx.high_threshold) + decrease_preferred_cpus(); + else if (steal_ratio <=3D sg_ctx.low_threshold) + increase_preferred_cpus(); + /* + * else: steal ratio is within bounds. Still do design checks so that + * module restores to active if CPU hotplug breaks those assumptions. + */ + if (!preferred_cpus_valid()) { + restore_preferred_to_active(); + return; + } + +requeue_work: + schedule_delayed_work(&sg_ctx.work, sg_ctx.delay); +} + static int __init steal_governor_init(void) { if (sg_ctx.low_threshold >=3D sg_ctx.high_threshold) { @@ -117,6 +261,10 @@ static int __init steal_governor_init(void) } =20 sg_ctx.delay =3D msecs_to_jiffies(sg_ctx.interval_ms); + INIT_DELAYED_WORK(&sg_ctx.work, steal_governor_loop); + sg_ctx.steal =3D get_system_steal_time(); + sg_ctx.time =3D ktime_get(); + schedule_delayed_work(&sg_ctx.work, sg_ctx.delay); pr_info("enabled. interval: %ums, high_threshold: %u, low_threshold: %u\n= ", sg_ctx.interval_ms, sg_ctx.high_threshold, sg_ctx.low_threshold); =20 @@ -125,6 +273,7 @@ static int __init steal_governor_init(void) =20 static void __exit steal_governor_exit(void) { + disable_delayed_work_sync(&sg_ctx.work); restore_preferred_to_active(); pr_info("disabled\n"); } --=20 2.47.3 From nobody Tue Sep 29 06:07:42 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13F5E37DEBA; Wed, 12 Aug 2026 05:43:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513389; cv=none; b=PBfHgGVxNr/2LXtnAWlongPg49lySjZvl5FegpbRq0nix8YS/iux0WmWWH650ZhDxFJYaD/AfLg6y8KPfw6OdStYPP9pqeeXDxfrSdhCoT2FEsTzcJex2mAA36v8rqxfDQa33pjXASiszgk1DKvCvJV3Oz9y5q39843h8G/zfdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786513389; c=relaxed/simple; bh=TBYD3Ma107LANX8ztUdMoJjdekI0UIGp8ILn0r3D3HY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qZAF4AB7ISyF0x3lXhD46sTHt7ZeekPAgQBDI2pZsca/0saIdI3llFNPo7Wni3y2A9h5homIfDod0WTb+DV6WhNEVrfO+RZ/+5HNIv0yzhKFdPX4QhoNuLFyFhpeO/SJF2ZDurNxh/zZY7ltF9rKEg1cWaJ4tQWHceInIxkkCno= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=CCpotSEr; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="CCpotSEr" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 67C31eZI416548; Wed, 12 Aug 2026 05:42:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=BRkUi9dTiYNKZ4ssM Ju8kNyywnPRjg93Lb0gWTNKtqA=; b=CCpotSErOfINOci++OePtSZbPNQ/jR4zw FCyiOAUEAaWe93dwESxlEyscLrAgTtJTYYTRUXQDc1lGFe+zDiB6JHIKXeQyL3Ga W13LNJVGoOwzaxBBJbIdNrah7/hGwspK0oOFl0/Nu0Pe0tuClsKl4Pn9n1/3wira rzNIgsM3WRB2uYorH2ocMH9y7bO366tnsB6FN5aUK0eHgYkWK9vloSeiPXEkRv64 U9VfgvJaKkHZgQmui3LUVe9CBOCpnoPntMksYwJpEtQd+ZyzlgXZ64+BJER6DHqF h29QsfpwxuAl2vh3yNaVZW7W7XIRjBw/vRa4ow0+SemWeYBt9oGdw== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4fwvk00yns-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:44 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 67C5fHWb010376; Wed, 12 Aug 2026 05:42:43 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4fxh0gcj4s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Aug 2026 05:42:43 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 67C5gdsp32375210 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 12 Aug 2026 05:42:39 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C8CFC20040; Wed, 12 Aug 2026 05:42:39 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 77C6D20043; Wed, 12 Aug 2026 05:42:29 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.124.217.151]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 12 Aug 2026 05:42:29 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, corbet@lwn.net, meted@linux.ibm.com, ynorov@nvidia.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org, rdunlap@infradead.org, kernellwp@gmail.com, linux-doc@vger.kernel.org, jgross@suse.com, virtualization@lists.linux.dev Subject: [PATCH v10 12/12] virt/steal_governor: Enable the driver Date: Wed, 12 Aug 2026 11:10:33 +0530 Message-ID: <20260812054033.95658-13-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260812054033.95658-1-sshegde@linux.ibm.com> References: <20260812054033.95658-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfXwvucWGPicbZY NnxZI+LgMgIIUt+BXkVV3Bn0bZFHVXGtj7hJcAvIgB5U+I27Y9puMHLn1vcaTqYoWAVRvej+NMi 5X2fI36a295aKhKuFmj2R7XwYTMvBiLGTKB0KLWBZkPn4R5apyk9m87zLNs2FzWX1Yvl7xBSU6c +1sajpTcoStPcNg6/izONuxTj/ENHHosuJTHVHMAquZqbuQiM9dEEKZ+E8ItKxSTyDHPDhT/DhE m4DusMfQltl+2F5m/kkDnidniO7h6MjjTU58vhMlGvBeHyuhjltzFeASchF+AeBRxQWTgZAAA2y 7OU2FpWHCpdz9rwoscvXZD9WOmwME2XnPHeJnCMJaOh80/0JkJKRdFKs2jf/ZDcM2Va8C9NHXZA X4jERgMgbU5rDLNI4suCTED7J9iM5fNai00gjRnDEoPnbPSWj9D95LYFi4tF/MYUhSpxxWBEvg6 ajL3zWHZEulscAdoZ7w== X-Proofpoint-Spam-Info: AW1haW4tMjYwODEyMDA0MSBTYWx0ZWRfX/KM4XSZ583gb JC3PsJhGlkDg1ik9GwBtXJyYnzcwDAZsvJk+XnrJNHttX0y4cm4MxzcN8D3bmJCY+Kt48/rjR1W A0LuUHa34+DLAS5dnD5Qd/c0LHDl3dg= X-Authority-Analysis: v=2.4 cv=RqD16imK c=1 sm=1 tr=0 ts=6a7c07d5 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=Sv0fKeRqtYgA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=jPZz8W9ceouySDeeKuAA:9 X-Proofpoint-GUID: 74O_N2J7JsooKqFnOfo9qDp-x3ty-TP3 X-Proofpoint-ORIG-GUID: FqpydhhX7-4ecHn6p9n_gVrC8mogoktv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-08-12_01,2026-08-10_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 suspectscore=0 lowpriorityscore=0 clxscore=1015 adultscore=0 bulkscore=0 malwarescore=0 impostorscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2608120041 Content-Type: text/plain; charset="utf-8" Provide a config option to enable the steal_governor driver. Since the feature targets paravirtualized environments and requires SMP, enforce those dependencies. The driver selects CONFIG_PREFERRED_CPU for the core scheduler mechanisms to work. It is recommended to build the driver as a module (m) instead of built-in (y) due to the following reasons: - Module parameters are read-only after initialization. Building as a module allows updating these parameters by simply reloading the module. Default module parameters cannot work in all configurations. - The driver can be completely disabled by unloading the module. - This feature works best when all VMs operate in a cooperative manner. Requiring an explicit module load ensures intentional deployment across all VMs by the system administrator. Suggested-by: Yury Norov Signed-off-by: Shrikanth Hegde --- drivers/virt/Kconfig | 18 ++++++++++++++++++ drivers/virt/Makefile | 1 + 2 files changed, 19 insertions(+) diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index 52eb7e4ba71f..6681d06363f9 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -41,6 +41,24 @@ config FSL_HV_MANAGER 4) A kernel interface for receiving callbacks when a managed partition shuts down. =20 +config STEAL_GOVERNOR + tristate "Dynamic vCPU management based on steal time" + depends on PARAVIRT && SMP + select PREFERRED_CPU + default m + help + This driver helps to reduce the steal time in paravirtualized + environments, thereby reducing vCPU preemption. Reducing vCPU + preemption provides improved lock holder preemption and reduces + cost of vCPU preemption in the host. + + By default preferred CPUs will be same as active CPUs. Depending + on the steal time when steal_governor driver is enabled, + preferred CPUs could become subset of active CPUs. + + It is recommended to build it as module and load the module + to enable it. + source "drivers/virt/vboxguest/Kconfig" =20 source "drivers/virt/nitro_enclaves/Kconfig" diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile index f29901bd7820..05fb075ef5b8 100644 --- a/drivers/virt/Makefile +++ b/drivers/virt/Makefile @@ -5,6 +5,7 @@ =20 obj-$(CONFIG_FSL_HV_MANAGER) +=3D fsl_hypervisor.o obj-$(CONFIG_VMGENID) +=3D vmgenid.o +obj-$(CONFIG_STEAL_GOVERNOR) +=3D steal_governor.o obj-y +=3D vboxguest/ =20 obj-$(CONFIG_NITRO_ENCLAVES) +=3D nitro_enclaves/ --=20 2.47.3