From nobody Fri Dec 19 20:37:54 2025 Received: from mail-ua1-f99.google.com (mail-ua1-f99.google.com [209.85.222.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA1602F5469 for ; Wed, 3 Dec 2025 11:43:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762209; cv=none; b=KQF+f9RjlnW0Udm6VYfBoAHL8cgqcDw56IOUTW/+FKuq/NGZ9aiB+LAWD2nlalOT4kBTI0BEtJrlBC+aYQyCh9pTc1mwHMKbz3zzhRkpfgBgDAsDTwrxRIA5abhRoVDW7PALXauNrLBJqFxVbqcOaH4dAyTRNA2U0ke8+5klFSE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762209; c=relaxed/simple; bh=0haK8KYriPaTG6SlT0IxDG6YPomgA9ogsrkebaJgMvY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Src8SJ4u4nmSU7SQZdLa1GhKcRzYeK/WNZnOynHbnZYsPU/zYqla/KOemn4gknoSCFcAY0VmCwcajscGk444FtXaXPCgORcst4Z0NWaY6BLiiCQpIQJB9yULGPKvMIsKWNq2OpnTtw+5O5/BBbXGwWRw2RBKRM2ojS+2IF/d1t8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=c33GpDac; arc=none smtp.client-ip=209.85.222.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="c33GpDac" Received: by mail-ua1-f99.google.com with SMTP id a1e0cc1a2514c-93518a78d0aso3470049241.3 for ; Wed, 03 Dec 2025 03:43:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764762207; x=1765367007; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PH4ANwBhxoAuPZERs6KPB00evBiCDrU2sux4zg9pefE=; b=wVLzgiMQElO7SVNk2XvowBJjBKAcL1dU7StYOMi1kjFkwH3PgIVPC898ePter1g+1P d7g/bGJ+7eii1SqC0ATFRZj2lcBUMAYfqFZn+L3QaJega3pzccNDoljQgYV0Rk6R/VC9 nGTjcPCl4ciQPQKyPFiTzI3+BFv4EaglHoOet4loQwJa817feTFwgbg9/PI0Qy4H95r/ wEabFiHHqFmTdFcbo1tG+fRfGJIsl1LvUQSKRDT7+4frDa51xUQDtwpWupElShE1jtrW 5Eoe5uBU0pRFOzcVHcl6DGsPs40RNYBR3SDVj02dg/VI1udtnIIrd6EPZFIKtAuuOKdA t+AA== X-Forwarded-Encrypted: i=1; AJvYcCUDia7WY9Ed/iIk0T5kTJG7zfN25nwHxHMAMxfr8+FoiySWsHwf7lUaLW66ADYJBoilmPwo3c9Q+z2rz8E=@vger.kernel.org X-Gm-Message-State: AOJu0Yy7EENPdjAY4PC68bXbe20IoY0fBHNy7K2A6dtJywbAB5jdSVHh yOMu0n5eIeh7N3mep5AJJ2naO7xSYP1h8Xku9/EzMNcvgcDwBMQ01JyyEIRA7naT7DQkHkLWAH1 Rmm2+HV90zPWVS5dToPmgnIZVZUFxFr/YA7R+e5GwO+rAJE50t+T7a8KzlhjuDIgXQMCtPw3S9z iqpQx9gTUxDiXCinWCB7w7a6SqePRoMlk1b2YH9IUu0H9ADTkHZ1O5Yx/qNXShLHPizQ4m9lnkM y237YEjbcayns4vw9g= X-Gm-Gg: ASbGncsFLmxge4k2bEFI73KTzOvR8BaoKpClb9H1ruE6gJaM++ZFvRO5QS1NzGtxF2b COaM2i4dPC8eD8ogunNGrMFUfRvfnFV0FN1CoCkNcpUChPkW0+t54rjKgN7tpURWxjudBXzqrpd 7QYYzbcDrZK7xyn2GtNI11/ptFNjaLY0YX7SV9lB00FumNnUQBq8iTka+TsCeCZvAgQwnvPEs7H hMhjy70FinzWWa6ca3DLy+1qB6vlyHecSNCJNOTpPyIrz93YlYHOAs9wgCP4+kDPMhI2PUY52lj HnZVbAVe9EiEDisnvlOBPQvuhWYTKs6B4QFl9t3xvM6L2k42vWLAmqwtMpKA2rham2xRCSu1DKf 5XSTx+j1DbnBeJzEIsKkMeV3i64DitJXKrKviazuZqtu4DcnR2sYFPtUIBBdfXFPuWKtnvWRd6d HH86B/o4Qoxm9RospY1Jr8al74oTZvCmSECm738q92gw== X-Google-Smtp-Source: AGHT+IGJYu0qAH9E50S6K2W+VPo7xdzKsYgmuEfyk0r2qnGVjAeGcAoSkF1BIRU158zvLmJhQf6ZBfUKUl7H X-Received: by 2002:a05:6102:26ca:b0:5db:2828:c133 with SMTP id ada2fe7eead31-5e48e248052mr415775137.10.1764762206800; Wed, 03 Dec 2025 03:43:26 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-11.dlp.protect.broadcom.com. [144.49.247.11]) by smtp-relay.gmail.com with ESMTPS id a1e0cc1a2514c-93cd6c3c3cdsm501540241.1.2025.12.03.03.43.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:43:26 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-qk1-f198.google.com with SMTP id af79cd13be357-8b2217a9c60so2003975185a.3 for ; Wed, 03 Dec 2025 03:43:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764762206; x=1765367006; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PH4ANwBhxoAuPZERs6KPB00evBiCDrU2sux4zg9pefE=; b=c33GpDacu4sKVV4JVk/MiC3oBzKDaPajZvZN4Ke74wseoS+hHOLl4VXDIRVOmslLsa h4MG/15feVz7u4lRMHLSlbh9A5iSKA+s+ZyVvwYYNxzSIBQWLiZ9acmMvKsqnbS/XRd/ NNv7gqx6Z32tkuQHYdiW0ZHiJ5rZu7a2c/aGg= X-Forwarded-Encrypted: i=1; AJvYcCX+TEnscrhXIzviXxZtVWlbPxq1wAkt4hk8IwFlhSPvR50AP0n5D9WXsSMKgaZzHk/cveQCg4LFTBm0biA=@vger.kernel.org X-Received: by 2002:a05:620a:4049:b0:8b2:e87e:1093 with SMTP id af79cd13be357-8b5e47a151emr228686585a.3.1764762206314; Wed, 03 Dec 2025 03:43:26 -0800 (PST) X-Received: by 2002:a05:620a:4049:b0:8b2:e87e:1093 with SMTP id af79cd13be357-8b5e47a151emr228684385a.3.1764762205927; Wed, 03 Dec 2025 03:43:25 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id af79cd13be357-8b52a1b65bbsm1284727985a.33.2025.12.03.03.43.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:43:25 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Joseph Salisbury , Adam Li , Hazem Mohamed Abuelfotoh , Chris Mason Subject: [PATCH v6.1 1/4] sched/fair: Revert max_newidle_lb_cost bump Date: Wed, 3 Dec 2025 11:25:49 +0000 Message-Id: <20251203112552.1738424-2-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112552.1738424-1-ajay.kaher@broadcom.com> References: <20251203112552.1738424-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit d206fbad9328ddb68ebabd7cf7413392acd38081 upstream. Many people reported regressions on their database workloads due to: 155213a2aed4 ("sched/fair: Bump sd->max_newidle_lb_cost when newidle bala= nce fails") For instance Adam Li reported a 6% regression on SpecJBB. Conversely this will regress schbench again; on my machine from 2.22 Mrps/s down to 2.04 Mrps/s. Reported-by: Joseph Salisbury Reported-by: Adam Li Reported-by: Dietmar Eggemann Reported-by: Hazem Mohamed Abuelfotoh Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://lkml.kernel.org/r/20250626144017.1510594-2-clm@fb.com Link: https://lkml.kernel.org/r/006c9df2-b691-47f1-82e6-e233c3f91faf@oracle= .com Link: https://patch.msgid.link/20251107161739.406147760@infradead.org [ Ajay: Modified to apply on v6.1 ] Signed-off-by: Ajay Kaher --- kernel/sched/fair.c | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b6795bf15..f5a041bc3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10941,14 +10941,8 @@ static inline bool update_newidle_cost(struct sche= d_domain *sd, u64 cost) /* * Track max cost of a domain to make sure to not delay the * next wakeup on the CPU. - * - * sched_balance_newidle() bumps the cost whenever newidle - * balance fails, and we don't want things to grow out of - * control. Use the sysctl_sched_migration_cost as the upper - * limit, plus a litle extra to avoid off by ones. */ - sd->max_newidle_lb_cost =3D - min(cost, sysctl_sched_migration_cost + 200); + sd->max_newidle_lb_cost =3D cost; sd->last_decay_max_lb_cost =3D jiffies; } else if (time_after(jiffies, sd->last_decay_max_lb_cost + HZ)) { /* @@ -11630,17 +11624,10 @@ static int sched_balance_newidle(struct rq *this_= rq, struct rq_flags *rf) =20 t1 =3D sched_clock_cpu(this_cpu); domain_cost =3D t1 - t0; + update_newidle_cost(sd, domain_cost); + curr_cost +=3D domain_cost; t0 =3D t1; - - /* - * Failing newidle means it is not effective; - * bump the cost so we end up doing less of it. - */ - if (!pulled_task) - domain_cost =3D (3 * sd->max_newidle_lb_cost) / 2; - - update_newidle_cost(sd, domain_cost); } =20 /* --=20 2.40.4 From nobody Fri Dec 19 20:37:54 2025 Received: from mail-pl1-f227.google.com (mail-pl1-f227.google.com [209.85.214.227]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B2912F5A03 for ; Wed, 3 Dec 2025 11:43:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.227 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762212; cv=none; b=Yg3vghP9KzqcVROBFLfARPz8UR5gFn6RbFqdJbb0UXOViwYtxIXcXAIjPxFQedGsfXV1h2z/lCzivG1n2zwSWmCkBTlb61OwLG3gLeKQk7h5ZUYAgNI3X6YEanU+crYpylEnFzEJZGPWt0IsusBkvCfwCJ88OUOxkDrGYZnM5xw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762212; c=relaxed/simple; bh=6CPDYjvsBfV3Kcd80YC7yRDSERboeJo3FZe5VW1pFCQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uPonpTSDqc5XVeWGahHnZkNB1GfnO8VH/QFcQbYmgtTXMlD6p/xiFJCF9zJilrL3PZZiw+tHHSiZoHA20WlMQkbeIp2+VecKFzTNx+n4qFMoncnaR9r1f4I+dcOvR5WqBF5LTRn297SLmNeSkbQE7Y9yDr/QmQVPr8PJ0nDEl1k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=RDrzF7S+; arc=none smtp.client-ip=209.85.214.227 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="RDrzF7S+" Received: by mail-pl1-f227.google.com with SMTP id d9443c01a7336-297dd95ffe4so58329395ad.3 for ; Wed, 03 Dec 2025 03:43:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764762211; x=1765367011; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zG5ObQTvL5bWOa2rL3As4NqAHq0IGlQkowqc9ibYGAQ=; b=qKFrVMYhhMLMRZAtDnA9g78U2rQhfKecbGfswUxyNYWBi1HJZdw9ZanJKwj3Qs1uxV 5MT47bsoVG+fvf5vn5Pj6JHVOI8kT0aSnRXul5nAFxVk9LfhxVjDQXEABBtCf9bNwfES ajrZyxoVDPXFxyxsv6vBi04l8/UNjQValABnOCuYTkmfY7GH+bSD6ubWjMuvWWeGle5h 1CN+jUnZR+oRG1+XEE+gOBHmC8ft3XrCnwgScBX7kPeyCY9VNUj23a24mXiizLT1pkE3 GgaPxKcogwsLjhbsFoWuN9RmG2mNcyB2PInWUC7GOHaDWYqh0D6yEl+Y/K3Yqbur6WYd bG0g== X-Forwarded-Encrypted: i=1; AJvYcCVoT2bha6Im5js+12+4Y2U3c9oV3LCAG6ChTmUjE2wWLFGUgOCYC0Mz+gk5ANVcoV+M1fEQZRrM6T3VeSo=@vger.kernel.org X-Gm-Message-State: AOJu0Yy4ViqjNJVEBsTm57BSHiL/YM+erpBBJHY9KpfWXedo/bdQrBdq 13QKGVdpA4ebaXhLgXYVY40u2GXbZLvZdODPbb4JsWwcazGj1QU+/OLLTIILJN9jU1hffRZODCv JZS0dSLKRqo7Sqjpg5qQQgFLgrXEX1CHy/Ed6Z8Mn+R3/Lud3TazMBYGxJ7hSX1u/sdqmvR94ME 9pPzlODsA2Ojz744zpQ/o9bi0syZri2S7LckrRJdQmdGEetMtu4qOSKC+pkcIE98WdF3biUw0EB NIa4A+XIdZ+LI+v7po= X-Gm-Gg: ASbGncuKN4Ax6iRlpHMi9gmIKGgqgR/waYpYRTJ9/qpPDnYMI7BZj/3jCGqmsmGQLm8 CmAPGImuWvmCtmbxK4bBbuytVElAPpwl1N1/Ebs2qCNW1gBq581eCz05frpzKCOVo1l8iVpSf65 LIdt+h1cmTtUR4prs0rogXiQ0ov9jhBNhJsXT4B3hyX23/7GGrDS48hhRyBrL+ti0NPfim7hZut pC+AnQigWNPx71O/9JaIgAoXmNS3epXTI3gum4iCubXfkfIr5/Lxt7cGx2DN9v1nd8DxnskO/HU wKs3elSUxjWv0aOqsRuIOO8gaYVmUoZo6v/nhhz+Rk0aW7+j4FoTKGNXeBKvJp9LUpN/QPovPZL kdR90Ec6zjVqT0pMFkuFQh64rT00k7sZWaYnUTzFkKgCEpsBh/UqLOcKaHpVUjBT3tmN5gzk2+I NZAZ1FNkJFlbS0Fd9dWCYSnlkEyNvA4DFO9JxN/H0zrl4C X-Google-Smtp-Source: AGHT+IEiviN8Zjgz+aphRSFSB0qz3HmTHB7lRb/q5V7/C1cVkU4sdSwXHHOGfwe/eJo1iRocc0+aF5nvAfro X-Received: by 2002:a17:903:987:b0:298:33c9:eda1 with SMTP id d9443c01a7336-29d683f42f2mr19718745ad.43.1764762210655; Wed, 03 Dec 2025 03:43:30 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-103.dlp.protect.broadcom.com. [144.49.247.103]) by smtp-relay.gmail.com with ESMTPS id d9443c01a7336-29bceaac314sm25223545ad.29.2025.12.03.03.43.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:43:30 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-qk1-f199.google.com with SMTP id af79cd13be357-8b2e4b78e35so1277920285a.0 for ; Wed, 03 Dec 2025 03:43:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764762209; x=1765367009; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zG5ObQTvL5bWOa2rL3As4NqAHq0IGlQkowqc9ibYGAQ=; b=RDrzF7S+U/fsXbSOW2mHGkPI4c/dhEacqtxeabgXKZh0NRO/miYdEevVZlQDIBjGPz 6Kgy73myU/tp9pFxCxaEdC0UVLC5UFffsZcPEOdqnct3PcIKMyjPEyOKirdd+LN4UcWn SetzctYQvombdU8HifKyEQd2Y7jayoVWzHZjY= X-Forwarded-Encrypted: i=1; AJvYcCWZVjxlDse1HtJ/Q7SrZZ2mHyYKd1I2uQ+YiQIucYUbRSBpV6gGSTtMQQKwVkSKIMl4sGLNaQPcPIQ00eM=@vger.kernel.org X-Received: by 2002:a05:620a:40d4:b0:8b2:d26f:14a8 with SMTP id af79cd13be357-8b5e47a74f9mr206606485a.9.1764762209423; Wed, 03 Dec 2025 03:43:29 -0800 (PST) X-Received: by 2002:a05:620a:40d4:b0:8b2:d26f:14a8 with SMTP id af79cd13be357-8b5e47a74f9mr206596985a.9.1764762207862; Wed, 03 Dec 2025 03:43:27 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id af79cd13be357-8b52a1b65bbsm1284727985a.33.2025.12.03.03.43.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:43:27 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Chris Mason Subject: [PATCH v6.1 2/4] sched/fair: Small cleanup to sched_balance_newidle() Date: Wed, 3 Dec 2025 11:25:50 +0000 Message-Id: <20251203112552.1738424-3-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112552.1738424-1-ajay.kaher@broadcom.com> References: <20251203112552.1738424-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit e78e70dbf603c1425f15f32b455ca148c932f6c1 upstream. Pull out the !sd check to simplify code. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://patch.msgid.link/20251107161739.525916173@infradead.org [ Ajay: Modified to apply on v6.1 ] Signed-off-by: Ajay Kaher --- kernel/sched/fair.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index f5a041bc3..9a7aa83ca 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11589,14 +11589,15 @@ static int sched_balance_newidle(struct rq *this_= rq, struct rq_flags *rf) =20 rcu_read_lock(); sd =3D rcu_dereference_check_sched_domain(this_rq->sd); + if (!sd) { + rcu_read_unlock(); + goto out; + } =20 if (!READ_ONCE(this_rq->rd->overload) || - (sd && this_rq->avg_idle < sd->max_newidle_lb_cost)) { - - if (sd) - update_next_balance(sd, &next_balance); + this_rq->avg_idle < sd->max_newidle_lb_cost) { + update_next_balance(sd, &next_balance); rcu_read_unlock(); - goto out; } rcu_read_unlock(); --=20 2.40.4 From nobody Fri Dec 19 20:37:54 2025 Received: from mail-qk1-f226.google.com (mail-qk1-f226.google.com [209.85.222.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BD1C2F6167 for ; Wed, 3 Dec 2025 11:43:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.226 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762214; cv=none; b=MyZymZkooBGB30A6LH6vjLq6yQUm5BJGVJSu93LTZ86+7bGKRrKURGvdsOigqtoUoYyG7053QEOQNusHyvFMGBGyt+i7JWEJHX5/si8Znx5uoDUuEmFi2ZR73bE9uuSIWBd0Q6b+XAvbboQp6O7NxZrogXVzQ8ajfurd2uDH7cE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762214; c=relaxed/simple; bh=4J7PIolAnA+0ar2aFpYpjYwUmB3rJDZiGwj5VvC7eK4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WQqa88m2IzqhoqZZTk7pVXXxKhM87Am9VVm97l7Ka/Q2U2nGFp8p+lyoJ92nE29dkd5HU9D0/NVoI3LVSv2u+OAIA82k/AffF78gTHco90FGGKUkt88BANpb+1pZui0OVq48N0hjhBmYybDuGgXUSrV7/LXIJdT9qCnH9g/R144= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=eCvtRU7S; arc=none smtp.client-ip=209.85.222.226 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="eCvtRU7S" Received: by mail-qk1-f226.google.com with SMTP id af79cd13be357-8b2d7c38352so88245085a.0 for ; Wed, 03 Dec 2025 03:43:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764762212; x=1765367012; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ciwg4/3mhwvtZ0RPA5sXQX/xfpLeKOSvzNQ2tetDUpM=; b=vZ3LCQxwxsEtpryrvPOyYkd3CavF488qm5ymUozrk+Bd0DGBBKSw+wH+qOtnh5K51g ww9jHbzJWZ+R9xrQ22DpSAsVJGnOT438B+6pYPXFc7MIjcr2YaPjV+FjsaGEbX/g+UDO 5nMhqRTAiYhpffYfdRTRXEMeu28EstWIrlLr0i4bNLgrN/hYKJ7JV590AUNPJlZiTVGY /I38Et8B+WvUgYB2xZek3rZbBNOvf0EwVKe7BIux5KUY9xY5n9/eMJ4Dn66g+Z720B8J phGJesCyqbJDR4IbfvJnoeucatK+ihAKZ8Hek6bBDlbczo9/Q5Pi+74totgs+LsSc9jW OXkA== X-Forwarded-Encrypted: i=1; AJvYcCX31Sw6xnH11EK3t4jgAR03dBo1Mw0vyL26I9XORnT/VV1CwzZ3jTmcmKfhBwYw0axFZZeLC6fJPFJ69s8=@vger.kernel.org X-Gm-Message-State: AOJu0YwOdCcNEChTyQr5mN4uhp9WI7jBIr60fDAoe7hpXiloGrugrY+T dnmzY6X3RoOBYTi0uOrFltnafYZfxtHJjGIg7/BFrkODLxx5ZXDfhIEMqpym8EOkE5pzBtAHj2o FydbNlTs0g82lHfQFwdPmMAV41pgdql6tJwym/SkRTq5OR+U+qQi/k2aTXLy4FN8FDZ07mx1oda aFcy1dAwUrE9UNjBZn5mu1WmHAeO9EsT+vE7w23IbWeBYWuHiYiUyO7zL2WnPOAAkrYTWMm0iER 8SToFxB/b3Scfo/HeI= X-Gm-Gg: ASbGncsz9zPTZpjcxH8OIwBeo7rK3oDysBgp9+apfTnkr3tLb6eTXR+ruRdFT4t6Prf Gmz03Doj6wpCqE0xCvPacliKrcL0b9dHdIDobYxwDz1B40ue9YdEocAGv71FYJByk8OOsV5NQsQ trFRzskrV4YvkJVtc0ZcIJ23i/W5NuYj6Sf0pkHV7uYmyHq1sIcGmPvBn8uK7Kit2H/85gZiZFr OJOYdUYTBV0YcA9pV5kBliXCXCIyCgURnMFKK/chknCOMvWWIz748100hZqCVxH6wUn0oE0eXk7 dj1OJDnJ61vMrlc4kR4ukWUPDuvyjbwKb9fuoId6io1zkX8WOfsglDjPfff2FbRJVuKJeSU1XSP 4uVu+jOi5DYsXRAYavgCSlzmEHU7f2sg2h7dMQRqnQ16G1ZjppSnGtep5XWjld7OxJjivDe/owy qDrjDyk09pMQIlWBJGNn1AgtVFWciyjwAIh+4crgYDaA== X-Google-Smtp-Source: AGHT+IFUWAr9A8B5jlhTOa393+FIu6H5ZGdXbhm4tkfR5uog0aHPs8l0AbI84LsI0dY2/noUU4N1VuR0gtth X-Received: by 2002:a05:6214:53c9:b0:884:70ed:5414 with SMTP id 6a1803df08f44-88818d0bdf3mr35402466d6.17.1764762212037; Wed, 03 Dec 2025 03:43:32 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-11.dlp.protect.broadcom.com. [144.49.247.11]) by smtp-relay.gmail.com with ESMTPS id 6a1803df08f44-88652ae6291sm25177866d6.17.2025.12.03.03.43.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:43:32 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-qk1-f197.google.com with SMTP id af79cd13be357-8b55a033a2aso190795285a.0 for ; Wed, 03 Dec 2025 03:43:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764762211; x=1765367011; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ciwg4/3mhwvtZ0RPA5sXQX/xfpLeKOSvzNQ2tetDUpM=; b=eCvtRU7S2TvoYVerT7b6TmljL9Gma0+FVwnVaYKew/yGaH7ChZcJlSJjMoK+hn9Msu R1TJejhYZw+w9qyjT370GQkgsmC4s/sRnI5r6D2uxVCUuF5VZE2vOyns5rB2yMJFF4Wn faqPACpF8KPZ77Y+NMX1Am5+A3K/1tWdzbxOQ= X-Forwarded-Encrypted: i=1; AJvYcCUZ03axK5Il3VZ3gpoZd3Z+nKAlcv5GhpbRO4g0zIHRqR2b3WbeI18G9W2qbo8RYnuMN6D8WKDjv7pAU9g=@vger.kernel.org X-Received: by 2002:a05:620a:2681:b0:8b2:dd78:9288 with SMTP id af79cd13be357-8b5ac0684b3mr900416285a.13.1764762211384; Wed, 03 Dec 2025 03:43:31 -0800 (PST) X-Received: by 2002:a05:620a:2681:b0:8b2:dd78:9288 with SMTP id af79cd13be357-8b5ac0684b3mr900413085a.13.1764762210951; Wed, 03 Dec 2025 03:43:30 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id af79cd13be357-8b52a1b65bbsm1284727985a.33.2025.12.03.03.43.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:43:30 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Chris Mason Subject: [PATCH v6.1 3/4] sched/fair: Small cleanup to update_newidle_cost() Date: Wed, 3 Dec 2025 11:25:51 +0000 Message-Id: <20251203112552.1738424-4-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112552.1738424-1-ajay.kaher@broadcom.com> References: <20251203112552.1738424-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit 08d473dd8718e4a4d698b1113a14a40ad64a909b upstream. Simplify code by adding a few variables. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://patch.msgid.link/20251107161739.655208666@infradead.org [ Ajay: Modified to apply on v6.1 ] Signed-off-by: Ajay Kaher --- kernel/sched/fair.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 9a7aa83ca..2f296e2af 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10937,22 +10937,25 @@ void update_max_interval(void) =20 static inline bool update_newidle_cost(struct sched_domain *sd, u64 cost) { + unsigned long next_decay =3D sd->last_decay_max_lb_cost + HZ; + unsigned long now =3D jiffies; + if (cost > sd->max_newidle_lb_cost) { /* * Track max cost of a domain to make sure to not delay the * next wakeup on the CPU. */ sd->max_newidle_lb_cost =3D cost; - sd->last_decay_max_lb_cost =3D jiffies; - } else if (time_after(jiffies, sd->last_decay_max_lb_cost + HZ)) { + sd->last_decay_max_lb_cost =3D now; + + } else if (time_after(now, next_decay)) { /* * Decay the newidle max times by ~1% per second to ensure that * it is not outdated and the current max cost is actually * shorter. */ sd->max_newidle_lb_cost =3D (sd->max_newidle_lb_cost * 253) / 256; - sd->last_decay_max_lb_cost =3D jiffies; - + sd->last_decay_max_lb_cost =3D now; return true; } =20 --=20 2.40.4 From nobody Fri Dec 19 20:37:54 2025 Received: from mail-pj1-f97.google.com (mail-pj1-f97.google.com [209.85.216.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 762352F656A for ; Wed, 3 Dec 2025 11:43:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.97 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762217; cv=none; b=I5Bn4dJ4qlS8UdhiP7vUhtymlalCUlxKPtr55ISB8oktAJYWlNcsCi31Q2vft1RL1FMUMuNAQhDQX0ExzPIVJ6y15X/Gj/d18ST92gk7C8nV72M24AEiEiucPvhw+f4ABvEiotF/KxXBaaGlSf/AOtLxSMQoKOl3S66QvNzYAn4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762217; c=relaxed/simple; bh=UtdFMJwZBJWJFUXRVW5pS9k5RKZ7sSfvPSP4YuwCcxk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VdBpPucB+ddYlyVruM5TW4J6VEjYGMYW8NpVfFG/3yU+0MK6k38tUrSbOaqLfpk9NWRMfpse0b+MpGUUgCy1iKE1c+avVY0hA/xpGlugvxerMXkeIrTkBkqrxVxnbQ/TELqust6luBwTkm3CFEq8wULRqYtBo8ZdDuNI5X3tYMU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=ZjuHlBfM; arc=none smtp.client-ip=209.85.216.97 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="ZjuHlBfM" Received: by mail-pj1-f97.google.com with SMTP id 98e67ed59e1d1-3434700be69so9272389a91.1 for ; Wed, 03 Dec 2025 03:43:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764762215; x=1765367015; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=12d1g/kfvy76Un8DYq1Egc2YQIXGsJ6Y11du/XcNKeo=; b=s2nqREI1jxT2OVo+CHguaYk9fzu+W3eoykeguBaLtTu8zP+mkqDlsApu4OoAzZCnzt VEVasiOoJ1gERxMhL4Ku/DLH83CWI5ZVN7oKPYTYnc8DNvEm7C7gPZXVzNoGUTEIXnH4 KD9wZBSQEjxKFPUwBhAHxPy2Pt387xGHreouI01ip/69na1/qyR7qeeGYIkrE+KJpCTL BguyX0Q8UhYO9Slz/pz8WYQoAnx575ou67v+3vtEhizqPlJ1ijjCR6tvNkOj3v96rh7p mKAuUpI7XDt1x/vaDiDSguJpT0oblo33Ok7qpuSIDkTjniYKDrEEgJ4Qb9tBMmD4FP1N OzNg== X-Forwarded-Encrypted: i=1; AJvYcCVbiGGruNwR50KMwVFJWcG/z2G6EqirfgHXNgya+TU9qJ89o3C9n9dPO6KTLw1Vxfk53qM/hNfMZ/qMDI8=@vger.kernel.org X-Gm-Message-State: AOJu0YwXKetPotToqzXCWGpGbIsFSe0bXTLHmXGC5GKcYodP3Owqev89 5NUT1+vcJZ1X0fQ07Qyj6DN3n5N6HH/CEgnm83t6bCWny8Sz1/BVjg2o4PAf9I2yz8UvqwF5BBB Q26+Rzmlhv8BPN/C0Wri/nHrQEOb0HwnO57D/yDMKrxnrP+AbQjrY0loEMqFkyDBDZobcvzMdu7 0pkzXGQroDf9vl1WD4+41tnEyqUHvsXdNOdr4FxuvoSqpgD04FmPQ2AqloihdVNwwNcrtlKAIbm ixQkYSqZLRwGgkPQG4= X-Gm-Gg: ASbGncvGxAskZUXJCMYaf6iI13w3xmgNiJc29ljYsPBvtxBKa+4OnPTex1JGskVH3sG guPgJcQM1AArHUu8eXrbBnn8R1zAripv/eEAKADlyVtKvrWwHeInsAz7/wusfSj1wGPN5vDdU94 otuKLVb9hngZQigWEi8szlx444mbNOvglv/WeMq9xFoMCwIHJQWPr6askiVSdga0en6F+SXL2d0 GiBLChfeGPuzY3GghtMhXtKudtTQUNVSKiwIv7yP7tjs8iFU/uhVS3dxFdISBkhxgmi0+G3Z3xs zbaavypoGRz81pzmVvUEsR0WgCV3pFLw/DRZmFiFAJmshAZ9qyXqp1JBftcGDCY2uNT243L1vIl HGHigdLb7aycEZg1rJFIaPOiaWL66oihgoihMfQRyO3iPpRY2iP1nuurCzwt26WSHvl9vyMQg1x PUvN7rD3wFNXJSgcHQb4djOYBErtTEInOH9CM4oxvUcsuV X-Google-Smtp-Source: AGHT+IGts4sP6+HtK2g/F4hML138zGJXBQXEY21tceitxY0EBLW0FV7oiL7lexMsLjJH0hvjZYOzWTk6Af5x X-Received: by 2002:a17:90b:2f84:b0:340:ff89:8b62 with SMTP id 98e67ed59e1d1-349127f956amr2218286a91.21.1764762214715; Wed, 03 Dec 2025 03:43:34 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-103.dlp.protect.broadcom.com. [144.49.247.103]) by smtp-relay.gmail.com with ESMTPS id 98e67ed59e1d1-34910b1a9aesm260541a91.1.2025.12.03.03.43.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:43:34 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-qk1-f199.google.com with SMTP id af79cd13be357-8b22d590227so777741285a.1 for ; Wed, 03 Dec 2025 03:43:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764762213; x=1765367013; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=12d1g/kfvy76Un8DYq1Egc2YQIXGsJ6Y11du/XcNKeo=; b=ZjuHlBfMImMVIr3c5kHc3R0ugrEbwThQdtqqYJvFbVi/W+Ag2NmwGUPmiQGChnmmru F6bi6P+119vSL8KixZGbQVf9C2iAfjxBKYaKiPJfWBS4MxH0FJRpWD1DCzY2r66Yxs2S PfoHLtZJoVG+y7sGNPt2P9JYh6pogdQOmRKlY= X-Forwarded-Encrypted: i=1; AJvYcCV+v/1CmnAluXhwkyCbupVn9OLa3ua6qwRklWSy4vjJAuveVpV3ls8nb6JBYd+md2iklQs4PJ+Me8q10EE=@vger.kernel.org X-Received: by 2002:a05:620a:31a2:b0:8b2:6606:edaf with SMTP id af79cd13be357-8b5e5830538mr253363385a.37.1764762213308; Wed, 03 Dec 2025 03:43:33 -0800 (PST) X-Received: by 2002:a05:620a:31a2:b0:8b2:6606:edaf with SMTP id af79cd13be357-8b5e5830538mr253358885a.37.1764762212814; Wed, 03 Dec 2025 03:43:32 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id af79cd13be357-8b52a1b65bbsm1284727985a.33.2025.12.03.03.43.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:43:32 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Chris Mason Subject: [PATCH v6.1 4/4] sched/fair: Proportional newidle balance Date: Wed, 3 Dec 2025 11:25:52 +0000 Message-Id: <20251203112552.1738424-5-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112552.1738424-1-ajay.kaher@broadcom.com> References: <20251203112552.1738424-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit 33cf66d88306663d16e4759e9d24766b0aaa2e17 upstream. Add a randomized algorithm that runs newidle balancing proportional to its success rate. This improves schbench significantly: 6.18-rc4: 2.22 Mrps/s 6.18-rc4+revert: 2.04 Mrps/s 6.18-rc4+revert+random: 2.18 Mrps/S Conversely, per Adam Li this affects SpecJBB slightly, reducing it by 1%: 6.17: -6% 6.17+revert: 0% 6.17+revert+random: -1% Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://lkml.kernel.org/r/6825c50d-7fa7-45d8-9b81-c6e7e25738e2@meta.c= om Link: https://patch.msgid.link/20251107161739.770122091@infradead.org [ Ajay: Modified to apply on v6.1 ] Signed-off-by: Ajay Kaher --- include/linux/sched/topology.h | 3 +++ kernel/sched/core.c | 3 +++ kernel/sched/fair.c | 44 ++++++++++++++++++++++++++++++---- kernel/sched/features.h | 5 ++++ kernel/sched/sched.h | 7 ++++++ kernel/sched/topology.c | 6 +++++ 6 files changed, 64 insertions(+), 4 deletions(-) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index 816df6cc4..caeceec3e 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -106,6 +106,9 @@ struct sched_domain { unsigned int nr_balance_failed; /* initialise to 0 */ =20 /* idle_balance() stats */ + unsigned int newidle_call; + unsigned int newidle_success; + unsigned int newidle_ratio; u64 max_newidle_lb_cost; unsigned long last_decay_max_lb_cost; =20 diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 9b01fdceb..09ffe1b96 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -112,6 +112,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(sched_util_est_se_tp); EXPORT_TRACEPOINT_SYMBOL_GPL(sched_update_nr_running_tp); =20 DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues); +DEFINE_PER_CPU(struct rnd_state, sched_rnd_state); =20 #ifdef CONFIG_SCHED_DEBUG /* @@ -9632,6 +9633,8 @@ void __init sched_init_smp(void) { sched_init_numa(NUMA_NO_NODE); =20 + prandom_init_once(&sched_rnd_state); + /* * There's no userspace yet to cause hotplug operations; hence all the * CPU masks are stable and all blatant races in the below code cannot diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 2f296e2af..9f7c9083e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10935,11 +10935,27 @@ void update_max_interval(void) max_load_balance_interval =3D HZ*num_online_cpus()/10; } =20 -static inline bool update_newidle_cost(struct sched_domain *sd, u64 cost) +static inline void update_newidle_stats(struct sched_domain *sd, unsigned = int success) +{ + sd->newidle_call++; + sd->newidle_success +=3D success; + + if (sd->newidle_call >=3D 1024) { + sd->newidle_ratio =3D sd->newidle_success; + sd->newidle_call /=3D 2; + sd->newidle_success /=3D 2; + } +} + +static inline bool +update_newidle_cost(struct sched_domain *sd, u64 cost, unsigned int succes= s) { unsigned long next_decay =3D sd->last_decay_max_lb_cost + HZ; unsigned long now =3D jiffies; =20 + if (cost) + update_newidle_stats(sd, success); + if (cost > sd->max_newidle_lb_cost) { /* * Track max cost of a domain to make sure to not delay the @@ -10987,7 +11003,7 @@ static void rebalance_domains(struct rq *rq, enum c= pu_idle_type idle) * Decay the newidle max times here because this is a regular * visit to all the domains. */ - need_decay =3D update_newidle_cost(sd, 0); + need_decay =3D update_newidle_cost(sd, 0, 0); max_cost +=3D sd->max_newidle_lb_cost; =20 /* @@ -11621,6 +11637,22 @@ static int sched_balance_newidle(struct rq *this_r= q, struct rq_flags *rf) break; =20 if (sd->flags & SD_BALANCE_NEWIDLE) { + unsigned int weight =3D 1; + + if (sched_feat(NI_RANDOM)) { + /* + * Throw a 1k sided dice; and only run + * newidle_balance according to the success + * rate. + */ + u32 d1k =3D sched_rng() % 1024; + weight =3D 1 + sd->newidle_ratio; + if (d1k > weight) { + update_newidle_stats(sd, 0); + continue; + } + weight =3D (1024 + weight/2) / weight; + } =20 pulled_task =3D load_balance(this_cpu, this_rq, sd, CPU_NEWLY_IDLE, @@ -11628,10 +11660,14 @@ static int sched_balance_newidle(struct rq *this_= rq, struct rq_flags *rf) =20 t1 =3D sched_clock_cpu(this_cpu); domain_cost =3D t1 - t0; - update_newidle_cost(sd, domain_cost); - curr_cost +=3D domain_cost; t0 =3D t1; + + /* + * Track max cost of a domain to make sure to not delay the + * next wakeup on the CPU. + */ + update_newidle_cost(sd, domain_cost, weight * !!pulled_task); } =20 /* diff --git a/kernel/sched/features.h b/kernel/sched/features.h index ee7f23c76..0115183ee 100644 --- a/kernel/sched/features.h +++ b/kernel/sched/features.h @@ -99,5 +99,10 @@ SCHED_FEAT(UTIL_EST_FASTUP, true) =20 SCHED_FEAT(LATENCY_WARN, false) =20 +/* + * Do newidle balancing proportional to its success rate using randomizati= on. + */ +SCHED_FEAT(NI_RANDOM, true) + SCHED_FEAT(ALT_PERIOD, true) SCHED_FEAT(BASE_SLICE, true) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 95afded0b..6f66a9b1a 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -5,6 +5,7 @@ #ifndef _KERNEL_SCHED_SCHED_H #define _KERNEL_SCHED_SCHED_H =20 +#include #include #include #include @@ -1190,6 +1191,12 @@ static inline bool is_migration_disabled(struct task= _struct *p) } =20 DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues); +DECLARE_PER_CPU(struct rnd_state, sched_rnd_state); + +static inline u32 sched_rng(void) +{ + return prandom_u32_state(this_cpu_ptr(&sched_rnd_state)); +} =20 #define cpu_rq(cpu) (&per_cpu(runqueues, (cpu))) #define this_rq() this_cpu_ptr(&runqueues) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index d404b5d2d..9d6ec8311 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1584,6 +1584,12 @@ sd_init(struct sched_domain_topology_level *tl, =20 .last_balance =3D jiffies, .balance_interval =3D sd_weight, + + /* 50% success rate */ + .newidle_call =3D 512, + .newidle_success =3D 256, + .newidle_ratio =3D 512, + .max_newidle_lb_cost =3D 0, .last_decay_max_lb_cost =3D jiffies, .child =3D child, --=20 2.40.4