From nobody Fri Dec 19 21:12:21 2025 Received: from mail-oa1-f97.google.com (mail-oa1-f97.google.com [209.85.160.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 528B52F5467 for ; Wed, 3 Dec 2025 11:40:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.97 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762021; cv=none; b=mPOJVxryCP3p9VD/nPq+z2CGJ2BqWdo6kx/Cd6jWZ6skErRuU8j2C1bGaBMKCXb4sHzawWGvQdij4+7haJ9D94WWDuAjXN3If7z7NLAD6VS26VAApIQDFvc+rpnR0N/ErrakplPLOBectdH3fg7Jcv9FCw3ylzq86kaQjWxF/7k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762021; c=relaxed/simple; bh=1gdc28kwhY19UQraDdo+Rzp2Og6mCveOvOii9v+y3lw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aJd6sJRySwjI2EhycgguEWDnocUNkueGvppWpQaLLy6gdEpMWZvj343Z8YMu9NaNGeTa5c0DkYJ42z/QO+bKq6I0C1WEPQNc/sLB4DMMJmmfxUcIZstjN+k6vnMojpfhgGjxbb1dzcHbeqOVV29PSTTMLyW0yInZT9VGGB68/pM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=Fa9O9KeQ; arc=none smtp.client-ip=209.85.160.97 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="Fa9O9KeQ" Received: by mail-oa1-f97.google.com with SMTP id 586e51a60fabf-3f0cbfae787so1823580fac.3 for ; Wed, 03 Dec 2025 03:40:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764762019; x=1765366819; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Brn42TT1Ey4jdpd/BGskKy4rpvE127k8ANXujp7zLgY=; b=G0YfiZc+YR3J3a9vFhS7cJSZW4fodhZjWtEJJxGKSccgzY8bgljynL4k3VgzX8hhwB r7w+v6hHM0k0uixpXdhUuc7zwDp+qnmyXummCZ4mAWiMANif2mAr4jfL3TuiMstR+J+G hPpyQo2mUsvExOHmlDvcPZxmjUz60LIO7XK5x7MVxkkTPjbcA8D3fZPFFHeY8iYWCBbi zORORgUPwVuUpPUkSlfUjoauQ+Q4DIQICJ4+n+uBXr8TvK+8ahc7FlrQgJK9N0lQL8Fn CDTr6ezUkSNNfM9LARz+bE719IMSkHmLn6ydEvK3WOl1gwL9Zpu3J5dhuciZVrxqd/EI 7xRQ== X-Forwarded-Encrypted: i=1; AJvYcCV4PXvt6jQHJMohUJltmKKiEHKFpCGjuJ6NsTWkappZH/EbOBnXzlDvLYlkMD4aKr24Njnwqd1OBrwcSaA=@vger.kernel.org X-Gm-Message-State: AOJu0YzJf6SZCMoALIyG0A7axL7bkhv/vjClZyRnyJagdaNS9cSisuz2 sgZCbPLmea9ICuealrK0eLFS+dFEJhJBH+s99lhOVKL6zuk3VTgwBMIuuICJjyRldwlg7LkR7yd bs2OFLJJJ5hyhzMc0w0Lzmysi2z6ZVJ2e4jXI2rHBICrREbTEmCjACeaINUa6qBcBDeOs6lGTpR 7zMVkou+ZtitAvltwRTar+uGr3IKWoaE7Q0ZfYUyOC6UFkbcmdbeK09ey1kJgMMCIvOWCmoxXP0 +VoYLZ5YBMobZCZLj0= X-Gm-Gg: ASbGncv1/qwrzn8u3NxkwVXWLj/8aq5MDljEvs+tHngwNJKS+QHDTxGQA4a31Hei6fx 43eCGPi9Mzu3FgwH0tpyN/l4Ax6frmifEO3cCnRjso3/UxuVBkkjjDd0JTbRPPra2PPm2NLHXGJ WkJj7+7stZXnqFqvm5ZDDJFanED5win8G0okQygEflimF//9JpW+TqTAN/U6QKynCeqzb4t1FIj yLAZUIJaDDO5Pvie8PiYHDY9itCIcHCNxLmnkJ5apShdcRSgoZqfW11rnTc0hAkNplZGDCiX/TU MseXXa3WMziPmjt9aunujML3BrWl75hBk/kLkcIBhqSysdQIMYL+dHARgE2zvrJ2DUsTrdDHZlz +itGd8NsKmmO5I3KTheVJpyw1no9V8ftQyLUkYoD8v0i+egTUa4n6XptfOnfK/G+a6aUG3ARJ03 SpNuZXPBZ/INGiUH2vbHYMCk8ATNtd8ppVkT57FgiMZA== X-Google-Smtp-Source: AGHT+IGkL9ckYZGPw+1gLUMUioAhQM7RoTsYGDxbumSvjTLhxy3qAk4i1mUksfIsyatnEZawcAVTUXySG9+b X-Received: by 2002:a05:6871:811:b0:3f1:528e:c727 with SMTP id 586e51a60fabf-3f16940fef5mr1034354fac.47.1764762019179; Wed, 03 Dec 2025 03:40:19 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-11.dlp.protect.broadcom.com. [144.49.247.11]) by smtp-relay.gmail.com with ESMTPS id 586e51a60fabf-3f0dca40914sm1837536fac.8.2025.12.03.03.40.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:40:19 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pg1-f198.google.com with SMTP id 41be03b00d2f7-bf06c3426b7so3349383a12.3 for ; Wed, 03 Dec 2025 03:40:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764762017; x=1765366817; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Brn42TT1Ey4jdpd/BGskKy4rpvE127k8ANXujp7zLgY=; b=Fa9O9KeQJrmuJnYbn3aXC5HldDURzEwZIlhNIi0XbdcnHb+eJxWTLBVAEEaQ8MIDUB 9Vc4bR/Vu0f5Csz67C29cR3Ojh7zJnSmHKDfSdh5UCE35b3+gKEcokiaomEh/6AUnUVl wWb3tP51qeQqvMuvvkTrCbwJWOT2nyxjpuVs4= X-Forwarded-Encrypted: i=1; AJvYcCU/4iL0TP7795xNdCCYUYvlFCrJXoXppzGLdHKuLt4Q7Z0CWoaKsgIscgmaUfXCTe+U680BmLNXu6owJiE=@vger.kernel.org X-Received: by 2002:a05:7300:d09:b0:2a4:3593:9686 with SMTP id 5a478bee46e88-2ab92d39d8amr1444460eec.3.1764762017366; Wed, 03 Dec 2025 03:40:17 -0800 (PST) X-Received: by 2002:a05:7300:d09:b0:2a4:3593:9686 with SMTP id 5a478bee46e88-2ab92d39d8amr1444441eec.3.1764762016700; Wed, 03 Dec 2025 03:40:16 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-2a965b1ceeesm63324781eec.5.2025.12.03.03.40.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:40:16 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Joseph Salisbury , Adam Li , Hazem Mohamed Abuelfotoh , Chris Mason Subject: [PATCH v6.6 1/4] sched/fair: Revert max_newidle_lb_cost bump Date: Wed, 3 Dec 2025 11:22:52 +0000 Message-Id: <20251203112255.1738272-2-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112255.1738272-1-ajay.kaher@broadcom.com> References: <20251203112255.1738272-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit d206fbad9328ddb68ebabd7cf7413392acd38081 upstream. Many people reported regressions on their database workloads due to: 155213a2aed4 ("sched/fair: Bump sd->max_newidle_lb_cost when newidle bala= nce fails") For instance Adam Li reported a 6% regression on SpecJBB. Conversely this will regress schbench again; on my machine from 2.22 Mrps/s down to 2.04 Mrps/s. Reported-by: Joseph Salisbury Reported-by: Adam Li Reported-by: Dietmar Eggemann Reported-by: Hazem Mohamed Abuelfotoh Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://lkml.kernel.org/r/20250626144017.1510594-2-clm@fb.com Link: https://lkml.kernel.org/r/006c9df2-b691-47f1-82e6-e233c3f91faf@oracle= .com Link: https://patch.msgid.link/20251107161739.406147760@infradead.org [ Ajay: Modified to apply on v6.6 ] Signed-off-by: Ajay Kaher --- kernel/sched/fair.c | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 7f23b866c..842d54a91 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11710,14 +11710,8 @@ static inline bool update_newidle_cost(struct sche= d_domain *sd, u64 cost) /* * Track max cost of a domain to make sure to not delay the * next wakeup on the CPU. - * - * sched_balance_newidle() bumps the cost whenever newidle - * balance fails, and we don't want things to grow out of - * control. Use the sysctl_sched_migration_cost as the upper - * limit, plus a litle extra to avoid off by ones. */ - sd->max_newidle_lb_cost =3D - min(cost, sysctl_sched_migration_cost + 200); + sd->max_newidle_lb_cost =3D cost; sd->last_decay_max_lb_cost =3D jiffies; } else if (time_after(jiffies, sd->last_decay_max_lb_cost + HZ)) { /* @@ -12403,17 +12397,10 @@ static int sched_balance_newidle(struct rq *this_= rq, struct rq_flags *rf) =20 t1 =3D sched_clock_cpu(this_cpu); domain_cost =3D t1 - t0; + update_newidle_cost(sd, domain_cost); + curr_cost +=3D domain_cost; t0 =3D t1; - - /* - * Failing newidle means it is not effective; - * bump the cost so we end up doing less of it. - */ - if (!pulled_task) - domain_cost =3D (3 * sd->max_newidle_lb_cost) / 2; - - update_newidle_cost(sd, domain_cost); } =20 /* --=20 2.40.4 From nobody Fri Dec 19 21:12:21 2025 Received: from mail-pg1-f228.google.com (mail-pg1-f228.google.com [209.85.215.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 439E52F547F for ; Wed, 3 Dec 2025 11:40:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.228 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762022; cv=none; b=dH7TP9WE8rbTQWIM9h0eMfyDGmnPJmjV/4e9SPLGd8hG51BTvBzn06tFwFRccT5fzopGZ+cgOebYODvCDNBl4VF39lSyIspvfiN2lnZQmoV7wkP5LL/9/E4SzIlOy9yAeGjITXdNuTWvk6Yc6sC54Mzpkrc+cItrRsjINzMy9Mc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762022; c=relaxed/simple; bh=TXO5NRp5c5ShzA2cY2ZOV+GiFeT9eyDUZ2fy1Hg+chk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YpX3egZalQzSLetifq5E46avCimQvAe5d1HLR3PL/o9e6iUMflGNcELULZJlh1EUsYMWuG5hsXLZu/KkL5T3SRgJFM8QAAh2gVWeEvcVDPD69VJ0ypVrZ7rt21/8om4fIPYkVPb+kEpXD2lbNg4B8AWKcfDWH8o4XJ4TgYZFMNg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=ZR30DkVY; arc=none smtp.client-ip=209.85.215.228 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="ZR30DkVY" Received: by mail-pg1-f228.google.com with SMTP id 41be03b00d2f7-b4755f37c3eso4309485a12.3 for ; Wed, 03 Dec 2025 03:40:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764762020; x=1765366820; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7amVhtvbqglJSO4BYYfWbBRtTIKW47Hm7wnmb8VHueI=; b=cAWUAkLCA5MEbXTItYcIm3a34tLxy1kxsA4YiUZDTC68oq2rUGASmIwvF54oAAbueY oGrjyl/9oQV4TY9uo9RP9g3CfDt0XuYNkCmpBQBQYP7CqEI60nLfiTHZxzS6hKQnCpN6 2owWI+smVMFZd2mKEanwGoHmShEWoIoOf0uXlm3/jcNNFkzwi1aG9zLdNggYs0AgiPgF chJsUW+BsFGaxbboNWqjv0SU5jjwoS9lN3vA3keEbLPHgcrrL4pUQJpvfthJyY2pD97r l3WNoAfGjPfRb1dHStxvTP5tkjvVZ3jptilc/W6/469fz0zGLFKfljW5ENPy6YLtpWdB Tmnw== X-Forwarded-Encrypted: i=1; AJvYcCV1FCDwnvJT/hMsBZ7R8PJvEYrEzHlovl7owzrZ87kTE4EnhJxF0c5ha+mQ77wb9SRlRKvD3D1bqNB8lmg=@vger.kernel.org X-Gm-Message-State: AOJu0YySaygD8x+nwZNgv4kHW/Qjr3n10FwuaEImbH6pnkwdbH5juq6a ynDePBKzdCIHbqHRv8A9zfZ5OYbn1sJ6Pi7oT1TUZYS/2FpF/22/jlYBdZMtgx63iK/GmHjVNdQ 3sKQCfRcUua+XRq9DASrppJsfJkj/asRUi1WnmOmG1LkiTsmJuGQYF0xE1ftQGtlKMA8J2ya1VO ulCRKINrPkg4+OMpJsuqfnhFXD9yxVXBYOkmJHD2MTfCEBJgXcrZI2CUyWTnUi4pokCaMX9FZom zKcm7NxJmg3bu0z4ds= X-Gm-Gg: ASbGncu0qdt3NpPGeuM1zevLn6fJNajPqWz5sOhoGZzJYuqVJOLSi/YR2AH24ENBlgO o0qdBIH/lEHWxhVSyO5oh6/N7aUZIh+y+1N3m/YdBswePe3+TfAAHN34ybYvkCtRupTIWYEP5P9 ZeemmMn0DAdyFtHyIJaMNL8ecn0cE8Zc4qlA6UxIFcOJcZ1Xh8BdxCVpA9n2mdpah8c/Vmmdwnj 5cwbAlfFOmd9M05mhBsJOs+JJ5GuhW7GDQOrIXTNF+oII5PUbxtsGBaLwqnhGHIRnefl8Hlqo53 IU6mjDfp0P91TPs9CDBwiOpZvHYWUy0ZqJghGCFMNMvyeofbqAvLvBs04G/JnFRAPFJgo82mX3P 8houzDWwLuHdrCi5vAU0q4jINcQlnyEl0nxOzGNvO7TCTG1CZGCp5vfLdy2OOYIIImvklUaJaMw E7jV1QygKuDAsBahNwxunFG8eHQoiv/k/8TGP16g6Hsp7l X-Google-Smtp-Source: AGHT+IGPVODMQPOGLt3YYFR9bsSg4fiAj1eDOnr/nkb0YDXHI+iuXrygsf4wTZLSKVKoDkh9ce+2fZNMfeKZ X-Received: by 2002:a05:7022:6610:b0:11a:2ec0:dd02 with SMTP id a92af1059eb24-11df0c507e6mr1719707c88.33.1764762020354; Wed, 03 Dec 2025 03:40:20 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-103.dlp.protect.broadcom.com. [144.49.247.103]) by smtp-relay.gmail.com with ESMTPS id a92af1059eb24-11dcb033199sm2523724c88.4.2025.12.03.03.40.19 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:40:20 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pg1-f200.google.com with SMTP id 41be03b00d2f7-be8c77ecc63so5819940a12.2 for ; Wed, 03 Dec 2025 03:40:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764762018; x=1765366818; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7amVhtvbqglJSO4BYYfWbBRtTIKW47Hm7wnmb8VHueI=; b=ZR30DkVY36hEaEfuFdqDX72BWm6wWQsr5dgvaKIS9h64neBLnEJKZZWqjsANu6km1b NIyDKQDbntGtkvRM0IawWUgr/vfDgHWOn/uewg49V8IzolFNszdBDp3ElT686QgZqjNf 8EzpWV95snjPwJfxieTJE8axNinxLtFw00rfo= X-Forwarded-Encrypted: i=1; AJvYcCU8aNDA8pYSK+XlfP28bffWOQt5mx9WnReNYjWxmRRNvd+07ZY2fwcX/eCzb1K1rUaCvFqaMlxDUqhxzro=@vger.kernel.org X-Received: by 2002:a05:693c:800d:b0:2a4:617a:419f with SMTP id 5a478bee46e88-2ab92da3943mr1071054eec.2.1764762018407; Wed, 03 Dec 2025 03:40:18 -0800 (PST) X-Received: by 2002:a05:693c:800d:b0:2a4:617a:419f with SMTP id 5a478bee46e88-2ab92da3943mr1071036eec.2.1764762017869; Wed, 03 Dec 2025 03:40:17 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-2a965b1ceeesm63324781eec.5.2025.12.03.03.40.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:40:17 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Chris Mason Subject: [PATCH v6.6 2/4] sched/fair: Small cleanup to sched_balance_newidle() Date: Wed, 3 Dec 2025 11:22:53 +0000 Message-Id: <20251203112255.1738272-3-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112255.1738272-1-ajay.kaher@broadcom.com> References: <20251203112255.1738272-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit e78e70dbf603c1425f15f32b455ca148c932f6c1 upstream. Pull out the !sd check to simplify code. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://patch.msgid.link/20251107161739.525916173@infradead.org [ Ajay: Modified to apply on v6.6 ] Signed-off-by: Ajay Kaher --- kernel/sched/fair.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 842d54a91..e47bf8d6c 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12362,14 +12362,15 @@ static int sched_balance_newidle(struct rq *this_= rq, struct rq_flags *rf) =20 rcu_read_lock(); sd =3D rcu_dereference_check_sched_domain(this_rq->sd); + if (!sd) { + rcu_read_unlock(); + goto out; + } =20 if (!READ_ONCE(this_rq->rd->overload) || - (sd && this_rq->avg_idle < sd->max_newidle_lb_cost)) { - - if (sd) - update_next_balance(sd, &next_balance); + this_rq->avg_idle < sd->max_newidle_lb_cost) { + update_next_balance(sd, &next_balance); rcu_read_unlock(); - goto out; } rcu_read_unlock(); --=20 2.40.4 From nobody Fri Dec 19 21:12:21 2025 Received: from mail-pl1-f225.google.com (mail-pl1-f225.google.com [209.85.214.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DB8D2F5A13 for ; Wed, 3 Dec 2025 11:40:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.225 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762023; cv=none; b=gz5sfhL6VoKs1MQd0JL/3CHqGOW47p1lNNBRJFf5Ch5FYsriauEZAi2C8IMO+X0lcGppTJuCdwUxrLUxEPxTijoUJZykLOd691+cMWL9G1yehY1PcGQUc26Dx3ocKPlWcqJO6NSfVGfKMkuAlNeBZ2GpeWsGJ9I28KLDQSv0HpI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762023; c=relaxed/simple; bh=9EPDhCCtWaRo4rHvSMa36kDhj/Pz8zBcRzcppxUZU6c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bgqVvVplwPYsJktlMslOUKi5JmomWLrSSmBBaBjCflEAmvFq10zBtWYYc1+BMWkg6r/PEAJT2KZisNtFYFFQTSfiYVYBvzphs6loHsE0aCoib9T1TF7IoGiCoRSgiLDjHBpo4pDyZmH2e4YbPrmQ0Lr7aUCsVwXYEImXe/GL7QA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=AB8/2xrR; arc=none smtp.client-ip=209.85.214.225 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="AB8/2xrR" Received: by mail-pl1-f225.google.com with SMTP id d9443c01a7336-29ba9249e9dso73837435ad.3 for ; Wed, 03 Dec 2025 03:40:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764762021; x=1765366821; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1iH4y3Zrgl56i/VHQKgTMa01v/ge6GrCshnPXW5Ao8U=; b=O9v0HvEhXNYzQP68lDuJtDu1O1CbN9XvkupP9oIPtwQEFRVCnSTf4vJji9QKadbZn0 vKjjmiKmzlyMGLhbDTpBmS73lYQpK/2GM+SYBa0yR1D2GK4Qv8o5w/bBWx60y/UAgsy/ pyhmUSuBbbe9GfXfZhOwWb6Kq/QOv3qLlAknFZWp38q5lHAYfw8mhT1f/Px0piO9xKYH wcVBmSpFji9iAaF9zgO0oTLaXCDzir004xLOJa1i9rI6+YyavuSIy18WFaNqRjdxYANY 1Q2H5scqrZ3C5RWGl2bn6ZAluhIpSqAwk/+/Z/iSBCr1UW90e8EFJyOppwSmHa4oMTHb ajqQ== X-Forwarded-Encrypted: i=1; AJvYcCWwTmjXLaS4hh9Z0d7j1q5/eXEadyz2qTu1NqgxTWiSZ3f0RxOqTKe6PVtnkise8SiuI8qFdzARm6nUeoc=@vger.kernel.org X-Gm-Message-State: AOJu0YwahhsKHkzE2fq7ZmpAUps15Ai2PA1+2GuxEPZ115DEJbSKYwBo BqsKDeE4JAd64yvdIqSIQPMoVcBaAu1AUDQBud78FIsXlXW1RvJhr/e1B2RlXOWn0R3jg7QlTkL LKisjW36NR1QpIEAjD+4axlZIgTRtiJnAytMjHNdZ5+064f1zcidTQDeQVCMo36bnDegwxgaeq6 9R+amn/HY4pbJCgWa1HflVs54YEVwEFfm/jNmNsunmv6lsTwCR5LqRIqWjFCv1oE0p3YnoMHFCB RQirmcerAZf5rW21kk= X-Gm-Gg: ASbGncsPeGV1OSD1KPdXKWJGmxDVWwAhhEAhk0E3SWMy2yfhk7ADIUKKuDSew2gj25+ mzx+wntGYUdKIMIMmvnYvasjtZZxTdjEbHPY06v8S0WAlBoNqopgzg1RAg8Do4K3xteXOLRb2dL xxl7PYsWtElLDoDwt3J3Ros+z9aKYcBvtJq+SbfpBb8Doa/Yb46GfOPBKnqTQEWFkIfUG32cpS1 Qkc/DcSC/vfIyvHY8Q//KT/ianGyuCx/2nyqjCW1+jKubAsStwVlPncZfZ4KdYygwcTiaqJx3y9 C8bUJfOsG+vlTIUoJLuVhxnxHjEZVzQ/q8iZ0NmfKboOqLk1w2TWZ2rbgW+PFQ1isMc+avHasG2 YZYUf7fsXDvWRSqvvEnjM3FSNp6NuhGdfcCrRjnvpcR8Vo6h6oeGV7G1u0U3kqA6jVp8ric5eG4 IMBzTZojmpj/CZEH9YVuqrmvu4Z0W4QYh4itE273WTvw== X-Google-Smtp-Source: AGHT+IGJxp79l1IQkhy7FWwf6kKFdDo0iSi1boqbC70KB0B9bOgcAk1SJnokvqw2A1428b23Z0FpaZnuvEwO X-Received: by 2002:a17:902:daca:b0:298:529b:8956 with SMTP id d9443c01a7336-29d684767a9mr23236505ad.56.1764762021415; Wed, 03 Dec 2025 03:40:21 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-11.dlp.protect.broadcom.com. [144.49.247.11]) by smtp-relay.gmail.com with ESMTPS id d9443c01a7336-29bceb26697sm25971425ad.58.2025.12.03.03.40.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:40:21 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pg1-f198.google.com with SMTP id 41be03b00d2f7-bddf9ce4931so9746966a12.0 for ; Wed, 03 Dec 2025 03:40:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764762020; x=1765366820; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1iH4y3Zrgl56i/VHQKgTMa01v/ge6GrCshnPXW5Ao8U=; b=AB8/2xrRsmJY1SqLLMRtCGW5LrZAlJLipFOHhN06PQ1uk7N24w2hZRwHlIWBm6brzD YQFo3GuirYp7lT34BNz4Cjo7gVDPnHLmnsVssSGdl3Fnpba6MsdlQtokcA6uS+awux4f ivWhunC49HuOwmCIWXt7W5jqfZ0kHHJcSBuRI= X-Forwarded-Encrypted: i=1; AJvYcCVyZkXAALnd0nqVPZezvQ97RYp5lhhUeEvrmtC3YULxJW7rylnx1DkxPen3o2eBSgXEd4agqedz1UzTiiY=@vger.kernel.org X-Received: by 2002:a05:7300:6c89:b0:2a4:75f1:fb27 with SMTP id 5a478bee46e88-2ab92e37150mr1660502eec.25.1764762019545; Wed, 03 Dec 2025 03:40:19 -0800 (PST) X-Received: by 2002:a05:7300:6c89:b0:2a4:75f1:fb27 with SMTP id 5a478bee46e88-2ab92e37150mr1660479eec.25.1764762019027; Wed, 03 Dec 2025 03:40:19 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-2a965b1ceeesm63324781eec.5.2025.12.03.03.40.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:40:18 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Chris Mason Subject: [PATCH v6.6 3/4] sched/fair: Small cleanup to update_newidle_cost() Date: Wed, 3 Dec 2025 11:22:54 +0000 Message-Id: <20251203112255.1738272-4-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112255.1738272-1-ajay.kaher@broadcom.com> References: <20251203112255.1738272-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit 08d473dd8718e4a4d698b1113a14a40ad64a909b upstream. Simplify code by adding a few variables. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://patch.msgid.link/20251107161739.655208666@infradead.org [ Ajay: Modified to apply on v6.6 ] Signed-off-by: Ajay Kaher --- kernel/sched/fair.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e47bf8d6c..f93a6a12e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11706,22 +11706,25 @@ void update_max_interval(void) =20 static inline bool update_newidle_cost(struct sched_domain *sd, u64 cost) { + unsigned long next_decay =3D sd->last_decay_max_lb_cost + HZ; + unsigned long now =3D jiffies; + if (cost > sd->max_newidle_lb_cost) { /* * Track max cost of a domain to make sure to not delay the * next wakeup on the CPU. */ sd->max_newidle_lb_cost =3D cost; - sd->last_decay_max_lb_cost =3D jiffies; - } else if (time_after(jiffies, sd->last_decay_max_lb_cost + HZ)) { + sd->last_decay_max_lb_cost =3D now; + + } else if (time_after(now, next_decay)) { /* * Decay the newidle max times by ~1% per second to ensure that * it is not outdated and the current max cost is actually * shorter. */ sd->max_newidle_lb_cost =3D (sd->max_newidle_lb_cost * 253) / 256; - sd->last_decay_max_lb_cost =3D jiffies; - + sd->last_decay_max_lb_cost =3D now; return true; } =20 --=20 2.40.4 From nobody Fri Dec 19 21:12:21 2025 Received: from mail-dl1-f99.google.com (mail-dl1-f99.google.com [74.125.82.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D96192F6167 for ; Wed, 3 Dec 2025 11:40:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762025; cv=none; b=dB+MUuiuJDjUdy7vicdzCOIeVy324upVijlfVCuFsZp0wkinVdBDq2sLJuHzZ50NW9CYrPqm/Q8tQa4t8LzaEq8gs/CZxteXEdrFYc1uFmMUiRjhFYVPYtNl2LZvLazyyy5pAbucjqsHVlTUleiG8q/PubxqfihTDc7RfThXDkI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764762025; c=relaxed/simple; bh=02Yp53TuowJehhoJcF+Iw5981qDC8pXTcvwqF/knsI4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kTSgVjCxaTGs5D535ejNzSFGiEXGKA0wBf2uYKpz3u/k2KEAWrzwuf2D182x5+qSGMhDwNX4RcWvP+vlDBwVJQKf1Amw0olclLHld3I9QPGOHw+P7ZWiINP631K8sRJud3WkX3MHMrjYULe1JqPKkF7VwTXHU9xe1zJD7KOS06M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=PQfQtqB5; arc=none smtp.client-ip=74.125.82.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="PQfQtqB5" Received: by mail-dl1-f99.google.com with SMTP id a92af1059eb24-11beb0a7bd6so956630c88.1 for ; Wed, 03 Dec 2025 03:40:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764762023; x=1765366823; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=b/SKWEA8ISmyxBr9KByGjrpmJhdyEvH8fSbpB6G/k0w=; b=EkGKT6qzSAlEWvWzXOzYmXTGOky0Qd5LaOrIAXW98GaORjRMuSpq0Pa3eTxMrQh2iC lzHbaXmzc70gpUzCCGZEbMi+gDdDGW/cD1+4VH+NaCJWWk+MSimliC+yzmZmLYCXjs/s 5rRUhWWhPbSZxocFQnnxI+k77cErnI1K/c2lZQUeWsvvWfuA9bZlfaibfpDY6W66eXlv sPxSbe4sknx7t+yq4+X6++vYbQfr/UlI74ey/iRuKvnVhWlkCwK/mNupZ2XzZw8JwiDM YQ+/MbrLZWQC+ZWM6LMENVFcdpnf4DYP8OCq3K9X+vGIlqeyZJsDfSyMHRviWM4SsK6w TxWA== X-Forwarded-Encrypted: i=1; AJvYcCWS/h0rNhOvdsNZJm7vupypuf0kHt0vUcuKYhKiPKXTBG5XfKZiaYCidRJwFnfTSC+eaatkFkA9CBsFmdM=@vger.kernel.org X-Gm-Message-State: AOJu0YwGei0LtZ8A/US61Js09YjO92wewYavmD5wrIiPqDnp2+EnpppZ 1qlpUd3O48ntl5BmW8jmrhEYUVMb1hwtcUm8InCU/bkT4+UMHza8SlG8opgQyCJ+MvMhmCzcxRf rOQhSj55z16mQOZEnMfDzcLSj94pJxFd3l9NKEkZRX1spMBTh9/JSYnvcvaZe6FuVwu/ORzMd1r VKJBURXF+ThwbyDvUTNvRuILZMWpN32oxvfV8ucW12LNXzwUk0fsCFIszRK8sGZJ4vZf1s+37m0 ubKuVjPN++g2bCKyBI= X-Gm-Gg: ASbGncuXkYnN48U3sO/f0MM/Euk64LRv16kGqRF8Vxu3uWa6zhiO7vUoo/A5fXIzlah Y7gYAri6W+Xp2cdVPdZffjQu679O4vFz/9cO5HdAWuRtWa3xiCMLrMDOp++DsqA0pu+96NNrDIx u7YXegT2x3zrGXHjQHAeTyjVRNiZ61ep4xB8WiwK0EoCZ/MpNkYSK0nSocY8y1NW7lrG8RjqrgN 9meOApP7C+QwSrbhSaOF8UVZOlI287w6rYT9DRzoo+gzhZFT3nXHq/TaAQ4lB8wAto8A0gd8TbP P8tDgcAS6w7TpvQYMV4UDl0VtDLrmvlNbHmpN0lAJhMLfl2d9NXUWlrElxWreI3JuzMLUcndBpP aBpWh/i8gquvQC2zNs6eafksxyPXPrU+G21fKOX8DGQJuk7iUa+8ouPmZs7k9Tr8+WQY+LumUQ/ o0fT28/RhMpZ4OMiksL4BLP05VdFGUFRhJcBUMIEiMag== X-Google-Smtp-Source: AGHT+IEP1eI+IcGvKs97ffOFBLQKH4WwJ5jzCoD5EdEmN6l3o8e77MbWhORSr+htWX/EV3vQgfASLo3g7FNp X-Received: by 2002:a05:7022:2604:b0:119:e569:f85b with SMTP id a92af1059eb24-11de948a95amr4638640c88.18.1764762022713; Wed, 03 Dec 2025 03:40:22 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-11.dlp.protect.broadcom.com. [144.49.247.11]) by smtp-relay.gmail.com with ESMTPS id a92af1059eb24-11dcb03f38fsm2356024c88.6.2025.12.03.03.40.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:40:22 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-dy1-f200.google.com with SMTP id 5a478bee46e88-2a467c4e74bso1067671eec.1 for ; Wed, 03 Dec 2025 03:40:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764762021; x=1765366821; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b/SKWEA8ISmyxBr9KByGjrpmJhdyEvH8fSbpB6G/k0w=; b=PQfQtqB5M3kK8u318NO1Q8A2ZhMCieTbmATYvgwbDKFtzolSkhl9h+9QaEa3RYbcAv nOYuvybn1LfYVQ2m/8Fr6Mk8KdgtYUyf0Nit0ZWveoNO8rV33Sju7Em9x1fsA51WBHxq UNJ/t4WzBb9TYCGiOoA7WCqCvW/IHoxYvTqYY= X-Forwarded-Encrypted: i=1; AJvYcCWP6VRPMmaCC6JJ2jkxx9bQ9mswTNYVL9prr3g+5PoB2Jd0Bi6UZbl9x0+Lkp61U9eFd+YxKun2WK19gSY=@vger.kernel.org X-Received: by 2002:a05:693c:2b10:b0:2ab:9990:79a9 with SMTP id 5a478bee46e88-2ab99908053mr504400eec.17.1764762020794; Wed, 03 Dec 2025 03:40:20 -0800 (PST) X-Received: by 2002:a05:693c:2b10:b0:2ab:9990:79a9 with SMTP id 5a478bee46e88-2ab99908053mr504383eec.17.1764762020180; Wed, 03 Dec 2025 03:40:20 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-2a965b1ceeesm63324781eec.5.2025.12.03.03.40.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:40:19 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Chris Mason Subject: [PATCH v6.6 4/4] sched/fair: Proportional newidle balance Date: Wed, 3 Dec 2025 11:22:55 +0000 Message-Id: <20251203112255.1738272-5-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112255.1738272-1-ajay.kaher@broadcom.com> References: <20251203112255.1738272-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra (Intel) commit 33cf66d88306663d16e4759e9d24766b0aaa2e17 upstream. Add a randomized algorithm that runs newidle balancing proportional to its success rate. This improves schbench significantly: 6.18-rc4: 2.22 Mrps/s 6.18-rc4+revert: 2.04 Mrps/s 6.18-rc4+revert+random: 2.18 Mrps/S Conversely, per Adam Li this affects SpecJBB slightly, reducing it by 1%: 6.17: -6% 6.17+revert: 0% 6.17+revert+random: -1% Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://lkml.kernel.org/r/6825c50d-7fa7-45d8-9b81-c6e7e25738e2@meta.c= om Link: https://patch.msgid.link/20251107161739.770122091@infradead.org [ Ajay: Modified to apply on v6.6 ] Signed-off-by: Ajay Kaher --- include/linux/sched/topology.h | 3 +++ kernel/sched/core.c | 3 +++ kernel/sched/fair.c | 44 ++++++++++++++++++++++++++++++---- kernel/sched/features.h | 5 ++++ kernel/sched/sched.h | 7 ++++++ kernel/sched/topology.c | 6 +++++ 6 files changed, 64 insertions(+), 4 deletions(-) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index 9671b7234..197039bab 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -106,6 +106,9 @@ struct sched_domain { unsigned int nr_balance_failed; /* initialise to 0 */ =20 /* idle_balance() stats */ + unsigned int newidle_call; + unsigned int newidle_success; + unsigned int newidle_ratio; u64 max_newidle_lb_cost; unsigned long last_decay_max_lb_cost; =20 diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 1b5e4389f..c4a9797e9 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -116,6 +116,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(sched_util_est_se_tp); EXPORT_TRACEPOINT_SYMBOL_GPL(sched_update_nr_running_tp); =20 DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues); +DEFINE_PER_CPU(struct rnd_state, sched_rnd_state); =20 #ifdef CONFIG_SCHED_DEBUG /* @@ -9872,6 +9873,8 @@ void __init sched_init_smp(void) { sched_init_numa(NUMA_NO_NODE); =20 + prandom_init_once(&sched_rnd_state); + /* * There's no userspace yet to cause hotplug operations; hence all the * CPU masks are stable and all blatant races in the below code cannot diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index f93a6a12e..a10df85e4 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -11704,11 +11704,27 @@ void update_max_interval(void) max_load_balance_interval =3D HZ*num_online_cpus()/10; } =20 -static inline bool update_newidle_cost(struct sched_domain *sd, u64 cost) +static inline void update_newidle_stats(struct sched_domain *sd, unsigned = int success) +{ + sd->newidle_call++; + sd->newidle_success +=3D success; + + if (sd->newidle_call >=3D 1024) { + sd->newidle_ratio =3D sd->newidle_success; + sd->newidle_call /=3D 2; + sd->newidle_success /=3D 2; + } +} + +static inline bool +update_newidle_cost(struct sched_domain *sd, u64 cost, unsigned int succes= s) { unsigned long next_decay =3D sd->last_decay_max_lb_cost + HZ; unsigned long now =3D jiffies; =20 + if (cost) + update_newidle_stats(sd, success); + if (cost > sd->max_newidle_lb_cost) { /* * Track max cost of a domain to make sure to not delay the @@ -11756,7 +11772,7 @@ static void rebalance_domains(struct rq *rq, enum c= pu_idle_type idle) * Decay the newidle max times here because this is a regular * visit to all the domains. */ - need_decay =3D update_newidle_cost(sd, 0); + need_decay =3D update_newidle_cost(sd, 0, 0); max_cost +=3D sd->max_newidle_lb_cost; =20 /* @@ -12394,6 +12410,22 @@ static int sched_balance_newidle(struct rq *this_r= q, struct rq_flags *rf) break; =20 if (sd->flags & SD_BALANCE_NEWIDLE) { + unsigned int weight =3D 1; + + if (sched_feat(NI_RANDOM)) { + /* + * Throw a 1k sided dice; and only run + * newidle_balance according to the success + * rate. + */ + u32 d1k =3D sched_rng() % 1024; + weight =3D 1 + sd->newidle_ratio; + if (d1k > weight) { + update_newidle_stats(sd, 0); + continue; + } + weight =3D (1024 + weight/2) / weight; + } =20 pulled_task =3D load_balance(this_cpu, this_rq, sd, CPU_NEWLY_IDLE, @@ -12401,10 +12433,14 @@ static int sched_balance_newidle(struct rq *this_= rq, struct rq_flags *rf) =20 t1 =3D sched_clock_cpu(this_cpu); domain_cost =3D t1 - t0; - update_newidle_cost(sd, domain_cost); - curr_cost +=3D domain_cost; t0 =3D t1; + + /* + * Track max cost of a domain to make sure to not delay the + * next wakeup on the CPU. + */ + update_newidle_cost(sd, domain_cost, weight * !!pulled_task); } =20 /* diff --git a/kernel/sched/features.h b/kernel/sched/features.h index f77016823..48b104ab5 100644 --- a/kernel/sched/features.h +++ b/kernel/sched/features.h @@ -88,4 +88,9 @@ SCHED_FEAT(UTIL_EST_FASTUP, true) =20 SCHED_FEAT(LATENCY_WARN, false) =20 +/* + * Do newidle balancing proportional to its success rate using randomizati= on. + */ +SCHED_FEAT(NI_RANDOM, true) + SCHED_FEAT(HZ_BW, true) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 64634314a..e1913e253 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -5,6 +5,7 @@ #ifndef _KERNEL_SCHED_SCHED_H #define _KERNEL_SCHED_SCHED_H =20 +#include #include #include #include @@ -1205,6 +1206,12 @@ static inline bool is_migration_disabled(struct task= _struct *p) } =20 DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues); +DECLARE_PER_CPU(struct rnd_state, sched_rnd_state); + +static inline u32 sched_rng(void) +{ + return prandom_u32_state(this_cpu_ptr(&sched_rnd_state)); +} =20 #define cpu_rq(cpu) (&per_cpu(runqueues, (cpu))) #define this_rq() this_cpu_ptr(&runqueues) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index b87426b74..9fa77e7a6 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1600,6 +1600,12 @@ sd_init(struct sched_domain_topology_level *tl, =20 .last_balance =3D jiffies, .balance_interval =3D sd_weight, + + /* 50% success rate */ + .newidle_call =3D 512, + .newidle_success =3D 256, + .newidle_ratio =3D 512, + .max_newidle_lb_cost =3D 0, .last_decay_max_lb_cost =3D jiffies, .child =3D child, --=20 2.40.4