From nobody Fri Dec 19 21:03:51 2025 Received: from mail-qv1-f99.google.com (mail-qv1-f99.google.com [209.85.219.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50FF62F5339 for ; Wed, 3 Dec 2025 11:38:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764761889; cv=none; b=AHQqXXBo0vqZNkXtPkb67RVNMZbXyUyVtjRi31YR5woPkHcT6/egO3xKn52fXwt3GEGFkXrPWknY8v/y1FdRAr34luhnOWGffaOYehhSqEKjTOVQLhRh9Goi39Qs0E2f0TWuO/8NmYajAaO97wEnx6h9r1fddw3RqHTTbsISCsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764761889; c=relaxed/simple; bh=3ZfId0fg17jLpmXVAs6RDt/lgvIwJmUHkgZynliI29I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iXKjfVG7S/09YfcmEMru6QpVxVlH3bsxyBB7zrRAj+SGbuv7mWAIpQYrP+jy4mscKSLGiJ/pdshzkEtNXisPYvd3LeC0pzoSkgIRsshNe2v+/6+5sQCAH3cz9rkAzhIc6a/T1UFF070eZkfTCDXCvl3ZeQO3YijiepXGCVhubkY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=RubBXCwX; arc=none smtp.client-ip=209.85.219.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="RubBXCwX" Received: by mail-qv1-f99.google.com with SMTP id 6a1803df08f44-882379c0b14so51373456d6.1 for ; Wed, 03 Dec 2025 03:38:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764761887; x=1765366687; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QwoBIBoPcN27LB5ca5Kj/q9w5devdvomIojhATPdeUM=; b=WoBI8lqGhRbVQ8K1e9QHZNsDLWNSOKvYv4YPsxQB3Kk7NxV4jLT7REb032+uF/6X+Y FpdBrMeAZ2aC+RKMyGfv6OaCblOBxhga3lzBUXxwtPdE0d63uYBzBeZ8HDvBbs3YMx19 XYd2/3umjGmm7+l5n9gNLe0G/Umlk31L50nyVxMAiWCyzLahKq1vhU0ihFmlPFTXxygg NpCnXokfrieIerztQ3hzmDjbYLo61j8BNr+RIpD0FyN2WazuLklWhAEvlIBXD5HRS7bE vQHpItKOle/cSzthm9bXESMgQ454cC6lTtzUVLjqFZ41kyO0IYY4JxDElYubLtkQPyJT pPmA== X-Forwarded-Encrypted: i=1; AJvYcCVljV39U0sF8K3DzK8KVQ1ksHI2THKItOKsp+A3zj1iXsdv1yOcaOG9ViysYYubSQ2dmbs/wuOcPgfKDbM=@vger.kernel.org X-Gm-Message-State: AOJu0YwZJLCu0MXyG7rrzeK0i3/G559QNXSqpv1M9bbWoCWAxx0PpbHC fDxu4NWEgkjiXi2n3VDSl2pxCfIhJXDou/JNFk3tzD+5rrI2kj0XeHj5x7VmAcJJfaVH8Ez3wg8 UcO8AU4auWfAiGB5qijx8zfG6LNBKEREIOdpOXHvF5+V36//gCSyxFRWRvWBiRwjLfLkaLYIxPO Au0L3PdUnifmAXCKc53P8gN73Ghfvcl4v9dtbwhRD2AUZivznLH6w7cxl5FaGSU5hBLYUsdsFfb QxTxeXEfBtpOZ2YII4= X-Gm-Gg: ASbGncskan/F/DUIZC64M/xOKd0oXBFlypo7XIA4lFensGbFUyoZLX9VqgUH6cdsvXU FTS+Cfa0rl4TMPIvn1fRRgqXk6PWTCnq9FmqXcGSjgI1P0kTvBBKBFt3VzV7uHBDdkU086uGG6N /YiPBD4H9lYZK7lL72IBw3DW04xYzNGbtq8eL4hMb4uSu39lFX0uLsn8OavyCfyN0SUlTdYGQ2J NZuhDwK5ikDPLjCb5YX1I1VweEPBz+xpUAUyP8E9dBLa+VQ4atTnAP6ejqoeqUY1wsxX1R3Tu3O UtY3r2NOIhWz5vOKR8PAOFK4mDM02qqeGp+hn+5myKyS87CgMzUgZxwgpArS5StktpHfFZ7lDXZ ap4A1PMqh4+gNSvWWes9Ps0k7aA6dZdPZEQT35fGFwehJ7M5gmNgE0bhKXDVkhP+If+DKreoeqd FYeBi6SnJY58bx7IlJd8neXOi6Xg3bZv4fyRRljt8ZAiTK X-Google-Smtp-Source: AGHT+IEZk6a4sqbqi8awV6J5/22H3d+hfhlGOioqLSBOL4kMcwT2MZCz8Ix/XgjF8GgSMMzRtkZtZ3C6AxVj X-Received: by 2002:a05:6214:242a:b0:87c:2919:7db3 with SMTP id 6a1803df08f44-8881955b608mr27350876d6.51.1764761886977; Wed, 03 Dec 2025 03:38:06 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-117.dlp.protect.broadcom.com. [144.49.247.117]) by smtp-relay.gmail.com with ESMTPS id d75a77b69052e-4efd2eef4f0sm5823611cf.0.2025.12.03.03.38.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:38:06 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pf1-f198.google.com with SMTP id d2e1a72fcca58-7b9090d9f2eso10980958b3a.0 for ; Wed, 03 Dec 2025 03:38:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764761886; x=1765366686; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QwoBIBoPcN27LB5ca5Kj/q9w5devdvomIojhATPdeUM=; b=RubBXCwXV6eqT8DWnifQ/AefdMZqiYX9ybaKYsoG3zDHx2F1iKnFfrGqMONLOGzPbE sMONMT4ToazbNumYTJsLWptBi+EAyWJgsu7v2cnGmYH1jRuEmS2fqfhoM7/fdqa2UHoi ELhlykF8hExgHxigcJ1uD5n7wbvz7ZsS+H9w8= X-Forwarded-Encrypted: i=1; AJvYcCVzrCMXyDXyS3AVfxCdPsyzIF6i1Zo+E1Z1wpTcsPZTOkE7XwNH7TMI2pSF9IbuR31aSoY5774OF3tzAaA=@vger.kernel.org X-Received: by 2002:a05:701a:c965:b0:119:e56b:9593 with SMTP id a92af1059eb24-11df0cc504amr1158019c88.24.1764761885730; Wed, 03 Dec 2025 03:38:05 -0800 (PST) X-Received: by 2002:a05:701a:c965:b0:119:e56b:9593 with SMTP id a92af1059eb24-11df0cc504amr1157989c88.24.1764761885142; Wed, 03 Dec 2025 03:38:05 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-11dcb03c232sm83169465c88.6.2025.12.03.03.38.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:38:04 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Joseph Salisbury , Adam Li , Hazem Mohamed Abuelfotoh , Chris Mason Subject: [PATCH v6.12 1/4] sched/fair: Revert max_newidle_lb_cost bump Date: Wed, 3 Dec 2025 11:20:24 +0000 Message-Id: <20251203112027.1738141-2-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112027.1738141-1-ajay.kaher@broadcom.com> References: <20251203112027.1738141-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit d206fbad9328ddb68ebabd7cf7413392acd38081 upstream. Many people reported regressions on their database workloads due to: 155213a2aed4 ("sched/fair: Bump sd->max_newidle_lb_cost when newidle bala= nce fails") For instance Adam Li reported a 6% regression on SpecJBB. Conversely this will regress schbench again; on my machine from 2.22 Mrps/s down to 2.04 Mrps/s. Reported-by: Joseph Salisbury Reported-by: Adam Li Reported-by: Dietmar Eggemann Reported-by: Hazem Mohamed Abuelfotoh Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://lkml.kernel.org/r/20250626144017.1510594-2-clm@fb.com Link: https://lkml.kernel.org/r/006c9df2-b691-47f1-82e6-e233c3f91faf@oracle= .com Link: https://patch.msgid.link/20251107161739.406147760@infradead.org [ Ajay: Modified to apply on v6.12 ] Signed-off-by: Ajay Kaher --- kernel/sched/fair.c | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 8bdcb5df0..7ba5dd10e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12223,14 +12223,8 @@ static inline bool update_newidle_cost(struct sche= d_domain *sd, u64 cost) /* * Track max cost of a domain to make sure to not delay the * next wakeup on the CPU. - * - * sched_balance_newidle() bumps the cost whenever newidle - * balance fails, and we don't want things to grow out of - * control. Use the sysctl_sched_migration_cost as the upper - * limit, plus a litle extra to avoid off by ones. */ - sd->max_newidle_lb_cost =3D - min(cost, sysctl_sched_migration_cost + 200); + sd->max_newidle_lb_cost =3D cost; sd->last_decay_max_lb_cost =3D jiffies; } else if (time_after(jiffies, sd->last_decay_max_lb_cost + HZ)) { /* @@ -12935,17 +12929,10 @@ static int sched_balance_newidle(struct rq *this_= rq, struct rq_flags *rf) =20 t1 =3D sched_clock_cpu(this_cpu); domain_cost =3D t1 - t0; + update_newidle_cost(sd, domain_cost); + curr_cost +=3D domain_cost; t0 =3D t1; - - /* - * Failing newidle means it is not effective; - * bump the cost so we end up doing less of it. - */ - if (!pulled_task) - domain_cost =3D (3 * sd->max_newidle_lb_cost) / 2; - - update_newidle_cost(sd, domain_cost); } =20 /* --=20 2.40.4 From nobody Fri Dec 19 21:03:51 2025 Received: from mail-qv1-f100.google.com (mail-qv1-f100.google.com [209.85.219.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 421402F5480 for ; Wed, 3 Dec 2025 11:38:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.100 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764761890; cv=none; b=qOKArg3XO9DbFQxTMweoOW7zLYiukqK2w3LIavUrp4Bqey2QhJzgbFoCcLtyVbJtp/uS00znDNlVhIZpygYHXRl/LIuwD721deEUNxGEMD2wjxPiN4bvNUq1MOzWNrtoVacHc3MPJogHL0Sahd2M4EkgJUYH8hGbztFA5syw3mk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764761890; c=relaxed/simple; bh=2i0Nr3FF86hxTZ+uwa8LSg7RhEJAMHkbIJrtE6kLfCg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uROcWI8RbbSsLdb3G7xPXWwA2xpVJhzl1eCJ/x7wpuWBM1ojVp/JW2GihF8WbGwPOEsvPTt31jlfw0KYuodNUZFog3vfTF/gNkKiZx6g3PrxmFj6KdRZ6wSpeqH5qBG1n25EZII82I4kic+439vMl4O1acbdHYU367ZIk/lhrNM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=STLroqKn; arc=none smtp.client-ip=209.85.219.100 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="STLroqKn" Received: by mail-qv1-f100.google.com with SMTP id 6a1803df08f44-8823fa3bd95so7083656d6.1 for ; Wed, 03 Dec 2025 03:38:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764761888; x=1765366688; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=G5hKdYszeQOOdLcK2HxRZCwJUCsOXMz/ue3SXkECFJE=; b=wEuRufeMKlpnUsO1au/CjqDh4FA03IcSR455V8N+JR18W22AWmTnxUOO5L3kU+1n/T chsupZKw2HDLPxlEMdJoZjWvqpLNZ1uLCE+ELjhKJFWPVwx7syjC6CRfvSGBpWuzdfq4 6xJanSoGaQYOrebegpcOCcclBI2CxMgd1WPvod/s/jV7ezYWVh5qHEjGV3B2xTOBnckF HOyleOJ98mbvmSnqQOAF1l7EIWYgn6bufL62UCdPbqkhoWbfZ4ZM6Uvzhj6sLYycEbyi 5+/+o1tftWdXn84eTwpYGRchTtpALTuA4PiDtE5T9mhvVamVBB9xzn02CluAS/ahMJAP tcAQ== X-Forwarded-Encrypted: i=1; AJvYcCWNWo1mDGrDBvmZz3vHiJEXbLyYJ10QpoVWqWd4PedGGaohSEdL/4MhbYo2pHNG5fdsn6oQjK71HmUReVM=@vger.kernel.org X-Gm-Message-State: AOJu0Yx/topmGmuNajLEDlN7B9pYEi6Tn+FUYkIy42zBW7k6B9VBkyZR pVs3Jy1WWFoaLXDIjFEqp6rzX2AdAkzruxA03J27WgMmCR9Y6uQJ1EqRtiAHLLucDfj1B1kEpbi yrVywfoy7PZk8EnrVKOBYu/Agr3p7NUaDPvMiMNTLIDIiXKdejPl9udjiC9GDnQZiNLxJa1pXl4 LTLCS0F4LObpcj8eYfL2wsn7VmeD9jaA7raJK1s2WDP/6I+W+W4k67QbVztEF49/1Y+jetYzopX QM521iFPDwRElrIEjA= X-Gm-Gg: ASbGnct60+KYpoxaJQXfTvjWdtE2ro443OAsfxc4x6BzJKJqObJWd1BJYEkGkpw5szM dQWvPt8XcQYFBpy9DFW3TAJY7KCIMvEUKnLPIxRCCFea5sL2NHqPKS3kibW3GAzcYtktaqdPJms iT6cEjaWb5g0A3se/TLCqgbUm02gHMMou66gbygr0Un85RpZNV5aha57afGxSQzNxQkieK4IObv MxKhpzsAl7ACsxqZiZMsdJVVDYSgt2GBqErY4XE24sGvkRhW8CFDFlSHAW8TBAv4roW+M5Yis00 oO8+dhn4a03TlW7qLEbArpHBqQLFbFxUKn04PpOTv0bUoeDE5gp+Vqjpqg1D0dks/HrmVJtcMJ1 OcppDdt9WNEqzdryGzlbJM34JSn3Vz3XkGXF9bw+FDBIBd5HpLYJMwukfs8UW+qy45nZgrFyOW6 bRV4iXVQZ+3sjEnrdbWxH1wp9QB4BcCGjA8xnrMVzdDjUF X-Google-Smtp-Source: AGHT+IH+TVin+GhXVuC+joXRUnuz0kHU2yfAe07KEXivXcKXTRpJjAsZmAJAKNQwHo1iiGEyGDT5C68AuqDM X-Received: by 2002:a05:6214:212a:b0:884:6f86:e093 with SMTP id 6a1803df08f44-88818b7c63emr31642166d6.5.1764761888110; Wed, 03 Dec 2025 03:38:08 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-103.dlp.protect.broadcom.com. [144.49.247.103]) by smtp-relay.gmail.com with ESMTPS id 6a1803df08f44-88652afaaf5sm25832976d6.22.2025.12.03.03.38.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:38:08 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-dl1-f71.google.com with SMTP id a92af1059eb24-11ddcc9f85eso1260404c88.0 for ; Wed, 03 Dec 2025 03:38:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764761887; x=1765366687; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G5hKdYszeQOOdLcK2HxRZCwJUCsOXMz/ue3SXkECFJE=; b=STLroqKnv361Ygg4J8nI65gyJxGR3L0MtGYeB7d0NWYmHSyoWMjVjHoYLM+vr5aNIf lxB2LBf+5wVEkOXiMVGrUEvGL1Vih0/BmouS2vP8+rF0EBjcd7MRT9fAvFwznaeuGhFu 5xNHLx4SpXmK7S31BLa2yO5LcCphCo2T7WcLI= X-Forwarded-Encrypted: i=1; AJvYcCX8WCHTu7P+a1qupzkrk4wvu/q0AwLJ++Xqu3/uvtI1w1Tc9i9IDQtxmMndeZAbSHlL3uJWAl29EAQDaig=@vger.kernel.org X-Received: by 2002:a05:7022:6299:b0:11c:883d:1ef0 with SMTP id a92af1059eb24-11de94870femr3672951c88.15.1764761886896; Wed, 03 Dec 2025 03:38:06 -0800 (PST) X-Received: by 2002:a05:7022:6299:b0:11c:883d:1ef0 with SMTP id a92af1059eb24-11de94870femr3672933c88.15.1764761886322; Wed, 03 Dec 2025 03:38:06 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-11dcb03c232sm83169465c88.6.2025.12.03.03.38.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:38:06 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Chris Mason Subject: [PATCH v6.12 2/4] sched/fair: Small cleanup to sched_balance_newidle() Date: Wed, 3 Dec 2025 11:20:25 +0000 Message-Id: <20251203112027.1738141-3-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112027.1738141-1-ajay.kaher@broadcom.com> References: <20251203112027.1738141-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit e78e70dbf603c1425f15f32b455ca148c932f6c1 upstream. Pull out the !sd check to simplify code. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://patch.msgid.link/20251107161739.525916173@infradead.org [ Ajay: Modified to apply on v6.12 ] Signed-off-by: Ajay Kaher --- kernel/sched/fair.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 7ba5dd10e..b6637954e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12895,14 +12895,16 @@ static int sched_balance_newidle(struct rq *this_= rq, struct rq_flags *rf) =20 rcu_read_lock(); sd =3D rcu_dereference_check_sched_domain(this_rq->sd); + if (!sd) { + rcu_read_unlock(); + goto out; + } =20 if (!get_rd_overloaded(this_rq->rd) || - (sd && this_rq->avg_idle < sd->max_newidle_lb_cost)) { + this_rq->avg_idle < sd->max_newidle_lb_cost) { =20 - if (sd) - update_next_balance(sd, &next_balance); + update_next_balance(sd, &next_balance); rcu_read_unlock(); - goto out; } rcu_read_unlock(); --=20 2.40.4 From nobody Fri Dec 19 21:03:51 2025 Received: from mail-pl1-f226.google.com (mail-pl1-f226.google.com [209.85.214.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE85E2F5A1D for ; Wed, 3 Dec 2025 11:38:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.226 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764761892; cv=none; b=EDnH5UAD+g5Jmv33vgOVsai9QrlIBLInfFxOol/MshmGawtpWjWmegr5TN53Ec1x8d27E8Z8gNqq6MMF5egRvn559bzlyxXGvGNSoVGtOA3NiVjV9a2XxO5Y/atP92UTKfrQn029vYZnfJkV98roGKqSIDZm54tGceCjujryXa4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764761892; c=relaxed/simple; bh=/IlEpooVPRR4biFYwUdCr22YNedtY02/izr3qt8kbDU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ReNCxBlUhEjt6LpzyGOaWn111U8MuIbAJwXX5SosOoPSQpv59v0j3wXcI/ahJMmf9nNaAn2zU/esLfKDCZ56NH+37EzIr90IptNutC6lvRFXVQjvO8FKks4Dq9pc6gMPY4PlqW0D/ycidRRWOGRYbEHrpBd8QAhpJkpYAfqudxA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=cNXKv9Ua; arc=none smtp.client-ip=209.85.214.226 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="cNXKv9Ua" Received: by mail-pl1-f226.google.com with SMTP id d9443c01a7336-2984dfae043so57460855ad.0 for ; Wed, 03 Dec 2025 03:38:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764761890; x=1765366690; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZLfHsULW/WFUBHVOGU/nqYd2laF33Pp/+LBFX8T/D0c=; b=cAoBiuJ/AzHDkc4RZFTHrjkU68k2nNhbBtOHAuloCF/mucNgmyJzPWbWXJARse13gD /afcHy4rDhghPQhhnVGnp9mj6SaJTOZT1cfUfvgR1u5ZrC6ISkFFQhfw48S1trOlIqrx RxmLpKP77wAz7DhBnqIKgSOnOyKiIG9goP3VEFfIWA/qMElhEkosHBXCx1oFM//OJwLe crhSnnBq4wSmCo1PF6yKJlL2v+tSsf/1kw6qjILeHYssSTLY9FTo+o/q9krCU1kxHHDU hLDRpnR6u2Df+pN8O4qqprIypkD7cMxfg5hZ9gIH0R+uAs1v7avgVEmKmZ98puJ1bNyf x1rQ== X-Forwarded-Encrypted: i=1; AJvYcCVoKCP1FYwD3ENvX4e+QWp5drbNs0SSA9f49cFhtrX3XVCCOvRIIlwswagqq/KYLGtReaU5iWZYKAowPCU=@vger.kernel.org X-Gm-Message-State: AOJu0YxGl5kywSEtcacQhAFAn6/XJqxhrfNzpMB4236Ue2kwQaMtMyqV ND7KvDogpaWTjT7orBWVa3f7G55rYA+dcRYITWI2gkm8q73Uj3mczSsavrMNLj2w8yeRI1qUcqs A1qY2KMdZV4AyQILynroiDP/2XELyM/jWJx6Mrg9G0yL5U8dXSnPCQXcJk2304zPIqT5zoYHkdW eUzXBkjiygnmtLlHzImKnlg8uDpGOe3uU7aWLcgBgQgPCzS27R9hFx7aoPhoS/vLOVZd2WyzOMT SZDJ2qMPU3gMdvJGEw= X-Gm-Gg: ASbGncuIVTJovRyuakeCH5zxhRBficZCAmG8hXj/xz/abDlpjCHM7kkIEMU8Gz+TPhQ ixTmE4MZZ4z4D3phfZ6vpHHI/ljFtn+aH+vev2Z/WQtTm0Gg/n4a/+S5g5rV140AgMgi+gD9fRK osIZLjHFIS7djQL7ikiHlH4blS6L1syeqPWu4kDjiccmrf7XfT9L4OTU+klmKANIUOCPAzZj0Xi OiSJmLyiOlSW2OJpe3V3kjvaxjFihLCK8pIiVQUnWSLuoIpTRaUPq+wxkkDPFRnOzrIDZ77tM8m qH0/fZsrVfXXszqayV/f0Czpc9l5Ebvo8iZtHOt+rXrVHsyY57EWxzUy80/XOkMu002aJHbRAem xtFLm0sgR4AQ6ydB8xZmYkx9kp37EUk3l9JrvCkYFx3/I50pPApfjY86D4raXwUayZKWwY4ht8G kqhe5kDqn1i2MUBmyAmrdLU2d696QcJlKXjGiQLVmsxw== X-Google-Smtp-Source: AGHT+IFtIVrkQ5aP1hOd8dg7rETHZnCWu21yaJ65PUEju5KN1J7k/9Zx87wHA9b6AUSyqAOHE2ZB3L3QlKUl X-Received: by 2002:a17:903:2442:b0:295:7806:1d64 with SMTP id d9443c01a7336-29d6830aba1mr27159855ad.25.1764761890119; Wed, 03 Dec 2025 03:38:10 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-11.dlp.protect.broadcom.com. [144.49.247.11]) by smtp-relay.gmail.com with ESMTPS id d9443c01a7336-29bceb0d145sm25777315ad.56.2025.12.03.03.38.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:38:10 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pg1-f197.google.com with SMTP id 41be03b00d2f7-b6ce1b57b9cso6763708a12.1 for ; Wed, 03 Dec 2025 03:38:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764761888; x=1765366688; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZLfHsULW/WFUBHVOGU/nqYd2laF33Pp/+LBFX8T/D0c=; b=cNXKv9UayeM9Kp//+A/iNLVmRBnMSHyGtCtjAOUbae+h7b3Xot2jaeVYD8FCHxD+39 R3ytN5PSprHQ8FwV9lFjNhnVndkydax9UnMPaUT5fTeAiieKyxYHG3gRCqIeCCrwj3Uq eGYXbGZFZuzAf6gSpJH1FNbYdqPUzT66XarIw= X-Forwarded-Encrypted: i=1; AJvYcCXHVsI6vI59jKntnS+ekr4geiYWiKvZCnzqIjpObaJY88O5URAD6K+zDC51TqOFDfagb8gmBvg4HWk6umI=@vger.kernel.org X-Received: by 2002:a05:7022:412:b0:119:e56b:91d1 with SMTP id a92af1059eb24-11df0bb9dd7mr1558617c88.2.1764761888044; Wed, 03 Dec 2025 03:38:08 -0800 (PST) X-Received: by 2002:a05:7022:412:b0:119:e56b:91d1 with SMTP id a92af1059eb24-11df0bb9dd7mr1558586c88.2.1764761887489; Wed, 03 Dec 2025 03:38:07 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-11dcb03c232sm83169465c88.6.2025.12.03.03.38.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:38:07 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Chris Mason Subject: [PATCH v6.12 3/4] sched/fair: Small cleanup to update_newidle_cost() Date: Wed, 3 Dec 2025 11:20:26 +0000 Message-Id: <20251203112027.1738141-4-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112027.1738141-1-ajay.kaher@broadcom.com> References: <20251203112027.1738141-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra commit 08d473dd8718e4a4d698b1113a14a40ad64a909b upstream. Simplify code by adding a few variables. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://patch.msgid.link/20251107161739.655208666@infradead.org [ Ajay: Modified to apply on v6.12 ] Signed-off-by: Ajay Kaher --- kernel/sched/fair.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b6637954e..ae5da8f34 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12219,22 +12219,25 @@ void update_max_interval(void) =20 static inline bool update_newidle_cost(struct sched_domain *sd, u64 cost) { + unsigned long next_decay =3D sd->last_decay_max_lb_cost + HZ; + unsigned long now =3D jiffies; + if (cost > sd->max_newidle_lb_cost) { /* * Track max cost of a domain to make sure to not delay the * next wakeup on the CPU. */ sd->max_newidle_lb_cost =3D cost; - sd->last_decay_max_lb_cost =3D jiffies; - } else if (time_after(jiffies, sd->last_decay_max_lb_cost + HZ)) { + sd->last_decay_max_lb_cost =3D now; + + } else if (time_after(now, next_decay)) { /* * Decay the newidle max times by ~1% per second to ensure that * it is not outdated and the current max cost is actually * shorter. */ sd->max_newidle_lb_cost =3D (sd->max_newidle_lb_cost * 253) / 256; - sd->last_decay_max_lb_cost =3D jiffies; - + sd->last_decay_max_lb_cost =3D now; return true; } =20 --=20 2.40.4 From nobody Fri Dec 19 21:03:51 2025 Received: from mail-pl1-f228.google.com (mail-pl1-f228.google.com [209.85.214.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 123622F6165 for ; Wed, 3 Dec 2025 11:38:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.228 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764761893; cv=none; b=JC4R7EVIY+RcLVFG76UH0es6rro48qJLGzwFWEW2SkOKvvu50Ag5LHBLPcEg1fYx9RJ9G9JYkwK+vBKIewgEZONiRiv3Zxsmp2+HU1OUqJbqAOIKvPzE8QMg0b5c5Y5PUwU/YBTfGyBEHIyKYhUi7yYpsbkU0xy396BlwhWGqYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764761893; c=relaxed/simple; bh=ZG5mzj65CsSbn+sH8RL7YDJ3FtnnWDUbqNPW06zJwbA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rYx4pwyk24ji9fXeMxcFME7Id0j9kJH/WY5k5SSl+Zmn0SQGKklHappJpLP3XOkC8b1igFBRArV6xnLQCo6TamhdCRhN4Un5WYpYNdwvJBtJ98QFVx2zZUqAqqQN5n4kTnQgeGb3SmhpwAabbaQuowPHFTuAzKXJT6bDoYAakjs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=XRzimyoa; arc=none smtp.client-ip=209.85.214.228 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="XRzimyoa" Received: by mail-pl1-f228.google.com with SMTP id d9443c01a7336-2980d9b7df5so78030795ad.3 for ; Wed, 03 Dec 2025 03:38:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764761891; x=1765366691; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xaYvaRoI/zaMoFXpgyvVSoj5rbLS0T6qMiMhubQCjpU=; b=qEYX/fBhLwN4uiV/wFv8XKrewcEkubJPu1pcZo+12hNYVOOcaeZFOE+qPmdhSMy6wS CcDNVeVtxZnrwSqDdeT+btg30kL757UV6y2GJRNyiXql/5IUmPvT8qZLOLRHuMAGwgH/ tZ/Z0cUCMkxVOr6IrcyRF6nyNT4zNfuntDyYeGmW5W8cJKSRzTifvObFgz78y818modp mfdbOa/rhYaTuAtEHkjfdV0vbNq0QGulJBI+ExLVWQzduMAcyDWFY9vQPlPxpvlPGQBA TIgc07/y1JdKWNQprpW7hFmpm1gTTdIFOl5kQtFNTGlpe+WoqHdRvUgLHUTI1A3j3QWH KTXw== X-Forwarded-Encrypted: i=1; AJvYcCV/SR7nmrRIZ2JiaemDkXuR6tkLrKKFHlsbwNHp9ituz/nbfsNWTM2nEz4ImkPz5rOZuKCRcKaLFLCrRHY=@vger.kernel.org X-Gm-Message-State: AOJu0YypzApsK/N9y7BWXr5/H2YXWaIy2XAXH4GmR9HOCPY/jTKwsUoV qmQA/2frNcvdJlZ2emiuknl6tfE/z8xM8RyNAi/FFKbyeJ1K+cpwSHECtSp550bZC6s78+S9ajV VvnvKa1Ml1hyKPpVhmNDsIbfXpOjsVzzMJNmoSkqgxy9ygGvsMJ4BTt9i4sQoZVGFSKRrQMMLjH AGDegqSYdSiCylAjVKu9TgFbfVIBKO4CBcSwe8ZZriN67nENIhXuaZPtTo1nV0RFuDlIHxdip+W 6XgDB0QGj7qK3NiJLA= X-Gm-Gg: ASbGncscsm56mJ53jGD/PNuL/dwSGSR1OXtDIUw1h2s7CQdYLDMi2odzNU3njUPH027 wyxoaEPc+rZiJTS2cYYVjVIwXr54676cP9JHi/sRu/fO3gjKriQxDqFbzFxG02wIxRCKoSylAlu wAJPlSFxTRCEIPRxCq7CTMeMSEpO8KTknGD+NOdvHMOw1IBqXItxTJXRR4eQhm6USEzdarNXwjs OakQBfs7UiqF+sxORCMk9hBQT2ylZJK7RJLSUUwaYMBGF+6XzOi0G8rMteYFZK073IN+bp9YHNI RCFJoe340TN7J1kJ597gyvMNNUgxiGZWj8vS3QmBNdf+ucYgWlA1VL+WO6DhF5winl0kh/UN0gm R+VtVJKdIIIG6OMUOxlqjbxuZJUFUPuMfBot/daeIk2D3jtKxhKMMyVc9WvLFSfDZKGw4LyDTpl NWDbHa/ZJcIAd3cyl4ifvTqSFo2ErZyv9wDhGiSw07FhLy X-Google-Smtp-Source: AGHT+IEFlo65ERy01RVtmlheO8GqVjjDnPfUxcCJtlpP42JlCTz+Wk9igLpKO0c9NXZqDeSSiU+z/JHjgdJ+ X-Received: by 2002:a17:902:f60c:b0:294:cc1d:e2b6 with SMTP id d9443c01a7336-29d683e6c95mr23674095ad.59.1764761891250; Wed, 03 Dec 2025 03:38:11 -0800 (PST) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-117.dlp.protect.broadcom.com. [144.49.247.117]) by smtp-relay.gmail.com with ESMTPS id d9443c01a7336-29bce41b327sm24100325ad.6.2025.12.03.03.38.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Dec 2025 03:38:11 -0800 (PST) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-pl1-f197.google.com with SMTP id d9443c01a7336-295fbc7d4abso81066895ad.1 for ; Wed, 03 Dec 2025 03:38:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1764761889; x=1765366689; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xaYvaRoI/zaMoFXpgyvVSoj5rbLS0T6qMiMhubQCjpU=; b=XRzimyoaMKPExMb0l3iKOwitOheD2OMaKMpr227o6sGMdcft4trTuejuBJ2qTpeElk jv02tbiWKQ3q+TPTjQmv2J8Q984KjqgwHSGkAuwseYIEi0ogMxjgRzvycPguaXUn93uC MvjPpBuRWpLXv33iXTi3SLpRufD8xNodic7qQ= X-Forwarded-Encrypted: i=1; AJvYcCXvUQ7/vtXZrCtHKyxsBdO9Kz+cXnEfTetGZ7TzLbiZDzjD/hXflFDtMzgtePAXI/fA+xUEz13WVo+vWz4=@vger.kernel.org X-Received: by 2002:a05:7022:160a:b0:11b:9386:825d with SMTP id a92af1059eb24-11df0cebfdamr1730601c88.42.1764761889426; Wed, 03 Dec 2025 03:38:09 -0800 (PST) X-Received: by 2002:a05:7022:160a:b0:11b:9386:825d with SMTP id a92af1059eb24-11df0cebfdamr1730570c88.42.1764761888840; Wed, 03 Dec 2025 03:38:08 -0800 (PST) Received: from photon-dev-haas.. ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-11dcb03c232sm83169465c88.6.2025.12.03.03.38.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 03:38:08 -0800 (PST) From: Ajay Kaher To: stable@vger.kernel.org, gregkh@linuxfoundation.org Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, ajay.kaher@broadcom.com, alexey.makhalov@broadcom.com, yin.ding@broadcom.com, tapas.kundu@broadcom.com, Chris Mason Subject: [PATCH v6.12 4/4] sched/fair: Proportional newidle balance Date: Wed, 3 Dec 2025 11:20:27 +0000 Message-Id: <20251203112027.1738141-5-ajay.kaher@broadcom.com> X-Mailer: git-send-email 2.40.4 In-Reply-To: <20251203112027.1738141-1-ajay.kaher@broadcom.com> References: <20251203112027.1738141-1-ajay.kaher@broadcom.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra (Intel) commit 33cf66d88306663d16e4759e9d24766b0aaa2e17 upstream. Add a randomized algorithm that runs newidle balancing proportional to its success rate. This improves schbench significantly: 6.18-rc4: 2.22 Mrps/s 6.18-rc4+revert: 2.04 Mrps/s 6.18-rc4+revert+random: 2.18 Mrps/S Conversely, per Adam Li this affects SpecJBB slightly, reducing it by 1%: 6.17: -6% 6.17+revert: 0% 6.17+revert+random: -1% Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Dietmar Eggemann Tested-by: Dietmar Eggemann Tested-by: Chris Mason Link: https://lkml.kernel.org/r/6825c50d-7fa7-45d8-9b81-c6e7e25738e2@meta.c= om Link: https://patch.msgid.link/20251107161739.770122091@infradead.org [ Ajay: Modified to apply on v6.12 ] Signed-off-by: Ajay Kaher --- include/linux/sched/topology.h | 3 +++ kernel/sched/core.c | 3 +++ kernel/sched/fair.c | 44 ++++++++++++++++++++++++++++++---- kernel/sched/features.h | 5 ++++ kernel/sched/sched.h | 7 ++++++ kernel/sched/topology.c | 6 +++++ 6 files changed, 64 insertions(+), 4 deletions(-) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index 4237daa5a..3cf27591f 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -106,6 +106,9 @@ struct sched_domain { unsigned int nr_balance_failed; /* initialise to 0 */ =20 /* idle_balance() stats */ + unsigned int newidle_call; + unsigned int newidle_success; + unsigned int newidle_ratio; u64 max_newidle_lb_cost; unsigned long last_decay_max_lb_cost; =20 diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 4b1953b6c..b1895b330 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -118,6 +118,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(sched_update_nr_running_tp= ); EXPORT_TRACEPOINT_SYMBOL_GPL(sched_compute_energy_tp); =20 DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues); +DEFINE_PER_CPU(struct rnd_state, sched_rnd_state); =20 #ifdef CONFIG_SCHED_DEBUG /* @@ -8335,6 +8336,8 @@ void __init sched_init_smp(void) { sched_init_numa(NUMA_NO_NODE); =20 + prandom_init_once(&sched_rnd_state); + /* * There's no userspace yet to cause hotplug operations; hence all the * CPU masks are stable and all blatant races in the below code cannot diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index ae5da8f34..189681ab8 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12217,11 +12217,27 @@ void update_max_interval(void) max_load_balance_interval =3D HZ*num_online_cpus()/10; } =20 -static inline bool update_newidle_cost(struct sched_domain *sd, u64 cost) +static inline void update_newidle_stats(struct sched_domain *sd, unsigned = int success) +{ + sd->newidle_call++; + sd->newidle_success +=3D success; + + if (sd->newidle_call >=3D 1024) { + sd->newidle_ratio =3D sd->newidle_success; + sd->newidle_call /=3D 2; + sd->newidle_success /=3D 2; + } +} + +static inline bool +update_newidle_cost(struct sched_domain *sd, u64 cost, unsigned int succes= s) { unsigned long next_decay =3D sd->last_decay_max_lb_cost + HZ; unsigned long now =3D jiffies; =20 + if (cost) + update_newidle_stats(sd, success); + if (cost > sd->max_newidle_lb_cost) { /* * Track max cost of a domain to make sure to not delay the @@ -12269,7 +12285,7 @@ static void sched_balance_domains(struct rq *rq, en= um cpu_idle_type idle) * Decay the newidle max times here because this is a regular * visit to all the domains. */ - need_decay =3D update_newidle_cost(sd, 0); + need_decay =3D update_newidle_cost(sd, 0, 0); max_cost +=3D sd->max_newidle_lb_cost; =20 /* @@ -12927,6 +12943,22 @@ static int sched_balance_newidle(struct rq *this_r= q, struct rq_flags *rf) break; =20 if (sd->flags & SD_BALANCE_NEWIDLE) { + unsigned int weight =3D 1; + + if (sched_feat(NI_RANDOM)) { + /* + * Throw a 1k sided dice; and only run + * newidle_balance according to the success + * rate. + */ + u32 d1k =3D sched_rng() % 1024; + weight =3D 1 + sd->newidle_ratio; + if (d1k > weight) { + update_newidle_stats(sd, 0); + continue; + } + weight =3D (1024 + weight/2) / weight; + } =20 pulled_task =3D sched_balance_rq(this_cpu, this_rq, sd, CPU_NEWLY_IDLE, @@ -12934,10 +12966,14 @@ static int sched_balance_newidle(struct rq *this_= rq, struct rq_flags *rf) =20 t1 =3D sched_clock_cpu(this_cpu); domain_cost =3D t1 - t0; - update_newidle_cost(sd, domain_cost); - curr_cost +=3D domain_cost; t0 =3D t1; + + /* + * Track max cost of a domain to make sure to not delay the + * next wakeup on the CPU. + */ + update_newidle_cost(sd, domain_cost, weight * !!pulled_task); } =20 /* diff --git a/kernel/sched/features.h b/kernel/sched/features.h index 050d75030..da8ec0c23 100644 --- a/kernel/sched/features.h +++ b/kernel/sched/features.h @@ -122,3 +122,8 @@ SCHED_FEAT(WA_BIAS, true) SCHED_FEAT(UTIL_EST, true) =20 SCHED_FEAT(LATENCY_WARN, false) + +/* + * Do newidle balancing proportional to its success rate using randomizati= on. + */ +SCHED_FEAT(NI_RANDOM, true) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index cf541c450..78b40c540 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -5,6 +5,7 @@ #ifndef _KERNEL_SCHED_SCHED_H #define _KERNEL_SCHED_SCHED_H =20 +#include #include #include #include @@ -1348,6 +1349,12 @@ static inline bool is_migration_disabled(struct task= _struct *p) } =20 DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues); +DECLARE_PER_CPU(struct rnd_state, sched_rnd_state); + +static inline u32 sched_rng(void) +{ + return prandom_u32_state(this_cpu_ptr(&sched_rnd_state)); +} =20 #define cpu_rq(cpu) (&per_cpu(runqueues, (cpu))) #define this_rq() this_cpu_ptr(&runqueues) diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 4bd825c24..bd8b2b301 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1632,6 +1632,12 @@ sd_init(struct sched_domain_topology_level *tl, =20 .last_balance =3D jiffies, .balance_interval =3D sd_weight, + + /* 50% success rate */ + .newidle_call =3D 512, + .newidle_success =3D 256, + .newidle_ratio =3D 512, + .max_newidle_lb_cost =3D 0, .last_decay_max_lb_cost =3D jiffies, .child =3D child, --=20 2.40.4