From nobody Mon Feb 9 05:01:34 2026 Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB9CC1ABEBD for ; Tue, 20 Aug 2024 16:35:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171721; cv=none; b=VfyxX6nmJ3TVb6NSajnYO9yeKcjYVUFh0bjt06q0gLd47nWEzZrOdQlpqklVWrKnL8z03zNaCpg2bY3UwjEwmW5Cv+Wt6JIVP/end2eej9vuyVpP7SQ4XCMF1AASTqe7w2Qfc5MEB3pfpZC+HSnC70PE4eCcsO3vON9SN5uybPQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171721; c=relaxed/simple; bh=zQ47ImMy4Dn1DCDt3xGlBdnPJ6cBUL0LZB3cTsDPnpA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oJQ3RWSC0wZsTP4iLJB7YadkapCd/R661fYDNXUcWjA3nZK9VzLwp9gKSAoIgeGP/j9X9DIpometibsUAAeeFxzry/XQBPxngxlsZxlKcWv74YpTaJRzFE0LAtRAWQGcUlujm8UunvIKHTh4DdxDFcj8O0ajeLMU/nOgqmWRbWc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=B+yMZs56; arc=none smtp.client-ip=209.85.167.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="B+yMZs56" Received: by mail-lf1-f44.google.com with SMTP id 2adb3069b0e04-5332f488c6eso5297806e87.1 for ; Tue, 20 Aug 2024 09:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171718; x=1724776518; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ALljPJwr4PeLXzf4Mb7jserQc5yvW1LK2jxOSl+dUgk=; b=B+yMZs56DcUH8lwhBw7LjnX1UE4bUz+uXrsRAFZbq2wDl9WIxCE+7bQCJwVAV5dsvD 1HsSi9npTs5kShx7/YWBr/EBeHotOA4Dpbfrbr2FRXrBNy60r0xdiA3bwQ1g6i+Y/DNs C/bxMu8u18z+qiYdM/h/tJV3T9vRh/uxJxbCCKC2n9UDjv2RAhlDe9MPXAbNuxxPyfDq bMuW7p4nT+noroAo2WrnYcpYuAjP8ldg5f+Y0le8COnziAh0YLgg4T2nKPUipHxp1kK7 dILxGzzGxGrRXuCOJanfpKtPd+YKkTbvX4DuWhHIKs156AogCLT6SJAUMuiXA3HMrzCz U0nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171718; x=1724776518; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ALljPJwr4PeLXzf4Mb7jserQc5yvW1LK2jxOSl+dUgk=; b=EwcT1qPi7HU8zWUKQrFWdfBgQwb7yFGs8dyEhgI1rQ12AaQZIYI4tzjNVzH2LBtPLs dcRtiunO8BRzSG51MrKdgZkTI0bljYkrEXw0VI+zQsmRktTbbC5AtNU40xhAxa5SYEYw AJC9f7/xZ2UnuYniWrsle1wusYrusvzxshgkbMqL/OUTqVSVLva5fzcVcCWNhW6v+KIz sjMm4VioL7GsrqajRMImTt08ayHVYFmy1pZ2RfKNeNMmq7QJPEjGmlbHSpy+SeyMC1G9 sotOrHV73gOFjqjojZFZ6yC9pvNLjWsD/2sLh/AynBnoO/WQ2kbZb/1utSfQw82kRBCy GgIg== X-Forwarded-Encrypted: i=1; AJvYcCWorbH2El7Kz0P05IV2+Z/Biea0DvQLrkKqz/EzUvykx3n9bdXqouMgZ/F2n4xOPpTsS+5aDhX4h+Wsd+Lb0tcksXVa7CEz/0A9aZOm X-Gm-Message-State: AOJu0Yy8ALdIa3wWkJznhfA4w+kPV7WrVQcX8K8ztbsFNU4lQMcDa2nX F3da/DnUixt1F7rFIraUSqu2HtCI7donh62E5thyWtcdh4zGpxlaDBu7KNc+GSA= X-Google-Smtp-Source: AGHT+IGuDVH/ZjqmvSREiN6JjWHaWFgiSgGpgzFBXWMg8TVU1iExwbDmMv1TgZFfKYdZJrbLW1T/7A== X-Received: by 2002:a05:6512:1193:b0:533:4620:ebec with SMTP id 2adb3069b0e04-5334620eda4mr942030e87.3.1724171717578; Tue, 20 Aug 2024 09:35:17 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.35.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:35:17 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 01/16] sched: cpufreq: Rename map_util_perf to sugov_apply_dvfs_headroom Date: Tue, 20 Aug 2024 17:34:57 +0100 Message-Id: <20240820163512.1096301-2-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We are providing headroom for the utilization to grow until the next decision point to pick the next frequency. Give the function a better name and give it some documentation. It is not really mapping anything. Also move it to cpufreq_schedutil.c. This function relies on updating util signal appropriately to give a headroom to grow. This is tied to schedutil and scheduler and not something that can be shared with other governors. Acked-by: Viresh Kumar Acked-by: Rafael J. Wysocki Reviewed-by: Vincent Guittot Signed-off-by: Qais Yousef --- include/linux/sched/cpufreq.h | 5 ----- kernel/sched/cpufreq_schedutil.c | 20 +++++++++++++++++++- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/include/linux/sched/cpufreq.h b/include/linux/sched/cpufreq.h index bdd31ab93bc5..d01755d3142f 100644 --- a/include/linux/sched/cpufreq.h +++ b/include/linux/sched/cpufreq.h @@ -28,11 +28,6 @@ static inline unsigned long map_util_freq(unsigned long = util, { return freq * util / cap; } - -static inline unsigned long map_util_perf(unsigned long util) -{ - return util + (util >> 2); -} #endif /* CONFIG_CPU_FREQ */ =20 #endif /* _LINUX_SCHED_CPUFREQ_H */ diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedu= til.c index eece6244f9d2..575df3599813 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -178,12 +178,30 @@ static unsigned int get_next_freq(struct sugov_policy= *sg_policy, return cpufreq_driver_resolve_freq(policy, freq); } =20 +/* + * DVFS decision are made at discrete points. If CPU stays busy, the util = will + * continue to grow, which means it could need to run at a higher frequency + * before the next decision point was reached. IOW, we can't follow the ut= il as + * it grows immediately, but there's a delay before we issue a request to = go to + * higher frequency. The headroom caters for this delay so the system cont= inues + * to run at adequate performance point. + * + * This function provides enough headroom to provide adequate performance + * assuming the CPU continues to be busy. + * + * At the moment it is a constant multiplication with 1.25. + */ +static inline unsigned long sugov_apply_dvfs_headroom(unsigned long util) +{ + return util + (util >> 2); +} + unsigned long sugov_effective_cpu_perf(int cpu, unsigned long actual, unsigned long min, unsigned long max) { /* Add dvfs headroom to actual utilization */ - actual =3D map_util_perf(actual); + actual =3D sugov_apply_dvfs_headroom(actual); /* Actually we don't need to target the max performance */ if (actual < max) max =3D actual; --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A718B1B3F08 for ; Tue, 20 Aug 2024 16:35:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171722; cv=none; b=pJKnh70EkdNMhoro8nJs2qgYxaoVZsyIEZYq/PcvGvmBu+0FOc1CKAWBl6EIynSPdR3FcmVsI2iY85OBE7RuUdtFTPDyCyTXA42df/4jr3d5S0pWc0TNnsfrsxpAEqeeJrdqHYk//5wvbHRzNtkZaqL1QzQkk8AqFUE14wkTKM8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171722; c=relaxed/simple; bh=x61t08722IA9HiK/FzMTCXQkBhUf938GZOXU6aQsSSk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Yn9+Yv1L4MrSlF4unn8EEv9Fsx4LNfpzaEGSJ8Pc2N9cp1Zc3z14Mm39TLJvORKDS+E81funCEwTGEuHjobGp0ZE8ZKrK8vgPW2H/R8e9Gxzf2EIHujB6fX9mpTLQ6U4+vdNiW/RRTN5oLUnXmfr0lz82gETagZkoXTND5L0JNA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=V6TR+HEk; arc=none smtp.client-ip=209.85.208.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="V6TR+HEk" Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-5a10835487fso8653171a12.1 for ; Tue, 20 Aug 2024 09:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171719; x=1724776519; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P4GvNZ2q2vkKbUvc/NgCLEzCheBxpfsamb4EXfBM/Og=; b=V6TR+HEk6ch4QlU0saqGNMJ5DhHPrFkh0qnwkkIkhf/0rInDOAi1ajEM91s4akYkIR oCv9yClD3OTbLl8kyOeWbTIsGihOBFlKn6E4j2/q3BCrcVkcCnaUaQWkcTnfhCQ4Bb22 tAh3+7g+dIphmESa6y1cVQCKCRcsnnf14Vno3QKL5826xuLylVi2ErstQCLz/7+pMjwI mHaBz9VYiHsBHaMqUlvhhRc3hmcfFU/MG6BjizdwMowafdQfVNq+R1p8jot7WqQXmX6I MHw3IfTHV2sRUlUYcKN7fidLgJfpaWVgU3mbg5JzZ7KmIOi5NwgFAHSz4AASMOf6bcDZ 0UVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171719; x=1724776519; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P4GvNZ2q2vkKbUvc/NgCLEzCheBxpfsamb4EXfBM/Og=; b=RUENz0qtB1Ff4Zpbb0BuI5oR7BTg4A4mJmdH0e1tHg1ThFG4An7v9Pm64aDoByeLmi W7bAEE4VhfEty+I0PC4DuI1UAGlXTbiFLM8PhCyWenM5iMGqToLOm8FW2/LPuGsQTexW 4R4FdJerVg+s2OI0cLL4B0J19junmT5FrzVTS13pZE2WBantoxQKu+BM2e+KWx0GiIYM zidR7q1hErV7b5YJUwHPgKTQgDOly96vuBKUWukrQynTWOiOLUlfC//WPHwEz50b6CBJ PgEd1LY4XON6HxBeJhHJCBgAzJpKvknkE0iIqkNdzbhb0h9YA50IbLpD73QDQzmYTkbo j44Q== X-Forwarded-Encrypted: i=1; AJvYcCWVHcHXmrBCu+8SI+A4NFM/xYcyGNtYTdRzef28BMY1KQpS5fnMRKqj0FgTnnjYF0O8Yc4YhKlH5b1rbpg=@vger.kernel.org X-Gm-Message-State: AOJu0Yx+wNCEbniJnjDyGrPRqvVj77S/RPbdO8uJj5dpFwzIcWzdjY+x RFtu5l7pcx9n+FnewmwDtxkcwbs2xHpaBKFa0v2sMyqfsCVDxzzazb62xDzRmaM= X-Google-Smtp-Source: AGHT+IGANk0JtgzuFnotXZ6oaaNXX0v0ntO5GyaujzzJT2sPsn/z/iWuqqA4+lqIPu3rUA7Z15IshQ== X-Received: by 2002:a17:907:1c22:b0:a80:f6f2:e070 with SMTP id a640c23a62f3a-a83928a3333mr1164972466b.3.1724171718395; Tue, 20 Aug 2024 09:35:18 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.35.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:35:18 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 02/16] sched/pelt: Add a new function to approximate the future util_avg value Date: Tue, 20 Aug 2024 17:34:58 +0100 Message-Id: <20240820163512.1096301-3-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Given a util_avg value, the new function will return the future one given a runtime delta. This will be useful in later patches to help replace some magic margins with more deterministic behavior. Signed-off-by: Qais Yousef --- kernel/sched/pelt.c | 22 +++++++++++++++++++++- kernel/sched/sched.h | 1 + 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c index fa52906a4478..2ce83e880bd5 100644 --- a/kernel/sched/pelt.c +++ b/kernel/sched/pelt.c @@ -466,4 +466,24 @@ int update_irq_load_avg(struct rq *rq, u64 running) =20 return ret; } -#endif +#endif /* CONFIG_HAVE_SCHED_AVG_IRQ */ + +/* + * Approximate the new util_avg value assuming an entity has continued to = run + * for @delta us. + */ +unsigned long approximate_util_avg(unsigned long util, u64 delta) +{ + struct sched_avg sa =3D { + .util_sum =3D util * PELT_MIN_DIVIDER, + .util_avg =3D util, + }; + + if (unlikely(!delta)) + return util; + + accumulate_sum(delta, &sa, 1, 0, 1); + ___update_load_avg(&sa, 0); + + return sa.util_avg; +} diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 4c36cc680361..294c6769e330 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3064,6 +3064,7 @@ unsigned long sugov_effective_cpu_perf(int cpu, unsig= ned long actual, unsigned long min, unsigned long max); =20 +unsigned long approximate_util_avg(unsigned long util, u64 delta); =20 /* * Verify the fitness of task @p to run on @cpu taking into account the --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E7951B3F33 for ; Tue, 20 Aug 2024 16:35:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171723; cv=none; b=Ya+qCtRwPf8NZT4s4PHMvX/JzQ0kyIOQXWIQIatDOmGoQ7Gvmi3AX5Swzjs4QLGS/dkXmfEfisDnExiRQiB3Nid6+DZeD8SMy0ydlhpmo1Zl3SG1Tbie+A/p8kxgdoaSbEnKqosKK/Uf0iOnttlUKzo8CL55+9wQ+cUL31l9OoI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171723; c=relaxed/simple; bh=FF4nQxBM0nS1lqakhr4Q+sUzCZDVAFySaog/xxInf3M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ok+lExfqimQJetY1Y223sgogUU07DNUORaRnXZtBWCzT2GFgzSNKqJ20D8pZmqgzv8dbo3u2hyelgqD7NCxdyUeWNRFuZxjoxmD7nCcwy/Ly9Nne2Y3IZRifDWjddYakch2euBsqRPVDKZ0YUVTMZh0hZECHKXlsAU3PU7Dc01w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=I8RXwI2m; arc=none smtp.client-ip=209.85.167.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="I8RXwI2m" Received: by mail-lf1-f53.google.com with SMTP id 2adb3069b0e04-52f04b3cb33so2907819e87.0 for ; Tue, 20 Aug 2024 09:35:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171719; x=1724776519; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/BSe4KOyJw2YEYF8rnu6mjIl7u9y+oNGIQ3Hg6lKPvk=; b=I8RXwI2menTRCf+yMfWW84ydKaVgC8EaI0+dk0M56EXo1MH5w5kGeC4s2aOpnqQWZQ VK9w+FqoOQlkqcCO7ngOn4KWGi7Dk9rcqopIwm3NbpFrHtPkSMIjFLhpvgtwR11sl0n3 9uONdTjNa87gXkR4Z8g883SQgjTwYZorQuGWz/LvC9IWbmw2KUxkvsZzFZLpE3zmgh4d JQ20QBJlGdjA0uCPNyh5vaEAXt7DZzaeYNgAC81/il4bqs1+1d7qGzDuKrcaq7ySaRf1 9JTFsbjojBZvIOii/Uzl36hdwPQfw+PHkl4iIcRTkr5UutahNW9NWZk6uFpXt7Dn7GVc gWPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171719; x=1724776519; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/BSe4KOyJw2YEYF8rnu6mjIl7u9y+oNGIQ3Hg6lKPvk=; b=Q14D2RETVLuKLTnjInlzJbgZmyroMLJG7R6WILgKaEqAF+CeQc+14APpN0mjExAdPO gkk9SrBcmtQuLW+MyJ4FNAlTzkDTz5gZvLKvuiDVYIYd1zSvwXAvUlaZ2pICC5x01rm+ VTcrE2SQEorjrtpcbeh3IDow2dWXjoAN4F7MrED5/dl0vsCyk5xSAZZMpn23EWYsNSKR xlCkXLnJ3jjRWRNVWjW/VqrhN01QRQ3umVgE6wUotogYBSJ/aiCMVMc/39CBYr2BPYmp KenuLLhpta9tUXW9Ts/xmqQ1bw4OwqYnncNY8NC7X5xAEouypFmgPlqkOWhpAjqZpjUm TQLQ== X-Forwarded-Encrypted: i=1; AJvYcCXanHBR2HTy+jakdcDJ5Uk7P02EO/B+ltTkdKjtFype512LaRwlq/CgNqCiAJYjUqw0Zo2be9aICJqr5+w=@vger.kernel.org X-Gm-Message-State: AOJu0Yw7zu+9wdipO6hoRAYoaxQb89P1Zc/yg++4RKSWJQSyOsr1vzL4 nuu4EoMPSHCUV5SIpDMLheEQ7kn5TA2QFzO8r6ep3i9riP9zCI3trMPOE2bADyE= X-Google-Smtp-Source: AGHT+IFsK5sSF4zn+iA0VCf5c4osiSUBkZLZkTq8PaWwmO8mR0g9hawJ9xoV1658/XBy+x9/4qSzfw== X-Received: by 2002:a05:6512:3d28:b0:52c:8342:6699 with SMTP id 2adb3069b0e04-5331c6e4088mr12083886e87.55.1724171719232; Tue, 20 Aug 2024 09:35:19 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.35.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:35:18 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 03/16] sched/pelt: Add a new function to approximate runtime to reach given util Date: Tue, 20 Aug 2024 17:34:59 +0100 Message-Id: <20240820163512.1096301-4-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" It is basically the ramp-up time from 0 to a given value. Will be used later to implement new tunable to control response time for schedutil. Signed-off-by: Qais Yousef --- kernel/sched/pelt.c | 21 +++++++++++++++++++++ kernel/sched/sched.h | 1 + 2 files changed, 22 insertions(+) diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c index 2ce83e880bd5..06cb881ba582 100644 --- a/kernel/sched/pelt.c +++ b/kernel/sched/pelt.c @@ -487,3 +487,24 @@ unsigned long approximate_util_avg(unsigned long util,= u64 delta) =20 return sa.util_avg; } + +/* + * Approximate the required amount of runtime in ms required to reach @uti= l. + */ +u64 approximate_runtime(unsigned long util) +{ + struct sched_avg sa =3D {}; + u64 delta =3D 1024; // period =3D 1024 =3D ~1ms + u64 runtime =3D 0; + + if (unlikely(!util)) + return runtime; + + while (sa.util_avg < util) { + accumulate_sum(delta, &sa, 1, 0, 1); + ___update_load_avg(&sa, 0); + runtime++; + } + + return runtime; +} diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 294c6769e330..47f158b2cdc2 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3065,6 +3065,7 @@ unsigned long sugov_effective_cpu_perf(int cpu, unsig= ned long actual, unsigned long max); =20 unsigned long approximate_util_avg(unsigned long util, u64 delta); +u64 approximate_runtime(unsigned long util); =20 /* * Verify the fitness of task @p to run on @cpu taking into account the --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E23F51B8EA7 for ; Tue, 20 Aug 2024 16:35:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171723; cv=none; b=mNMM7/kKzG6jq2qk7A020cDTPE2vz/QMPPT1yX5EbV4nkJJchK/ApmHqK7GLLaIWJjuNfTyFrTRXQY3uyo84Sq/pDKq7ahJ5QVAIuQshMJQ9S0vAShDCr174JXjJx3l9dbJPwrEBS6IpJrNEG+la4yAeNDmXbz9refroP39A8oY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171723; c=relaxed/simple; bh=iywCMvLV4TtRo7p9sFYjyRIh592/LLot9ueypZVp58c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GD+ex6hp2QdFUreptapd12RdQ6t4EMIHs8jHRujthh+tUVeyGRkXFO1WEg6bCbjQhRpyUqxaXN62tiEaLY0Nlbc6PYcrJLAWqg7DmK9IipLBSQNTXGzWoRVKXXp0zKy6ju3gf2rM+TPkYwoPHK7Dssktc6FXS7Vx7GH63ffxvEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=FoFhAMWf; arc=none smtp.client-ip=209.85.221.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="FoFhAMWf" Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-3717de33d58so3689655f8f.1 for ; Tue, 20 Aug 2024 09:35:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171720; x=1724776520; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4F1WOdF23Pe6zUBd4kovJU9RvUMFw9YRtUZ0tQf1aSc=; b=FoFhAMWfB3718FijD5y0b0aIYk+gikZSkfUeJJ8Yu49C4AgRG8JGKeGBSFm6+wDlU5 s9Z6BTE5bDwj4MjhuSaRWcw6meo0nk6rGoknxGWTvnXEYltGR0/T1Bh+mKwyUKRJY5mf HADhrMKtY3E3Xqzono+H0556XGUdoLZ5K3ntUsnX36lce/UEyTuCB24+tmRsSeZ3qDHE e7rplvTGfuwsDB9jjY2cDmr0GHaM5h8tW3678gRvZZpOIQGAOQYlQSSK/dOGRR3840bY J6bV4uJyckwa7Dl7ySRG0fMzW8rABf4YBK2LuT5AlQQlKtxxjQCzXkJvYAkw+QmzUsyc QVYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171720; x=1724776520; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4F1WOdF23Pe6zUBd4kovJU9RvUMFw9YRtUZ0tQf1aSc=; b=Khe/iyclNteVZ/I+3WOHAARauKSQkK9WNj+qS+9dy6aaL1jh62PtAuxSpoibEXYAfX hVssXM6uP/6emLrVdIN2aBXD1Fkz5Bp0GAXdmpDmjZScxTGyFRDk+YQroThLAMWw6+2j YYExL5yahAVlzJDP9CgIe56dEbc1acoago4Bzbwpsltun6FVwK8FycLhLH5Hm8sicwfe daUJrF4WB3q3TFNmO3tKDzIAtTbjotyA5EVC5hmuaIsCdLKWpXo0CImlKwYLk0UQTlXf x0185NJxuYtE1mJXZAp5MS5oeCi0tPQds37xItWIp7NQ0M2aqLL+l882ZHdSn7vYLjiZ 4BCQ== X-Forwarded-Encrypted: i=1; AJvYcCXBhdpbEnnQeC2voOZUys6GJ4M5uDJLmCOiEONj05UPyP+vVypmku4ZzMCbNBwrjl9820uLy9nPikEUD7LOzOidRGq2lN9sDqrGE//R X-Gm-Message-State: AOJu0Yxth/nwAjl058NtLsKaAlUtn0e0ys0b3Uf/ShTtiPEP/f76snx4 /Y8dUTuyAuTFT5Q4H2+sjn71rb7ZhIr6FX6z3vmn+LhMYj5ZD/mxhOJ6egomHEI= X-Google-Smtp-Source: AGHT+IEu45k94fu5QhYe3wBpe9rK5a5C2AuIr7eJpYJMNhrKU40wfFxlLbRCcxUSZTh9bJvNjgDxDA== X-Received: by 2002:a5d:400d:0:b0:367:8847:5bf4 with SMTP id ffacd0b85a97d-37194317679mr9919698f8f.10.1724171720072; Tue, 20 Aug 2024 09:35:20 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.35.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:35:19 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 04/16] sched/fair: Remove magic hardcoded margin in fits_capacity() Date: Tue, 20 Aug 2024 17:35:00 +0100 Message-Id: <20240820163512.1096301-5-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace hardcoded margin value in fits_capacity() with better dynamic logic. 80% margin is a magic value that has served its purpose for now, but it no longer fits the variety of systems that exist today. If a system is over powered specifically, this 80% will mean we leave a lot of capacity unused before we decide to upmigrate on HMP system. On many systems the little cores are under powered and ability to migrate faster away from them is desired. Redefine misfit migration to mean the utilization threshold at which the task would become misfit at the next load balance event assuming it becomes an always running task. To calculate this threshold, we use the new approximate_util_avg() function to find out the threshold, based on arch_scale_cpu_capacity() the task will be misfit if it continues to run for a TICK_USEC which is our worst case scenario for when misfit migration will kick in. Signed-off-by: Qais Yousef --- kernel/sched/core.c | 1 + kernel/sched/fair.c | 40 ++++++++++++++++++++++++++++++++-------- kernel/sched/sched.h | 1 + 3 files changed, 34 insertions(+), 8 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 6d35c48239be..402ee4947ef0 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -8266,6 +8266,7 @@ void __init sched_init(void) rq->sd =3D NULL; rq->rd =3D NULL; rq->cpu_capacity =3D SCHED_CAPACITY_SCALE; + rq->fits_capacity_threshold =3D SCHED_CAPACITY_SCALE; rq->balance_callback =3D &balance_push_callback; rq->active_balance =3D 0; rq->next_balance =3D jiffies; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 9057584ec06d..e5e986af18dc 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -95,11 +95,15 @@ int __weak arch_asym_cpu_priority(int cpu) } =20 /* - * The margin used when comparing utilization with CPU capacity. - * - * (default: ~20%) + * fits_capacity() must ensure that a task will not be 'stuck' on a CPU wi= th + * lower capacity for too long. This the threshold is the util value at wh= ich + * if a task becomes always busy it could miss misfit migration load balan= ce + * event. So we consider a task is misfit before it reaches this point. */ -#define fits_capacity(cap, max) ((cap) * 1280 < (max) * 1024) +static inline bool fits_capacity(unsigned long util, int cpu) +{ + return util < cpu_rq(cpu)->fits_capacity_threshold; +} =20 /* * The margin used when comparing CPU capacities. @@ -4978,14 +4982,13 @@ static inline int util_fits_cpu(unsigned long util, unsigned long uclamp_max, int cpu) { - unsigned long capacity =3D capacity_of(cpu); unsigned long capacity_orig; bool fits, uclamp_max_fits; =20 /* * Check if the real util fits without any uclamp boost/cap applied. */ - fits =3D fits_capacity(util, capacity); + fits =3D fits_capacity(util, cpu); =20 if (!uclamp_is_used()) return fits; @@ -9592,12 +9595,33 @@ static void update_cpu_capacity(struct sched_domain= *sd, int cpu) { unsigned long capacity =3D scale_rt_capacity(cpu); struct sched_group *sdg =3D sd->groups; + struct rq *rq =3D cpu_rq(cpu); + u64 limit; =20 if (!capacity) capacity =3D 1; =20 - cpu_rq(cpu)->cpu_capacity =3D capacity; - trace_sched_cpu_capacity_tp(cpu_rq(cpu)); + rq->cpu_capacity =3D capacity; + trace_sched_cpu_capacity_tp(rq); + + /* + * Calculate the util at which the task must be considered a misfit. + * + * We must ensure that a task experiences the same ramp-up time to + * reach max performance point of the system regardless of the CPU it + * is running on (due to invariance, time will stretch and task will + * take longer to achieve the same util value compared to a task + * running on a big CPU) and a delay in misfit migration which depends + * on TICK doesn't end up hurting it as it can happen after we would + * have crossed this threshold. + * + * To ensure that invaraince is taken into account, we don't scale time + * and use it as-is, approximate_util_avg() will then let us know the + * our threshold. + */ + limit =3D approximate_runtime(arch_scale_cpu_capacity(cpu)) * USEC_PER_MS= EC; + limit -=3D TICK_USEC; /* sd->balance_interval is more accurate */ + rq->fits_capacity_threshold =3D approximate_util_avg(0, limit); =20 sdg->sgc->capacity =3D capacity; sdg->sgc->min_capacity =3D capacity; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 47f158b2cdc2..ab4672675b84 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1093,6 +1093,7 @@ struct rq { struct sched_domain __rcu *sd; =20 unsigned long cpu_capacity; + unsigned long fits_capacity_threshold; =20 struct balance_callback *balance_callback; =20 --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF9081BA87C for ; Tue, 20 Aug 2024 16:35:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171725; cv=none; b=dJ3BfwtimbSfJ5KmHdDu2brnHFnLu/av3/5WL8x6O4TAWbYSK8MNFZ/yl+J4LQ9Qei2Gvp6jkOmDdLWoKJsNuV3UDiI9c+ra/JSKQ+/9OZyiW/pLki5kE0RXUAw5AgygCtAclL2ihAUuLnm9DZzwS5jcaSUVnAz+hdKKR8WnwxA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171725; c=relaxed/simple; bh=ZmhmvAh2ezCJJ4PmFxbu0s4/c7f7Y3hO+tlLfSwsztc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YdOvuQEUJz+JSQ9ZrrM3CCwglWZHTfwDZLpGj7iXC1XfVE6YUE7iUFFdS+iIdaGlamLav3Hy7O8NfISdhA5QBWeWd/fPmgHCHPYZJhqIPRSmb7j5Q5mrVor3ZlzB674AL3vot6CPjr2VEOz6/yN7Qa2jgbDPTh1kpgAH2oz9sGU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=ZN42eiHZ; arc=none smtp.client-ip=209.85.218.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="ZN42eiHZ" Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-a7abe5aa9d5so615137666b.1 for ; Tue, 20 Aug 2024 09:35:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171721; x=1724776521; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xQ1DK2daPE9ncuhHLypAOQVYnkEU114ByewzWSw0TA4=; b=ZN42eiHZCPvTwREOEJqJbGsuizb3xhvMQU+MZwMM7zwdpGQXJPR6SZZfQc6p3D/mN9 OBtJ23dNwC5DWwNJpmSbUXMusS5vccpj7YQVygqKPqeiXdhZCn6gccNgy+Gsbjf9j3+2 cZr7ggSGTuB73fANZfHQXOmh7FiwA/eHPo6fXHEpu35ASUbEC6aqWRzlXNeo80nRJ6To RXOcI+E/vyHC7ycySs3+7f5fDKDj9NTqZ47GICpek8hUBtP3p9vP67+UURg1Eq+ZT+EW 3KNhD8the0zCN95BVUUf05GwYDBQ2KWn4kkjDTFQLurhYDTtbS/ZqF54VpcpbRIAxth/ kq3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171721; x=1724776521; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xQ1DK2daPE9ncuhHLypAOQVYnkEU114ByewzWSw0TA4=; b=UGpoqI5mpwyn0vkaBSnUneIu1fhopod1ShXxn96xQiJEAiQ52Zi8UDZ7IE38Ixb7bM 5k74kpbVOyiuGuluigOEmB47oNxQgsm0TtHUUB1DdZMslAiLpAcajNy73DiLBGFOUZ43 F38eCuMSSXwzQyTYN6EXdEXutviFxri/+yN/dsQL12hP4jeZ4IEnNrz0gs5sTen0Z6/6 jzv1VAJoJPaRMrL3957NnFsldsBRAm2zzfXOltZ00IeP5hfPpq2jnrjx05bL8Jm4AO8o 2ylyOGS7FhbCyuorMbKC69pY7YXa6QBPUQTi0aDuaJ2Jn3Kkme0kVM+QqweDdHOV4THZ ahug== X-Forwarded-Encrypted: i=1; AJvYcCWMKiqCb3zTPLkZau5Z5IxiiK/kNRKShoi6dq/y5yoMZ6lhnkIfU9H0gVDH4r11aAAU+BHoYxphdXadVW0=@vger.kernel.org X-Gm-Message-State: AOJu0YynwcFKzKE0jJy+wgjIXSc2NMPxCeKVSGRYdtfMlRtlKz7mD0kV rjZLeOa2VDWUc3Dy2XHzt+l/11dIW8C2NfPiZTwZo2f2lVqs4gF0t34gQGuXql0= X-Google-Smtp-Source: AGHT+IGJIEiqfLHFXG+Ko/nMDgS5J7g2UwUVQhLX/Pve1qWCglcSM9ewImEX+qVu2VF8smMRoWdXuQ== X-Received: by 2002:a17:907:72cb:b0:a6f:27e6:8892 with SMTP id a640c23a62f3a-a8392a38c8dmr1018867866b.60.1724171720898; Tue, 20 Aug 2024 09:35:20 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.35.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:35:20 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 05/16] sched: cpufreq: Remove magic 1.25 headroom from sugov_apply_dvfs_headroom() Date: Tue, 20 Aug 2024 17:35:01 +0100 Message-Id: <20240820163512.1096301-6-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace 1.25 headroom in sugov_apply_dvfs_headroom() with better dynamic logic. Instead of the magical 1.25 headroom, use the new approximate_util_avg() to provide headroom based on the dvfs_update_delay, which is the period at which the cpufreq governor will send DVFS updates to the hardware, or min(curr.se.slice, TICK_USEC) which is the max delay for util signal to change and promote a cpufreq update; whichever is higher. Add a new percpu dvfs_update_delay that can be cheaply accessed whenever sugov_apply_dvfs_headroom() is called. We expect cpufreq governors that rely on util to drive its DVFS logic/algorithm to populate these percpu variables. schedutil is the only such governor at the moment. The behavior of schedutil will change. Some systems will experience faster dvfs rampup (because of higher TICK or rate_limit_us), others will experience slower rampup. The impact on performance should not be visible if not for the black hole effect of utilization invariance. A problem that will be addressed in later patches. Later patches will also address how to provide better control of how fast or slow the system should respond to allow userspace to select their power/perf/thermal trade-off. Signed-off-by: Qais Yousef --- kernel/sched/core.c | 1 + kernel/sched/cpufreq_schedutil.c | 36 ++++++++++++++++++++++++++------ kernel/sched/sched.h | 9 ++++++++ 3 files changed, 40 insertions(+), 6 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 402ee4947ef0..7099e40cc8bd 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -118,6 +118,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(sched_update_nr_running_tp= ); EXPORT_TRACEPOINT_SYMBOL_GPL(sched_compute_energy_tp); =20 DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues); +DEFINE_PER_CPU_READ_MOSTLY(u64, dvfs_update_delay); =20 #ifdef CONFIG_SCHED_DEBUG /* diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedu= til.c index 575df3599813..303b0ab227e7 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -187,13 +187,28 @@ static unsigned int get_next_freq(struct sugov_policy= *sg_policy, * to run at adequate performance point. * * This function provides enough headroom to provide adequate performance - * assuming the CPU continues to be busy. + * assuming the CPU continues to be busy. This headroom is based on the + * dvfs_update_delay of the cpufreq governor or min(curr.se.slice, TICK_US= ), + * whichever is higher. * - * At the moment it is a constant multiplication with 1.25. + * XXX: Should we provide headroom when the util is decaying? */ -static inline unsigned long sugov_apply_dvfs_headroom(unsigned long util) +static inline unsigned long sugov_apply_dvfs_headroom(unsigned long util, = int cpu) { - return util + (util >> 2); + struct rq *rq =3D cpu_rq(cpu); + u64 delay; + + /* + * What is the possible worst case scenario for updating util_avg, ctx + * switch or TICK? + */ + if (rq->cfs.h_nr_running > 1) + delay =3D min(rq->curr->se.slice/1000, TICK_USEC); + else + delay =3D TICK_USEC; + delay =3D max(delay, per_cpu(dvfs_update_delay, cpu)); + + return approximate_util_avg(util, delay); } =20 unsigned long sugov_effective_cpu_perf(int cpu, unsigned long actual, @@ -201,7 +216,7 @@ unsigned long sugov_effective_cpu_perf(int cpu, unsigne= d long actual, unsigned long max) { /* Add dvfs headroom to actual utilization */ - actual =3D sugov_apply_dvfs_headroom(actual); + actual =3D sugov_apply_dvfs_headroom(actual, cpu); /* Actually we don't need to target the max performance */ if (actual < max) max =3D actual; @@ -579,15 +594,21 @@ rate_limit_us_store(struct gov_attr_set *attr_set, co= nst char *buf, size_t count struct sugov_tunables *tunables =3D to_sugov_tunables(attr_set); struct sugov_policy *sg_policy; unsigned int rate_limit_us; + int cpu; =20 if (kstrtouint(buf, 10, &rate_limit_us)) return -EINVAL; =20 tunables->rate_limit_us =3D rate_limit_us; =20 - list_for_each_entry(sg_policy, &attr_set->policy_list, tunables_hook) + list_for_each_entry(sg_policy, &attr_set->policy_list, tunables_hook) { + sg_policy->freq_update_delay_ns =3D rate_limit_us * NSEC_PER_USEC; =20 + for_each_cpu(cpu, sg_policy->policy->cpus) + per_cpu(dvfs_update_delay, cpu) =3D rate_limit_us; + } + return count; } =20 @@ -868,6 +889,9 @@ static int sugov_start(struct cpufreq_policy *policy) memset(sg_cpu, 0, sizeof(*sg_cpu)); sg_cpu->cpu =3D cpu; sg_cpu->sg_policy =3D sg_policy; + + per_cpu(dvfs_update_delay, cpu) =3D sg_policy->tunables->rate_limit_us; + cpufreq_add_update_util_hook(cpu, &sg_cpu->update_util, uu); } return 0; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index ab4672675b84..c2d9fba6ea7a 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3068,6 +3068,15 @@ unsigned long sugov_effective_cpu_perf(int cpu, unsi= gned long actual, unsigned long approximate_util_avg(unsigned long util, u64 delta); u64 approximate_runtime(unsigned long util); =20 +/* + * Any governor that relies on util signal to drive DVFS, must populate th= ese + * percpu dvfs_update_delay variables. + * + * It should describe the rate/delay at which the governor sends DVFS freq + * update to the hardware in us. + */ +DECLARE_PER_CPU_READ_MOSTLY(u64, dvfs_update_delay); + /* * Verify the fitness of task @p to run on @cpu taking into account the * CPU original capacity and the runtime/deadline ratio of the task. --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-lj1-f172.google.com (mail-lj1-f172.google.com [209.85.208.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E7B91BD507 for ; Tue, 20 Aug 2024 16:35:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171726; cv=none; b=gcPwaXwIUjjBuAOxWCDyMglImcqXtEdnGiCNdiHMy4hhuoeQNtPcVggQNHmwFPjAydsJLaeKSQU0TQxpnX3WV2NSmcnQWTtiBrbVESgtIUk4jsy4Rq9OkT2TOGe3ckjv4gZh3JGfRSw7jAbtYwH9taXApBvcHh+xMt9AFdWl3tw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171726; c=relaxed/simple; bh=tgYyWJNFjQme0TMeNdmbWnROtVj9eTJRm8pByyeiq9c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=m4szwvcTagqIxOiRPb79IkgVgl7s5IplBPUmWez9wkanVih76F5OUSASv+ux79gHHZpjHiieH2Bfg++PcFytX0Oj/7RBCuI1LGVcqxkuaioiab5afCavieHy3qAI81IL7QtWXVOJDGVUvy12mWsseP+znCKVw+qyHxE63DF8IMk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=1LzhH+1Z; arc=none smtp.client-ip=209.85.208.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="1LzhH+1Z" Received: by mail-lj1-f172.google.com with SMTP id 38308e7fff4ca-2f029e9c9cfso30807841fa.2 for ; Tue, 20 Aug 2024 09:35:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171722; x=1724776522; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cnyuo1aGTE5JLvwJSoqPF/e83EBO8SWfb0UwiymD/r8=; b=1LzhH+1ZQWYEG0T9WKKxRjlxjwI1DixubVZuDM68kOGY9SbrW/L+0Qt0YOagX+VT02 8FTzWCt3EiLhQh3gvezxNoHAptgaBXX5K6aMec5FiJqS+CLLGmMgq92IbG51tzXgVKXm 9AmmI9R69cQYu+cFGEpUcpcb7DsgDEqZsxFRvD7FkZGNf/VuOUR9n6Z5xVIqShTFQpQs l4OrL2047I9YKhOWczNebbIbzlHMpVVU8MVyV0I62MtCYBS1maOkC5Jzq5mmyB2WC2zG +NRHBx2lQBChqmgqswTGyPkzbCYCtFAGoPRf0hxgfqNdCuZGxIsdjG8K4s79htcjv3Bn bFvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171722; x=1724776522; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cnyuo1aGTE5JLvwJSoqPF/e83EBO8SWfb0UwiymD/r8=; b=J/mmZ1k9qVkbEW507ZKoOua2LrqKaBSSA2F/WXKr10HwIKCazAFdpdrL1r89tXO2Jm qwgWaoj1RfbFpqFMibN5J1V4BGVqHXoKV6UVMEaWQ8EpOAAfHSc6EEQVM+E9vbG2uURB KA/9aSNEpPKzOMLeWNWVCgDzfYC0Wn5ahfpbcL+H79mSWr21FiYg4C1pKvljL+7wm+K9 K6hqkfQq2feD2HFhBeFv+ftyYDXtGa0oAMeVgoZdxDaHBzfs2gzKaJqCswUYRr1LJpIg k0ytni6CZKaMYQfLGmsJ4u6zxVp8eTI8SdLhxwZeOu3DyJay3lRy9pt6ieu1IwJrZ8/x BeKg== X-Forwarded-Encrypted: i=1; AJvYcCWDL0vPAX7cKCOD3PiM/eD80mE28Gfu5jp4eR+l5fuNdA5m4tSotq6AnrSQQEH9hbb5hIMkDZpAdO2LlaS4x7I/DB5bS+Go4obfbTQ3 X-Gm-Message-State: AOJu0YwXzXqW6IHR5T/0xlW/pNC/SQvIwlobAqzljX/D71htaeeefO7f fYgsFf3VfvkMuzDsHakoKQCevXsyR6WxZZ3E1TI1+Bf5Y43sw3Bj0weIY3+QtcM= X-Google-Smtp-Source: AGHT+IFyy57O8ICffhFQegIIFct0J1HR2fD4S7+D4GtgQH4PdLNsbpsWoXm759jhmaZ/Fqu9rrA55w== X-Received: by 2002:a2e:84a:0:b0:2f3:ee58:eaf with SMTP id 38308e7fff4ca-2f3ee580f92mr20700721fa.23.1724171721872; Tue, 20 Aug 2024 09:35:21 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.35.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:35:21 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 06/16] sched/schedutil: Add a new tunable to dictate response time Date: Tue, 20 Aug 2024 17:35:02 +0100 Message-Id: <20240820163512.1096301-7-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The new tunable, response_time_ms, allow us to speed up or slow down the response time of the policy to meet the perf, power and thermal characteristic desired by the user/sysadmin. There's no single universal trade-off that we can apply for all systems even if they use the same SoC. The form factor of the system, the dominant use case, and in case of battery powered systems, the size of the battery and presence or absence of active cooling can play a big role on what would be best to use. The new tunable provides sensible defaults, but yet gives the power to control the response time to the user/sysadmin, if they wish to. This tunable is applied before we apply the DVFS headroom. The default behavior of applying 1.25 headroom can be re-instated easily now. But we continue to keep the min required headroom to overcome hardware limitation in its speed to change DVFS. And any additional headroom to speed things up must be applied by userspace to match their expectation for best perf/watt as it dictates a type of policy that will be better for some systems, but worse for others. There's a whitespace clean up included in sugov_start(). Signed-off-by: Qais Yousef --- Documentation/admin-guide/pm/cpufreq.rst | 17 +++- drivers/cpufreq/cpufreq.c | 4 +- include/linux/cpufreq.h | 3 + kernel/sched/cpufreq_schedutil.c | 115 ++++++++++++++++++++++- 4 files changed, 132 insertions(+), 7 deletions(-) diff --git a/Documentation/admin-guide/pm/cpufreq.rst b/Documentation/admin= -guide/pm/cpufreq.rst index 6adb7988e0eb..fa0d602a920e 100644 --- a/Documentation/admin-guide/pm/cpufreq.rst +++ b/Documentation/admin-guide/pm/cpufreq.rst @@ -417,7 +417,7 @@ is passed by the scheduler to the governor callback whi= ch causes the frequency to go up to the allowed maximum immediately and then draw back to the value returned by the above formula over time. =20 -This governor exposes only one tunable: +This governor exposes two tunables: =20 ``rate_limit_us`` Minimum time (in microseconds) that has to pass between two consecutive @@ -427,6 +427,21 @@ This governor exposes only one tunable: The purpose of this tunable is to reduce the scheduler context overhead of the governor which might be excessive without it. =20 +``respone_time_ms`` + Amount of time (in milliseconds) required to ramp the policy from + lowest to highest frequency. Can be decreased to speed up the + responsiveness of the system, or increased to slow the system down in + hope to save power. The best perf/watt will depend on the system + characteristics and the dominant workload you expect to run. For + userspace that has smart context on the type of workload running (like + in Android), one can tune this to suite the demand of that workload. + + Note that when slowing the response down, you can end up effectively + chopping off the top frequencies for that policy as the util is capped + to 1024. On HMP systems this chopping effect will only occur on the + biggest core whose capacity is 1024. Don't rely on this behavior as + this is a limitation that can hopefully be improved in the future. + This governor generally is regarded as a replacement for the older `ondema= nd`_ and `conservative`_ governors (described below), as it is simpler and more tightly integrated with the CPU scheduler, its overhead in terms of CPU co= ntext diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index a45aac17c20f..5dc44c3694fe 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -533,8 +533,8 @@ void cpufreq_disable_fast_switch(struct cpufreq_policy = *policy) } EXPORT_SYMBOL_GPL(cpufreq_disable_fast_switch); =20 -static unsigned int __resolve_freq(struct cpufreq_policy *policy, - unsigned int target_freq, unsigned int relation) +unsigned int __resolve_freq(struct cpufreq_policy *policy, + unsigned int target_freq, unsigned int relation) { unsigned int idx; =20 diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h index 20f7e98ee8af..c14ffdcd8933 100644 --- a/include/linux/cpufreq.h +++ b/include/linux/cpufreq.h @@ -622,6 +622,9 @@ int cpufreq_driver_target(struct cpufreq_policy *policy, int __cpufreq_driver_target(struct cpufreq_policy *policy, unsigned int target_freq, unsigned int relation); +unsigned int __resolve_freq(struct cpufreq_policy *policy, + unsigned int target_freq, + unsigned int relation); unsigned int cpufreq_driver_resolve_freq(struct cpufreq_policy *policy, unsigned int target_freq); unsigned int cpufreq_policy_transition_delay_us(struct cpufreq_policy *pol= icy); diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedu= til.c index 303b0ab227e7..94e35b7c972d 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -8,9 +8,12 @@ =20 #define IOWAIT_BOOST_MIN (SCHED_CAPACITY_SCALE / 8) =20 +DEFINE_PER_CPU_READ_MOSTLY(unsigned long, response_time_mult); + struct sugov_tunables { struct gov_attr_set attr_set; unsigned int rate_limit_us; + unsigned int response_time_ms; }; =20 struct sugov_policy { @@ -22,6 +25,7 @@ struct sugov_policy { raw_spinlock_t update_lock; u64 last_freq_update_time; s64 freq_update_delay_ns; + unsigned int freq_response_time_ms; unsigned int next_freq; unsigned int cached_raw_freq; =20 @@ -59,6 +63,70 @@ static DEFINE_PER_CPU(struct sugov_cpu, sugov_cpu); =20 /************************ Governor internals ***********************/ =20 +static inline u64 sugov_calc_freq_response_ms(struct sugov_policy *sg_poli= cy) +{ + int cpu =3D cpumask_first(sg_policy->policy->cpus); + unsigned long cap =3D arch_scale_cpu_capacity(cpu); + unsigned int max_freq, sec_max_freq; + + max_freq =3D sg_policy->policy->cpuinfo.max_freq; + sec_max_freq =3D __resolve_freq(sg_policy->policy, + max_freq - 1, + CPUFREQ_RELATION_H); + + /* + * We will request max_freq as soon as util crosses the capacity at + * second highest frequency. So effectively our response time is the + * util at which we cross the cap@2nd_highest_freq. + */ + cap =3D sec_max_freq * cap / max_freq; + + return approximate_runtime(cap + 1); +} + +static inline void sugov_update_response_time_mult(struct sugov_policy *sg= _policy) +{ + unsigned long mult; + int cpu; + + if (unlikely(!sg_policy->freq_response_time_ms)) + sg_policy->freq_response_time_ms =3D sugov_calc_freq_response_ms(sg_poli= cy); + + mult =3D sg_policy->freq_response_time_ms * SCHED_CAPACITY_SCALE; + mult /=3D sg_policy->tunables->response_time_ms; + + if (SCHED_WARN_ON(!mult)) + mult =3D SCHED_CAPACITY_SCALE; + + for_each_cpu(cpu, sg_policy->policy->cpus) + per_cpu(response_time_mult, cpu) =3D mult; +} + +/* + * Shrink or expand how long it takes to reach the maximum performance of = the + * policy. + * + * sg_policy->freq_response_time_ms is a constant value defined by PELT + * HALFLIFE and the capacity of the policy (assuming HMP systems). + * + * sg_policy->tunables->response_time_ms is a user defined response time. = By + * setting it lower than sg_policy->freq_response_time_ms, the system will + * respond faster to changes in util, which will result in reaching maximum + * performance point quicker. By setting it higher, it'll slow down the am= ount + * of time required to reach the maximum OPP. + * + * This should be applied when selecting the frequency. + */ +static inline unsigned long +sugov_apply_response_time(unsigned long util, int cpu) +{ + unsigned long mult; + + mult =3D per_cpu(response_time_mult, cpu) * util; + + return mult >> SCHED_CAPACITY_SHIFT; +} + static bool sugov_should_update_freq(struct sugov_policy *sg_policy, u64 t= ime) { s64 delta_ns; @@ -215,7 +283,10 @@ unsigned long sugov_effective_cpu_perf(int cpu, unsign= ed long actual, unsigned long min, unsigned long max) { - /* Add dvfs headroom to actual utilization */ + /* + * Speed up/slow down response timee first then apply DVFS headroom. + */ + actual =3D sugov_apply_response_time(actual, cpu); actual =3D sugov_apply_dvfs_headroom(actual, cpu); /* Actually we don't need to target the max performance */ if (actual < max) @@ -614,8 +685,42 @@ rate_limit_us_store(struct gov_attr_set *attr_set, con= st char *buf, size_t count =20 static struct governor_attr rate_limit_us =3D __ATTR_RW(rate_limit_us); =20 +static ssize_t response_time_ms_show(struct gov_attr_set *attr_set, char *= buf) +{ + struct sugov_tunables *tunables =3D to_sugov_tunables(attr_set); + + return sprintf(buf, "%u\n", tunables->response_time_ms); +} + +static ssize_t +response_time_ms_store(struct gov_attr_set *attr_set, const char *buf, siz= e_t count) +{ + struct sugov_tunables *tunables =3D to_sugov_tunables(attr_set); + struct sugov_policy *sg_policy; + unsigned int response_time_ms; + + if (kstrtouint(buf, 10, &response_time_ms)) + return -EINVAL; + + /* XXX need special handling for high values? */ + + tunables->response_time_ms =3D response_time_ms; + + list_for_each_entry(sg_policy, &attr_set->policy_list, tunables_hook) { + if (sg_policy->tunables =3D=3D tunables) { + sugov_update_response_time_mult(sg_policy); + break; + } + } + + return count; +} + +static struct governor_attr response_time_ms =3D __ATTR_RW(response_time_m= s); + static struct attribute *sugov_attrs[] =3D { &rate_limit_us.attr, + &response_time_ms.attr, NULL }; ATTRIBUTE_GROUPS(sugov); @@ -803,11 +908,13 @@ static int sugov_init(struct cpufreq_policy *policy) goto stop_kthread; } =20 - tunables->rate_limit_us =3D cpufreq_policy_transition_delay_us(policy); - policy->governor_data =3D sg_policy; sg_policy->tunables =3D tunables; =20 + tunables->rate_limit_us =3D cpufreq_policy_transition_delay_us(policy); + tunables->response_time_ms =3D sugov_calc_freq_response_ms(sg_policy); + sugov_update_response_time_mult(sg_policy); + ret =3D kobject_init_and_add(&tunables->attr_set.kobj, &sugov_tunables_kt= ype, get_governor_parent_kobj(policy), "%s", schedutil_gov.name); @@ -867,7 +974,7 @@ static int sugov_start(struct cpufreq_policy *policy) void (*uu)(struct update_util_data *data, u64 time, unsigned int flags); unsigned int cpu; =20 - sg_policy->freq_update_delay_ns =3D sg_policy->tunables->rate_limit_us * = NSEC_PER_USEC; + sg_policy->freq_update_delay_ns =3D sg_policy->tunables->rate_limit_us *= NSEC_PER_USEC; sg_policy->last_freq_update_time =3D 0; sg_policy->next_freq =3D 0; sg_policy->work_in_progress =3D false; --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1CE2A1ABEBD for ; Tue, 20 Aug 2024 16:36:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171807; cv=none; b=Q8WteqLzD/UbH+hsWgl4YLVnL7kY9o6+ra7s0AgXj1xL+wgQUfT3MGdhjKKw8lf6/RD9ld5fdZF8B7W9iJ046WBka5NivdgrO3sdxbOLyipuDujf5XrYesfCk15S1P8L1vZ14Fgxqg3QeHuFayp2kmAFC9l+r1O1jdKDaQi0EJk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171807; c=relaxed/simple; bh=ggQ0xHMfPwWR5W6MLhFpsoF3X1RHmOFsfCjj+44KrkY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NMe6SvjobvxPq8I31kf6u7pNyfx1g5PTZbZEs+p6fRi/UPZmSL4y4wNSdqKUlV3B0rByM0bxNByblDHD7Bq5I3U3fpwkp4EVa2P+n+UM+aZsTL7FGS/BhTgu/jORcNuK1dxi4aAOEY06JiraS4V6jxBKuoHykL2RL56hETn8tRo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=wsyA3i1I; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="wsyA3i1I" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-a7aa4ca9d72so741083366b.0 for ; Tue, 20 Aug 2024 09:36:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171804; x=1724776604; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1wet2+g0g75UUnmsKQrb/VZONF2cRQanUuyFsF+8Vjk=; b=wsyA3i1IbjWoPW9j3p0vNvUGwPtjDlNT45+LMvpLkR7Gh+g1Ft4yn9bcG6cHDY6Fx7 EskjTsaMfaEWcarR6Xqt1tnXRL+7pkPpjAlgUab7PvpOrpZbl2ph7izXNPl7oLISini8 V4ljBcawl8Sf7NAfs1prsOlVMV9YYOL0v+8HXCSOh1Jr1bnvLd6Ku1IjRgSIUBusiYEp r084fR18jdRdhPY4bsDBe2NmYEqcLYCTg9X4e7/HH2BtfV9GHjqzHR7A6eYyy1REsQRq 0wdm6G189GSe9YpKNQ8GF0GCTNPMj8Yt4Zf7NoW335B8v6ZY/TFYj7N1dCKjvlLGo8OY bzvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171804; x=1724776604; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1wet2+g0g75UUnmsKQrb/VZONF2cRQanUuyFsF+8Vjk=; b=pd6vIg31DGAm3ZnoULwDa3MCAzq+t5fIowEF3ITq85R3WGdYoQN6i+9c8IEQTkd3LJ /L2ifcU5P8Xp68r37c8aspUEwBGrxrmPokhk8pvyKAu/Bm4Kk6zkBWCfpgS/hU8NB55j F9xh9k4PCqbhOO/0XZ+uDogDd0D9iqWEmOXkA7QcXPnMpu5MvXUXj+69kfzkjdjEss4r l0fSuKE5Cxrwa23coDwPqMvf7gUQ3HdL+1HTlQpM7RCifbVUCXQkjb2h7eBy9NrWYzsP JF+/LSkPWF3UyaI2au5MJt5UEekbRtnRdntVUQREvxfXEk5jfEpcqV/5Qn+OtriD9GQV jynw== X-Forwarded-Encrypted: i=1; AJvYcCUNKs4oWNPdw5g+IPmmLtokwRZ1EE8922t8I39WQr28XNWBL+qpkLWFyRnGJYBvDv/Diq6JH50upNxDgPk5H/2jb+eRB/plW57lEaOe X-Gm-Message-State: AOJu0YwoTV5hPv2H9yAjvIlHJz9PVnco6TBXFrJM/qbtl+ZZzRzvJsII YGTCYLH3AaGgf/vvSmUEZ0WkAt5AqHXWQ9gu13MUlyeQrMiu6CLtWU8s/IakzlvrLiX/riqyUB4 4 X-Google-Smtp-Source: AGHT+IFEuSDJpZmH2vuXQPhm8UldWkmv4L2KE1jsWvuj7bhXQKIf0BaaBicVLq/zLot7W5U3e4x0BA== X-Received: by 2002:a17:906:d7c7:b0:a7a:bece:6222 with SMTP id a640c23a62f3a-a83928a9f25mr1027872166b.10.1724171804254; Tue, 20 Aug 2024 09:36:44 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.35.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:43 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 07/16] sched/pelt: Introduce PELT multiplier boot time parameter Date: Tue, 20 Aug 2024 17:35:03 +0100 Message-Id: <20240820163512.1096301-8-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The param is set as read only and can only be changed at boot time via kernel.sched_pelt_multiplier=3D[1, 2, 4] PELT has a big impact on the overall system response and reactiveness to change. Smaller PELT HF means it'll require less time to reach the maximum performance point of the system when the system become fully busy; and equally shorter time to go back to lowest performance point when the system goes back to idle. This faster reaction impacts both DVFS response and migration time between clusters in HMP system. Smaller PELT values (higher multiplier) are expected to give better performance at the cost of more power. Under-powered systems can particularly benefit from faster response time. Powerful systems can still benefit from response time if they want to be tuned towards perf more and power is not the major concern for them. This combined with response_time_ms from schedutil should give the user and sysadmin a deterministic way to control the triangular power, perf and thermals for their system. The default response_time_ms will half as PELT HF halves. Update approximate_{util_avg, runtime}() to take into account the PELT HALFLIFE multiplier. Signed-off-by: Vincent Guittot [qyousef: Commit message and boot param] Signed-off-by: Qais Yousef --- kernel/sched/pelt.c | 62 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 58 insertions(+), 4 deletions(-) diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c index 06cb881ba582..536575757420 100644 --- a/kernel/sched/pelt.c +++ b/kernel/sched/pelt.c @@ -24,6 +24,9 @@ * Author: Vincent Guittot */ =20 +static __read_mostly unsigned int sched_pelt_lshift; +static unsigned int sched_pelt_multiplier =3D 1; + /* * Approximate: * val * y^n, where y^32 ~=3D 0.5 (~1 scheduling period) @@ -180,6 +183,7 @@ static __always_inline int ___update_load_sum(u64 now, struct sched_avg *sa, unsigned long load, unsigned long runnable, int running) { + int time_shift; u64 delta; =20 delta =3D now - sa->last_update_time; @@ -195,12 +199,17 @@ ___update_load_sum(u64 now, struct sched_avg *sa, /* * Use 1024ns as the unit of measurement since it's a reasonable * approximation of 1us and fast to compute. + * On top of this, we can change the half-time period from the default + * 32ms to a shorter value. This is equivalent to left shifting the + * time. + * Merge both right and left shifts in one single right shift */ - delta >>=3D 10; + time_shift =3D 10 - sched_pelt_lshift; + delta >>=3D time_shift; if (!delta) return 0; =20 - sa->last_update_time +=3D delta << 10; + sa->last_update_time +=3D delta << time_shift; =20 /* * running is a subset of runnable (weight) so running can't be set if @@ -468,6 +477,51 @@ int update_irq_load_avg(struct rq *rq, u64 running) } #endif /* CONFIG_HAVE_SCHED_AVG_IRQ */ =20 +static int set_sched_pelt_multiplier(const char *val, const struct kernel_= param *kp) +{ + int ret; + + ret =3D param_set_int(val, kp); + if (ret) + goto error; + + switch (sched_pelt_multiplier) { + case 1: + fallthrough; + case 2: + fallthrough; + case 4: + WRITE_ONCE(sched_pelt_lshift, + sched_pelt_multiplier >> 1); + break; + default: + ret =3D -EINVAL; + goto error; + } + + return 0; + +error: + sched_pelt_multiplier =3D 1; + return ret; +} + +static const struct kernel_param_ops sched_pelt_multiplier_ops =3D { + .set =3D set_sched_pelt_multiplier, + .get =3D param_get_int, +}; + +#ifdef MODULE_PARAM_PREFIX +#undef MODULE_PARAM_PREFIX +#endif +/* XXX: should we use sched as prefix? */ +#define MODULE_PARAM_PREFIX "kernel." +module_param_cb(sched_pelt_multiplier, &sched_pelt_multiplier_ops, &sched_= pelt_multiplier, 0444); +MODULE_PARM_DESC(sched_pelt_multiplier, "PELT HALFLIFE helps control the r= esponsiveness of the system."); +MODULE_PARM_DESC(sched_pelt_multiplier, "Accepted value: 1 32ms PELT HALIF= E - roughly 200ms to go from 0 to max performance point (default)."); +MODULE_PARM_DESC(sched_pelt_multiplier, " 2 16ms PELT HALIF= E - roughly 100ms to go from 0 to max performance point."); +MODULE_PARM_DESC(sched_pelt_multiplier, " 4 8ms PELT HALIF= E - roughly 50ms to go from 0 to max performance point."); + /* * Approximate the new util_avg value assuming an entity has continued to = run * for @delta us. @@ -482,7 +536,7 @@ unsigned long approximate_util_avg(unsigned long util, = u64 delta) if (unlikely(!delta)) return util; =20 - accumulate_sum(delta, &sa, 1, 0, 1); + accumulate_sum(delta << sched_pelt_lshift, &sa, 1, 0, 1); ___update_load_avg(&sa, 0); =20 return sa.util_avg; @@ -494,7 +548,7 @@ unsigned long approximate_util_avg(unsigned long util, = u64 delta) u64 approximate_runtime(unsigned long util) { struct sched_avg sa =3D {}; - u64 delta =3D 1024; // period =3D 1024 =3D ~1ms + u64 delta =3D 1024 << sched_pelt_lshift; // period =3D 1024 =3D ~1ms u64 runtime =3D 0; =20 if (unlikely(!util)) --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0848319414E for ; Tue, 20 Aug 2024 16:36:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171809; cv=none; b=vD9NEEn1AFjHPPeMDmDF8k+3afB0+zUvv6bFfm/BikMmmZWxyDtq3ScIX6PTvv6gKxznlHRAvjid0o39z+1B3Nn8s5gDpK528OvOVXkLn7KehOoynjJNrgfni3nnizyAd1ICopM+tQf+UNz5HgbTQsOW2GlxlUSCxjbKFu8lrZo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171809; c=relaxed/simple; bh=NWGl6xmypDOUzvDbdd6/ZYozVxHaZS6LctziiRw3r/4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=ORxUBjG8Jo5scv0hRw6rgQ0qEe+8eg58MAJK4E0V6/DiqEXk0KWZSZkf5baRHpIsPQbZbNDJKlZiqJ1Utl2fTF6cHLaAIQ0x9vKyDSh7S64VgQmdR8wF/DgTlrQR+OfVPR38f/AydZHvebUtPf7CKpismXhkPZg2R7Gbviyq+N8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=RXtjyE/P; arc=none smtp.client-ip=209.85.208.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="RXtjyE/P" Received: by mail-ed1-f42.google.com with SMTP id 4fb4d7f45d1cf-5bb85e90ad5so4253852a12.3 for ; Tue, 20 Aug 2024 09:36:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171805; x=1724776605; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MlOaXaSxe4t1PPDAFFFwKxv22YBKt/FKkTrJcGZpyCM=; b=RXtjyE/P3BG/2Se4+wuqZn3Cf+Eru8kvIkt31wZ5fw+B8afAMb7hG0T6xvTJUx7Wbr zoUrFjiJiuXtxr9Z+MqmRYNKrAE33RJh8GhoPs+yEgDPvAU4HjxwxnBEsrBhvm1Op+iB MsqB62OBwp60MSPF6566vcj9AsfrPUInlpHhmjdyA3tAV9AUUITrl1T3f/ZD4SiGfbu/ 6uWsY2WyLKUEJeUMedut17OPFs4qjrtj+z/bBKEb4YGQrewhdnn2JJwzhLS+6xFmNgbn m2fv1pOjeajBEdvplOIEsRIkAhMWh66E/XSeviooXq6y/lUHoDntlNRhxiwvaPQgOuOb Ek6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171805; x=1724776605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MlOaXaSxe4t1PPDAFFFwKxv22YBKt/FKkTrJcGZpyCM=; b=meeGYsfuiULBihdumE7zDxUolqxZwiJ5Seo4Ab7cNpcDBXl8BYDXFNc9cixxXA+0c+ e7ZV3fDSO8ZM8zG7V2raVVpEqYmZOQgXBeMa+QDM2fO6bmW/AOLGZ2QUD67flNa2cO5U x1sf7wn18cxvCH13i2cZuy2swn9V6KKXKUH8uQJYOBodpj8Z2y+f2sImZfrjyiuYtL2M pInOkY5JP2Ke8evPNLFT3AROZh6IzJv3WJWusc2Mrh6q33O3JgcdzuasqodAR+DlA4fN Agl8WXJf1CrlCL3wH+wl9hSqrNfPdaPVXMmxBX0F0+fykKrH/3FafrxT82bvm6HcWL3W DRpA== X-Forwarded-Encrypted: i=1; AJvYcCW2q/n0Hxgmcj4SCXyszdBJcpyVZsJprg24odbhw2juzy+wnrjq3t2DNslXITec2L07mErD783vbvWsLrE=@vger.kernel.org X-Gm-Message-State: AOJu0Yw2on7LFokf7vPFxX46i0x8EZxqypGz0+uemkauBQAwjs1olH7G Hvjxm4xY2Ad/l0MZVwsktKi2uf9hTKvkkSVJS/KAURMY7RY1ZHAxrdCI/0ZsQPk= X-Google-Smtp-Source: AGHT+IGd0lN/68+It+jeiTE+VFnqKtHnby+yBIZv2MbicBv5kJa2yRauuDHdIXSJbNXUDYwoNgDzbA== X-Received: by 2002:a17:907:c7d7:b0:a7a:be06:d8e3 with SMTP id a640c23a62f3a-a8392a22287mr1003338666b.46.1724171805332; Tue, 20 Aug 2024 09:36:45 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.36.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:44 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 08/16] sched/fair: Extend util_est to improve rampup time Date: Tue, 20 Aug 2024 17:35:04 +0100 Message-Id: <20240820163512.1096301-9-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Utilization invariance can cause big delays. When tasks are running, accumulate non-invairiant version of utilization to help tasks to settle down to their new util_avg values faster. Keep track of delta_exec during runnable across activations to help update util_est for a long running task accurately. util_est shoudl still behave the same at enqueue/dequeue. Before this patch the a busy task tamping up would experience the following transitions, running on M1 Mac Mini rampup-6338 util_avg running =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=90 986.0=E2=94=A4 = =E2=96=84=E2=96=84=E2=96=84=E2=96=84=E2=96=84=E2=96=9F=E2=96=80=E2=96=80= =E2=96=80=E2=96=80=E2=94=82 =E2=94=82 =E2= =96=97=E2=96=84=E2=96=84=E2=96=9F=E2=96=80=E2=96=80=E2=96=80=E2=96=98 = =E2=94=82 =E2=94=82 =E2=96=97= =E2=96=84=E2=96=9F=E2=96=80=E2=96=80 =E2=94=82 =E2=94=82 =E2=96=84=E2= =96=9F=E2=96=80=E2=96=80 =E2=94=82 739.5=E2=94=A4 =E2=96=84=E2=96= =9F=E2=96=80=E2=96=98 =E2=94=82 =E2=94=82 =E2=96=97=E2=96=84= =E2=96=9B=E2=96=98 =E2=94=82 =E2=94=82 =E2=96=97=E2=96=9F= =E2=96=80 =E2=94=82 493.0=E2=94=A4 =E2=96=97=E2=96=9B=E2= =96=80 =E2=94=82 =E2=94=82 =E2=96=97=E2=96=84=E2=96= =9B=E2=96=80 =E2=94=82 =E2=94=82 =E2=96=84=E2=96=9F=E2=96=80= =E2=94=82 =E2=94=82 =E2=96=84=E2=96=9B=E2=96=98 = =E2=94=82 246.5=E2=94=A4 =E2=96=97=E2=96=9F=E2=96=80=E2= =96=98 =E2=94=82 =E2=94=82 =E2=96=84=E2=96=9F=E2=96=80=E2=96= =80 =E2=94=82 =E2=94=82 =E2=96=97=E2=96=84=E2=96=84=E2=96=9B=E2= =96=98 =E2=94=82 =E2=94=82 =E2=96=97=E2=96=84=E2=96=84=E2=96=84=E2=96= =9F=E2=96=80 =E2=94=82 0.0=E2=94=A4 =E2=96=97 =E2=96=97=E2=96=84=E2=96=84=E2=96=9F=E2= =96=80=E2=96=80 =E2= =94=82 =E2=94=94=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=AC= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=AC=E2= =94=98 1.700 1.733 1.767 1.800 1.833 1.867 1.900 1.933 1.967 2= .000 =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80 rampup-6338 util_avg running residency (ms) =E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 0.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 5.5 15.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87 7.9 36.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87 8.0 57.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87 8.0 78.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87 7.9 98.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87 5.0 117.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87 5.0 137.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87 5.0 156.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87 4.0 176.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87 3.0 191.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87 4.0 211.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87 4.0 230.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87 3.0 248.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87 3.0 266.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87 2.0 277.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87 3.0 294.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.6 311.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.4 327.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87 2.0 340.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87 3.0 358.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87 2.0 371.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87 1.0 377.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87 2.0 389.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87 2.0 401.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87 2.0 413.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87 3.0 431.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87 2.0 442.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87 2.0 456.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87 1.0 =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= Sum Time Running on CPU (ms) =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80 CPU0.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 90.39 CPU4.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 1156.= 93 6338 rampup CPU0.0 Frequency =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=90 2.06=E2=94=A4 =E2=96=9B=E2=96=80=E2=96=80 = =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 1.70=E2=94=A4 =E2=96=9B=E2=96=80=E2=96=80=E2=96= =98 =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 1.33=E2=94=A4 =E2=96=97=E2=96=84=E2=96=84=E2=96=84= =E2=96=8C =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 0.97=E2=94=A4 =E2=96=97=E2=96=84=E2=96=84=E2=96=84=E2= =96=9F =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 0.60=E2=94=A4 =E2=96=97 =E2=96=97=E2=96=84=E2=96=84=E2=96=84=E2=96= =84=E2=96=84=E2=96=84=E2=96=84=E2=96=84=E2=96=9F = =E2=94=82 =E2=94=94=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=AC=E2=94=98 1.700 1.733 1.767 1.800 1.833 1.867 1.900 1.933 1.967 2= .000 6338 rampup CPU4.0 Frequency =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=90 3.20=E2=94=A4 =E2=96=90= =E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2= =96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96= =80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=9B=E2= =96=80=E2=96=80 =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 2.78=E2=94=A4 =E2=96=90=E2=96= =80=E2=96=80=E2=96=98 =E2=94=82 =E2=94=82 =E2=96=97=E2=96= =84=E2=96=9F =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 2.35=E2=94=A4 =E2=96=97=E2=96=84= =E2=96=84=E2=96=8C =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=84=E2=96=84=E2= =96=9F =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 1.93=E2=94=A4 =E2=96=97=E2=96=84=E2=96= =84=E2=96=8C =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 1.50=E2=94=A4 =E2=96=97=E2=96=84=E2=96=84= =E2=96=9F =E2=94=82 =E2=94=94=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=AC=E2=94=98 1.700 1.733 1.767 1.800 1.833 1.867 1.900 1.933 1.967 2= .000 =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80 6338 rampup CPU0.0 Frequency residency (ms) =E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 0.6 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 37.300000000000004 0.972 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 15.0 1.332 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 15.0 1.704 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= 11.0 2.064 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87 12.1 =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80 6338 rampup CPU4.0 Frequency residency (ms) =E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 1.5 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87 11.9 1.956 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87 10.0 2.184 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87 10.0 2.388 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87 11.0 2.592 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87 10.0 2.772 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87 10.0 2.988 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87 10.0 3.204 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87 85.3 After the patch the response is improved to rampup frequencies faster and migrate from little quicker rampup-2234 util_avg running =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=90 984=E2=94=A4 = =E2=96=97=E2=96=84=E2=96=84=E2=96=84=E2=96=84=E2=96=84=E2=96=9B=E2=96=80= =E2=96=80=E2=96=80=E2=96=80=E2=94=82 =E2=94=82 =E2= =96=84=E2=96=84=E2=96=9F=E2=96=80=E2=96=80=E2=96=80=E2=96=80 =E2= =94=82 =E2=94=82 =E2=96=84= =E2=96=84=E2=96=9F=E2=96=80=E2=96=80 =E2=94=82 =E2=94=82 =E2=96=84=E2= =96=9F=E2=96=80=E2=96=98 =E2=94=82 738=E2=94=A4 =E2=96=84=E2=96= =9F=E2=96=80=E2=96=98 =E2=94=82 =E2=94=82 =E2=96=97=E2=96=9F= =E2=96=80=E2=96=98 =E2=94=82 =E2=94=82 =E2=96=97=E2=96=9F=E2= =96=80 =E2=94=82 492=E2=94=A4 =E2=96=97=E2=96=9F=E2= =96=80 =E2=94=82 =E2=94=82 =E2=96=97=E2=96=9F=E2=96= =80 =E2=94=82 =E2=94=82 =E2=96=9F=E2=96=80 = =E2=94=82 =E2=94=82 =E2=96=84=E2=96=9B=E2=96=98 = =E2=94=82 246=E2=94=A4 =E2=96=97=E2=96=9F=E2=96=98 = =E2=94=82 =E2=94=82 =E2=96=97=E2=96=9F=E2=96=80 = =E2=94=82 =E2=94=82 =E2=96=97=E2=96=9F=E2=96=80 = =E2=94=82 =E2=94=82 =E2=96=97=E2=96=9F=E2=96=80 = =E2=94=82 0=E2=94=A4 =E2=96=84=E2=96=84=E2=96=84=E2=96=9B=E2= =96=80 =E2=94=82 =E2=94=94=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=98 1.700 1.733 1.767 1.800 1.833 1.867 1.900 1.933 1.967 2= .000 =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80 rampup-2234 util_avg running residency (ms) =E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 0.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 5.6000000000000005 15.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 8.0 39.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87 5.0 61.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87 4.0 85.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 99.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 3.0 120.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 3.0 144.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 160.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 176.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 192.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 210.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 228.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 246.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 263.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 282.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 1.0 291.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 309.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 327.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 344.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 1.0 354.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 373.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 1.0 382.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 400.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 1.0 408.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 425.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 1.0 434.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 2.0 452.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 1.0 2234 rampup CPU1.0 Frequency =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=90 2.06=E2=94=A4 =E2=96=90=E2=96=80 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 1.70=E2=94=A4 =E2=96=9B=E2=96=80 = =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 1.33=E2=94=A4 =E2=96=84=E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 0.97=E2=94=A4 =E2=96=97=E2=96=84=E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 0.60=E2=94=A4 =E2=96=97=E2=96=84=E2=96=84=E2=96=9F = =E2=94=82 =E2=94=94=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=AC=E2=94=98 1.700 1.733 1.767 1.800 1.833 1.867 1.900 1.933 1.967 2= .000 2234 rampup CPU4.0 Frequency =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=90 3.10=E2=94=A4 = =E2=96=90=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2= =96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=94=82 =E2=94=82 =E2=96=9B=E2= =96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96=80=E2=96= =80=E2=96=80=E2=96=80 =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=90=E2=96=80= =E2=96=80=E2=96=80=E2=96=80=E2=96=98 =E2=94=82 2.70=E2=94=A4 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=90=E2=96=80=E2= =96=80=E2=96=80=E2=96=80 =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 2.30=E2=94=A4 =E2=96=9B=E2=96=80=E2=96= =80 =E2=94=82 =E2=94=82 =E2=96=8C = =E2=94=82 =E2=94=82 =E2=96=90=E2=96=80=E2=96=80= =E2=96=98 =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 1.90=E2=94=A4 =E2=96=90=E2=96=80=E2=96=80 = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 =E2=94=82 =E2=96=97=E2=96=84=E2=96=9F = =E2=94=82 =E2=94=82 =E2=96=90 = =E2=94=82 1.50=E2=94=A4 =E2=96=97=E2=96=9F = =E2=94=82 =E2=94=94=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=AC=E2=94=98 1.700 1.733 1.767 1.800 1.833 1.867 1.900 1.933 1.967 2= .000 =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= Sum Time Running on CPU (ms) =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80 CPU1.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87 32.53 CPU4.0 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87 540.3 =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80 2234 rampup CPU1.0 Frequency residency (ms) =E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 0.6 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87 12.1 0.972 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 6.5 1.332 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 3.7 1.704 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 5.5 2.064 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87 4.8 =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80 2234 rampup CPU4.0 Frequency residency (ms) =E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 1.5 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 4.0 1.728 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87 8.0 1.956 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 9.0 2.184 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 9.0 2.388 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 11.0 2.592 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 16.0 2.772 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87 18.0 2.988 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87 47.0 3.096 =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2= =96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96= =87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87=E2=96=87= =E2=96=87=E2=96=87=E2=96=87 53.4 Signed-off-by: Qais Yousef --- include/linux/sched.h | 1 + kernel/sched/core.c | 1 + kernel/sched/fair.c | 43 +++++++++++++++++++++++++++++++------------ 3 files changed, 33 insertions(+), 12 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 90691d99027e..8db8f4085d84 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -544,6 +544,7 @@ struct sched_entity { unsigned int on_rq; =20 u64 exec_start; + u64 delta_exec; u64 sum_exec_runtime; u64 prev_sum_exec_runtime; u64 vruntime; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 7099e40cc8bd..e2b4b87ec2b7 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4318,6 +4318,7 @@ static void __sched_fork(unsigned long clone_flags, s= truct task_struct *p) =20 p->se.on_rq =3D 0; p->se.exec_start =3D 0; + p->se.delta_exec =3D 0; p->se.sum_exec_runtime =3D 0; p->se.prev_sum_exec_runtime =3D 0; p->se.nr_migrations =3D 0; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e5e986af18dc..a6421e4032c0 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1118,6 +1118,7 @@ static s64 update_curr_se(struct rq *rq, struct sched= _entity *curr) =20 curr->exec_start =3D now; curr->sum_exec_runtime +=3D delta_exec; + curr->delta_exec =3D delta_exec; =20 if (schedstat_enabled()) { struct sched_statistics *stats; @@ -1126,7 +1127,6 @@ static s64 update_curr_se(struct rq *rq, struct sched= _entity *curr) __schedstat_set(stats->exec_max, max(delta_exec, stats->exec_max)); } - return delta_exec; } =20 @@ -4890,16 +4890,20 @@ static inline void util_est_update(struct cfs_rq *c= fs_rq, if (!sched_feat(UTIL_EST)) return; =20 - /* - * Skip update of task's estimated utilization when the task has not - * yet completed an activation, e.g. being migrated. - */ - if (!task_sleep) - return; - /* Get current estimate of utilization */ ewma =3D READ_ONCE(p->se.avg.util_est); =20 + /* + * If a task is running, update util_est ignoring utilization + * invariance so that if the task suddenly becomes busy we will rampup + * quickly to settle down to our new util_avg. + */ + if (!task_sleep) { + ewma &=3D ~UTIL_AVG_UNCHANGED; + ewma =3D approximate_util_avg(ewma, p->se.delta_exec / 1000); + goto done; + } + /* * If the PELT values haven't changed since enqueue time, * skip the util_est update. @@ -4968,6 +4972,14 @@ static inline void util_est_update(struct cfs_rq *cf= s_rq, trace_sched_util_est_se_tp(&p->se); } =20 +static inline void util_est_update_running(struct cfs_rq *cfs_rq, + struct task_struct *p) +{ + util_est_dequeue(cfs_rq, p); + util_est_update(cfs_rq, p, false); + util_est_enqueue(cfs_rq, p); +} + static inline unsigned long get_actual_cpu_capacity(int cpu) { unsigned long capacity =3D arch_scale_cpu_capacity(cpu); @@ -5164,13 +5176,13 @@ static inline int sched_balance_newidle(struct rq *= rq, struct rq_flags *rf) =20 static inline void util_est_enqueue(struct cfs_rq *cfs_rq, struct task_struct *p) {} - static inline void util_est_dequeue(struct cfs_rq *cfs_rq, struct task_struct *p) {} - static inline void -util_est_update(struct cfs_rq *cfs_rq, struct task_struct *p, - bool task_sleep) {} +util_est_update(struct cfs_rq *cfs_rq, struct task_struct *p, bool task_sl= eep) {} +static inline void +util_est_update_running(struct cfs_rq *cfs_rq, struct task_struct *p) {} + static inline void update_misfit_status(struct task_struct *p, struct rq *= rq) {} =20 #endif /* CONFIG_SMP */ @@ -6906,6 +6918,8 @@ static void dequeue_task_fair(struct rq *rq, struct t= ask_struct *p, int flags) rq->next_balance =3D jiffies; =20 dequeue_throttle: + if (task_sleep) + p->se.delta_exec =3D 0; util_est_update(&rq->cfs, p, task_sleep); hrtick_update(rq); } @@ -8546,6 +8560,9 @@ pick_next_task_fair(struct rq *rq, struct task_struct= *prev, struct rq_flags *rf set_next_entity(cfs_rq, se); } =20 + if (prev->on_rq) + util_est_update_running(&rq->cfs, prev); + goto done; simple: #endif @@ -12710,6 +12727,8 @@ static void task_tick_fair(struct rq *rq, struct ta= sk_struct *curr, int queued) entity_tick(cfs_rq, se, queued); } =20 + util_est_update_running(&rq->cfs, curr); + if (static_branch_unlikely(&sched_numa_balancing)) task_tick_numa(rq, curr); =20 --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F0361B8EA8 for ; Tue, 20 Aug 2024 16:36:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171810; cv=none; b=HJXEx2K/rqRwzUGyR1/oJ4M6NLSTbg+qa3pIPvtBnZNcGiGF9kWyGLDayYPJoehJCO5R5q23d+DYW4kNXV6bgBudye92iL/NCt65JqLQPr1YEfQdgUHMuJU/DEhlIzChP3enOmhHHrFmTir2JtdW1DMvFT3oWQEVws6tqWin+uM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171810; c=relaxed/simple; bh=GQPbKPJztlK6T882OcegVi96fmYLqoqnWFn1+Q/Iecs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IDm+ubxQqvDCyuGYs0b/SMGWYE4bELyW1el7bbfbLhHe+Iw8Khd/ScQle667VdmBghD+dtEVB36/TNT84FMjb4EvHui8UHBXhVPc+gRTPsUiUBLBhg1JcaCIRzw/Vac7q6CTHd4ENVtXgiMo/qARxmtakWMRG4NIXgiIAsJgZYs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=H0X/u4lk; arc=none smtp.client-ip=209.85.208.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="H0X/u4lk" Received: by mail-ed1-f47.google.com with SMTP id 4fb4d7f45d1cf-5bed83489c3so4345967a12.3 for ; Tue, 20 Aug 2024 09:36:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171806; x=1724776606; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Gs4Ks0Kkm98Csg47MQJPALx1GhqbRV6StE7nBUKSXR0=; b=H0X/u4lkYVxFQFnzat5w3riIAUl6WVHxxPByZomWAcSHNL+Ptim7h5zZhLrJV3CCx9 amArqi5HN+roUdYPOTMBeL5bhMY7AIEJvdql9T4gYy/vL3HajjoFV1hh4wxInVuZQdOH VtM7oPIiEfB2R+DYHdaHwup+R0WFfFrtXODtyV1MbX/cbEARy5X2gXMLIP0Tzi1+zAzz BzkbYZFei4WSydr7Od7zo370kfNVvrIY+AiYwBv5Z6ZaLPVFFqtpt2P7PBOvOGM7G2oH CQ9e8wkYgph2Swc0WrTN7m0TLt+PP6waRY9AQQQcT2x+F26lpeB5aLQG7dezsNHHI36L KhPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171806; x=1724776606; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gs4Ks0Kkm98Csg47MQJPALx1GhqbRV6StE7nBUKSXR0=; b=EG93c5xBmCzRspzXOun6XRdFumxy0X3DopLWWaC2ndixnjLtsBc63F3LMtb+/4pRBX i6msTA+KWLl8qGPA1wrWq/ITrbdicdzivHMLUhuGWRyHE5CuQyVbTK/715BGJk9UJX5H JIVIa6mNd+PeTUzCFGhch5bNGGOoXziEJS2gkDfYuPIXYf/YLSEd81ofzUXwl3WdV4Yj iN/XssRJuVwPPyYmz1KLIqoNbSYbHAiNlX8zqbAtB3YcYKzME0283ffba3detDTLI5pY Ij07GQW4hTRqxRS4rMMxiRmhWkbafh4a0G/Vh85zDpKxtG88soy+et2sy+m/RzPn9yWM KYRg== X-Forwarded-Encrypted: i=1; AJvYcCWtIFV0M9ZEwRlB1PSlqSAUWOEuHLev4JIBbu4fEVbHil6fYuBhhq8aRSCIZZ4hmXtlggCcvFKUwkzoE66g0OYRzUP7kyKV9bQJMZfV X-Gm-Message-State: AOJu0YynRt+9qfW7t1uBgaOVtHOt3cjS6zAGvVvN7pdPkBX6gxfi8flC TblZ2pmEF3A2UqTeEBR8dnxJYcY3c2UdElG/eQyju5O9uogczuMayHIfUWarwMo= X-Google-Smtp-Source: AGHT+IGwNTmVgxwt0g3+fIC/ibzEl8NcSRLlNxBxLL/CCcYzyggVVh9aXnQfRJ+/Pw7OoMUPFxuxmw== X-Received: by 2002:a17:907:6d20:b0:a7d:2f42:db54 with SMTP id a640c23a62f3a-a8392a49851mr940380666b.65.1724171806188; Tue, 20 Aug 2024 09:36:46 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.36.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:45 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 09/16] sched/fair: util_est: Take into account periodic tasks Date: Tue, 20 Aug 2024 17:35:05 +0100 Message-Id: <20240820163512.1096301-10-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The new faster rampup is great for performance. But terrible for power. We want the faster rampup to be only applied for tasks that are transitioning from one periodic/steady state to another periodic/steady state. But if they are stably periodic, then the faster rampup doesn't make sense as util_avg describes their computational demand accurately and we can rely on that to make accurate decision. And preserve the power savings from being exact with the resources we give to this task (ie: smaller DVFS headroom). We detect periodic tasks based on util_avg across util_est_update() calls. If it is rising, then the task is going through a transition. We rely on util_avg being stable for periodic tasks with very little variations around one stable point. Signed-off-by: Qais Yousef --- include/linux/sched.h | 2 ++ kernel/sched/core.c | 2 ++ kernel/sched/fair.c | 17 ++++++++++++++--- 3 files changed, 18 insertions(+), 3 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 8db8f4085d84..2e8c5a9ffa76 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -829,6 +829,8 @@ struct task_struct { struct uclamp_se uclamp[UCLAMP_CNT]; #endif =20 + unsigned long util_avg_dequeued; + struct sched_statistics stats; =20 #ifdef CONFIG_PREEMPT_NOTIFIERS diff --git a/kernel/sched/core.c b/kernel/sched/core.c index e2b4b87ec2b7..c91e6a62c7ab 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4331,6 +4331,8 @@ static void __sched_fork(unsigned long clone_flags, s= truct task_struct *p) p->se.cfs_rq =3D NULL; #endif =20 + p->util_avg_dequeued =3D 0; + #ifdef CONFIG_SCHEDSTATS /* Even if schedstat is disabled, there should not be garbage */ memset(&p->stats, 0, sizeof(p->stats)); diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a6421e4032c0..0c10e2afb52d 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4832,6 +4832,11 @@ static inline unsigned long task_util(struct task_st= ruct *p) return READ_ONCE(p->se.avg.util_avg); } =20 +static inline unsigned long task_util_dequeued(struct task_struct *p) +{ + return READ_ONCE(p->util_avg_dequeued); +} + static inline unsigned long task_runnable(struct task_struct *p) { return READ_ONCE(p->se.avg.runnable_avg); @@ -4899,9 +4904,12 @@ static inline void util_est_update(struct cfs_rq *cf= s_rq, * quickly to settle down to our new util_avg. */ if (!task_sleep) { - ewma &=3D ~UTIL_AVG_UNCHANGED; - ewma =3D approximate_util_avg(ewma, p->se.delta_exec / 1000); - goto done; + if (task_util(p) > task_util_dequeued(p)) { + ewma &=3D ~UTIL_AVG_UNCHANGED; + ewma =3D approximate_util_avg(ewma, p->se.delta_exec / 1000); + goto done; + } + return; } =20 /* @@ -4914,6 +4922,9 @@ static inline void util_est_update(struct cfs_rq *cfs= _rq, /* Get utilization at dequeue */ dequeued =3D task_util(p); =20 + if (!task_on_rq_migrating(p)) + p->util_avg_dequeued =3D dequeued; + /* * Reset EWMA on utilization increases, the moving average is used only * to smooth utilization decreases. --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A82181AE04F for ; Tue, 20 Aug 2024 16:36:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171810; cv=none; b=ER+HM26FSViMT8RNAxThWyDw5VxbaZVJ8ATp7X+c2HLkMqtgTyfvrzU3pIqYY2liSh0JwohnFYLGIR6wu3Nml5ygXuhBZvC3p4xmRhiuDQRy65ZkNLNaJIKe07SKm4Sfzx4RI64AEqRBkt2nP/80BRsh7yWhG36aPhKfRPXyqdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171810; c=relaxed/simple; bh=H2v4wPkY9/S00PuSqnRFW7BMeliOkRRZuCIYsWw2Xl0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DlWibJTkKYxwoBgHHROzTMBEwGH2V1IcUA/Zl8QTfq1EbyHF3cKP+6cxxiYoOEeR2nMA5mmxq/GQ30t14ZXYWJuofaOLMxXWdfhYObMqowlU4pAN9lbljjsw9nuGIuh+haHWAR37hELcwGqhq1XZ0mSdDjZ0RFSsSbGTxl5f5AM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=SEdLHilB; arc=none smtp.client-ip=209.85.218.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="SEdLHilB" Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-a8643a6bd55so153743166b.3 for ; Tue, 20 Aug 2024 09:36:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171807; x=1724776607; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JT5j0PZ9X9xu3k6v8CfXi6RGxKCSRQvQX4tJvcwiDgI=; b=SEdLHilBdGd+HV3kLIAFgh5Tco/aov6L03RWLcXoDAxxwECbqowStf7e7Nn4W00uNA 8ou3G4Dx/Qe935abWpyN261HVj/GzOCQ3TdT5w/Ei88D7RX7h/hPDGY513givXkFJOiL FlhSh8Oqovtucp01VF8KkxGGsubtzyP4MExPV5wMARKfzjNOIg/rNu6Uq2XcVvXM/1Gn NotMSmAekdwHEjUvlRlecHIuTf82M0IC/dNIXbEHRGht3TJroZomTcVEcZ7gFL/bP5aF B309BAeGdVe2jdane30E7qGJcyQLmu1IZuO0fGUiO0esf7UfyDmBCjvU04zCUWbsviB1 NlBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171807; x=1724776607; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JT5j0PZ9X9xu3k6v8CfXi6RGxKCSRQvQX4tJvcwiDgI=; b=pMAnukXQ053vuhxoeriF+vgqGC6g+osCAmkvfv/4mdFz+HO3yUQfny3CrnKkCp47ek p8IH+D/B/YUC0cDCgKfvccSYiAnXNe7Bgtw+RQAJA9MzE5qunOO2HfDVVmGsaw6JyuO3 QSEOdEJY1GuOmh86TMrnbhsC2okAZ/+/7npvEnZeEAGrG+C8jqfTFum19+MyLKZhlH9g 49cRsa3G2fL5O+A4Ibo1PkjnaL73RSm7ydGKZvvTTYyAV0+vCyFYD2wvMvvrSKqoSav6 WdXfU9r8xDL6ln4AZNwgAffb8lvq4OEQmDKoM3ub5jPKLWaVNwy9N/t/po+bhA83Df3R BMJA== X-Forwarded-Encrypted: i=1; AJvYcCX42N/lBaddGIxsXsANAL3HqdTzVV/8ksqiEBL37GOOgbrBVMxpbpaQt3rJPIqGj2+5zcFM0GMhESG9Fbo=@vger.kernel.org X-Gm-Message-State: AOJu0YzcRUA9h6ofecUIyMmns10osb5Mb0ZLrv2fH9s+bJ0exOFgh49M vP05EEOzpuoB8x+hUO4/ad5/HE9iuByVyCIaH3p7Lm/Ua3CP3uNNyKbHfhmQ4SU= X-Google-Smtp-Source: AGHT+IFCqy8udzUPQSToeARc04ldQIgYbEcEvnYLXxCQCr6hah66sD1+LHqReW2Q/Kq1330Y8OrsKQ== X-Received: by 2002:a17:907:efc8:b0:a7a:c197:8701 with SMTP id a640c23a62f3a-a83929534e5mr1074429266b.31.1724171807015; Tue, 20 Aug 2024 09:36:47 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.36.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:46 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 10/16] sched/qos: Add a new sched-qos interface Date: Tue, 20 Aug 2024 17:35:06 +0100 Message-Id: <20240820163512.1096301-11-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The need to describe the conflicting demand of various workloads hasn't been higher. Both hardware and software have moved rapidly in the past decade and system usage is more diverse and the number of workloads expected to run on the same machine whether on Mobile or Server markets has created a big dilemma on how to better manage those requirements. The problem is that we lack mechanisms to allow these workloads to describe what they need, and then allow kernel to do best efforts to manage those demands based on the hardware it is running on transparently and current system state. Example of conflicting requirements that come across frequently: 1. Improve wake up latency for SCHED_OTHER. Many tasks end up using SCHED_FIFO/SCHED_RR to compensate for this shortcoming. RT tasks lack power management and fairness and can be hard and error prone to use correctly and portably. 2. Prefer spreading vs prefer packing on wake up for a group of tasks. Geekbench-like workloads would benefit from parallelising on different CPUs. hackbench type of workloads can benefit from waking on up same CPUs or a CPU that is closer in the cache hierarchy. 3. Nice values for SCHED_OTHER are system wide and require privileges. Many workloads would like a way to set relative nice value so they can preempt each others, but not be impact or be impacted by other tasks belong to different workloads on the system. 4. Provide a way to tag some tasks as 'background' to keep them out of the way. SCHED_IDLE is too strong for some of these tasks but yet they can be computationally heavy. Example tasks are garbage collectors. Their work is both important and not important. 5. Provide a way to improve DVFS/upmigration rampup time for specific tasks that are bursty in nature and highly interactive. Whether any of these use cases warrants an additional QoS hint is something to be discussed individually. But the main point is to introduce an interface that can be extendable to cater for potentially those requirements and more. rampup_multiplier to improve DVFS/upmigration for bursty tasks will be the first user in later patch. It is desired to have apps (and benchmarks!) directly use this interface for optimal perf/watt. But in the absence of such support, it should be possible to write a userspace daemon to monitor workloads and apply these QoS hints on apps behalf based on analysis done by anyone interested in improving the performance of those workloads. Signed-off-by: Qais Yousef --- Documentation/scheduler/index.rst | 1 + Documentation/scheduler/sched-qos.rst | 44 ++++++++++++++++++ include/uapi/linux/sched.h | 4 ++ include/uapi/linux/sched/types.h | 46 +++++++++++++++++++ kernel/sched/syscalls.c | 3 ++ .../trace/beauty/include/uapi/linux/sched.h | 4 ++ 6 files changed, 102 insertions(+) create mode 100644 Documentation/scheduler/sched-qos.rst diff --git a/Documentation/scheduler/index.rst b/Documentation/scheduler/in= dex.rst index 43bd8a145b7a..f49b8b021d97 100644 --- a/Documentation/scheduler/index.rst +++ b/Documentation/scheduler/index.rst @@ -21,6 +21,7 @@ Scheduler sched-rt-group sched-stats sched-debug + sched-qos =20 text_files =20 diff --git a/Documentation/scheduler/sched-qos.rst b/Documentation/schedule= r/sched-qos.rst new file mode 100644 index 000000000000..0911261cb124 --- /dev/null +++ b/Documentation/scheduler/sched-qos.rst @@ -0,0 +1,44 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Scheduler QoS +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +1. Introduction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Different workloads have different scheduling requirements to operate +optimally. The same applies to tasks within the same workload. + +To enable smarter usage of system resources and to cater for the conflicti= ng +demands of various tasks, Scheduler QoS provides a mechanism to provide mo= re +information about those demands so that scheduler can do best-effort to +honour them. + + @sched_qos_type what QoS hint to apply + @sched_qos_value value of the QoS hint + @sched_qos_cookie magic cookie to tag a group of tasks for which the QoS + applies. If 0, the hint will apply globally system + wide. If not 0, the hint will be relative to tasks that + has the same cookie value only. + +QoS hints are set once and not inherited by children by design. The +rationale is that each task has its individual characteristics and it is +encouraged to describe each of these separately. Also since system resourc= es +are finite, there's a limit to what can be done to honour these requests +before reaching a tipping point where there are too many requests for +a particular QoS that is impossible to service for all of them at once and +some will start to lose out. For example if 10 tasks require better wake +up latencies on a 4 CPUs SMP system, then if they all wake up at once, only +4 can perceive the hint honoured and the rest will have to wait. Inheritan= ce +can lead these 10 to become a 100 or a 1000 more easily, and then the QoS +hint will lose its meaning and effectiveness rapidly. The chances of 10 +tasks waking up at the same time is lower than a 100 and lower than a 1000. + +To set multiple QoS hints, a syscall is required for each. This is a +trade-off to reduce the churn on extending the interface as the hope for +this to evolve as workloads and hardware get more sophisticated and the +need for extension will arise; and when this happen the task should be +simpler to add the kernel extension and allow userspace to use readily by +setting the newly added flag without having to update the whole of +sched_attr. diff --git a/include/uapi/linux/sched.h b/include/uapi/linux/sched.h index 3bac0a8ceab2..67ef99f64ddc 100644 --- a/include/uapi/linux/sched.h +++ b/include/uapi/linux/sched.h @@ -102,6 +102,9 @@ struct clone_args { __aligned_u64 set_tid_size; __aligned_u64 cgroup; }; + +enum sched_qos_type { +}; #endif =20 #define CLONE_ARGS_SIZE_VER0 64 /* sizeof first published struct */ @@ -132,6 +135,7 @@ struct clone_args { #define SCHED_FLAG_KEEP_PARAMS 0x10 #define SCHED_FLAG_UTIL_CLAMP_MIN 0x20 #define SCHED_FLAG_UTIL_CLAMP_MAX 0x40 +#define SCHED_FLAG_QOS 0x80 =20 #define SCHED_FLAG_KEEP_ALL (SCHED_FLAG_KEEP_POLICY | \ SCHED_FLAG_KEEP_PARAMS) diff --git a/include/uapi/linux/sched/types.h b/include/uapi/linux/sched/ty= pes.h index 90662385689b..55e4b1e79ed2 100644 --- a/include/uapi/linux/sched/types.h +++ b/include/uapi/linux/sched/types.h @@ -94,6 +94,48 @@ * scheduled on a CPU with no more capacity than the specified value. * * A task utilization boundary can be reset by setting the attribute to -1. + * + * Scheduler QoS + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * + * Different workloads have different scheduling requirements to operate + * optimally. The same applies to tasks within the same workload. + * + * To enable smarter usage of system resources and to cater for the confli= cting + * demands of various tasks, Scheduler QoS provides a mechanism to provide= more + * information about those demands so that scheduler can do best-effort to + * honour them. + * + * @sched_qos_type what QoS hint to apply + * @sched_qos_value value of the QoS hint + * @sched_qos_cookie magic cookie to tag a group of tasks for which the Q= oS + * applies. If 0, the hint will apply globally system + * wide. If not 0, the hint will be relative to tasks that + * has the same cookie value only. + * + * QoS hints are set once and not inherited by children by design. The + * rationale is that each task has its individual characteristics and it is + * encouraged to describe each of these separately. Also since system reso= urces + * are finite, there's a limit to what can be done to honour these requests + * before reaching a tipping point where there are too many requests for + * a particular QoS that is impossible to service for all of them at once = and + * some will start to lose out. For example if 10 tasks require better wake + * up latencies on a 4 CPUs SMP system, then if they all wake up at once, = only + * 4 can perceive the hint honoured and the rest will have to wait. Inheri= tance + * can lead these 10 to become a 100 or a 1000 more easily, and then the Q= oS + * hint will lose its meaning and effectiveness rapidly. The chances of 10 + * tasks waking up at the same time is lower than a 100 and lower than a 1= 000. + * + * To set multiple QoS hints, a syscall is required for each. This is a + * trade-off to reduce the churn on extending the interface as the hope for + * this to evolve as workloads and hardware get more sophisticated and the + * need for extension will arise; and when this happen the task should be + * simpler to add the kernel extension and allow userspace to use readily = by + * setting the newly added flag without having to update the whole of + * sched_attr. + * + * Details about the available QoS hints can be found in: + * Documentation/scheduler/sched-qos.rst */ struct sched_attr { __u32 size; @@ -116,6 +158,10 @@ struct sched_attr { __u32 sched_util_min; __u32 sched_util_max; =20 + __u32 sched_qos_type; + __s64 sched_qos_value; + __u32 sched_qos_cookie; + }; =20 #endif /* _UAPI_LINUX_SCHED_TYPES_H */ diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index ae1b42775ef9..a7d4dfdfed43 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -668,6 +668,9 @@ int __sched_setscheduler(struct task_struct *p, return retval; } =20 + if (attr->sched_flags & SCHED_FLAG_QOS) + return -EOPNOTSUPP; + /* * SCHED_DEADLINE bandwidth accounting relies on stable cpusets * information. diff --git a/tools/perf/trace/beauty/include/uapi/linux/sched.h b/tools/per= f/trace/beauty/include/uapi/linux/sched.h index 3bac0a8ceab2..67ef99f64ddc 100644 --- a/tools/perf/trace/beauty/include/uapi/linux/sched.h +++ b/tools/perf/trace/beauty/include/uapi/linux/sched.h @@ -102,6 +102,9 @@ struct clone_args { __aligned_u64 set_tid_size; __aligned_u64 cgroup; }; + +enum sched_qos_type { +}; #endif =20 #define CLONE_ARGS_SIZE_VER0 64 /* sizeof first published struct */ @@ -132,6 +135,7 @@ struct clone_args { #define SCHED_FLAG_KEEP_PARAMS 0x10 #define SCHED_FLAG_UTIL_CLAMP_MIN 0x20 #define SCHED_FLAG_UTIL_CLAMP_MAX 0x40 +#define SCHED_FLAG_QOS 0x80 =20 #define SCHED_FLAG_KEEP_ALL (SCHED_FLAG_KEEP_POLICY | \ SCHED_FLAG_KEEP_PARAMS) --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B81431BC066 for ; Tue, 20 Aug 2024 16:36:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171815; cv=none; b=tZrNP/tmVyvhCLhMDeQ2txJJBw1UnmQWLpVEeeD4oHJXNkpr8JCPcN0xJVc+MgtdWm9VXSPgs6ouLzU0aT1yBJUMkvUJE04nQxzS+kD400Qwn+xr6668j9IgbTGXXZ0o/kgCc8tJVMOjf30vRm17wAmU4NW7BC+U19H22rIvEXI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171815; c=relaxed/simple; bh=6C3oq3HrJLkJWrLpo+1dSeHCEUbPFUKzQa1BwujeAi0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AZEZPYiaphx5HqpUa/ZYAbcr791i7skzVJx1Z1H2E6lVjAK2jys0p1fpEkAzaaPZxWNS91ibeJwxpUrTQCYT8UmJ0SaY8nO/KIefBWDd7h25lhYUvM0H9L2BWkLFrAAXr6K+w/60df78m45n2NGU7RsvFfHE48SNjYCk+8LuMJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=fjzVqkVB; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="fjzVqkVB" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a7a9e25008aso709589366b.0 for ; Tue, 20 Aug 2024 09:36:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171808; x=1724776608; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D/d/uli9PIV9KsPRWvcOGLBXbZ0uKNuetYque1eIsGo=; b=fjzVqkVBxMlFXs/sf5pfn83HKxZQ5kmtbDyuCPqCWoT989vqp5FYBgRHckO175VAK6 afp5xA1Kt3xPIl/EhBbsiMUTi+KZe3lMFTUMw+wF/gfH0N2Zh3ZBp76G84juM/GieE1P ATcMhYlRgSR49mkGkASHqiIi+kzig0aJhlL0J45kH3IWYiQvLA/z6824GVWUeBy/jwTF OO8c65IReOgiNzWEsfrQUUHkZpYEopWYA3pATdKuwtFZSAj/Hv1olpkpvTw5AUJgVZID hDC8O3YuV0TvVW+0Y/aHZC5FzOTc4xELh9NvIMwBPR0MtX1auZ+ymrHXTFERLFwTi9sp YFRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171808; x=1724776608; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D/d/uli9PIV9KsPRWvcOGLBXbZ0uKNuetYque1eIsGo=; b=VMYCm9TjaSPo5SuPP14bqfE5mxxgzpLB0B0hWDYjYcsudnkKPBz22v8mRBRkiXOdQU it+QLwzVUU6v49/Zjz05fY2/nBF+KJ7qJHqZ0X+L8P9eWjdOLXZcjOZkX9CiO0AOzmhX HUEjxJse6/r+4Hn5/xKIp1EHPPDARiyXZ43o/QMM1jCgY8ww2p4DY1XhRBXW4W1uaAZT y0VZOFxfozvcyD+o7pCI/fP6DnpL25Fr0ewu740AWhmaHPoxGC7TE2roXYdHm2PmBF1s 5j3CBJitopTx5k8zXcblMcCnZMh5GZ8/1qhEXnXJZCIQA6s5gjLdPF5rXSV+RP08r8HR Z6gQ== X-Forwarded-Encrypted: i=1; AJvYcCXhnSVDlVlwBB4PNzQ61OAw2fqBTWomJ3pBb9xjskjY4vBeUA98TKX9PvUeaibxYBjjFQTC+PN/9qq4pkij7ECdpFZwRtWOrrOo2+Da X-Gm-Message-State: AOJu0YxU3kXMUKlz3EYBnNOwVOjdPdKP2KhyrdysWkd8wh9NyrGfUVVV M+42bLxLQPv6tXFrNzMLVfsnkqF4gl96pJZ54RvveZJwdrXvGgwndZajXfO265o= X-Google-Smtp-Source: AGHT+IEZPw2L8xpcAT+hcydRU41CPHFT8ONkE8EI/GuWh2TzpVwCtCSqfce21OS/S3Nt2vjzA7lj8A== X-Received: by 2002:a17:907:f193:b0:a80:f893:51bb with SMTP id a640c23a62f3a-a8392a4c515mr1029504866b.68.1724171807827; Tue, 20 Aug 2024 09:36:47 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.36.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:47 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 11/16] sched/qos: Add rampup multiplier QoS Date: Tue, 20 Aug 2024 17:35:07 +0100 Message-Id: <20240820163512.1096301-12-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Bursty tasks are hard to predict. To use resources efficiently, the system would like to be exact as much as possible. But this poses a challenge for these bursty tasks that need to get access to more resources quickly. The new SCHED_QOS_RAMPUP_MULTIPLIER allows userspace to do that. As the name implies, it only helps them to transition to a higher performance state when they get _busier_. That is perfectly periodic tasks by definition are not going through a transition and will run at a constant performance level. It is the tasks that need to transition from one periodic state to another periodic state that is at a higher level that this rampup_multiplier will help with. It also slows down the ewma decay of util_est which should help those bursty tasks to keep their faster rampup. This should work complimentary with uclamp. uclamp tells the system about min and max perf requirements which can be applied immediately. rampup_multiplier is about reactiveness of the task to change. Specifically to a change for a higher performance level. The task might necessary need to have a min perf requirements, but it can have sudden burst of changes that require higher perf level and it needs the system to provide this faster. TODO: update the sched_qos docs Signed-off-by: Qais Yousef --- include/linux/sched.h | 7 ++++ include/uapi/linux/sched.h | 2 ++ kernel/sched/core.c | 66 ++++++++++++++++++++++++++++++++++++++ kernel/sched/fair.c | 6 ++-- kernel/sched/syscalls.c | 38 ++++++++++++++++++++-- 5 files changed, 115 insertions(+), 4 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 2e8c5a9ffa76..a30ee43a25fb 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -404,6 +404,11 @@ struct sched_info { #endif /* CONFIG_SCHED_INFO */ }; =20 +struct sched_qos { + DECLARE_BITMAP(user_defined, SCHED_QOS_MAX); + unsigned int rampup_multiplier; +}; + /* * Integer metrics need fixed point arithmetic, e.g., sched/fair * has a few: load, load_avg, util_avg, freq, and capacity. @@ -882,6 +887,8 @@ struct task_struct { =20 struct sched_info sched_info; =20 + struct sched_qos sched_qos; + struct list_head tasks; #ifdef CONFIG_SMP struct plist_node pushable_tasks; diff --git a/include/uapi/linux/sched.h b/include/uapi/linux/sched.h index 67ef99f64ddc..0baba91ba5b8 100644 --- a/include/uapi/linux/sched.h +++ b/include/uapi/linux/sched.h @@ -104,6 +104,8 @@ struct clone_args { }; =20 enum sched_qos_type { + SCHED_QOS_RAMPUP_MULTIPLIER, + SCHED_QOS_MAX, }; #endif =20 diff --git a/kernel/sched/core.c b/kernel/sched/core.c index c91e6a62c7ab..54faa845cb29 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -152,6 +152,8 @@ __read_mostly int sysctl_resched_latency_warn_once =3D = 1; */ const_debug unsigned int sysctl_sched_nr_migrate =3D SCHED_NR_MIGRATE_BREA= K; =20 +unsigned int sysctl_sched_qos_default_rampup_multiplier =3D 1; + __read_mostly int scheduler_running; =20 #ifdef CONFIG_SCHED_CORE @@ -4488,6 +4490,47 @@ static int sysctl_schedstats(struct ctl_table *table= , int write, void *buffer, #endif /* CONFIG_SCHEDSTATS */ =20 #ifdef CONFIG_SYSCTL +static void sched_qos_sync_sysctl(void) +{ + struct task_struct *g, *p; + + guard(rcu)(); + for_each_process_thread(g, p) { + struct rq_flags rf; + struct rq *rq; + + rq =3D task_rq_lock(p, &rf); + if (!test_bit(SCHED_QOS_RAMPUP_MULTIPLIER, p->sched_qos.user_defined)) + p->sched_qos.rampup_multiplier =3D sysctl_sched_qos_default_rampup_mult= iplier; + task_rq_unlock(rq, p, &rf); + } +} + +static int sysctl_sched_qos_handler(struct ctl_table *table, int write, + void *buffer, size_t *lenp, loff_t *ppos) +{ + unsigned int old_rampup_mult; + int result; + + old_rampup_mult =3D sysctl_sched_qos_default_rampup_multiplier; + + result =3D proc_dointvec(table, write, buffer, lenp, ppos); + if (result) + goto undo; + if (!write) + return 0; + + if (old_rampup_mult !=3D sysctl_sched_qos_default_rampup_multiplier) { + sched_qos_sync_sysctl(); + } + + return 0; + +undo: + sysctl_sched_qos_default_rampup_multiplier =3D old_rampup_mult; + return result; +} + static struct ctl_table sched_core_sysctls[] =3D { #ifdef CONFIG_SCHEDSTATS { @@ -4534,6 +4577,13 @@ static struct ctl_table sched_core_sysctls[] =3D { .extra2 =3D SYSCTL_FOUR, }, #endif /* CONFIG_NUMA_BALANCING */ + { + .procname =3D "sched_qos_default_rampup_multiplier", + .data =3D &sysctl_sched_qos_default_rampup_multiplier, + .maxlen =3D sizeof(unsigned int), + .mode =3D 0644, + .proc_handler =3D sysctl_sched_qos_handler, + }, }; static int __init sched_core_sysctl_init(void) { @@ -4543,6 +4593,21 @@ static int __init sched_core_sysctl_init(void) late_initcall(sched_core_sysctl_init); #endif /* CONFIG_SYSCTL */ =20 +static void sched_qos_fork(struct task_struct *p) +{ + /* + * We always force reset sched_qos on fork. These sched_qos are treated + * as finite resources to help improve quality of life. Inheriting them + * by default can easily lead to a situation where the QoS hint become + * meaningless because all tasks in the system have it. + * + * Every task must request the QoS explicitly if it needs it. No + * accidental inheritance is allowed to keep the default behavior sane. + */ + bitmap_zero(p->sched_qos.user_defined, SCHED_QOS_MAX); + p->sched_qos.rampup_multiplier =3D sysctl_sched_qos_default_rampup_multip= lier; +} + /* * fork()/clone()-time setup: */ @@ -4562,6 +4627,7 @@ int sched_fork(unsigned long clone_flags, struct task= _struct *p) p->prio =3D current->normal_prio; =20 uclamp_fork(p); + sched_qos_fork(p); =20 /* * Revert to default priority/policy on fork if requested. diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0c10e2afb52d..3d9794db58e1 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4906,7 +4906,7 @@ static inline void util_est_update(struct cfs_rq *cfs= _rq, if (!task_sleep) { if (task_util(p) > task_util_dequeued(p)) { ewma &=3D ~UTIL_AVG_UNCHANGED; - ewma =3D approximate_util_avg(ewma, p->se.delta_exec / 1000); + ewma =3D approximate_util_avg(ewma, (p->se.delta_exec/1000) * p->sched_= qos.rampup_multiplier); goto done; } return; @@ -4974,6 +4974,8 @@ static inline void util_est_update(struct cfs_rq *cfs= _rq, * 0.25, thus making w=3D1/4 ( >>=3D UTIL_EST_WEIGHT_SHIFT) */ ewma <<=3D UTIL_EST_WEIGHT_SHIFT; + if (p->sched_qos.rampup_multiplier) + last_ewma_diff /=3D p->sched_qos.rampup_multiplier; ewma -=3D last_ewma_diff; ewma >>=3D UTIL_EST_WEIGHT_SHIFT; done: @@ -9643,7 +9645,7 @@ static void update_cpu_capacity(struct sched_domain *= sd, int cpu) * on TICK doesn't end up hurting it as it can happen after we would * have crossed this threshold. * - * To ensure that invaraince is taken into account, we don't scale time + * To ensure that invariance is taken into account, we don't scale time * and use it as-is, approximate_util_avg() will then let us know the * our threshold. */ diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index a7d4dfdfed43..dc7d7bcaae7b 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -543,6 +543,35 @@ static void __setscheduler_uclamp(struct task_struct *= p, const struct sched_attr *attr) { } #endif =20 +static inline int sched_qos_validate(struct task_struct *p, + const struct sched_attr *attr) +{ + switch (attr->sched_qos_type) { + case SCHED_QOS_RAMPUP_MULTIPLIER: + if (attr->sched_qos_cookie) + return -EINVAL; + if (attr->sched_qos_value < 0) + return -EINVAL; + break; + default: + return -EINVAL; + } + + return 0; +} + +static void __setscheduler_sched_qos(struct task_struct *p, + const struct sched_attr *attr) +{ + switch (attr->sched_qos_type) { + case SCHED_QOS_RAMPUP_MULTIPLIER: + set_bit(SCHED_QOS_RAMPUP_MULTIPLIER, p->sched_qos.user_defined); + p->sched_qos.rampup_multiplier =3D attr->sched_qos_value; + default: + break; + } +} + /* * Allow unprivileged RT tasks to decrease priority. * Only issue a capable test if needed and only once to avoid an audit @@ -668,8 +697,11 @@ int __sched_setscheduler(struct task_struct *p, return retval; } =20 - if (attr->sched_flags & SCHED_FLAG_QOS) - return -EOPNOTSUPP; + if (attr->sched_flags & SCHED_FLAG_QOS) { + retval =3D sched_qos_validate(p, attr); + if (retval) + return retval; + } =20 /* * SCHED_DEADLINE bandwidth accounting relies on stable cpusets @@ -799,7 +831,9 @@ int __sched_setscheduler(struct task_struct *p, __setscheduler_params(p, attr); __setscheduler_prio(p, newprio); } + __setscheduler_uclamp(p, attr); + __setscheduler_sched_qos(p, attr); =20 if (queued) { /* --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D0301BD515 for ; Tue, 20 Aug 2024 16:36:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171812; cv=none; b=V88hlzcnsZpIk5950jyP4yFUZJUw6ZBdvtO2YCMHEHBI6uSTx7aWNbv5PiHTOFzvKTz1AkczJXG7eHNEGTxp3s5SgyM318xfygn0o7qiLGf58fxyfeJJ25JvjoY8ykc1vHpYh+iZqn05fe128EIrIwGDdv5a7W1baP6dFlMvbEw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171812; c=relaxed/simple; bh=OqsaIQh4uctp/oFXsRzpZsyZxGxfrG6iSiwYiGo8PH8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fx5gjF024Uv1nbC2QlVmLBveZXDPqlOGv7RWg3gmo+w+G867QL8bWYotNP9HDG1bWLzwfsG+vi53VT4Zj87nVy5hIAsPw51QTGrwlHSWoSLGWSwTI/xKxjtbkUgMokkAD56V/O6aHHHpw7O8yJG+QgTfoKmnA2ccrduwfG/dAHo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=RTMnOQ6Y; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="RTMnOQ6Y" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a7aa086b077so595078866b.0 for ; Tue, 20 Aug 2024 09:36:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171809; x=1724776609; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=njOkbGyQ961EBslTvE6EDoi/18VK+01g5BSPEHY0GfY=; b=RTMnOQ6YK46DwBqw1h2uMXzX+6ia50MIzmrFzdRyi606L4gpe9mgVOQLjyiHT0/inH qQy/5Ws94ElnG0lNwn3558zunrUbtYXHvreSFkFHs58r9TmMlqvmJib1bSe1TbExfO+g ZQnypvquHj8Yk5Jzuvr3A5FM8ynUFKwQjOzQmyk7Rdgg5UBUILfsCKCNBdN3DCoH6ndF D87/NM4OjWar68x+o8aX5UnG/wHwTMmtviaVvc7Btr0Wl/zNsghZoWY0EaZZiUD+bRfl AW9MtTMT1ZM4vXJwMSsBI0Eu+rXIXkS62s9DLOEJ/43Gd2E5XFVkNIGxWYwtTC4JbwiP Cxfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171809; x=1724776609; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=njOkbGyQ961EBslTvE6EDoi/18VK+01g5BSPEHY0GfY=; b=BAuQRFX4nvz0bpdYIdUQUnYZOr5oGt/uGATuBHwauJI+/8Q+5JluWLWEEqGFHBvjZD Csm4hpaeNoOQ5Ku17OIPINmfly5ceE0vW2pO4ALe/QMwm/Z6XCDHa9xRKIoMuxNn+Zey b8nMWyXFzDLEHj/AX471DPSWO+pCx4s9PBuGlKAoJ6v2QeFqqxGv1ryWPFbagGCcyU8f mIY1ki/TMQZf1R8JnJAZCCKOK9ZU1kGG0YxPaRcp4N11utTksXNFeszBrSeMAUC1Q65i vO9B7vSMgRAiDRmWa91qZvRuC/xVdJVJfMCoY+K1O4MHU4YmKw9VVJxCdGsvPDpSjnLU 2o8w== X-Forwarded-Encrypted: i=1; AJvYcCV0k26hbMoYGLxTqzINrutaBZkCGvEEIpnSnOwzNBrN+xsPSUBxJ3EEo1Am4rfhOMxybQKFUSkbD9FdeYIaeen5be/DA90hPRBVBkYi X-Gm-Message-State: AOJu0YyVzQEBzitXARv+mnTEqcGxjFezzN5YgheQ31LPMOV1mmlxYc8B eiOEPLrqMdEbbbKbzAcd1V1sQfJCpHPvZBSKSo8sGMgrjRHZfLxMCxyapaybbg86jjuU18CGIEl Q X-Google-Smtp-Source: AGHT+IGAOpAp1NhyW1kkfvm5+3bpeVK+LP152tiTM71qKtdLXPrb3x+lQlXhi6Bkd3u5CZOa4S8bpQ== X-Received: by 2002:a17:906:f591:b0:a7a:c106:3659 with SMTP id a640c23a62f3a-a8647b6c7a6mr242678966b.60.1724171808835; Tue, 20 Aug 2024 09:36:48 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.36.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:48 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 12/16] sched/pelt: Add new waiting_avg to record when runnable && !running Date: Tue, 20 Aug 2024 17:35:08 +0100 Message-Id: <20240820163512.1096301-13-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This info will be useful to understand how long tasks end up waiting behind other tasks. This info is recorded for tasks only, and added/subtracted from root cfs_rq on __update_load_avg_se(). It also helps to decouple util_avg which indicates tasks computational demand from the fact that the CPU might need to run faster to reduce the waiting time. It has been a point of confusion in the past while discussing uclamp and util_avg and the fact that not keeping freq high means tasks will take longer to run and cause delays. Isolating the source of delay into its own signal would be a better way to take this source of delay into account when making decisions independently of task's/CPU's computational demands. It is not used now. But will be used later to help drive DVFS headroom. It could become a helpful metric to help us manage waiting latencies in general, for example in load balance. TODO: waiting_avg should use rq_clock_task() as it doesn't care about invariance. Waiting time should reflect actual wait in realtime as this is the measure of latency that users care about. Signed-off-by: Qais Yousef --- include/linux/sched.h | 2 ++ kernel/sched/debug.c | 5 +++++ kernel/sched/fair.c | 32 +++++++++++++++++++++++++++++- kernel/sched/pelt.c | 45 ++++++++++++++++++++++++++++++------------- 4 files changed, 70 insertions(+), 14 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index a30ee43a25fb..f332ce5e226f 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -477,10 +477,12 @@ struct sched_avg { u64 last_update_time; u64 load_sum; u64 runnable_sum; + u64 waiting_sum; u32 util_sum; u32 period_contrib; unsigned long load_avg; unsigned long runnable_avg; + unsigned long waiting_avg; unsigned long util_avg; unsigned int util_est; } ____cacheline_aligned; diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index c1eb9a1afd13..5fa2662a4a50 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -528,6 +528,7 @@ static void print_cfs_group_stats(struct seq_file *m, i= nt cpu, struct task_group P(se->avg.load_avg); P(se->avg.util_avg); P(se->avg.runnable_avg); + P(se->avg.waiting_avg); #endif =20 #undef PN_SCHEDSTAT @@ -683,6 +684,8 @@ void print_cfs_rq(struct seq_file *m, int cpu, struct c= fs_rq *cfs_rq) cfs_rq->avg.load_avg); SEQ_printf(m, " .%-30s: %lu\n", "runnable_avg", cfs_rq->avg.runnable_avg); + SEQ_printf(m, " .%-30s: %lu\n", "waiting_avg", + cfs_rq->avg.waiting_avg); SEQ_printf(m, " .%-30s: %lu\n", "util_avg", cfs_rq->avg.util_avg); SEQ_printf(m, " .%-30s: %u\n", "util_est", @@ -1071,9 +1074,11 @@ void proc_sched_show_task(struct task_struct *p, str= uct pid_namespace *ns, #ifdef CONFIG_SMP P(se.avg.load_sum); P(se.avg.runnable_sum); + P(se.avg.waiting_sum); P(se.avg.util_sum); P(se.avg.load_avg); P(se.avg.runnable_avg); + P(se.avg.waiting_avg); P(se.avg.util_avg); P(se.avg.last_update_time); PM(se.avg.util_est, ~UTIL_AVG_UNCHANGED); diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 3d9794db58e1..a8dbba0b755e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4726,6 +4726,22 @@ static void detach_entity_load_avg(struct cfs_rq *cf= s_rq, struct sched_entity *s trace_pelt_cfs_tp(cfs_rq); } =20 +static inline void add_waiting_avg(struct cfs_rq *cfs_rq, struct sched_ent= ity *se) +{ + unsigned long waiting_avg; + waiting_avg =3D READ_ONCE(cfs_rq->avg.waiting_avg); + waiting_avg +=3D READ_ONCE(se->avg.waiting_avg); + WRITE_ONCE(cfs_rq->avg.waiting_avg, waiting_avg); +} + +static inline void sub_waiting_avg(struct cfs_rq *cfs_rq, struct sched_ent= ity *se) +{ + unsigned long waiting_avg; + waiting_avg =3D READ_ONCE(cfs_rq->avg.waiting_avg); + waiting_avg -=3D min(waiting_avg, READ_ONCE(se->avg.waiting_avg)); + WRITE_ONCE(cfs_rq->avg.waiting_avg, waiting_avg); +} + /* * Optional action to be done while updating the load average */ @@ -4744,8 +4760,15 @@ static inline void update_load_avg(struct cfs_rq *cf= s_rq, struct sched_entity *s * Track task load average for carrying it to new CPU after migrated, and * track group sched_entity load average for task_h_load calculation in m= igration */ - if (se->avg.last_update_time && !(flags & SKIP_AGE_LOAD)) + if (se->avg.last_update_time && !(flags & SKIP_AGE_LOAD)) { + bool update_rq_waiting_avg =3D entity_is_task(se) && se_runnable(se); + + if (update_rq_waiting_avg) + sub_waiting_avg(&rq_of(cfs_rq)->cfs, se); __update_load_avg_se(now, cfs_rq, se); + if (update_rq_waiting_avg) + add_waiting_avg(&rq_of(cfs_rq)->cfs, se); + } =20 decayed =3D update_cfs_rq_load_avg(now, cfs_rq); decayed |=3D propagate_entity_load_avg(se); @@ -5182,6 +5205,11 @@ attach_entity_load_avg(struct cfs_rq *cfs_rq, struct= sched_entity *se) {} static inline void detach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se) {} =20 +static inline void +add_waiting_avg(struct cfs_rq *cfs_rq, struct sched_entity *se) {} +static inline void +sub_waiting_avg(struct cfs_rq *cfs_rq, struct sched_entity *se) {} + static inline int sched_balance_newidle(struct rq *rq, struct rq_flags *rf) { return 0; @@ -6786,6 +6814,7 @@ enqueue_task_fair(struct rq *rq, struct task_struct *= p, int flags) * estimated utilization, before we update schedutil. */ util_est_enqueue(&rq->cfs, p); + add_waiting_avg(&rq->cfs, se); =20 /* * If in_iowait is set, the code below may not trigger any cpufreq @@ -6874,6 +6903,7 @@ static void dequeue_task_fair(struct rq *rq, struct t= ask_struct *p, int flags) bool was_sched_idle =3D sched_idle_rq(rq); =20 util_est_dequeue(&rq->cfs, p); + sub_waiting_avg(&rq->cfs, se); =20 for_each_sched_entity(se) { cfs_rq =3D cfs_rq_of(se); diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c index 536575757420..f0974abf8566 100644 --- a/kernel/sched/pelt.c +++ b/kernel/sched/pelt.c @@ -103,7 +103,8 @@ static u32 __accumulate_pelt_segments(u64 periods, u32 = d1, u32 d3) */ static __always_inline u32 accumulate_sum(u64 delta, struct sched_avg *sa, - unsigned long load, unsigned long runnable, int running) + unsigned long load, unsigned long runnable, int running, + bool is_task) { u32 contrib =3D (u32)delta; /* p =3D=3D 0 -> delta < 1024 */ u64 periods; @@ -118,6 +119,7 @@ accumulate_sum(u64 delta, struct sched_avg *sa, sa->load_sum =3D decay_load(sa->load_sum, periods); sa->runnable_sum =3D decay_load(sa->runnable_sum, periods); + sa->waiting_sum =3D decay_load((u64)(sa->waiting_sum), periods); sa->util_sum =3D decay_load((u64)(sa->util_sum), periods); =20 /* @@ -147,6 +149,8 @@ accumulate_sum(u64 delta, struct sched_avg *sa, sa->runnable_sum +=3D runnable * contrib << SCHED_CAPACITY_SHIFT; if (running) sa->util_sum +=3D contrib << SCHED_CAPACITY_SHIFT; + if (is_task && runnable && !running) + sa->waiting_sum +=3D contrib << SCHED_CAPACITY_SHIFT; =20 return periods; } @@ -181,7 +185,8 @@ accumulate_sum(u64 delta, struct sched_avg *sa, */ static __always_inline int ___update_load_sum(u64 now, struct sched_avg *sa, - unsigned long load, unsigned long runnable, int running) + unsigned long load, unsigned long runnable, int running, + bool is_task) { int time_shift; u64 delta; @@ -232,7 +237,7 @@ ___update_load_sum(u64 now, struct sched_avg *sa, * Step 1: accumulate *_sum since last_update_time. If we haven't * crossed period boundaries, finish. */ - if (!accumulate_sum(delta, sa, load, runnable, running)) + if (!accumulate_sum(delta, sa, load, runnable, running, is_task)) return 0; =20 return 1; @@ -272,6 +277,7 @@ ___update_load_avg(struct sched_avg *sa, unsigned long = load) */ sa->load_avg =3D div_u64(load * sa->load_sum, divider); sa->runnable_avg =3D div_u64(sa->runnable_sum, divider); + sa->waiting_avg =3D div_u64(sa->waiting_sum, divider); WRITE_ONCE(sa->util_avg, sa->util_sum / divider); } =20 @@ -303,7 +309,7 @@ ___update_load_avg(struct sched_avg *sa, unsigned long = load) =20 int __update_load_avg_blocked_se(u64 now, struct sched_entity *se) { - if (___update_load_sum(now, &se->avg, 0, 0, 0)) { + if (___update_load_sum(now, &se->avg, 0, 0, 0, false)) { ___update_load_avg(&se->avg, se_weight(se)); trace_pelt_se_tp(se); return 1; @@ -314,10 +320,17 @@ int __update_load_avg_blocked_se(u64 now, struct sche= d_entity *se) =20 int __update_load_avg_se(u64 now, struct cfs_rq *cfs_rq, struct sched_enti= ty *se) { + bool is_task =3D entity_is_task(se); + + if (is_task) + rq_of(cfs_rq)->cfs.avg.waiting_avg -=3D se->avg.waiting_avg; + if (___update_load_sum(now, &se->avg, !!se->on_rq, se_runnable(se), - cfs_rq->curr =3D=3D se)) { + cfs_rq->curr =3D=3D se, is_task)) { =20 ___update_load_avg(&se->avg, se_weight(se)); + if (is_task) + rq_of(cfs_rq)->cfs.avg.waiting_avg +=3D se->avg.waiting_avg; cfs_se_util_change(&se->avg); trace_pelt_se_tp(se); return 1; @@ -331,7 +344,8 @@ int __update_load_avg_cfs_rq(u64 now, struct cfs_rq *cf= s_rq) if (___update_load_sum(now, &cfs_rq->avg, scale_load_down(cfs_rq->load.weight), cfs_rq->h_nr_running, - cfs_rq->curr !=3D NULL)) { + cfs_rq->curr !=3D NULL, + false)) { =20 ___update_load_avg(&cfs_rq->avg, 1); trace_pelt_cfs_tp(cfs_rq); @@ -357,7 +371,8 @@ int update_rt_rq_load_avg(u64 now, struct rq *rq, int r= unning) if (___update_load_sum(now, &rq->avg_rt, running, running, - running)) { + running, + false)) { =20 ___update_load_avg(&rq->avg_rt, 1); trace_pelt_rt_tp(rq); @@ -383,7 +398,8 @@ int update_dl_rq_load_avg(u64 now, struct rq *rq, int r= unning) if (___update_load_sum(now, &rq->avg_dl, running, running, - running)) { + running, + false)) { =20 ___update_load_avg(&rq->avg_dl, 1); trace_pelt_dl_tp(rq); @@ -414,7 +430,8 @@ int update_hw_load_avg(u64 now, struct rq *rq, u64 capa= city) if (___update_load_sum(now, &rq->avg_hw, capacity, capacity, - capacity)) { + capacity, + false)) { ___update_load_avg(&rq->avg_hw, 1); trace_pelt_hw_tp(rq); return 1; @@ -462,11 +479,13 @@ int update_irq_load_avg(struct rq *rq, u64 running) ret =3D ___update_load_sum(rq->clock - running, &rq->avg_irq, 0, 0, - 0); + 0, + false); ret +=3D ___update_load_sum(rq->clock, &rq->avg_irq, 1, 1, - 1); + 1, + false); =20 if (ret) { ___update_load_avg(&rq->avg_irq, 1); @@ -536,7 +555,7 @@ unsigned long approximate_util_avg(unsigned long util, = u64 delta) if (unlikely(!delta)) return util; =20 - accumulate_sum(delta << sched_pelt_lshift, &sa, 1, 0, 1); + accumulate_sum(delta << sched_pelt_lshift, &sa, 1, 0, 1, false); ___update_load_avg(&sa, 0); =20 return sa.util_avg; @@ -555,7 +574,7 @@ u64 approximate_runtime(unsigned long util) return runtime; =20 while (sa.util_avg < util) { - accumulate_sum(delta, &sa, 1, 0, 1); + accumulate_sum(delta, &sa, 1, 0, 1, false); ___update_load_avg(&sa, 0); runtime++; } --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 045201C0DED for ; Tue, 20 Aug 2024 16:36:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171814; cv=none; b=c2XEfiJpG9fsZJVIGoU41GpXuWa+Pfubj0ZywOJSbRuy97Qy+H/56LvkygmBP7RfX3h14zCfcMiUQfUJ23aEKM3lbJACY89ffLUIFtBfCFGC/rtJhbTkcaC+aVBR+m5sh0zDYv6KwpmwtzNygRwulSTk9PWUZ8mKOhPTeWgdMwg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171814; c=relaxed/simple; bh=peGOTrMpJU6Y/7bYHyR4tSGHI6Lbtst36QPON3uOXj0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PiYHbA3MdF8tZ/+NLuYJeijJXbV+V/mVZJwYjcl3Cx9W5AdhgMTxIojYjcEty8/EKqVvlHi9B86FRO1uri1Ga3v/3IFRzWfEFxu37E1b2LFTItJmaCbA5GzlBfLEXghm1oH3uDhZIZT2R5TQag98yjzPeDHntsiALAaysr43Mck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=3A3uUyoa; arc=none smtp.client-ip=209.85.218.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="3A3uUyoa" Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-a8647056026so125241466b.0 for ; Tue, 20 Aug 2024 09:36:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171810; x=1724776610; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nawo0iXyk8rugawGjPThZSFWb4byKstZYwvxJoh01/U=; b=3A3uUyoa2XVh8IQpUMWF9oo+EJnAtvGnVkgTOkd3SrhbRtIYlpsyGX8V8H0KcR/V1e nC3fH2Ff2pHhFdNpRiGMeGkoOLXJJ3ZVk4/EDrag/Ntp/+MGPqI7avbvbJDjJjbxxTPg lzc0t5Agrk7BhTMPRV3uFnEnFfmr9nLTrrUJP5okbQOVen/wZpjKsrHUhEPuIHUCDEJE 3uapYNNelcMfaAL2Lb+dTBqsLyUESbDBWE47bOvfOOznohFRNZ8BkOduG711d+zvGaf7 FoDW+9myWedFeQHli9yAucsOfnybJAoZ/G8ZUmlROJcvzHe4zoCdGL7ccOb1BtYf75LJ FFJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171810; x=1724776610; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nawo0iXyk8rugawGjPThZSFWb4byKstZYwvxJoh01/U=; b=lcTkEuHMI5xVV4ImYKtWYaFCN6gkj59Wf4lx5/iloMSPUtY1EKtY487CHbhN36aWNm vHBvwwaXcBKVvuJSURHobMxYhQfrA++Ue0ahmHRwmJnjnz1ErK7gRuDY7ZC9dlVQ1UAS Xo5h//XIz22U3Fycp81dQTUNwMfuPfcJbKuNb0ZZJ/mFHKFaOuraEqUyDA4Ybp0eFLMB uC8XIvTVjGht/TdfJSoSXG8xjYOFbw+H2pDJZz2SMujlhZ/+MXnXd1DkT07ImcJrX3sj qo7QvHxPMU1PYfpA5EUodmzSpXTEUFzfWzAz4Zkhoug78PeO9OIMTlKnxynVkD5wFjYZ RWdw== X-Forwarded-Encrypted: i=1; AJvYcCUq9aXjoIW5DiisepwPpC8yOAXmTIP+xbexpByR2vYOMXaXc1TUGchOp/5/ks3NI2+Jvg8N+YirIJtQDbI=@vger.kernel.org X-Gm-Message-State: AOJu0YwQyrStln0HrXA9cyHhD2eryYxVBe4Z9Ef1HRfaReVnrAulsBnx QNallrtKVmPchmUckYRLFq5lrQPms/3FfTz0AGsa7M/TphR3JsL37xmN5PxWT0gyTSZqm8RXkSe U X-Google-Smtp-Source: AGHT+IGIadSEibAssyW3zb8AhXXDHexQCIIATFiI9hph0UEKoDLXeaAbkr5C1JABp/pKbDbwR99llQ== X-Received: by 2002:a17:907:d3c9:b0:a7a:b643:654f with SMTP id a640c23a62f3a-a839292f3ccmr1052502366b.15.1724171810258; Tue, 20 Aug 2024 09:36:50 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.36.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:49 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 13/16] sched/schedutil: Take into account waiting_avg in apply_dvfs_headroom Date: Tue, 20 Aug 2024 17:35:09 +0100 Message-Id: <20240820163512.1096301-14-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We now have three sources of delays. 1. How often we send cpufreq_updates 2. How often we update util_avg 3. How long tasks wait in RUNNABLE to become RUNNING The headroom should cater for all this type of delays to ensure the system is running at adequate performance point. We want to pick the maximum headroom required by any of these sources of delays. TODO: the signal should use task clock not pelt as this should be real time based and we don't care about invariance. Signed-off-by: Qais Yousef --- kernel/sched/cpufreq_schedutil.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedu= til.c index 94e35b7c972d..318b09bc4ab1 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -259,10 +259,15 @@ static unsigned int get_next_freq(struct sugov_policy= *sg_policy, * dvfs_update_delay of the cpufreq governor or min(curr.se.slice, TICK_US= ), * whichever is higher. * + * Also take into accounting how long tasks have been waiting in runnable = but + * !running state. If it is high, it means we need higher DVFS headroom to + * reduce it. + * * XXX: Should we provide headroom when the util is decaying? */ static inline unsigned long sugov_apply_dvfs_headroom(unsigned long util, = int cpu) { + unsigned long update_headroom, waiting_headroom; struct rq *rq =3D cpu_rq(cpu); u64 delay; =20 @@ -276,7 +281,10 @@ static inline unsigned long sugov_apply_dvfs_headroom(= unsigned long util, int c delay =3D TICK_USEC; delay =3D max(delay, per_cpu(dvfs_update_delay, cpu)); =20 - return approximate_util_avg(util, delay); + update_headroom =3D approximate_util_avg(util, delay); + waiting_headroom =3D util + READ_ONCE(rq->cfs.avg.waiting_avg); + + return max(update_headroom, waiting_headroom); } =20 unsigned long sugov_effective_cpu_perf(int cpu, unsigned long actual, --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7FAA1C2332 for ; Tue, 20 Aug 2024 16:36:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171814; cv=none; b=f/OhqY/OSSs0x8++mrfsuKqu+kaNi3Y2HnOBJga+RadB7LoLLHFoe/w4s8NPN5pLPUmZpwLbGCXr8d04JSVhGi6B8RnGoaB0ohU3RQ/yJjXk8IEKUyQsNNejGXprQlpx5WvjeAgmE+7ydS9x9jmlsnQeHII0ZEOhDQnzY0oCeQ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171814; c=relaxed/simple; bh=9ZPSZBuNvWFZgZLOCmzGhftDorxtai3m2giRjex3pcY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=as0KfUvf0VjXdXLPLwrK1klUl4KRblJvlRBoUU+N32lcsz/fw5dHJ8f0/8FtdzoUB9+WbEROAgzQ8sdNKD2rTB+7Xku+WSigJ9YAvXP1nroiAKqKCcQlJYXmD/Aw7pNAK1veCDdPEicsaRIJfjz+DpxJCKv4AE+OXybH+BerwlA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=OYm/b04C; arc=none smtp.client-ip=209.85.218.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="OYm/b04C" Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-a8643235f99so163172266b.3 for ; Tue, 20 Aug 2024 09:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171811; x=1724776611; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9h5cxC5T/07ttsRJWHESo6Dab0VfuQh9Cth3dLGuihw=; b=OYm/b04CrUWrJrdjX1an4E8erpwS2OhGYQZkI6WK68VaOwmsESIYHq5xbTJGzlFKGX NsXyrOzuTP8zggrwTcWzP4ej1rOOneaPjb/nVsCdLwiJ/gi2jZv9ypuEtrmw99ELT455 ljeDskts8uzbrDKuMxLP+7M0luwGiSt+VqA3Ei/4M1wuI5QACp7Xk6C/y/IkUo5Mztal aXSjKYLMtM+QJTuEZlTfM7uuQsJWTl4OUFEIPupjgcdSLHH3/srpjxVjNAxzWjYc0ZpY crJjifgsxSAZjBgFo0CWXUjNxbgQAKA4cLhF8NP2tPKhNEaoncfr7tk1WBbyZfgv9OWl 0xaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171811; x=1724776611; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9h5cxC5T/07ttsRJWHESo6Dab0VfuQh9Cth3dLGuihw=; b=Dy01LsCJk5aNe8vp6kiVIv8rsRH+wZFeW18F4AcNW4Lue9h8DJalyTZYUhEzN2gvAO OWdPolAqZZa7BjrwuSEWpE5D0QcRDeKao13PHzIf9D4+l2BBCCgjOwh1596i7pwQpsGG RvSC1MZgM8kjx51qrGw5joKnujX1eI6zAVuRhkqvnSnrux6++/DR2TPgP8xiI+uMiUF7 bizPAATSl5rzgpL1kxZxfRkakRB6g0tnl0Hy+w2WMOezK7SN+7yutqrYThSvsqYoUkRt 24Gw7aMklAp91oOvuUYenXLar+KL576nZLULqtktNmloLvKWiIm3LRaNnWzYF6h2dzqm 0YNA== X-Forwarded-Encrypted: i=1; AJvYcCW6jjLkyN/C0l3WyqB6n0KlE6YykYq78XKZ+jcMeXtBUUMBmCAUJTd/pryZXdSnUJnVQvBEKokC+qrnwC7qYw07OtNvd2ApgRbF2YA0 X-Gm-Message-State: AOJu0Yzvh4GsphTulx+eHYt6VdCFP4IgE1sJwtBGMFokX12yB+t2F/7F r0AtDLb673NvqYHFsrX+bcH+odddngCXnmWExDffeXTjd4Yjci+5Q+yFolGDVuI= X-Google-Smtp-Source: AGHT+IG0xeJvN3XiOQlKOh7QN7lETQdxPtqwJVb6er+yMgBCCOTRa61cwW2df57Oc6uhYemybizG5Q== X-Received: by 2002:a17:907:97d2:b0:a77:cdaa:88a3 with SMTP id a640c23a62f3a-a86479e45efmr210115366b.27.1724171811016; Tue, 20 Aug 2024 09:36:51 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.36.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:50 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 14/16] sched/schedutil: Ignore dvfs headroom when util is decaying Date: Tue, 20 Aug 2024 17:35:10 +0100 Message-Id: <20240820163512.1096301-15-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" It means we're being idling or doing less work and are already running at a higher value. No need to apply any dvfs headroom in this case. Signed-off-by: Qais Yousef --- kernel/sched/cpufreq_schedutil.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedu= til.c index 318b09bc4ab1..4a1a8b353d51 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -9,6 +9,7 @@ #define IOWAIT_BOOST_MIN (SCHED_CAPACITY_SCALE / 8) =20 DEFINE_PER_CPU_READ_MOSTLY(unsigned long, response_time_mult); +DEFINE_PER_CPU(unsigned long, last_update_util); =20 struct sugov_tunables { struct gov_attr_set attr_set; @@ -262,15 +263,19 @@ static unsigned int get_next_freq(struct sugov_policy= *sg_policy, * Also take into accounting how long tasks have been waiting in runnable = but * !running state. If it is high, it means we need higher DVFS headroom to * reduce it. - * - * XXX: Should we provide headroom when the util is decaying? */ static inline unsigned long sugov_apply_dvfs_headroom(unsigned long util, = int cpu) { - unsigned long update_headroom, waiting_headroom; + unsigned long update_headroom, waiting_headroom, prev_util; struct rq *rq =3D cpu_rq(cpu); u64 delay; =20 + prev_util =3D per_cpu(last_update_util, cpu); + per_cpu(last_update_util, cpu) =3D util; + + if (util < prev_util) + return util; + /* * What is the possible worst case scenario for updating util_avg, ctx * switch or TICK? --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9821C1BB6B8 for ; Tue, 20 Aug 2024 16:36:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171815; cv=none; b=LsB7lPw2ssgw1qr24VRvQfe69TA0B7oz3yKVy+KAogD1xtopLp5aCpx/k8zFg9vxGTvePCQeFeQyfF+H5dBvUNK2MJNjqqY3Wb+RHBA1CTFBh0YzjiaCb+svPlJHkMR2x1ENcsHZHuyHQSQpyzClW2FxuVs+WJ3zEa2WARAvN80= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171815; c=relaxed/simple; bh=nw2KAMga+N9SBRpuwZhMNhiboc0xudR9Yyh15QBkKfY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ETU3nbCm3bLaXQhKeJALG0UJw8PnEgb5TKPmKUzrz8phpEAr8psSMJxjMjsGgr8I1Wht7PEXiZfbifubBl3T5c9yhgIOSoCcQpr0wzVlpFlx1Xg1u+3wvieWy087gv3BgdSMBq5eh0pYYfTcoSDZxIjH0CKovAXXA5WcUQTrWEw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=pl/50XDE; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="pl/50XDE" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a7a81bd549eso466830066b.3 for ; Tue, 20 Aug 2024 09:36:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171812; x=1724776612; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E2cahoYdlsFx3NF9sR068SfEnyVTIFV9fZk/j7POxiM=; b=pl/50XDENJksbaqzw4PK5B0o7XTiHGuKR+mRP5tKuYaEta1OmQv7i/eD5lK6XkHlDQ a/qfJSbzn+aezmGgu3sBOB/ghRpQgJSjHVXk7XrxOWGCHsxT4g7tS0RyKAJh4qfNnDZe MrNA2HLqNIk9236txVQ7FNfsYLglwn9dBCOt0xhSRwP02jP7XJZzLfjxPYhqbbUttXdt fs61SypRsU8SK/DY47EIbeDI9VXXP3Tj0GH0UdrNKLRgpaneR22f3Sj4BGRC8ahMKkQ1 PXNEaT3SxDXRqfxmSSiF8oqyCp06nslixr/cvoqlHye8xbUUc+TdxdL2vQRNt7xk+TMG r8uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171812; x=1724776612; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E2cahoYdlsFx3NF9sR068SfEnyVTIFV9fZk/j7POxiM=; b=mEzptgw3W790Mq+ZjMq0Xy9oBToRhdMYqfJ06aLhKCIP6Xwdi8SgApNOBEQelEgyia XYvhOPhF2tSwUMVkinDOkwx1cGuwLtW9M3fESRrUCPHs6BEsfzXTarOaaUFzJU4Sk247 qRbySit1eyQ0u6Nt7ZkN4TGGs4mfu9gzFm2eiHdixDOJfrL55VtVu6jtre2mKAEDpddF K7b99Aw9lUR9uQ1owgbG/y+bvfiEsr7svyhkOXFMQkoYBt9U1XElmAmAwV2AOF4sqkDz K5WcJwuuqVg4OXrZ+737mDtQn1ayINeaCncgsXZRjdBYl586fvq8eLOEmyO1J+cGUWc/ PUvQ== X-Forwarded-Encrypted: i=1; AJvYcCXgr2vFwBI/jTHWIHxGAP9s2cYQ8pwEMKWrcScb7hh+II+lFxW9H4fovEhRgExYluQR4LQ8yT0KR5rrkAM=@vger.kernel.org X-Gm-Message-State: AOJu0Ywn/QR4hMtO8Vq6D986mIcs8tndXNk2PLEdm1XMP2hVVG0LVxK4 AmAp9BoVJljQYsh22pJ02vmYji5ekqCybnIt/Y/pT+7+gbXUE6yJoP44zAaueWE= X-Google-Smtp-Source: AGHT+IGYJnTxET1FI2CgmIVhGwHAm0Rvsu+OSAWgb5x2sIlIex+roRD/rs+GaRWocmTPkLXygBK5rw== X-Received: by 2002:a17:907:2d1e:b0:a7a:a557:454b with SMTP id a640c23a62f3a-a8392a47a85mr998867766b.66.1724171811750; Tue, 20 Aug 2024 09:36:51 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.36.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:51 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 15/16] sched/fair: Enable disabling util_est via rampup_multiplier Date: Tue, 20 Aug 2024 17:35:11 +0100 Message-Id: <20240820163512.1096301-16-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" util_est is a great feature to enable busy tasks with long sleep time to maintain their perf level. But it can also be expensive in terms of power for tasks that have no such perf requirements and just happened to be busy in the last activation. If a task sets its rampup_multiplier to 0, then it indicates that it is happy to glide along with system default response and doesn't require responsiveness. We can use that to further imply that the task is happy to decay its util for long sleep too and disable util_est. XXX: This could be overloading this QoS. We could add a separate more explicit QoS to disable util_est for tasks that don't care. Signed-off-by: Qais Yousef --- kernel/sched/fair.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a8dbba0b755e..ad72db5a266c 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4918,6 +4918,14 @@ static inline void util_est_update(struct cfs_rq *cf= s_rq, if (!sched_feat(UTIL_EST)) return; =20 + /* + * rampup_multiplier =3D 0 indicates util_est is disabled. + */ + if (!p->sched_qos.rampup_multiplier) { + ewma =3D 0; + goto done; + } + /* Get current estimate of utilization */ ewma =3D READ_ONCE(p->se.avg.util_est); =20 --=20 2.34.1 From nobody Mon Feb 9 05:01:34 2026 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 983A81C4635 for ; Tue, 20 Aug 2024 16:36:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171816; cv=none; b=pyLQmF2xfR+5PNUgR/yUv3qWz2oI2fG0UbXf5QEasIiyTjhBQh7RA4GKKCJfqqkNVGda8OC7TS8O2+kqNpl8MNVnFXWimK2+42bXrt7HFNk1mBT1yec2ee4xj8SOjlyx35Iazkm/X4njPRH92hw7QrnPdYtgaAG2aMkXr6DYgeQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724171816; c=relaxed/simple; bh=FcQoU+XSuInJQQOSIHfVyJzGsFgsU0Fz0b3FqV2urj0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oCbAG6WiGjH2cGp4fkNRKyc0QKft/qRcA4KADYl9QykEYzSty5H49o9Ijqzd1v+uDS9YZzOtwz3kNxRBoRzVPGhJGfZtcZREgeqD9BkSMZTMDiP2+6UGJ0EFNJKu09183PWvbKhKinkcSTngv63u+6UpYGYgxyX+jLT++55REMg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=Fsybhqe7; arc=none smtp.client-ip=209.85.218.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="Fsybhqe7" Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-a80eab3945eso584643666b.1 for ; Tue, 20 Aug 2024 09:36:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1724171813; x=1724776613; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tc420hGJWxSmQzyq9r4KlOxaBiNHLqomnZxiY/aUtfM=; b=Fsybhqe7muuGhbmgEdTCITd2P88EV0KF/kH75xdvPkU8GWKNa07xoXRRDpOMicwFwi vnw0Q2B/jcxC1RuJO/XDYFVkPxJsw/xaCoeRULoBrYZlMW752AMBar6kGPqf6FFNedhL Lu72rWH9TFTi0KkxHwvdyNfht1QcpE/GV6jLq+LKsijg8xXCeG9VVJFUz2dyysQNFAs6 5kWy52wB6U6vmT+hkRIJITVzIcGFPQfHxRuWMV9PbqxrGRtmHqYL8Y/i96xKFcEoiDTo KV96gXw6UIQj1afHF05y+JLNaQQpNLM9+ipA9I/wFwlsot+O10899CQnR/nTDqbLry7a EuWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724171813; x=1724776613; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tc420hGJWxSmQzyq9r4KlOxaBiNHLqomnZxiY/aUtfM=; b=teeFSwato2zTHCmlDEpIA/OfQDwwUf3UmwsBFuZTPWTKI0G/Na87bBpDBPF644+Qds JtixOSi9uXzob1mUdqLU39QnnFJB3299Uo51VO9PSPVtGAMRjSSmGcy1nXgwpFHoAAE1 15H0feiUgjDGcqm7lkl4InlnQwVS+Lpj7moXpuHxx3TyZWxJEUHDvfQD2DUOTaXEeozl 2R7vLf0NvIkACSpat9pfaevEVUi9kLnPsycrpEM5Hj5S2uRaFihQ3VbllqmgKk/N+49y K1KMUsmuslzIA9yBoNR1prYHQS3CPp31AQeK4UkuEogvh7fxVntVxD/ypDNOzYPU/8Ad nKWA== X-Forwarded-Encrypted: i=1; AJvYcCVZen3PgGxuCz0Vo/Y4G4v1rqMYnCZdhKn/ueo/f79e4/lhATUFoaYkOp2e4JNlH4aiNL/kIHFMrcgYQ+7Dyg9miu0cqAkqabL/lClT X-Gm-Message-State: AOJu0YxgHZMLU4Xm12EP61CKDzi2ItB4SgaExzh0lXSuiZTmbvhoj4o+ zybmVCQ9lyJaWS1IhQj67IArdRrZSptzWPWKCea0hSfXpzq1VgTHeS8B67WK9e8= X-Google-Smtp-Source: AGHT+IFrFBtEWI8G3mIhZShOA/h8coVH75w3HUcYb7DxNvB76QnrvRr68I5ag+1Mh1T6RnbU58v1zg== X-Received: by 2002:a17:907:d581:b0:a7a:8cb9:7490 with SMTP id a640c23a62f3a-a8392a15b30mr1057699366b.47.1724171812633; Tue, 20 Aug 2024 09:36:52 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a83838cfb5esm780728766b.59.2024.08.20.09.36.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Aug 2024 09:36:52 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , "Rafael J. Wysocki" , Viresh Kumar Cc: Juri Lelli , Steven Rostedt , Dietmar Eggemann , John Stultz , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Qais Yousef Subject: [RFC PATCH 16/16] sched/fair: Don't mess with util_avg post init Date: Tue, 20 Aug 2024 17:35:12 +0100 Message-Id: <20240820163512.1096301-17-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240820163512.1096301-1-qyousef@layalina.io> References: <20240820163512.1096301-1-qyousef@layalina.io> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The extrapolation logic for util_avg for newly forked tasks tries to crystal ball the task's demand. This has worked well when the system didn't have the means to help these tasks otherwise. But now we do have util_est that will rampup faster. And uclamp_min to ensure a good starting point if they really care. Since we really can't crystal ball the behavior, and giving the same starting value for all tasks is more consistent behavior for all forked tasks, and it helps to preserve system resources for tasks to compete to get them if they truly care, set the initial util_avg to be 0 when util_est feature is enabled. This should not impact workloads that need best single threaded performance (like geekbench) given the previous improvements introduced to help with faster rampup to reach max perf point more coherently and consistently across systems. Signed-off-by: Qais Yousef --- kernel/sched/fair.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index ad72db5a266c..45be77d1112f 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1031,6 +1031,19 @@ void init_entity_runnable_average(struct sched_entit= y *se) } =20 /* + * When util_est is used, the tasks can rampup much faster by default. And= with + * the rampup_multiplier, tasks can ask for faster rampup after fork. And = with + * uclamp, they can ensure a min perf requirement. Given all these factors= , we + * keep util_avg at 0 as we can't crystal ball the task demand after fork. + * Userspace have enough ways to ensure good perf for tasks after fork. Ke= eping + * the util_avg to 0 is good way to ensure a uniform start for all tasks. = And + * it is good to preserve precious resources. Truly busy forked tasks can + * compete for the resources without the need for initial 'cheat' to ramp = them + * up automagically. + * + * When util_est is not present, the extrapolation logic below will still + * apply. + * * With new tasks being created, their initial util_avgs are extrapolated * based on the cfs_rq's current util_avg: * @@ -1080,6 +1093,12 @@ void post_init_entity_util_avg(struct task_struct *p) return; } =20 + /* + * Tasks can rampup faster with util_est, so don't mess with util_avg. + */ + if (sched_feat(UTIL_EST)) + return; + if (cap > 0) { if (cfs_rq->avg.util_avg !=3D 0) { sa->util_avg =3D cfs_rq->avg.util_avg * se_weight(se); --=20 2.34.1