From nobody Wed Sep 10 09:29:27 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19D5BC0015E for ; Tue, 1 Aug 2023 19:41:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232583AbjHATlW (ORCPT ); Tue, 1 Aug 2023 15:41:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229707AbjHATlQ (ORCPT ); Tue, 1 Aug 2023 15:41:16 -0400 Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DE1919A8; Tue, 1 Aug 2023 12:41:13 -0700 (PDT) Received: from localhost (127.0.0.1) (HELO v370.home.net.pl) by /usr/run/smtp (/usr/run/postfix/private/idea_relay_lmtp) via UNIX with SMTP (IdeaSmtpServer 5.2.0) id 2278a89ce16b93ab; Tue, 1 Aug 2023 21:41:12 +0200 Authentication-Results: v370.home.net.pl; spf=softfail (domain owner discourages use of this host) smtp.mailfrom=rjwysocki.net (client-ip=195.136.19.94; helo=[195.136.19.94]; envelope-from=rjw@rjwysocki.net; receiver=) Received: from kreacher.localnet (unknown [195.136.19.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by v370.home.net.pl (Postfix) with ESMTPSA id 77F806621DD; Tue, 1 Aug 2023 21:41:11 +0200 (CEST) From: "Rafael J. Wysocki" To: Linux PM , Peter Zijlstra , Anna-Maria Behnsen Cc: LKML , Frederic Weisbecker , Kajetan Puchalski Subject: [RFC/RFT][PATCH v1 1/2] cpuidle: teo: Do not call tick_nohz_get_sleep_length() upfront Date: Tue, 01 Aug 2023 21:39:42 +0200 Message-ID: <1870608.tdWV9SEqCh@kreacher> In-Reply-To: <4511619.LvFx2qVVIh@kreacher> References: <4511619.LvFx2qVVIh@kreacher> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CLIENT-IP: 195.136.19.94 X-CLIENT-HOSTNAME: 195.136.19.94 X-VADE-SPAMSTATE: clean X-VADE-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedviedrjeeigddufeeiucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecujffqoffgrffnpdggtffipffknecuuegrihhlohhuthemucduhedtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufffkfgjfhgggfgtsehtufertddttdejnecuhfhrohhmpedftfgrfhgrvghlucflrdcuhgihshhotghkihdfuceorhhjfiesrhhjfiihshhotghkihdrnhgvtheqnecuggftrfgrthhtvghrnhepvdffueeitdfgvddtudegueejtdffteetgeefkeffvdeftddttdeuhfegfedvjefhnecukfhppeduleehrddufeeirdduledrleegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepudelhedrudefiedrudelrdelgedphhgvlhhopehkrhgvrggthhgvrhdrlhhotggrlhhnvghtpdhmrghilhhfrhhomhepfdftrghfrggvlhculfdrucghhihsohgtkhhifdcuoehrjhifsehrjhifhihsohgtkhhirdhnvghtqedpnhgspghrtghpthhtohepiedprhgtphhtthhopehlihhnuhigqdhpmhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehpvghtvghriiesihhnfhhrrgguvggrugdrohhrghdprhgtphhtthhopegrnhhnrgdqmhgrrhhirgeslhhinhhuthhrohhnihigrdguvgdprhgtphhtthhopehlihhnuhigqdhkvghrnhgvlhesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehfrhgv uggvrhhitgeskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepkhgrjhgvthgrnhdrphhutghhrghlshhkihesrghrmhdrtghomh X-DCC--Metrics: v370.home.net.pl 1024; Body=6 Fuz1=6 Fuz2=6 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Rafael J. Wysocki Because the cost of calling tick_nohz_get_sleep_length() may increase in the future, reorder the code in teo_select() so it first uses the statistics to pick up a candidate idle state and applies the utilization heuristic to it and only then calls tick_nohz_get_sleep_length() to obtain the sleep length value and refine the selection if necessary. This change by itself does not cause tick_nohz_get_sleep_length() to be called less often, but it prepares the code for subsequent changes that will do so. Signed-off-by: Rafael J. Wysocki --- drivers/cpuidle/governors/teo.c | 102 ++++++++++++++++-------------------= ----- 1 file changed, 43 insertions(+), 59 deletions(-) Index: linux-pm/drivers/cpuidle/governors/teo.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- linux-pm.orig/drivers/cpuidle/governors/teo.c +++ linux-pm/drivers/cpuidle/governors/teo.c @@ -306,15 +306,10 @@ static void teo_update(struct cpuidle_dr cpu_data->total +=3D PULSE; } =20 -static bool teo_time_ok(u64 interval_ns) +static bool teo_state_ok(int i, struct cpuidle_driver *drv) { - return !tick_nohz_tick_stopped() || interval_ns >=3D TICK_NSEC; -} - -static s64 teo_middle_of_bin(int idx, struct cpuidle_driver *drv) -{ - return (drv->states[idx].target_residency_ns + - drv->states[idx+1].target_residency_ns) / 2; + return !tick_nohz_tick_stopped() || + drv->states[i].target_residency_ns >=3D TICK_NSEC; } =20 /** @@ -354,6 +349,7 @@ static int teo_select(struct cpuidle_dri { struct teo_cpu *cpu_data =3D per_cpu_ptr(&teo_cpus, dev->cpu); s64 latency_req =3D cpuidle_governor_latency_req(dev->cpu); + ktime_t delta_tick =3D TICK_NSEC / 2; unsigned int idx_intercept_sum =3D 0; unsigned int intercept_sum =3D 0; unsigned int idx_recent_sum =3D 0; @@ -363,7 +359,6 @@ static int teo_select(struct cpuidle_dri int constraint_idx =3D 0; int idx0 =3D 0, idx =3D -1; bool alt_intercepts, alt_recent; - ktime_t delta_tick; bool cpu_utilized; s64 duration_ns; int i; @@ -374,9 +369,11 @@ static int teo_select(struct cpuidle_dri } =20 cpu_data->time_span_ns =3D local_clock(); - - duration_ns =3D tick_nohz_get_sleep_length(&delta_tick); - cpu_data->sleep_length_ns =3D duration_ns; + /* + * Set the expected sleep length to infinity in case of an early + * return. + */ + cpu_data->sleep_length_ns =3D KTIME_MAX; =20 /* Check if there is any choice in the first place. */ if (drv->state_count < 2) { @@ -384,11 +381,8 @@ static int teo_select(struct cpuidle_dri goto out_tick; } =20 - if (!dev->states_usage[0].disable) { + if (!dev->states_usage[0].disable) idx =3D 0; - if (drv->states[1].target_residency_ns > duration_ns) - goto out_tick; - } =20 cpu_utilized =3D teo_cpu_is_utilized(dev->cpu, cpu_data); /* @@ -398,7 +392,7 @@ static int teo_select(struct cpuidle_dri */ if (drv->state_count < 3 && cpu_utilized) { /* The CPU is utilized, so assume a short idle duration. */ - duration_ns =3D teo_middle_of_bin(0, drv); + duration_ns =3D drv->states[1].target_residency_ns / 2; /* * If state 0 is enabled and it is not a polling one, select it * right away unless the scheduler tick has been stopped, in @@ -408,7 +402,7 @@ static int teo_select(struct cpuidle_dri * anyway. */ if ((!idx && !(drv->states[0].flags & CPUIDLE_FLAG_POLLING) && - teo_time_ok(duration_ns)) || dev->states_usage[1].disable) { + teo_state_ok(0, drv)) || dev->states_usage[1].disable) { idx =3D 0; goto out_tick; } @@ -417,13 +411,7 @@ static int teo_select(struct cpuidle_dri goto end; } =20 - /* - * Find the deepest idle state whose target residency does not exceed - * the current sleep length and the deepest idle state not deeper than - * the former whose exit latency does not exceed the current latency - * constraint. Compute the sums of metrics for early wakeup pattern - * detection. - */ + /* Compute the sums of metrics for early wakeup pattern detection. */ for (i =3D 1; i < drv->state_count; i++) { struct teo_bin *prev_bin =3D &cpu_data->state_bins[i-1]; struct cpuidle_state *s =3D &drv->states[i]; @@ -439,19 +427,15 @@ static int teo_select(struct cpuidle_dri if (dev->states_usage[i].disable) continue; =20 - if (idx < 0) { - idx =3D i; /* first enabled state */ - idx0 =3D i; - } - - if (s->target_residency_ns > duration_ns) - break; + if (idx < 0) + idx0 =3D i; /* first enabled state */ =20 idx =3D i; =20 if (s->exit_latency_ns <=3D latency_req) constraint_idx =3D i; =20 + /* Save the sums for the current state. */ idx_intercept_sum =3D intercept_sum; idx_hit_sum =3D hit_sum; idx_recent_sum =3D recent_sum; @@ -479,13 +463,11 @@ static int teo_select(struct cpuidle_dri * all of the deeper states, or the sum of the numbers of recent * intercepts over all of the states shallower than the candidate one * is greater than a half of the number of recent events taken into - * account, the CPU is likely to wake up early, so find an alternative - * idle state to select. + * account, a shallower idle state is likely to be a better choice. */ alt_intercepts =3D 2 * idx_intercept_sum > cpu_data->total - idx_hit_sum; alt_recent =3D idx_recent_sum > NR_RECENT / 2; if (alt_recent || alt_intercepts) { - s64 first_suitable_span_ns =3D duration_ns; int first_suitable_idx =3D idx; =20 /* @@ -494,44 +476,39 @@ static int teo_select(struct cpuidle_dri * cases (both with respect to intercepts overall and with * respect to the recent intercepts only) in the past. * - * Take the possible latency constraint and duration limitation - * present if the tick has been stopped already into account. + * Take the possible duration limitation present if the tick + * has been stopped already into account. */ intercept_sum =3D 0; recent_sum =3D 0; =20 for (i =3D idx - 1; i >=3D 0; i--) { struct teo_bin *bin =3D &cpu_data->state_bins[i]; - s64 span_ns; =20 intercept_sum +=3D bin->intercepts; recent_sum +=3D bin->recent; =20 - span_ns =3D teo_middle_of_bin(i, drv); - if ((!alt_recent || 2 * recent_sum > idx_recent_sum) && (!alt_intercepts || 2 * intercept_sum > idx_intercept_sum)) { - if (teo_time_ok(span_ns) && - !dev->states_usage[i].disable) { + /* + * Use the current state unless it is too + * shallow or disabled, in which case take the + * first enabled state that is deep enough. + */ + if (teo_state_ok(i, drv) && + !dev->states_usage[i].disable) idx =3D i; - duration_ns =3D span_ns; - } else { - /* - * The current state is too shallow or - * disabled, so take the first enabled - * deeper state with suitable time span. - */ + else idx =3D first_suitable_idx; - duration_ns =3D first_suitable_span_ns; - } + break; } =20 if (dev->states_usage[i].disable) continue; =20 - if (!teo_time_ok(span_ns)) { + if (!teo_state_ok(i, drv)) { /* * The current state is too shallow, but if an * alternative candidate state has been found, @@ -543,7 +520,6 @@ static int teo_select(struct cpuidle_dri break; } =20 - first_suitable_span_ns =3D span_ns; first_suitable_idx =3D i; } } @@ -562,14 +538,22 @@ static int teo_select(struct cpuidle_dri * not sufficiently large. */ if (cpu_utilized) { - s64 span_ns; + i =3D teo_find_shallower_state(drv, dev, idx, KTIME_MAX, true); + if (teo_state_ok(i, drv)) + idx =3D i; + } =20 - i =3D teo_find_shallower_state(drv, dev, idx, duration_ns, true); - span_ns =3D teo_middle_of_bin(i, drv); - if (teo_time_ok(span_ns)) { + duration_ns =3D tick_nohz_get_sleep_length(&delta_tick); + cpu_data->sleep_length_ns =3D duration_ns; + + /* + * If the closest expected timer is before the terget residency of the + * candidate state, a shallower one needs to be found. + */ + if (drv->states[idx].target_residency_ns > duration_ns) { + i =3D teo_find_shallower_state(drv, dev, idx, duration_ns, false); + if (teo_state_ok(i, drv)) idx =3D i; - duration_ns =3D span_ns; - } } =20 end: