From nobody Tue Dec 2 02:05:20 2025 Received: from fra-out-001.esa.eu-central-1.outbound.mail-perimeter.amazon.com (fra-out-001.esa.eu-central-1.outbound.mail-perimeter.amazon.com [18.156.205.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 468AC30E822 for ; Thu, 20 Nov 2025 10:20:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=18.156.205.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763634044; cv=none; b=JMs3Ez/M1dcOaQFM77VEImUBeK95jxfOyjyTntTLjANmSxqyTq1fi2hbUOZU6dcdWFPJOKVvWcCPH7EsJBeMqRfiJu8/VDg/d4Uo2Ococ8rQjkbGsNwB9Z0pRfra3EJwdHr4XD4mazBoG3cZAZ8Vo81pbx1mNwUfoBvP4USi8Jo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763634044; c=relaxed/simple; bh=BA3IJ5aD6ZoWuhWvwUrCoOrq+Yz8RcZtews/XE0QYp4=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=P31q5rdGDTEpGZ2XQvGNtdvyN2xtqDE8QwKRqN9DPbAtcNRyDHd5o+NNb2ztgQ/zCr4igYFFaRk63sMww3E/m/qWyuOAYKWB+SbBagcrK+qyuEN/1wIF05Nk0CxekIpT+LUjNHXQ3blOXuo9Rnkn4FXrsINA2MRils8PNShOxm0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (2048-bit key) header.d=amazon.com header.i=@amazon.com header.b=qrVn9KpX; arc=none smtp.client-ip=18.156.205.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=amazon.com header.i=@amazon.com header.b="qrVn9KpX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazoncorp2; t=1763634041; x=1795170041; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=MdZlWQF+vpNTUbqWTd9r2ZBYcrW8xT2Hc0LSPhqCPMo=; b=qrVn9KpXy5H0GqTFsFDw2wUMrT8qYRSksIw8fggHKN28BclSye2MoSIf AV2F+Chtr3oe2uCsgYBprMRtqDTTVVcfv3YC5bkbpQkkMIAL38gTObDZV WMup123sI8+jgvfZpzR59aAuQji+DEPJTcetdPdMK4ytHE8BmwWqshydG AV2aArrzEoBBgpCjWogO1R/1EDg7Jn7KWOFiTozqgKbc36YvouHhODvj6 g3Z/YC4BkZhLht8I2avaal4wfhTgUG0QPkURKcoEv2FojFkOIJEfEiRcL SUvpgM+AyQlREgQVMLic4BxH+CJ6hboZxNv+VR5QHJQXWwgE2UtpcuBXP A==; X-CSE-ConnectionGUID: zihFw9X3QpmXh1m4Q+0LhA== X-CSE-MsgGUID: o25NdanpQi+3wujguAO7vg== X-IronPort-AV: E=Sophos;i="6.19,317,1754956800"; d="scan'208";a="5503721" Received: from ip-10-6-6-97.eu-central-1.compute.internal (HELO smtpout.naws.eu-central-1.prod.farcaster.email.amazon.dev) ([10.6.6.97]) by internal-fra-out-001.esa.eu-central-1.outbound.mail-perimeter.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2025 10:20:22 +0000 Received: from EX19MTAEUA002.ant.amazon.com [54.240.197.232:4964] by smtpin.naws.eu-central-1.prod.farcaster.email.amazon.dev [10.0.42.72:2525] with esmtp (Farcaster) id 4e4e30a5-5e3b-416e-b6b9-52acfeecb970; Thu, 20 Nov 2025 10:20:22 +0000 (UTC) X-Farcaster-Flow-ID: 4e4e30a5-5e3b-416e-b6b9-52acfeecb970 Received: from EX19D003EUB001.ant.amazon.com (10.252.51.97) by EX19MTAEUA002.ant.amazon.com (10.252.50.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.29; Thu, 20 Nov 2025 10:20:21 +0000 Received: from u5934974a1cdd59.ant.amazon.com (10.1.212.9) by EX19D003EUB001.ant.amazon.com (10.252.51.97) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.29; Thu, 20 Nov 2025 10:20:13 +0000 From: Fernand Sieber To: , CC: , , , , , , , , , , , Subject: [PATCH] sched/fair: Add more core cookie check in wake up fast path Date: Thu, 20 Nov 2025 12:19:55 +0200 Message-ID: <20251120101955.968586-1-sieberf@amazon.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: EX19D043UWC003.ant.amazon.com (10.13.139.240) To EX19D003EUB001.ant.amazon.com (10.252.51.97) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The fast path in select_idle_sibling() can place tasks on CPUs without considering core scheduling constraints, potentially causing immediate force idle when the sibling runs an incompatible task. Add cookie compatibility checks before selecting a CPU in the fast path. This prevents placing waking tasks on CPUs where the sibling is running an incompatible task, reducing force idle occurrences. Testing =3D=3D=3D=3D=3D=3D=3D Perf testing using 10 threads on 8 CPUs on a platform with each core sporting two hyperthreads. Each thread is running 1200 iterations of a cycle consisting of a random period of work between 0-10ms followed by a random period of sleep 0-30ms. The goal of this configuration is to apply a light load on the system (33%), with randomization causing opportunities for scheduling placement including all cases of idle cores, partially idle cores and fully busy cores. Each test configuration is run 3 times and time to work completion is measured and averaged. First configuration doesn't use cookies, second configuration groups the 10 threads in 5 pairs of cookies, third configuration also groups the 10 threads in 5 pairs of cookies, but with this patch applied. Test Results (seconds) Configuration Run 1 Run 2 Run 3 Average Overhead No cookie 24.653 24.454 24.586 24.564 0.000 Cookie 25.896 26.256 25.979 26.044 1.480 Cookie + patch 25.346 25.007 25.042 25.132 0.568 The patch reduces cookie overhead by 61.7% (0.568/1.480 =3D 0.383) Signed-off-by: Fernand Sieber Reviewed-by: K Prateek Nayak --- kernel/sched/fair.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 5b752324270b..90ceb3da2251 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7647,7 +7647,7 @@ static int select_idle_smt(struct task_struct *p, str= uct sched_domain *sd, int t */ if (!cpumask_test_cpu(cpu, sched_domain_span(sd))) continue; - if (available_idle_cpu(cpu) || sched_idle_cpu(cpu)) + if (__select_idle_cpu(cpu, p) !=3D -1) return cpu; } @@ -7841,6 +7841,7 @@ static int select_idle_sibling(struct task_struct *p,= int prev, int target) lockdep_assert_irqs_disabled(); if ((available_idle_cpu(target) || sched_idle_cpu(target)) && + sched_core_cookie_match(cpu_rq(target), p) && asym_fits_cpu(task_util, util_min, util_max, target)) return target; @@ -7849,6 +7850,7 @@ static int select_idle_sibling(struct task_struct *p,= int prev, int target) */ if (prev !=3D target && cpus_share_cache(prev, target) && (available_idle_cpu(prev) || sched_idle_cpu(prev)) && + sched_core_cookie_match(cpu_rq(prev), p) && asym_fits_cpu(task_util, util_min, util_max, prev)) { if (!static_branch_unlikely(&sched_cluster_active) || @@ -7881,6 +7883,7 @@ static int select_idle_sibling(struct task_struct *p,= int prev, int target) recent_used_cpu !=3D target && cpus_share_cache(recent_used_cpu, target) && (available_idle_cpu(recent_used_cpu) || sched_idle_cpu(recent_used_cp= u)) && + sched_core_cookie_match(cpu_rq(recent_used_cpu), p) && cpumask_test_cpu(recent_used_cpu, p->cpus_ptr) && asym_fits_cpu(task_util, util_min, util_max, recent_used_cpu)) { -- 2.34.1 Amazon Development Centre (South Africa) (Proprietary) Limited 29 Gogosoa Street, Observatory, Cape Town, Western Cape, 7925, South Africa Registration Number: 2004 / 034463 / 07