From nobody Wed Apr 8 12:36:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C649C6FA82 for ; Sat, 10 Sep 2022 10:54:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229547AbiIJKyR (ORCPT ); Sat, 10 Sep 2022 06:54:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229541AbiIJKyP (ORCPT ); Sat, 10 Sep 2022 06:54:15 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2045.outbound.protection.outlook.com [40.107.223.45]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B446475498 for ; Sat, 10 Sep 2022 03:54:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PyBaSELu43LhtNyplQaJbp9k/6PQyhyXf/Kx6KDxCSCnyjiaPfzACYLkYd5OX0uZHD2YX/bBJqRJAd30KKbNf9MmOnlxWUbH/ROpZ50qJSCGm8uQ0A4QriQJ1N9353zkSND4AgAEsbW6lV3eh++20PcADk6EnQdKgvOiMiLZLCRDfrFQCS5Kp/jR3ffGmhOlaHCeL2gow06JgBNtxnlOFxdHlL/Vr/HbiAswicgwVL99nb7TM0uFttKrwbUuHGnbgrxRlJjNo6wNhL71Ka3KpzSy3a/loSoGQ5I/Go7512lZ35zsP8gEMvGYX3Zj+meYJlF74mpJOKwvlc+70uT1ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VyfB89CyX9zg46zzrPCIK7yeupDiGt/ISg+1JvemhQw=; b=eMfON8e69T1GzTXPD73N782GEu0ljhea+JyCMpjoaH/BRHvWoySGE85ap4PdVhKv76M2Afo/YrJ/mqPuNr5oY0uq4GvHHIVRTCc9MAhhPjKTR+B1IaJFpwyoe2XcPLY0Vxj6oOTWJJ1fXAUybL8Ht2CsW+uorPG/muCHis0QxsPDk/YqdYv2mpZmkHoKleFFwDf5Z0buDBSXx8X+WdBbrahzj7ucTuxnC97KKuXH6ISv6ci5/m1RdFz1IuAGGkTgJ8KfAGhVd2QoLfdIXbkd5DTmeoOJvK/rOZMWWacon5kQktQztBknooeuTuiNAQG0UOTMewG/+myYLC2uiWdPpw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VyfB89CyX9zg46zzrPCIK7yeupDiGt/ISg+1JvemhQw=; b=d5v+DfguYhQDUt0vhCYV1eENHPk0IcjLCTH8ebaVWJmvzcNBYR+BQNbqcjniw3/5HScS0K6JdwVq8yvMKvSPE6cdf2jX+kZ71IxI8P7IoNoba/R8kSNdXYQlD5gpVffeIAnNqTuxlHW02qA1TlE5mxIbgppAXDg1JTRx+t7BhrM= Received: from DM6PR04CA0003.namprd04.prod.outlook.com (2603:10b6:5:334::8) by PH7PR12MB5806.namprd12.prod.outlook.com (2603:10b6:510:1d2::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.17; Sat, 10 Sep 2022 10:54:12 +0000 Received: from DM6NAM11FT024.eop-nam11.prod.protection.outlook.com (2603:10b6:5:334:cafe::7f) by DM6PR04CA0003.outlook.office365.com (2603:10b6:5:334::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12 via Frontend Transport; Sat, 10 Sep 2022 10:54:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT024.mail.protection.outlook.com (10.13.172.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.13 via Frontend Transport; Sat, 10 Sep 2022 10:54:11 +0000 Received: from BLR5CG134614W.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Sat, 10 Sep 2022 05:54:05 -0500 From: K Prateek Nayak To: CC: , , , , , , , , , , , , , , , Subject: [PATCH 1/5] task_struct: Add field for per-task userspace scheduler hint Date: Sat, 10 Sep 2022 16:23:22 +0530 Message-ID: <20220910105326.1797-2-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220910105326.1797-1-kprateek.nayak@amd.com> References: <20220910105326.1797-1-kprateek.nayak@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT024:EE_|PH7PR12MB5806:EE_ X-MS-Office365-Filtering-Correlation-Id: 28ddc50b-b5af-414f-32eb-08da931acb6f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ptiPvvitnscJu3qSXrptfxRo3miqB/xZBMp0t8VaCdD8rsyX9oEHh2fH4k4UyH6EVcH3teDt/vgF32/EhoHoOYl3EWYaT2+qL0kZWQPke/EVr+xEIVn/K7Zf9fTeshTTRrnJqIjFaLE+SFfmQRexDC/Gup3mZF90lEoI36kXIH9acR4EBgWaa7LngVLZIRil76MU1PQtoM7PpKg9jcuvImK1O1caiLr/IGDGa7sLE9j3C0OsJXIQMluHroxYyLJjrm3ga12Km6wk5HUHdGWc0uMeFhwbRGG7MYpKkyooeiSIe5GSMTYHSYo/nPZR9qQ5QrWe2djV7LHB19vQaJwOigHq7MjJL0ZiM/ydDcddpm4O38v6zktR+9z5SHUty2cYTDzRfoJktc+Poi6N2XyDbh7C9oTKCa4CYJbqzd0I8KVYaJb7wJGEF9T2+4V7Fv96haAh9GSyS7feiC8nPLygbQIQg5XOz5GM8qGlfFfObm+MECxh0i+FEitMHs4+p4shGNASiZFt0LZJilNVis+z0nDyvRorVl24VUJMzWZG/N6c00vVWZO5vxNr6bgK8/81BFQL4+Tw8PByqQnW90i/ghBZ/xX9K+dZYfoo8byRvpXxWDclREuXVHUJunTXJl9GL51s2j5jCMB1i3x8kFMrXvZdhaR2B/n3tVUJnW4M0qCx9icrg5gj2XG0HWYw7MWrYsqL0H9zHZQr3nA8XhVJTQfFtQJNZztuiFXCymFKHozXEaaBzqwBmifKYtdmBhiibLLpK04yWC/JWTk84/ZgsxSfCHHqYpO42xJqQiHM+kw1ifoGYX4LNAKn585kjn+B X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(136003)(396003)(376002)(46966006)(40470700004)(36840700001)(86362001)(81166007)(186003)(47076005)(336012)(36860700001)(6666004)(7696005)(2616005)(16526019)(478600001)(26005)(1076003)(426003)(70586007)(5660300002)(8936002)(40480700001)(41300700001)(4326008)(7416002)(8676002)(2906002)(70206006)(36756003)(82310400005)(356005)(316002)(82740400003)(54906003)(40460700003)(6916009)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2022 10:54:11.6747 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 28ddc50b-b5af-414f-32eb-08da931acb6f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT024.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5806 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" - Add a field named "hint" to store the per-task userspace scheduler hint set via the prctl() interface. - Hints are inherited from parent on fork. Inherited hints can be reset from the same prctl() interface used to set hint to default or any other value. - Hints are best effort case and scheduler doesn't make any guarantees of strictly adhering to it. Signed-off-by: K Prateek Nayak --- include/linux/sched.h | 1 + init/init_task.c | 1 + 2 files changed, 2 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index e7b2f8a5c711..fc953c9e956a 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -769,6 +769,7 @@ struct task_struct { int wake_cpu; #endif int on_rq; + unsigned int hint; =20 int prio; int static_prio; diff --git a/init/init_task.c b/init/init_task.c index ff6c4b9bfe6b..f5208e6a1934 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -79,6 +79,7 @@ struct task_struct init_task .static_prio =3D MAX_PRIO - 20, .normal_prio =3D MAX_PRIO - 20, .policy =3D SCHED_NORMAL, + .hint =3D 0, .cpus_ptr =3D &init_task.cpus_mask, .user_cpus_ptr =3D NULL, .cpus_mask =3D CPU_MASK_ALL, --=20 2.25.1 From nobody Wed Apr 8 12:36:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD5A5C6FA82 for ; Sat, 10 Sep 2022 10:54:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229558AbiIJKyq (ORCPT ); Sat, 10 Sep 2022 06:54:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229541AbiIJKyn (ORCPT ); Sat, 10 Sep 2022 06:54:43 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABE0FDF99 for ; Sat, 10 Sep 2022 03:54:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HumKqN8dKiC92tjduLHaiS08BJmTegfUTQhqoDc2We/ZNxK7d/mniMmIVmB2g3VU2gSZat1OqPVZk0SCV3H5VZWkW/TutwCJp7Qk2ryLhncTp03Vq/GpmchFlkPFuftgBzKPu3b4aZHUVCj+POKoYdeSdBycdC8S7DFzjExLoR9yhGMtxVnjGkY5EY3muAtOmE7pIG+7f75dWS0mNAmo6hI22+yhGwK9+Yh38vJ0QQSQRiA8kwA66jto5ZIL+UTwhWuodr8AlzefBsAhIWBPZS94ThxQRggHdNw40hvhY4RCTFwQpCSxjv26RGGXDVx0wY7Dn42Pwg0buZbRwyya7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ulG8kJYPeTFTgoE8ugNXZPJcGjjkZCu97EVgyc/ztfE=; b=Vf3e6AW8jSpfDhztvf9apjWHQuo/hl74uO9RMxi0EqP1HP69IGnml1BrJGjw6w8SRLGqd7bfUZ7/wK6k+VRMyCwOrvtGlR7nwhLW4acQfAVEkDr5nyKsHUmx1QW8BMrOjJ2oWDFvlbkGen2P7AgJR3CqTZrLQ1K49wpeMOOuoWmqDoQj2EClAp1Duy7ZtPe+fbPoI+t9iEDGJCN//bqFXdqkUHZ4Pbs6wbKL/e7yqxlXkPbMDZWbDJxCC1Xc2WzSc46S9e8Gbi8FPe+cJiFQsbxVj54bhOFPB+YbhTEEd8gRrgy/Wbs+D+/cZuoqVoJHKYORO1deNc9dnBtW3NjWsg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ulG8kJYPeTFTgoE8ugNXZPJcGjjkZCu97EVgyc/ztfE=; b=B2QVqrJKNK7Ou6RVGuN1hZUpfVyZT5lx0QrZYVd1UFjgMSmA8sjqlBeloeIo2ZIwreCGIyEJFDv0HdH9qm+Oz/qLMs5RQpAtL8+rIBOxd9XsewwYq85XIUb9zjfCPTRiuCL7lCVpTEA2UiFucz7M7OR50o/Lw8vWpJH7TwizFeA= Received: from DM6PR11CA0058.namprd11.prod.outlook.com (2603:10b6:5:14c::35) by PH7PR12MB6442.namprd12.prod.outlook.com (2603:10b6:510:1fa::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12; Sat, 10 Sep 2022 10:54:38 +0000 Received: from DM6NAM11FT106.eop-nam11.prod.protection.outlook.com (2603:10b6:5:14c:cafe::ad) by DM6PR11CA0058.outlook.office365.com (2603:10b6:5:14c::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.14 via Frontend Transport; Sat, 10 Sep 2022 10:54:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT106.mail.protection.outlook.com (10.13.172.229) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.13 via Frontend Transport; Sat, 10 Sep 2022 10:54:37 +0000 Received: from BLR5CG134614W.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Sat, 10 Sep 2022 05:54:30 -0500 From: K Prateek Nayak To: CC: , , , , , , , , , , , , , , , Subject: [PATCH 2/5] prctl: Add interface and helper functions to set hints Date: Sat, 10 Sep 2022 16:23:23 +0530 Message-ID: <20220910105326.1797-3-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220910105326.1797-1-kprateek.nayak@amd.com> References: <20220910105326.1797-1-kprateek.nayak@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT106:EE_|PH7PR12MB6442:EE_ X-MS-Office365-Filtering-Correlation-Id: 6f505895-8a97-440c-e6e8-08da931adae5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7+EUIFrC7enGeky+N0fdyqKc4JX3Qd+oHdXBaBIU8i5e2ncWPFBrFm7UNgcDHpbDBeq19dOIRUX8veMECD7xX8+aLESR0Y9DjmmiUeu9bgnukAFqY0N1BJamcGOgN8u0RdDi1rKxrdCkB1kfItCKUUrgK2POUPkD6ZYfSCTkdYK//7xRY5dLBiJ7tfZAskcc5pn0KT5c8DVYKW+K2+R+kr2bwKhD85xTQclNCuILvmJxNhvIYdpHWv8LXh+LvNiYvGhitih3tH35gnbYvymRqTmGg2+MVt1WXYc2wkSiAEdbavPrVkLVx1tVia+i8LZksd3ZoH7M56/kTaMJquvMuowR864b1o7anLMlORypOIzn500lnXq1FzfCyf7EzZ/o0vrtviSUWNpAksHBmQ6m0sCSJBCIz+cdyKSJgHLRPyRlEJZVNcCKGTMVvJfy2oJyZrsMxCa6yw6IYLHIGLxcSf9OM/Waz+sVuqvSbnDuJ5LeY0fdPOQB7WjZwvnWUm1Gj7xDl4nhfJlInNwaU60RD/WVHJuoo+Na1IMsHc5CMjGom4cpjVzAVw+stj2naXietV0ayeaWRlIPLHiSlzi8GmVH9MlbFsyCiMaYAuMdcMeim+50hKxQryAMly8yc6DvUBOxE66FEPrJ0datD4cbE4F4vlqNjmPcTHbmyFMR5rU3anZ8uzqMPRtLiKgg/3/9X/VbutuiGiyTq8SVJ4NYGhWCjkdVOsfFDqLTTWEWbInvQO8o+mRFAIMIz+d35X3K+Ojw9xSMonVTvqH/vPRiPjBMtk2We76G9GgWotGREbDGdNfxgKps0yUUKoMinYbQOW3Sj/98q+7OUv+r3QfvBuIaO6k5vfLomZlWrXy+P08= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(396003)(136003)(376002)(346002)(39860400002)(36840700001)(40470700004)(46966006)(4326008)(86362001)(8676002)(70586007)(426003)(2906002)(83380400001)(70206006)(478600001)(47076005)(336012)(40460700003)(6666004)(1076003)(186003)(26005)(8936002)(16526019)(7416002)(2616005)(36756003)(41300700001)(81166007)(82310400005)(356005)(82740400003)(7696005)(316002)(54906003)(5660300002)(6916009)(40480700001)(36860700001)(121494005)(557034005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2022 10:54:37.5873 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6f505895-8a97-440c-e6e8-08da931adae5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT106.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6442 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Hints are low-level knobs that can influence task placement decisions at various scheduler decision points. - Design of wakeup hints Hints are of two kinds: o fork time hints: These hints influence initial placement. It is observed that a correct initial placement can have a long lasting effect on workload performance. These hints override scheduler behavior when the system is not heavily loaded to overlook any pre-configured bias in scheduler (such as NUMA Imbalance) and place tasks in a way user finds beneficial. These hints are of the form PR_SCHED_HINT_FORK_* namely: - PR_SCHED_HINT_FORK_AFFINE - PR_SCHED_HINT_FORK_SPREAD o wakeup hints: These hints target a specific MC Domain during wakeup. The user can choose to bias the placement towards waker's LLC if we believe the waker-wakee follow a producer-consumer pattern. The user can also choose to bias the placement towards the MC domain where the tasks previously ran if we believe waker's signaling is just for synchronization and the wakee will continue to consume the data, it produced during its last run. These hints are of the form PR_SCHED_HINT_WAKE_* namely: - PR_SCHED_HINT_WAKE_AFFINE - PR_SCHED_HINT_WAKE_HOLD Only one hint of each type can be set at once currently. Failure to do so will lead to prctl() call returning -EINVAL. - API Design An example of setting hint PR_SCHED_HINT_FORK_AFFINE and PR_SCHED_HINT_WAKE_AFFINE for the current running process is a follows: #include prctl(PR_SCHED_HINT /* prctl() cmd */, PR_SCHED_HINT_WAKE_AFFINE | PR_SCHED_HINT_FORK_AFFINE /* Hints */, 0 /* pid of task to set hint for. pid 0 sets hint current task */, 0, 0); The above command sets PR_SCHED_HINT_WAKE_AFFINE and PR_SCHED_HINT_FORK_AFFINE for the task calling the prctl(). For a hint to be set, the task should be ftrace-able. As hints can be inherited by childern, one can wrap the runner script with a hint to avail the benefits and need not change the workload. Signed-off-by: K Prateek Nayak --- include/linux/sched.h | 1 + include/uapi/linux/prctl.h | 8 ++++ kernel/sched/core.c | 81 ++++++++++++++++++++++++++++++++++++++ kernel/sys.c | 5 +++ 4 files changed, 95 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index fc953c9e956a..84a630d7c529 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2405,5 +2405,6 @@ static inline void sched_core_fork(struct task_struct= *p) { } #endif =20 extern void sched_set_stop_task(int cpu, struct task_struct *stop); +extern int sched_set_hint(unsigned int hint, pid_t pid); =20 #endif diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index a5e06dcbba13..0a5dd42f1eab 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -284,4 +284,12 @@ struct prctl_mm_map { #define PR_SET_VMA 0x53564d41 # define PR_SET_VMA_ANON_NAME 0 =20 +/* Hint the scheduler of the expected task behavior */ +#define PR_SCHED_HINT 65 +# define PR_SCHED_HINT_DEFAULT 0 +# define PR_SCHED_HINT_FORK_AFFINE (1U << 0) /* Initial placement close to= forking CPU */ +# define PR_SCHED_HINT_FORK_SPREAD (1U << 1) /* Initial placement biased t= owards idlest group */ +# define PR_SCHED_HINT_WAKE_AFFINE (1U << 2) /* Subsequent wakeup target w= aker's MC domain */ +# define PR_SCHED_HINT_WAKE_HOLD (1U << 3) /* Subsequent wakeup target las= t run's MC domain */ + #endif /* _LINUX_PRCTL_H */ diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 7d289d87acf7..60bee250d7b4 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -71,6 +71,7 @@ # endif #endif =20 +#include #include =20 #include @@ -7323,6 +7324,86 @@ static void __setscheduler_params(struct task_struct= *p, set_load_weight(p, true); } =20 +/* + * Check whether the hints are valid and clear. + * Returns 0 if the hints are invalid of if + * the set hints are ambiguous and can lead to + * inconsistent behavior. + * Returns 1 for valid hint. + */ +static int valid_hint(unsigned int hint) { + if (!hint) + return 1; + + /* + * Largest value of hint can be calculated by setting the hints + * with the largest numeric value in each independent category. + */ + if (hint > (PR_SCHED_HINT_WAKE_HOLD | PR_SCHED_HINT_FORK_SPREAD)) + return 0; + + /* + * Only one of the fork time hints must be + * set. Consistent behavior cannot be + * guarenteed with conflicting hints. + */ + if ((hint & PR_SCHED_HINT_FORK_AFFINE) && + (hint & PR_SCHED_HINT_FORK_SPREAD)) + return 0; + + /* + * Only one of the wakeup hints must be + * set for the same reason stated above. + */ + if ((hint & PR_SCHED_HINT_WAKE_AFFINE) && + (hint & PR_SCHED_HINT_WAKE_HOLD)) + return 0; + + return 1; +} + +/* Called from prctl interface: PR_SCHED_HINT */ +int sched_set_hint(unsigned int hint, pid_t pid) +{ + struct task_struct *task; + int err =3D 0; + + /* + * Make sure hint is valid and the user has not + * requested for conflicting behavior at any + * given decision point. + */ + if (!valid_hint(hint)) + return -EINVAL; + + rcu_read_lock(); + if (pid =3D=3D 0) { + task =3D current; + } else { + task =3D find_task_by_vpid(pid); + if (!task) { + rcu_read_unlock(); + return -ESRCH; + } + } + get_task_struct(task); + rcu_read_unlock(); + + /* + * Check if this process has the right to modify the specified + * process. Use the regular "ptrace_may_access()" checks. + */ + if (!ptrace_may_access(task, PTRACE_MODE_READ_REALCREDS)) { + err =3D -EPERM; + goto out; + } + + WRITE_ONCE(task->hint, hint); +out: + put_task_struct(task); + return err; +} + /* * Check the target process has a UID that matches the current process's: */ diff --git a/kernel/sys.c b/kernel/sys.c index b911fa6d81ab..505ceea548bd 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -2623,6 +2623,11 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, a= rg2, unsigned long, arg3, case PR_SET_VMA: error =3D prctl_set_vma(arg2, arg3, arg4, arg5); break; + case PR_SCHED_HINT: + if (arg4 || arg5) + return -EINVAL; + error =3D sched_set_hint(arg2, arg3); + break; default: error =3D -EINVAL; break; --=20 2.25.1 From nobody Wed Apr 8 12:36:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C8A5C6FA82 for ; Sat, 10 Sep 2022 10:55:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229569AbiIJKzH (ORCPT ); Sat, 10 Sep 2022 06:55:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53632 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229561AbiIJKzF (ORCPT ); Sat, 10 Sep 2022 06:55:05 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2059.outbound.protection.outlook.com [40.107.244.59]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B76522B30 for ; Sat, 10 Sep 2022 03:55:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PgCIhvbxs1XKRL1sSNEczaP5nmYwPyMWrDr7tgZe7ex9chmMOrZMzekJnOCH1en8Dbt0OIbKDn+/inPpVChR3UcUvGJkncLQFfTTjRChVFTSJqiY7gFpNn6J4vMDA6VpAg0Q+GhzSjfwbEI4IzQTvWuUPmMm6LwP4M/Gs+v93CMGn34jJByme2X5lPykixz/zoX9Vj/X2SgrYdmhx+eisBLFg8kcTWdpUXP3YhRPlws38OO2zEmknPllO6e+yFEtdnXTFSySDhL8FpskEqKk5bbEBFrZt8pdkOBurDGaOgwoR4B36nFJWMUZx20TJNZHKTIk47NXEVd6pk94ZQwopg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mGwFg3aZaN8m5w++6OxvSrlMJ3pjF4/mPPFOhSsc9uE=; b=CWkb5iKG9mcKmre1P9tN9913JqcuHOHgHJ30nY7nP/ifh+1BYlJH6Jkc6mEKMw2AG+ZtdjMhm2HRFL1AKXUjaogC7jhASYNz3QVB0fCtbXprA2eqan9gCmJ61kpk2iBU4bHxrQBTbnR9HNxSLiESeKDXD2ZaO4O+7QYND4ntL4QOGsyfKrb2uJTaatw6VRaoL24hf7NS6vzqjgw1MUwraqP1NcThj1fwpgdknldHyMEW6S/AQIxczYy5drgPsHN/C2FVRsixHBE6hwa9s7lj5OhAW1ksHYPDIg/L7Qfag6DQU3Os6uiuV/RdnohPLc2p3CxHcbXx259ieSs5+NItzg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mGwFg3aZaN8m5w++6OxvSrlMJ3pjF4/mPPFOhSsc9uE=; b=CfRWhaExr+z4NiEMk9WUzwPK4yoPA0IhGLEHYkTnKWLRTQv90EpPlPAWkNWvsgb+xKbxNhoeEbiTYWbc0HvRFPPo8S4qSX8NYgMKSeEYv8VJt88ik2m2gjHa7OODF8cRE9ar/yDz7DzQtay8cjSCgzYYAqzl+GKULHk3t73DuE0= Received: from DS7PR06CA0029.namprd06.prod.outlook.com (2603:10b6:8:54::29) by CH0PR12MB5203.namprd12.prod.outlook.com (2603:10b6:610:ba::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12; Sat, 10 Sep 2022 10:55:02 +0000 Received: from DM6NAM11FT097.eop-nam11.prod.protection.outlook.com (2603:10b6:8:54:cafe::d2) by DS7PR06CA0029.outlook.office365.com (2603:10b6:8:54::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.18 via Frontend Transport; Sat, 10 Sep 2022 10:55:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT097.mail.protection.outlook.com (10.13.172.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.13 via Frontend Transport; Sat, 10 Sep 2022 10:55:01 +0000 Received: from BLR5CG134614W.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Sat, 10 Sep 2022 05:54:55 -0500 From: K Prateek Nayak To: CC: , , , , , , , , , , , , , , , Subject: [PATCH 3/5] sched/fair: Add support for hints in the subsequent wakeup path Date: Sat, 10 Sep 2022 16:23:24 +0530 Message-ID: <20220910105326.1797-4-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220910105326.1797-1-kprateek.nayak@amd.com> References: <20220910105326.1797-1-kprateek.nayak@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT097:EE_|CH0PR12MB5203:EE_ X-MS-Office365-Filtering-Correlation-Id: 2d245d28-0bbd-4945-4e2b-08da931ae956 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iU+33jNXNMIpSMaqQ+Xle8+yVtSbWL2WF9bg5OtS0kceE53rpUsa7Nm2Lkp2BJWmPvNf9YEX5QDi9/Ywtc9r4IBAtWCKJEO6RtQO3gEMhEC7bZ+h/dBdts4inHpQZVwmgWNzdVb90Yldz6a56G+b1JX7Lz8hDTMG7CB9xYo3MnTE2Cmv9/vEcvF2N6CLhP8C0ASGjCTcOBFui4ofT9WxicdbL4vcNe0CWxmydgInxCFagCsKJ9JTjE1idVlGhSgU2KFpD7mIKFrEu9NA+paA+FtvkNSUWjctWhV7FvSCUP3oKzdTBcsc6ZCv5wzoavdTinZ3bzTTCVquh9TtDhK+pHf+SQDUUkBsL9CBhAEUBXwLaT5WIZ3YnrhIJrqpmtwxebkCS97G4FajakQ3yT2Vp9IE6ueXJhSqJ1AYdidzKJqMhglIBjEgT1Z4AkoAurhCh2laoYPxb7lxYQQiIF2mgV1+dFQzr2/Lx63HLKEI7GUv+GGEgoLNsW01MPa6ESUfs0TaS9QNTWLTp0In+gHdDfX4yrM75TzBWDupym5mHFcqfSyala2gIo3KZ8uJ7BZxkVWkzODNl9Kjg2g9GUgC/ChNl30psQrhwqULE7f55GyMswp6Bu5BS7dVpMt/31zCAf2D9d5bJF5XCRucF4Y1Q/2eQgS43fBX002AFdpvTz04KN8pdIDtxW4JT00bBbE1pLPPG2ab84cYv6a0QzgAc2Uf8LhEjXhDkko0uLS8MLcZhLuV9TyltkrKOTCDOqKnX+nDzGXTTRlGa3rYn36HzUJ14woQ4rqDkDx4c4Bx7QpKsZicjFMahGJeP0AEWa7R X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(136003)(376002)(396003)(346002)(39860400002)(40470700004)(36840700001)(46966006)(5660300002)(336012)(16526019)(2906002)(47076005)(426003)(36756003)(83380400001)(41300700001)(1076003)(186003)(36860700001)(8936002)(7416002)(2616005)(70586007)(7696005)(26005)(82740400003)(70206006)(82310400005)(8676002)(4326008)(6666004)(316002)(6916009)(54906003)(40480700001)(81166007)(86362001)(356005)(40460700003)(478600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2022 10:55:01.8589 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2d245d28-0bbd-4945-4e2b-08da931ae956 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT097.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5203 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Hints are adhered to as long as there are idle cores in the target MC domain. Beyond that, the default behavior is followed. - Hinting flow in the wakeup path Following is the flow with wakeup hints: o Check if the task has a wakeup hint set and whether the current CPU and the CPU where the task previously ran are on two different LLCs. If either is false, bail out and follow the default logic. o Check whether the previous CPU or the current CPU is the desired CPU according to the set hint. o Test for idle cores in the MC domain of the hinted CPU. o If yes, set the desired CPU as the target for wakeup. The scheduler will then look for an idle CPU withing the MC domain of the target. o If test_idle_cores returns false, follow the default wakeup path. PR_SCHED_HINT_WAKE_AFFINE will favor an affine wakeup if the MC where the waker is running advertises idle core. PR_SCHED_HINT_WAKE_HOLD will bias the wakeup to MC domain where the task previously ran. - Results Following are results from running hackbench with only wakeup hints on a dual socket Zen3 system in NPS1 mode: o Hackbench Test: tip no-hint wake_= affine wake_hold 1-groups: 4.31 (0.00 pct) 4.46 (-3.48 pct) 4.20 (2= .55 pct) 4.11 (4.64 pct) 2-groups: 4.93 (0.00 pct) 4.85 (1.62 pct) 4.74 (3= .85 pct) 5.15 (-4.46 pct) 4-groups: 5.38 (0.00 pct) 5.35 (0.55 pct) 5.04 (6= .31 pct) 4.54 (15.61 pct) 8-groups: 5.59 (0.00 pct) 5.49 (1.78 pct) 5.39 (3= .57 pct) 5.71 (-2.14 pct) 16-groups: 7.18 (0.00 pct) 7.38 (-2.78 pct) 7.24 (-= 0.83 pct) 7.76 (-8.07 pct) As we can observe, the hint PR_SCHED_HINT_WAKE_AFFINE helps performance across all hackbench configurations. PR_SCHED_HINT_WAKE_HOLD does not show any consistent behavior and can lead to unpredictable behavior in hackbench. - Shortcomings In schbench, the delay to indicate that no idle core is available in target MC domain leads to pileup and severe degradation in p99 latency o schbench workers: tip no-hint wake_affine= wake_hold 1: 37.00 (0.00 pct) 38.00 (-2.70 pct) 18.00 (51.35 p= ct) 32.00 (13.51 pct) 2: 39.00 (0.00 pct) 36.00 (7.69 pct) 18.00 (53.84 p= ct) 36.00 (7.69 pct) 4: 41.00 (0.00 pct) 41.00 (0.00 pct) 21.00 (48.78 p= ct) 33.00 (19.51 pct) 8: 53.00 (0.00 pct) 54.00 (-1.88 pct) 31.00 (41.50 p= ct) 51.00 (3.77 pct) 16: 73.00 (0.00 pct) 74.00 (-1.36 pct) 2636.00 (-3510.9= 5 pct) 75.00 (-2.73 pct) 32: 116.00 (0.00 pct) 124.00 (-6.89 pct) 15696.00 (-13431.= 03 pct) 124.00 (-6.89 pct) 64: 217.00 (0.00 pct) 215.00 (0.92 pct) 15280.00 (-6941.4= 7 pct) 224.00 (-3.22 pct) 128: 477.00 (0.00 pct) 440.00 (7.75 pct) 14800.00 (-3002.7= 2 pct) 493.00 (-3.35 pct) 256: 1062.00 (0.00 pct) 1026.00 (3.38 pct) 15696.00 (-1377.9= 6 pct) 1026.00 (3.38 pct) 512: 47552.00 (0.00 pct) 47168.00 (0.80 pct) 60736.00 (-27.72 = pct) 49856.00 (-4.84 pct) Wake hold seems to still do well by reducing the larger latency samples that we observe during task migration. - Potential Solution One potential solution is to atomically read nr_busy_cpus member of sched_domain_shared struct but the performance impact of this is yet to be evaluated in the wakeup path. Signed-off-by: K Prateek Nayak --- kernel/sched/fair.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index efceb670e755..90e523cd8de8 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -51,6 +51,8 @@ =20 #include =20 +#include + #include "sched.h" #include "stats.h" #include "autogroup.h" @@ -7031,6 +7033,10 @@ select_task_rq_fair(struct task_struct *p, int prev_= cpu, int wake_flags) int want_affine =3D 0; /* SD_flags and WF_flags share the first nibble */ int sd_flag =3D wake_flags & 0xF; + bool use_hint =3D false; + unsigned int task_hint =3D READ_ONCE(p->hint); + unsigned int wakeup_hint =3D task_hint & + (PR_SCHED_HINT_WAKE_AFFINE | PR_SCHED_HINT_WAKE_HOLD); =20 /* * required for stable ->cpus_allowed @@ -7046,6 +7052,37 @@ select_task_rq_fair(struct task_struct *p, int prev_= cpu, int wake_flags) new_cpu =3D prev_cpu; } =20 + /* + * Handle the case where a hint is set and the current CPU + * and the previous CPU where task ran don't share caches. + */ + if (wakeup_hint && !cpus_share_cache(cpu, prev_cpu)) { + /* + * Start by assuming the hint is PR_SCHED_HINT_WAKE_AFFINE + * setting the target_cpu to the current CPU. + */ + int target_cpu =3D cpu; + + /* + * If the hint is PR_SCHED_HINT_WAKE_HOLD + * change target_cpu to the prev_cpu. + */ + + if (wakeup_hint & PR_SCHED_HINT_WAKE_HOLD) + target_cpu =3D prev_cpu; + + /* + * If a wakeup hint is set, try to bias the + * task placement towards the preferred node + * as long as there is an idle core in the + * targetted LLC. + */ + if (test_idle_cores(target_cpu, false)) { + use_hint =3D true; + new_cpu =3D target_cpu; + } + } + want_affine =3D !wake_wide(p) && cpumask_test_cpu(cpu, p->cpus_ptr); } =20 @@ -7057,7 +7094,11 @@ select_task_rq_fair(struct task_struct *p, int prev_= cpu, int wake_flags) */ if (want_affine && (tmp->flags & SD_WAKE_AFFINE) && cpumask_test_cpu(prev_cpu, sched_domain_span(tmp))) { - if (cpu !=3D prev_cpu) + /* + * In case it is optimal to follow the hints, + * do not re-evaluate the target CPU. + */ + if (cpu !=3D prev_cpu && !use_hint) new_cpu =3D wake_affine(tmp, p, cpu, prev_cpu, sync); =20 sd =3D NULL; /* Prefer wake_affine over balance flags */ --=20 2.25.1 From nobody Wed Apr 8 12:36:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06E2FECAAD5 for ; Sat, 10 Sep 2022 10:55:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229582AbiIJKzh (ORCPT ); Sat, 10 Sep 2022 06:55:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229566AbiIJKzf (ORCPT ); Sat, 10 Sep 2022 06:55:35 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2063.outbound.protection.outlook.com [40.107.93.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E14D3E754 for ; Sat, 10 Sep 2022 03:55:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gJkGPWwU/8r/SfLFIphC8czIPnhNahE0gkYOI8yuqrGCBnxaqJal8qsxi3TMSWKC4Snfc0iesP7FyDAxeWAKPzt2MRvsDDSuDvhEGeXSyX+eEcPeWrASfXnaqOGyexaDnUYqN8J4wNUckyftYjUwxfjxFiEyhTELlEwZ1qWvIQa6tEiRKEGBPz+IHfPlcjpUruY1AgttSrI79G7crOkzoXL+o/6R4kRekkQuaj7hPEdbWoPuCVujuWDZeqtn5mPOnnjrD2p8PeJ+eQPVMuDsLUDFf4zJjy7yrTZP0j+sHb6DgI3M1iIH1psO1YC/vfe9YQJ/7LItWHIZ72OtJk9sdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RR+hODHNRtmmtv8nY5zuwhWWokgI7eCNKilMB5xtB7w=; b=gB8/sOwaqECnrfk/ugebbDwxhpjscRJO0cK/fJNJbCxv6z0YWWL7AbJ7QKx7JvsTMK9KXkvkiWhFBxAqQQT2TM5oA/FJorXbvq3kvjbDJXyIC+k0XMycFC9XqoQVj1djz5iXPmtKOLHx3I3pKS6ZvQwzO6lKcvf0YnFq00bqslzLvhi404Xp+AJ4cngEhSROomW+ye3sSNhAl35R3+Jpj0qKiubQsIAFcXHneaIKbLiG5Y4bRFANc5dplVRKQJ2jMAM+hi7wt6gtjAIYU6g5nDtsBzS5pPl/uiVLhFxcqn8S0I6LzfqUt7ifOL4YkTxe9qDx6RiReIbvANbw5D7lpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RR+hODHNRtmmtv8nY5zuwhWWokgI7eCNKilMB5xtB7w=; b=DntdcvGZq5EbscCMWVtmDXvQRaIl5kbgBLZqIBYBOSvMw7T094Z2G323Oytasfld1tMfEPxxuMfPwv1zVyS5daTEq65owl44YMEXLIDJmn5XGS0ZQ+K4I9Ud11VuIK4pHI7p5oARAt5X/iAOzzX+5MYir7yqNCO7iOMVhN04nn8= Received: from DS7PR07CA0008.namprd07.prod.outlook.com (2603:10b6:5:3af::21) by LV2PR12MB5920.namprd12.prod.outlook.com (2603:10b6:408:172::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.14; Sat, 10 Sep 2022 10:55:28 +0000 Received: from DM6NAM11FT110.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3af:cafe::91) by DS7PR07CA0008.outlook.office365.com (2603:10b6:5:3af::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12 via Frontend Transport; Sat, 10 Sep 2022 10:55:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT110.mail.protection.outlook.com (10.13.173.205) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.13 via Frontend Transport; Sat, 10 Sep 2022 10:55:28 +0000 Received: from BLR5CG134614W.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Sat, 10 Sep 2022 05:55:21 -0500 From: K Prateek Nayak To: CC: , , , , , , , , , , , , , , , Subject: [PATCH 4/5] sched/fair: Consider hints in the initial task wakeup path Date: Sat, 10 Sep 2022 16:23:25 +0530 Message-ID: <20220910105326.1797-5-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220910105326.1797-1-kprateek.nayak@amd.com> References: <20220910105326.1797-1-kprateek.nayak@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT110:EE_|LV2PR12MB5920:EE_ X-MS-Office365-Filtering-Correlation-Id: 4bce9af0-c5b2-4b8c-d544-08da931af8ef X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FLGjKzaAab9WmVnYzH3e4mk3kEKSDhR7bwxqLmwsTnFkA+YJBNTYBNQrrEl803JEdfWo6pYh+FNE7IBG+aJ+kmkBdngmtxh/5gPQyjxnovIivAmYL92H/IBTkyqcyTwl2YKTBfOekhJYOz4+yjS/c08L5L1cmk1Vm5z/+wJjGFdqTbuizmR2IOVCrIfxSJfFZGyieGnuWFYTJV16M20x7AwGX1NPWPeRL7gP2ZdeC8gfxnOUBGqyPu8m8JQXWqeIXfzEpNi34aPA+/c4iYtQxHC8cPTEwAAGzxFivGN0WgzRSk3za39tCNU7savE3hfd/W9S0PpL/QbIXacZpsVyLFObufHZGL0se6G5o8qTmXDvTRIhaWZdXy1RoPfpiNu5k4WdQF+yGsZzff5r33w3l1RGUmvRs+LLZiE/Psmy0gJQL8Oo9fuQO30WnvhPWpU8pj9oUr8kMxvjaydSI1pjAagtcKUxRub+A8paCwHfohl2aGXnQkbG1QMqE/kesbAu9O99eilIdr5eC4CCohMKD5fPYExLlqyXGxdoqLWR1O92T/woyRFOujQsAErrGsdg6FFoc0mX0FXop+Fdt9W1V5sWgmcROw0Yzisf4yEhMiLv2o1QBZkoafFErQfFVxQhhTLVAFowCNcQZ3z36Zbfi7aM83ugBiLK1IU8YATbUL4LRr46o45eWhbVuhDiVwLyrxQS728P2nuD84e4O8uceiYZlyFbulwsV3lA8Y/v5oXMp6MXnGIpnqApIgtX6gSFOLPAHJ6bMMvRPdp8/+1OeX5WZZ4ObI8UDsEPDwSnZrQ= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(376002)(39860400002)(136003)(396003)(40470700004)(36840700001)(46966006)(426003)(186003)(7696005)(2616005)(47076005)(16526019)(336012)(1076003)(70586007)(8936002)(70206006)(4326008)(2906002)(8676002)(5660300002)(7416002)(82310400005)(40480700001)(36756003)(26005)(86362001)(83380400001)(41300700001)(478600001)(40460700003)(81166007)(356005)(82740400003)(36860700001)(316002)(6916009)(54906003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2022 10:55:28.0152 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4bce9af0-c5b2-4b8c-d544-08da931af8ef X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT110.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5920 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" These hints influence the behavior of the initial task placement and bias the placement towards or away from the CPU where the task is forked. The flow is as follows: - When a fork time hint is set, the NUMA biases are overlooked and only the sched_group's statistics computed by update_sg_wakeup_stats() (Number of idle CPUs and total utilization of the group) for the local group and the idlest group is considered while making initial task placement decision when both groups have idle CPUs. - In case a bias towards local group is hinted, go for the local group as long as an equivalent of idle core is present. Note: The current implements assume the system running the patch is SMT-2. Further optimizations can be made for systems with SMT-4, SMT-8, or with no SMT. - If a hint for spread is set, and there is a tie in number of idle CPUs in local group and idlest group, use the utilization of group as the tie breaking metric. PR_SCHED_HINT_FORK_AFFINE enables consolidation until half of the local group is filled. PR_SCHED_HINT_FORK_SPREAD will choose the target group based on the utilization if there is a tie in number of idle CPUs. These hints can be set individually in addition to wakeup hints. - Results Following are results from using individual fork time hints and combination of fork time hints and wakeup hints on various benchmark on a dual socket Zen3 system: o Only fork time hint: - Hackbench Test: tip no-hint fork_a= ffine fork_spread 1-groups: 4.31 (0.00 pct) 4.46 (-3.48 pct) 4.27 (0.= 92 pct) 4.28 (0.69 pct) 2-groups: 4.93 (0.00 pct) 4.85 (1.62 pct) 4.91 (0.= 40 pct) 5.15 (-4.46 pct) 4-groups: 5.38 (0.00 pct) 5.35 (0.55 pct) 5.36 (0.= 37 pct) 5.31 (1.30 pct) 8-groups: 5.59 (0.00 pct) 5.49 (1.78 pct) 5.51 (1.= 43 pct) 5.51 (1.43 pct) 16-groups: 7.18 (0.00 pct) 7.38 (-2.78 pct) 7.31 (-1= .81 pct) 7.25 (-0.97 pct) - schbench workers: tip no-hint fork_affine 1: 37.00 (0.00 pct) 38.00 (-2.70 pct) 17.00 (54.05 pct) 2: 39.00 (0.00 pct) 36.00 (7.69 pct) 21.00 (46.15 pct) 4: 41.00 (0.00 pct) 41.00 (0.00 pct) 28.00 (31.70 pct) 8: 53.00 (0.00 pct) 54.00 (-1.88 pct) 39.00 (26.41 pct) 16: 73.00 (0.00 pct) 74.00 (-1.36 pct) 68.00 (6.84 pct) 32: 116.00 (0.00 pct) 124.00 (-6.89 pct) 113.00 (2.58 pct) 64: 217.00 (0.00 pct) 215.00 (0.92 pct) 205.00 (5.52 pct) 128: 477.00 (0.00 pct) 440.00 (7.75 pct) 445.00 (6.70 pct) 256: 1062.00 (0.00 pct) 1026.00 (3.38 pct) 1007.00 (5.17 pct) 512: 47552.00 (0.00 pct) 47168.00 (0.80 pct) 47296.00 (0.53 pct) - tbench Clients: tip no-hint fork_affine = fork_spread 1 573.26 (0.00 pct) 572.29 (-0.16 pct) 572.70 (-0.09 pct)= 569.64 (-0.63 pct) 2 1131.19 (0.00 pct) 1119.57 (-1.02 pct) 1131.97 (0.06 pct)= 1101.03 (-2.66 pct) 4 2100.07 (0.00 pct) 2070.66 (-1.40 pct) 2094.80 (-0.25 pct= ) 2011.64 (-4.21 pct) 8 3809.88 (0.00 pct) 3784.16 (-0.67 pct) 3458.94 (-9.21 pct= ) 3867.70 (1.51 pct) 16 6560.72 (0.00 pct) 6449.64 (-1.69 pct) 6342.78 (-3.32 pct= ) 6700.50 (2.13 pct) 32 12203.23 (0.00 pct) 12180.02 (-0.19 pct) 10411.44 (-14.68 p= ct) 13104.29 (7.38 pct) 64 22389.81 (0.00 pct) 23084.51 (3.10 pct) 16614.14 (-25.79 p= ct) 24353.76 (8.77 pct) 128 32449.37 (0.00 pct) 33561.28 (3.42 pct) 19971.67 (-38.45 p= ct) 36201.16 (11.56 pct) 256 58962.40 (0.00 pct) 59118.43 (0.26 pct) 26836.13 (-54.48 p= ct) 61721.06 (4.67 pct) 512 59608.71 (0.00 pct) 60246.78 (1.07 pct) 36889.55 (-38.11 p= ct) 59696.57 (0.14 pct) 1024 58037.02 (0.00 pct) 58532.41 (0.85 pct) 39936.06 (-31.18 p= ct) 57445.62 (-1.01 pct) All these benchmarks show noticeable improvements only with a slightly different initial placement. A placement in line with benchmark behavior improves benchmark results. o Combination of hints - Hackbench Test: tip no-hint fork_affine += wake_affine fork_spread + wake_hold 1-groups: 4.31 (0.00 pct) 4.46 (-3.48 pct) 4.20 (2.5= 5 pct) 4.81 (-11.60 pct) 2-groups: 4.93 (0.00 pct) 4.85 (1.62 pct) 4.74 (3.8= 5 pct) 5.09 (-3.24 pct) 4-groups: 5.38 (0.00 pct) 5.35 (0.55 pct) 5.01 (6.8= 7 pct) 5.62 (-4.46 pct) 8-groups: 5.59 (0.00 pct) 5.49 (1.78 pct) 5.38 (3.7= 5 pct) 5.69 (-1.78 pct) 16-groups: 7.18 (0.00 pct) 7.38 (-2.78 pct) 7.25 (-0.= 97 pct) 7.97 (-11.00 pct) Hackbench improves further with pairing of correct wakeup hint with correct fork time hint. The regression is equally bad with wrong hints set. Signed-off-by: K Prateek Nayak --- kernel/sched/fair.c | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 90e523cd8de8..4c61bd0e93b3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9262,6 +9262,7 @@ find_idlest_group(struct sched_domain *sd, struct tas= k_struct *p, int this_cpu) struct sg_lb_stats local_sgs, tmp_sgs; struct sg_lb_stats *sgs; unsigned long imbalance; + unsigned int task_hint, fork_hint; struct sg_lb_stats idlest_sgs =3D { .avg_load =3D UINT_MAX, .group_type =3D group_overloaded, @@ -9365,8 +9366,14 @@ find_idlest_group(struct sched_domain *sd, struct ta= sk_struct *p, int this_cpu) break; =20 case group_has_spare: + task_hint =3D READ_ONCE(p->hint); + fork_hint =3D task_hint & + (PR_SCHED_HINT_FORK_SPREAD | PR_SCHED_HINT_FORK_AFFINE); #ifdef CONFIG_NUMA - if (sd->flags & SD_NUMA) { + /* + * If a hint is set, override any NUMA preference behavior. + */ + if ((sd->flags & SD_NUMA) && !fork_hint) { int imb_numa_nr =3D sd->imb_numa_nr; #ifdef CONFIG_NUMA_BALANCING int idlest_cpu; @@ -9406,14 +9413,37 @@ find_idlest_group(struct sched_domain *sd, struct t= ask_struct *p, int this_cpu) } #endif /* CONFIG_NUMA */ =20 + /* + * FIXME: Currently the system is assumed to be SMT-2 + * and that the number of cores in a group can be + * estimated by halving the group_weight. Determine a + * more generic logic for other SMT possibilities or + * derive it at runtime from the topology. + */ + if ((task_hint & PR_SCHED_HINT_FORK_AFFINE) && + local_sgs.idle_cpus > local->group_weight / 2) + return NULL; /* * Select group with highest number of idle CPUs. We could also * compare the utilization which is more stable but it can end * up that the group has less spare capacity but finally more * idle CPUs which means more opportunity to run task. */ - if (local_sgs.idle_cpus >=3D idlest_sgs.idle_cpus) + if (local_sgs.idle_cpus > idlest_sgs.idle_cpus) + return NULL; + + if (local_sgs.idle_cpus =3D=3D idlest_sgs.idle_cpus) { + /* + * In case of a tie between number of idle CPUs and if + * the task hints a benefit from spreading, go with the + * group with the lesser utilization. + */ + if ((task_hint & PR_SCHED_HINT_FORK_SPREAD) && + local_sgs.group_util > idlest_sgs.group_util) + return idlest; + return NULL; + } break; } =20 --=20 2.25.1 From nobody Wed Apr 8 12:36:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80A5BECAAD5 for ; Sat, 10 Sep 2022 10:56:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229586AbiIJK4B (ORCPT ); Sat, 10 Sep 2022 06:56:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229566AbiIJKz5 (ORCPT ); Sat, 10 Sep 2022 06:55:57 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2081.outbound.protection.outlook.com [40.107.223.81]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F101B40BDC for ; Sat, 10 Sep 2022 03:55:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ILXKVo7RiSXbR3IdLGgufI/fYyXqMPljbkroQwqha7hEcpprkR8Tl/Ur7/lpAZvbsYsLBevMZGh+bDYCmZAOQXalB48vyC0wT34B0UXR0fS7i2MLAn5nZo0CVy0K0IqqRHWPH5mNwgyujJKTQ/1F46V/j1Ytz+OIwimP+6jGv8ZMKL1UyCEY2rV6/zTUpdBOjYtuMiXD09BFybx4PXs2eGLWriENq87JBCbdcGC+coY4tJiDayAMiuG3fuqTEWaIZDQEm0lWU+nzNI144fe3Ic5zo818vU4+5+jOLXKAZhnx7a157VBKqpiRqz4LsPFk7COpZdcqHqaY00yq0RPj3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JH2TqdR7mrw9mV8XHYulf55CwYSKeriybmUS4AdUcPc=; b=AnX5o2+xxtFUC2NS3JOpIfmQyySqYOlZADE0jUhIiLGxsdLj+6dn1/13+pWb3h3pQs0akQTxy474P4AUvJTmyvYGk9oP3ph1ZPMA9IwZn4IguykCDpl2BTjtmYEayV31jCtDJ1NVX7auoXk5sbyXfzhY0cK7wBMNym4tMWYhCuc2Odw+1kRS5NOBKM+KZl4msDC6gUYyQ3XlJNoZLQhjrbph5w7vTGysSip9AWpwyMzz30mFtvxbp50Nr+XHIMB1aB+6voS5aPL5JBjdz4MmmqoYkj07LErq8e39kXXVJ4v2XM1KpTnKCKGkGbdLZJNSyYJ5v4Rn4UKhROWqY4wpow== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JH2TqdR7mrw9mV8XHYulf55CwYSKeriybmUS4AdUcPc=; b=t5NI9AE6DKfqOi6bOOs61JkL3ArvzBImKiPcGaaLarpox9K/mqcKYZKp8p7J7orO4CaQlAO44hCqQHJ/RQHlcbRiEdDf3TQ6D6uESLUDDPU9d9NHsgf5m7t6uPEbe02GSNE3WGNsp5UephszbkAk7ad98ZB5gUAGAbr37l19v50= Received: from DM5PR06CA0090.namprd06.prod.outlook.com (2603:10b6:3:4::28) by IA1PR12MB6436.namprd12.prod.outlook.com (2603:10b6:208:3ac::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.17; Sat, 10 Sep 2022 10:55:53 +0000 Received: from DM6NAM11FT098.eop-nam11.prod.protection.outlook.com (2603:10b6:3:4:cafe::81) by DM5PR06CA0090.outlook.office365.com (2603:10b6:3:4::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12 via Frontend Transport; Sat, 10 Sep 2022 10:55:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT098.mail.protection.outlook.com (10.13.173.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.13 via Frontend Transport; Sat, 10 Sep 2022 10:55:53 +0000 Received: from BLR5CG134614W.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Sat, 10 Sep 2022 05:55:46 -0500 From: K Prateek Nayak To: CC: , , , , , , , , , , , , , , , Subject: [PATCH 5/5] sched/fair: Add exception for hints in load balancing path Date: Sat, 10 Sep 2022 16:23:26 +0530 Message-ID: <20220910105326.1797-6-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220910105326.1797-1-kprateek.nayak@amd.com> References: <20220910105326.1797-1-kprateek.nayak@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT098:EE_|IA1PR12MB6436:EE_ X-MS-Office365-Filtering-Correlation-Id: d159c172-ed10-4567-afd6-08da931b07d7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NtWIXWxTSGuBPKmWir0YRMcr0XlQ33KD+CY037p5F9/ai4IJWJTegTwH0GiBLy2Wb995yjZqZ2Cy9stQVOT24jj3Dz+/5rX1XuIAfmvQBmSuodSEoHryFQQm7xabm141N4Hk+IwhXo08bFQz0mZlH1896NBvnwNagdK8hR6BSZcQudiFf3OGPZioqeFDaPvACVkEWva+ebQINcyPzNFWCIbvlI8y0l1vt4HckIJwy7qkyCgwZ9iLwQ2nKLXauIb4sNAFXO6sgQkkGp8ekzNBaz9ZOkOlmsn8gXmN4tDqlw8s/bo2H1MTVYtsFQib55CYfqZbtGmQHRtHiwQPgvJkpEOajPvELYM6iHKKi5aFM0UKMJ7kGZRXmvXafyN5TR+1LSOxnpu9uVFP1IBsFogd7s2dMX2qYU+Zn0pIhckZZspZrEDC6Ywt6FsaC7Bp+X64KTU6/C5yBcPWfk6DeanrINdTm2lowob8yfe5mOT8qQ593hfNPTmBMAzeWopyas4c5bPrbfkHIAMXNJFnsfC5nU4wjBLHNDbqigpELfjtYd/cXkvrec/NEsZQaKRWk3cKPcUEsszpBO3caeJepFWgKdlOuJQLR8e7TF2uffJzi/BrjOxxoyctw06R9M/kFm50JQ/S2+bpqZ51LSIi3ka6LZpU4urTd7s6E3/hQx7wMO7Z4EZV5lz7fUJTxdpRmtICEqbPqLPrtb8qnZCnlK6muTTMpHY16h/7D6oEItDGc0IyFry+f35FEEGoTxoRfr09x7HyRULgXEMJtW8oren/QYT+XVutXBKrZ0o9bh3ATy3MrcRLGjt/HSq40FFC1qhe X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(396003)(136003)(376002)(39860400002)(46966006)(36840700001)(40470700004)(36860700001)(41300700001)(7696005)(81166007)(83380400001)(86362001)(2616005)(4326008)(40480700001)(70586007)(70206006)(8676002)(26005)(478600001)(1076003)(356005)(8936002)(2906002)(7416002)(5660300002)(6666004)(54906003)(40460700003)(426003)(316002)(82310400005)(47076005)(82740400003)(6916009)(36756003)(186003)(336012)(16526019)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2022 10:55:53.0384 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d159c172-ed10-4567-afd6-08da931b07d7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT098.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6436 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" - Load balancing considerations If we have more tasks than the CPUs in the MC Domain, ignore the hint set by the user. This prevents losing the consolidation done at the wakeup time. - Considerations Few trial and errors were done to find a good threshold to ignore hints. Following are some of the wins and woes: o Ignore hint if MC domain of src CPU does not have an idle core: This metric is not very accurate and led to losing consolidation early on. o Ignore hint if sd_shared->nr_llc_scan is 0: This too, like the has_idle core metric was not always accurate. o An atomic read of sd_shared->nr_busy_cpus doesn't encapsulate overloaded run queues. Best results were found by scanning LLC and finding the number of running tasks and comparing it with size of LLC. If the LLC is beyond fully loaded, safely ignore hint. - Possible Improvements o Consider the status of hint: If a wake affine hint was ignored in the wakeup path, consider ignoring in the load balancer path as well as the running LLC is not the desired LLC in fact. Signed-off-by: K Prateek Nayak --- kernel/sched/fair.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 4c61bd0e93b3..8e1679b784fb 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7810,6 +7810,9 @@ struct lb_env { unsigned int loop_break; unsigned int loop_max; =20 + /* Indicator to ignore hint if LLC is overloaded */ + int ignore_hint; + enum fbq_type fbq_type; enum migration_type migration_type; struct list_head tasks; @@ -7977,6 +7980,21 @@ int can_migrate_task(struct task_struct *p, struct l= b_env *env) return 0; } =20 + /* + * Hints are followed only if the MC Domain is still ideal + * for the task. + */ + if (!env->ignore_hint) { + /* + * Only consider the hints from the wakeup path to maintain + * data locality. + */ + if (READ_ONCE(p->hint) & + (PR_SCHED_HINT_WAKE_AFFINE | PR_SCHED_HINT_WAKE_HOLD)) + return 0; + } + + /* Record that we found at least one task that could run on dst_cpu */ env->flags &=3D ~LBF_ALL_PINNED; =20 @@ -10182,6 +10200,7 @@ static int load_balance(int this_cpu, struct rq *th= is_rq, .cpus =3D cpus, .fbq_type =3D all, .tasks =3D LIST_HEAD_INIT(env.tasks), + .ignore_hint =3D 1, }; =20 cpumask_and(cpus, sched_domain_span(sd), cpu_active_mask); @@ -10213,6 +10232,30 @@ static int load_balance(int this_cpu, struct rq *t= his_rq, env.src_cpu =3D busiest->cpu; env.src_rq =3D busiest; =20 + /* + * Check if the hints can be followed during + * this load balancing cycle. + */ + if (!(sd->flags & SD_SHARE_PKG_RESOURCES)) { + struct sched_domain *src_sd_llc =3D rcu_dereference(per_cpu(sd_llc, env.= src_cpu)); + + if (src_sd_llc) { + int cpu, nr_llc_running =3D 0, llc_size =3D per_cpu(sd_llc_size, env.sr= c_cpu); + + for_each_cpu_wrap(cpu, sched_domain_span(src_sd_llc), env.src_cpu) { + struct rq *rq =3D cpu_rq(cpu); + nr_llc_running +=3D rq->nr_running - rq->cfs.idle_h_nr_running; + } + + /* + * Don't ignore hint if we can have one task + * per CPU in the LLC of the src_cpu. + */ + if (nr_llc_running <=3D llc_size) + env.ignore_hint =3D 0; + } + } + ld_moved =3D 0; /* Clear this flag as soon as we find a pullable task */ env.flags |=3D LBF_ALL_PINNED; @@ -10520,6 +10563,7 @@ static int active_load_balance_cpu_stop(void *data) .src_rq =3D busiest_rq, .idle =3D CPU_IDLE, .flags =3D LBF_ACTIVE_LB, + .ignore_hint =3D sd->flags & SD_SHARE_PKG_RESOURCES, }; =20 schedstat_inc(sd->alb_count); --=20 2.25.1