From nobody Tue Apr 7 04:34:56 2026 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010045.outbound.protection.outlook.com [52.101.85.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 077723DBD77; Thu, 2 Apr 2026 11:23:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.85.45 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129043; cv=fail; b=XJ0e1+qfwA/y3I0e8bECEt79n8Djav3G28UrOf62e32iirlXHz8BjOSLlLcf0ko8eqsUnQk/z58h0Jz9/S3v2c7DNmTpa6hAQwIfl2OZywXzZdEWqVaoAGXfPqyRa0+44xRVHGRcwzwaT9rqViGCmC7dlTq4xQX/98NTfjcw4qg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129043; c=relaxed/simple; bh=Da07juv+zBrc2dBIMH7jHgDcmgx8/ZOHCqOtnV0CDpA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=LbkEpI2id2zZ8vhGcmCWGZ+HVZO/WCZVXvobQgoSziF7y1jevkMlSBmhFPs6VKstEUDg8Ej7CxE7ZnIMY4+9fNtA9bLeL5FGu0T1EcKpcKRmoOfygWirqo3WzmcNXmoZUApIvEPDlBm4jJIEKNC6EOqTpjE08AZS5Ap5kzyjBQM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=NFKHCRrM; arc=fail smtp.client-ip=52.101.85.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="NFKHCRrM" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=TZZDXgk3f9e/8BZObCr83LKhMEkCsRtvol81Cu83ysvuHxUlMpJOtLlhhvDtpz+548wIC7NOu9vkzdW+n2Pbb3w/u2EdsbNHAz5U/ABgjmxdMjX5tS9A1zJqnDSfJYeoVyxPGgy5RbrzSWiWTA5006kmDgxbXm8PN9efQxShFUiJwZuj4hn5BqJsKzfZDazQ8PoWkgb6vTnCXltruNTwwzjwL/xDwtJOK3jxr8Zqh4DEbM9CctrxeC4tIhR6BxSsU+8m2zATxITMJ6WHVbhzfmXQC8gKG4n/mvdJ+QBR2PxOLWEra1y0NvUwx4CsdsY9BE3G9AlNZRBu9kOeKaNFgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RWzVcVka8vdMp49l9TneevMDQw/RlcqNsmTol/lDRZU=; b=M/yDesAihEK/NfoNk1DUWC3xCCB+derSOADX50/TkafwmjMKUqL+sPKpoD5cB1uFKFUf8HKs7kh448I4EYaxoNQwQJlLa8MNyPMRG9HntmE8ySZNxwXZvYTD6OTk4yLQFLp9VWKt/F1y8Ac8X4/itXXK3NCBxrhG98eMCQtbKF914bGLmKH8ePPSjeh0KSdbzoHmWngc01rbZJKu6pSTCSrd/VuXWLsip46StJhjcsx70aLOBrZlsokfJRhUVmkYOV+vTNOlxZvq6TlyqTSKKB4l1cwZJM6kQGKG0DEiWBPoavpuhtSv1cGzBVqCyzReF8YuzfuDsuYDz2W4oEcVng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RWzVcVka8vdMp49l9TneevMDQw/RlcqNsmTol/lDRZU=; b=NFKHCRrMsbajMULhu2YdweS5YsWeb6nFXGaCBo6gIFWEnzivrx+JLWyA6RqdB61eZ9+4cPccztls0I9ombZMQNn1AG038He2enjXuXdKwLbQJ7/E6csQHtq5/Q/zGhL0gCvEZ2sCAheCk67s5//tPJsteUSlO4WF6TcG2rS4wNw= Received: from SN7PR18CA0028.namprd18.prod.outlook.com (2603:10b6:806:f3::11) by DM4PR12MB6009.namprd12.prod.outlook.com (2603:10b6:8:69::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.19; Thu, 2 Apr 2026 11:23:44 +0000 Received: from SN1PEPF0002636A.namprd02.prod.outlook.com (2603:10b6:806:f3:cafe::3b) by SN7PR18CA0028.outlook.office365.com (2603:10b6:806:f3::11) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9769.18 via Frontend Transport; Thu, 2 Apr 2026 11:23:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SN1PEPF0002636A.mail.protection.outlook.com (10.167.241.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Thu, 2 Apr 2026 11:23:43 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Thu, 2 Apr 2026 06:23:38 -0500 From: K Prateek Nayak To: Thomas Gleixner , Ingo Molnar , "Peter Zijlstra" , Sebastian Andrzej Siewior , Borislav Petkov , Dave Hansen , CC: Darren Hart , Davidlohr Bueso , =?UTF-8?q?Andr=C3=A9=20Almeida?= , , , , , , K Prateek Nayak Subject: [PATCH v3 1/7] x86/runtime-const: Introduce runtime_const_mask_32() Date: Thu, 2 Apr 2026 11:22:44 +0000 Message-ID: <20260402112250.2138-2-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260402112250.2138-1-kprateek.nayak@amd.com> References: <20260402112250.2138-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb08.amd.com (10.181.42.217) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636A:EE_|DM4PR12MB6009:EE_ X-MS-Office365-Filtering-Correlation-Id: 6269188e-c658-4a53-830c-08de90aa4c8d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|36860700016|82310400026|1800799024|13003099007|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: hDN4xww+2Mh5DXtTqZlUXFbdIAwaVBlnYkB5ncfxuKVdh1AOj0a3/TwF8/7eDWgeoU/sWCwz+9On1Ec1w4GO83Op7L3b1oMvzGi/KWPNL1k7X5fOL0Vlf28CpfCxSPpSg4hsHesrxTpy/VnRns1q4FCR86kuH2w4dfB95HKncgswRTsL+K3s3mrwm4XsCJ9ACzZz8jT68T2wqoRaviEqnt1jzrBpU8KK25W/RhIwd8TbdQtCMJAVCLoRF6WjcqkZOgpoPPXoMAR5v2Z0ykU38RC1zcKfImGKFGzinxLutoraqOqS7Msoagt/V2mWpb2iolMm6H+aZgqdjVjY9/zERVOnAv3yFCeFSIhKI6EXO0dw11pMWEHjqQmyM8h/X6uX6fxCX9wDRLt3XUlKoq2W3fXzj77jQeryFmQ0LI7GBJ58JpOrwkv8W8h1S7sAhE5w+vvOXN6C/3gHg/9vhUNBpx2ZxX0iEh9oQNbQFf3n12Nh802NUOdCckl7udjsXcc4f6yB8XB9mJybqG0MucArPKJxBAlj5t5OwRr/1mnCQ95RTtw0+hpcL8L1yOp1Wlk/OMCQSnWYEFYcYNxZlN5v7FlJ03pHn+RXUsZXPmz2X3wWf+yZSmEREYQWF4lsRcAK9uRjO343G65ysCCB1WF1ACQClNLVyxuJ3Sv8xlKJyIQiG6UFJ8qDKEfCGMCZ8dldNEJ2aiOozwUffLJALyQTzLVz9SjTBBg7y4/q72srgKg= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(7416014)(36860700016)(82310400026)(1800799024)(13003099007)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: SIHUoJ62BypWPsuudTNDfHK35aZSMa1qfzBVlJzFDYKwLdt6j/SVBVyDKp4bD41S9dw5yNf7VtlFsdHNdOsI1j05FVCxDq7Z5KItPkKijybAt+4aj3X/E/hB88PYTT0Q8Dff4Wy8Iu3Y/aAP6MfVmor3A3N6yyUjIDMXw2eF2SqY9axEIj1l9Hpr0BO3XlBPD+/QxsD2mVr+guep+JZA8V6/NwcVebD3KxEkVkfrElMrc8bRtKBsC5a08k7BCGaF/YPyEMxCan5Gzrh1VK+0F1QUQ4vttI6nkYThg8RwqGfVtlQDLs9ZdvZVyrV3BDlcht2Ud4jkRiuvBCMYiuvf2/AzV0pSfz5w0B0NEX7MxorxuIm5CUzBkLycMgyFg+cRNeTMh+OJl76rGt7l1kmJqnyZLMSGFMgKagFij9shkYMfO8FABjkQITmZVeVvXARf X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2026 11:23:43.5718 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6269188e-c658-4a53-830c-08de90aa4c8d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636A.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6009 Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra Futex hash computation requires a mask operation with read-only after init data that will be converted to a runtime constant in the subsequent commit. Introduce runtime_const_mask_32 to further optimize the mask operation in the futex hash computation hot path. [ prateek: Broke off the x86 chunk, commit message. ] Link: https://patch.msgid.link/20260227161841.GH606826@noisy.programming.ki= cks-ass.net Not-yet-signed-off-by: Peter Zijlstra Signed-off-by: K Prateek Nayak --- Changelog v2..v3: o No changes. --- arch/x86/include/asm/runtime-const.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/arch/x86/include/asm/runtime-const.h b/arch/x86/include/asm/ru= ntime-const.h index 4cd94fdcb45e..b13f7036c1c9 100644 --- a/arch/x86/include/asm/runtime-const.h +++ b/arch/x86/include/asm/runtime-const.h @@ -41,6 +41,15 @@ :"+r" (__ret)); \ __ret; }) =20 +#define runtime_const_mask_32(val, sym) ({ \ + typeof(0u+(val)) __ret =3D (val); \ + asm_inline("and $0x12345678, %k0\n1:\n" \ + ".pushsection runtime_mask_" #sym ",\"a\"\n\t"\ + ".long 1b - 4 - .\n" \ + ".popsection" \ + : "+r" (__ret)); \ + __ret; }) + #define runtime_const_init(type, sym) do { \ extern s32 __start_runtime_##type##_##sym[]; \ extern s32 __stop_runtime_##type##_##sym[]; \ @@ -65,6 +74,11 @@ static inline void __runtime_fixup_shift(void *where, un= signed long val) *(unsigned char *)where =3D val; } =20 +static inline void __runtime_fixup_mask(void *where, unsigned long val) +{ + *(unsigned int *)where =3D val; +} + static inline void runtime_const_fixup(void (*fn)(void *, unsigned long), unsigned long val, s32 *start, s32 *end) { --=20 2.34.1 From nobody Tue Apr 7 04:34:56 2026 Received: from MW6PR02CU001.outbound.protection.outlook.com (mail-westus2azon11012006.outbound.protection.outlook.com [52.101.48.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 384EF3DB64E; Thu, 2 Apr 2026 11:24:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.48.6 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129063; cv=fail; b=iu4AgA4mWPG2HKNUnoUDE1GDYv2dd/i3hZbhZbddwuTYwYRkGbvATU+4ZrbGXYS/8NXeZBA+E6w9Izw+p8II4CXAp7r3MPXd9w52pG4nw48XyHO6tlRtAdChHBknkQ9Jd3tjeFwmkuOUHZe9RSfQcooFIqi7V5soJFuvicXwSUM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129063; c=relaxed/simple; bh=Sd/xcCRGOsdz9/JPfCOXTE0uj2aufo5aXoDz1WMJw+8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WwD+TK0uiKwSmMqMWE3vurUApKsSkaw9SJXd8IQCSnFu442I1JWBbth9/Q2DbJiRxx6aymMbzhWrNu/Mkq/VYM0dlSVIjyz5ChG4x+72pehJs8W16anbgetoV7Z7cIb3uzjFRGaAeFTzRkqa8F3TTdDnrcphls7subVlYDQvU1g= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=guTOkRyP; arc=fail smtp.client-ip=52.101.48.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="guTOkRyP" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mhMAo/uBtV8TJiOiGSsc7P3zghLhv0uS/eXJWH2Ck6mTjOY84iKmWGpO3yziFmjib95IUxtgOLsN4vQNUVFKyXVuOwibV/CvAbQ6bx/vZmZ89LM6VKsYxABIlY18YSe15sCINTZvrNH8C8OGo1DBZuo1LDx44SRI8obkFkAS+sfWti+Jcrg3ZcGOUEGOLgJ1TPIzG50dem0v18Z7Hxg/75/+i9rKSRbJCFE1gpfQwzZl7AbzYeEMLhSvf2vRE4xp4lVQjOWc2Ct+0Zl5i1X7Ko3LUEG801MOvnO+8C1G/RxGY1n3Eb//pBK5j/DAcFrmF3wYizzzPo329irIo/F0KQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=g36eWQBKA0M4/KKIDiJJgYvfQhbALtKtILY1VXZSYI4=; b=t0be+cG++3al8pvWFJOUb8LUKHEBpaGyQvbbSFaI0oXuKa93x+Yt2NT9RFxjyJnBPQCIIJkKW9dKZV9WDR40eyNPx5D8nOcIqRDMK2W2MicHtz5S3nFmTcZcgNR00j3ajlE1ig4aXmHPYXTWVW0Xa3Yn5kpYuXwKfRxLJ73cxUTxkB4+/JbOHs5s2tYwHYZQsqoetgbIifnAcp7tsJFV1p2ggC1nFeZWNVZWDuknOiSj/r0CcbaLRDrFOSkHh22X8cvBhWt/pthGpufmgIYRrONIzN+aJdrVzliwcjY6fks9Suxa9E0X6NTh/tf2GB1uPmlKjO9LsYbzsaBvE/BSIw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g36eWQBKA0M4/KKIDiJJgYvfQhbALtKtILY1VXZSYI4=; b=guTOkRyPtKSOdd9Yh8wD27UCyzTJCLBL4Zvym7kIqyTmsrPv8kvqxb+ypszgXxWp1W277rvLydUt6WTDzmdEJTFTIlavxvc4mEHLXZFbEWNHJbItkG32zhFaQ1tShap/jxojYXZMn1xtVi38sVrieEtTNg66TkKWG/B0IrriIK8= Received: from SN6PR16CA0047.namprd16.prod.outlook.com (2603:10b6:805:ca::24) by IA4PR12MB9833.namprd12.prod.outlook.com (2603:10b6:208:55b::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17; Thu, 2 Apr 2026 11:24:02 +0000 Received: from SN1PEPF0002636E.namprd02.prod.outlook.com (2603:10b6:805:ca:cafe::12) by SN6PR16CA0047.outlook.office365.com (2603:10b6:805:ca::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9769.18 via Frontend Transport; Thu, 2 Apr 2026 11:24:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SN1PEPF0002636E.mail.protection.outlook.com (10.167.241.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Thu, 2 Apr 2026 11:24:02 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Thu, 2 Apr 2026 06:23:56 -0500 From: K Prateek Nayak To: Thomas Gleixner , Ingo Molnar , "Peter Zijlstra" , Sebastian Andrzej Siewior , Catalin Marinas , "Will Deacon" , David Laight CC: Darren Hart , Davidlohr Bueso , =?UTF-8?q?Andr=C3=A9=20Almeida?= , , , , , , K Prateek Nayak , Jisheng Zhang Subject: [PATCH v3 2/7] arm64/runtime-const: Use aarch64_insn_patch_text_nosync() for patching Date: Thu, 2 Apr 2026 11:22:45 +0000 Message-ID: <20260402112250.2138-3-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260402112250.2138-1-kprateek.nayak@amd.com> References: <20260402112250.2138-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb08.amd.com (10.181.42.217) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636E:EE_|IA4PR12MB9833:EE_ X-MS-Office365-Filtering-Correlation-Id: 58c7883a-d2b4-451d-2010-08de90aa57f4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|7416014|376014|82310400026|1800799024|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: 57WAwVfJupASLRI23Sk26Hh7sFY2ZOamkYBQpkfbxSoDK8XQZwUsZwJdRUVZZB948rUGN5bRY2YxyMJEeVek2lzXMDyqjVNAj6JHFtN1MleDb2FF/f+sPqzUEFFvvQU6t+Uq7G60JmH2Rku3uih6wfSCzksmdLLDKoGcPLWSUmY9BcB3fSOCdN6DzWD6wPNeHVtSrxQIJIjM2IJAOxb6e0lZ1/crZ3unqEAVvW+mZSrcDswyZalDfCezav2Pw4RI9HrCUXIXeUXI4g/QPGe9YpW6t9yw6Kv2asjwYMCs3cm1hgDIQpa2xQBxx1Hnr+4sOSXdldKrmMMt6omCBqR7AGTbWLoS80ZL11FTgLxJzO1UiBGeOfyTrhtA1uE0+VoZcKCvkMxuH3L3RIDoL8vV63r4W9JGZAisyeSjJ6N/uqwvUgg6w6ZgYFadSYhonabK1mwRSDYiGR3eDIVJmtLIDkQ6kREV+VEkiKoLryhnGdwaxdGwevz5weubtZ4WCq6+RYOmm/zZCfWL825biLp4vxn1bN3JHAy5WD1N64vMyUzhNv0VMH8tjGkb6rwEMsXDlq01aT08W8UImcMx5W8I8YADvv848q5y7urUVQafWUVY//qpCHSZl4J0gYcZyq4ldZ0+OfZJn+YKCqbreWQqfza+sHqbOE9iOKYtdGTBlI0JOlARslFIxNwIG4tRvNB7LrX7chBAx/DtTAEgTHdk5Gdmfxs61YpUz7X3WXX0QMi95uuMpjJnu35KYXBFvjj7oi1wz8kjw4SklL/kAo/aVg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700016)(7416014)(376014)(82310400026)(1800799024)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: P0mR6wZ15wWfGhVeWC3XWlkWi7wr8m0+wZod16bBnEherWF5OKD00EjqhvfNg022psio0hzKF9FKbC5trxJzIqGp7x+TM4LRIOa6EzgaeI9QUIShnScArS2CDtbmkMT2tq5iIYHiVRq+XvZjhmlxLgmvm5AXksBgmeevRazwJXbRd7L9Ey2rc7fTfBX+yv67otPQVy//sB5wZWjLKJJVrbCXSziTvN8DcEvgzhQVzUzOLt34tzSvE49fDgkGQusuX2orY7uxSpn3mHKvReTQJfGCpl49Dhray8g7vF8n1SkiW5ZJUW3C5oih3UL7L4TauPG425EmmL9bBwt6nRDPiZY3NA27HdN5s+MkmTTS1MEU4CIMl8ew5PAie8v3ObDHrt79Y2c6opC2A0vBSCcj4nx5hwE5Zg/zlfcZ6XICmQeS8Z+rKSoD95iuKFFcJ8Li X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2026 11:24:02.6995 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 58c7883a-d2b4-451d-2010-08de90aa57f4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636E.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA4PR12MB9833 Content-Type: text/plain; charset="utf-8" The current scheme to directly patch the kernel text for runtime constants runs into the following issue with futex adapted to using runtime constants on arm64: Unable to handle kernel write to read-only memory at virtual address ... The pc points to the *p assignment in the following call chain: futex_init() runtime_const_init(shift, __futex_shift) __runtime_fixup_shift() *p =3D cpu_to_le32(insn); which suggests that core_initcall() is too late to patch the kernel text directly unlike the "d_hash_shift" which is initialized during vfs_caches_init_early() before the protections are in place. Use aarch64_insn_patch_text_nosync() to patch the runtime constants instead of doing it directly to allow runtime_const_init() slightly later into the boot. Since aarch64_insn_patch_text_nosync() calls caches_clean_inval_pou() internally, __runtime_fixup_caches() ends up being redundant. runtime_const_init() are rare and the overheads of multiple calls to caches_clean_inval_pou() instead of batching them together should be negligible in practice. The cpu_to_le32() conversion of instruction isn't necessary since it is handled later in the aarch64_insn_patch_text_nosync() call-chain: aarch64_insn_patch_text_nosync(addr, insn) aarch64_insn_write(addr, insn) __aarch64_insn_write(addr, cpu_to_le32(insn)) Signed-off-by: K Prateek Nayak --- Changelog v2..v3: o Reordered this to come before the introduction of runtime_const_mask_32(). (David) o Trimmed down the commit message to be more precise. --- arch/arm64/include/asm/runtime-const.h | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/runtime-const.h b/arch/arm64/include/as= m/runtime-const.h index c3dbd3ae68f6..a3106f80912b 100644 --- a/arch/arm64/include/asm/runtime-const.h +++ b/arch/arm64/include/asm/runtime-const.h @@ -7,6 +7,7 @@ #endif =20 #include +#include =20 /* Sigh. You can still run arm64 in BE mode */ #include @@ -50,13 +51,7 @@ static inline void __runtime_fixup_16(__le32 *p, unsigne= d int val) u32 insn =3D le32_to_cpu(*p); insn &=3D 0xffe0001f; insn |=3D (val & 0xffff) << 5; - *p =3D cpu_to_le32(insn); -} - -static inline void __runtime_fixup_caches(void *where, unsigned int insns) -{ - unsigned long va =3D (unsigned long)where; - caches_clean_inval_pou(va, va + 4*insns); + aarch64_insn_patch_text_nosync(p, insn); } =20 static inline void __runtime_fixup_ptr(void *where, unsigned long val) @@ -66,7 +61,6 @@ static inline void __runtime_fixup_ptr(void *where, unsig= ned long val) __runtime_fixup_16(p+1, val >> 16); __runtime_fixup_16(p+2, val >> 32); __runtime_fixup_16(p+3, val >> 48); - __runtime_fixup_caches(where, 4); } =20 /* Immediate value is 6 bits starting at bit #16 */ @@ -76,8 +70,7 @@ static inline void __runtime_fixup_shift(void *where, uns= igned long val) u32 insn =3D le32_to_cpu(*p); insn &=3D 0xffc0ffff; insn |=3D (val & 63) << 16; - *p =3D cpu_to_le32(insn); - __runtime_fixup_caches(where, 1); + aarch64_insn_patch_text_nosync(p, insn); } =20 static inline void runtime_const_fixup(void (*fn)(void *, unsigned long), --=20 2.34.1 From nobody Tue Apr 7 04:34:56 2026 Received: from SN4PR2101CU001.outbound.protection.outlook.com (mail-southcentralusazon11012022.outbound.protection.outlook.com [40.93.195.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A4D23CCFBA; Thu, 2 Apr 2026 11:24:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.195.22 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129080; cv=fail; b=QkGcmIm5iTq8suVYxnD8QIBb2EdySA0eFNDwmrrvzxwPbc9/4iqJrZUOLXXR6cWfqGlFinXP9BNEKoJvpAC9Y+ER2evFLjD08HUAUMK0uTdvCInGA4GnJF6Xmc9g1JoURhRsTXaj8NsAzJC1TtI//nU6LWjtH5+AFQxP1DGwYAI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129080; c=relaxed/simple; bh=cHgyYE2hKzAP04kG4BQncj6GNjlER+OVmWXlcIFYgvU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=t3GgpxiFA041EexI3vc1pxQh2PcNKTXL04o4cKffRJ7d3/T+vQpr4h7f+BRjAgBvdOLWI4wCzGq6ybpRR6Wa1LDWVy8Yu1AkuLKQl/D/XhcqzJOG/5tiNYuvwV6AeJ9xmcnZ8gWqW9spvC9VyBKNRPqkszS8LxfS21mcOTlpibw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Q9f4ez2j; arc=fail smtp.client-ip=40.93.195.22 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Q9f4ez2j" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=F8L7o1WCMDkTwbCQcS5RKz51M123286PJNjiEZRqY2BORCJm7Je5M8YkFInsvj2pCyG3m07IMiUuTJjmVS2Odr3EhStV0ucIwZsTE1cyUjbldvoHSSOCogf7GNvwXexq/iR1x/4UsT2GHvOSOuzh9cMx0/DQsZJVnt+g3RHwmM57If1e0DU+qCYR2CHegfcgr69P7mJIcl/DjGTXnhL2WOnrxjGSmb41lvNwO9q74Jb55uaDymhB9IciVwufJgZdkZPETMIUKOwiEWNL5jdOBkmOd8XTX/CNiZ0bCjrflUkAsT/DNtbGeJMtE9MXgYK521jOEGAF/6N47j6IAWVOCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CHd3Frej8BOMPcom8zD7zCOwGTywG+aA8p8YQ0sbL6s=; b=IFp8Ma9iLTLh5SSDlr17b9CJ0sCmiyiyQYAPqrdOGDIghTWbTmgnqeRYtmM9f+YZBr/fRRc+VEmomsmrlIkY9LeRhUUpy9AYIzLAWn4aHhAn5zUu3SCrUH+pDLCVywkX4mSy75vgh4pRO9FqQqW7k2tPPr44zJzKSfDBHdJv+Yjl+oN1Jd1imJLcopZuDB0Hr00LfkRi7PVgk/RBybd9fwpcrCu6/RwyV4b+pK2icqu608DJ/oUGXb8YbxxNqW8FZCfMpJ/dk+Yz3OTmemMXQi/OU1m5KKjEXBF1iNkYqcjfmev8W8jAzdp4OxPWFg+4bwvumtwSl+aOMVplC5q7GA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CHd3Frej8BOMPcom8zD7zCOwGTywG+aA8p8YQ0sbL6s=; b=Q9f4ez2jcOw1yY0shB8kiqE+fsOKD0Ful005mwyf9zXRIwceH4pbuAJ2l7499x3LpgSAHn5ec5d5vAngH/BUedohamGeRgtHtQ8u7pBT2jeoTfFiUZK5Wgd2tw63GACIbtx+bZNsUkfhMrnCQ1mTYHG4Kf5Bb/18grpGLcexdVs= Received: from SN7P220CA0012.NAMP220.PROD.OUTLOOK.COM (2603:10b6:806:123::17) by SJ5PPF75EAF8F39.namprd12.prod.outlook.com (2603:10b6:a0f:fc02::999) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.6; Thu, 2 Apr 2026 11:24:25 +0000 Received: from SN1PEPF0002636C.namprd02.prod.outlook.com (2603:10b6:806:123:cafe::71) by SN7P220CA0012.outlook.office365.com (2603:10b6:806:123::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9769.18 via Frontend Transport; Thu, 2 Apr 2026 11:24:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SN1PEPF0002636C.mail.protection.outlook.com (10.167.241.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Thu, 2 Apr 2026 11:24:24 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Thu, 2 Apr 2026 06:24:19 -0500 From: K Prateek Nayak To: Thomas Gleixner , Ingo Molnar , "Peter Zijlstra" , Sebastian Andrzej Siewior , Catalin Marinas , "Will Deacon" , David Laight CC: Darren Hart , Davidlohr Bueso , =?UTF-8?q?Andr=C3=A9=20Almeida?= , , , , , , K Prateek Nayak , Jisheng Zhang Subject: [PATCH v3 3/7] arm64/runtime-const: Introduce runtime_const_mask_32() Date: Thu, 2 Apr 2026 11:22:46 +0000 Message-ID: <20260402112250.2138-4-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260402112250.2138-1-kprateek.nayak@amd.com> References: <20260402112250.2138-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb08.amd.com (10.181.42.217) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636C:EE_|SJ5PPF75EAF8F39:EE_ X-MS-Office365-Filtering-Correlation-Id: 0e1d557e-a307-45bd-5eb3-08de90aa6535 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|82310400026|1800799024|36860700016|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: Kw2yM0yYJQ9vgG08dRfEmXoipSrlJilbZ14XgvbNb5CjtDqwJ5Tpsz0xsjaeco8eYlwVhw0fd5FCPrtYSrhRD4wOFR8ujrIIx8thDdfs67dSifP7GrwliJM9XHU2cATewPTXaPwtjgmTf8rR9itRV5X5VpjpC/gjSZyE5QRV532NYCbXYvbAexaLHsOr4Z6zKUO5ihXIYdZK8IKv4zlUgEhyYWUsTQRD7O7A4Jl1UDqeTGGzo/Xu5k1k/x7OitJQhtQLJwH70DghZ3mcfG3Wp/XtXyw5ysyGaqFeXs6XeNgukcDUh7+pS9+MhzGX8tpwhcGdb7U1WTyB3TPpT1LaEiA3BPcc+Lo1oyyT95DnlzVFZ4outTDMSfqc10784Ad/W3ZghYkkWIQdo7QOuzkR1vbpz9aLJVU/6JMZN1vTj25bb8HWJwzpxeiH3thPC6eGFgMJvmpyqWUO7USMz1Ka2ixrcdpvjmkzwY+BSzRd9Mi+BTpe3D4hebbUs3OKTTcr9ef5/zsUNxhfl5Jl38vObV5PgoSufci8cNQgNBxv3zopofI+6xXdT3QkUB/1IFjICqlrjc6euGsmjiArEL2QlDA99NUho5er4Fd0V7FJHXWQYqaKU6A4FT8QbPZgWTZVQ8R5oRb7W3D+5nMZHRheDhI8LLOx9+Pl/py8sv51iPjDYIRk3NIjIy5NGFFlHKaAMbQT42KH5z3ttQY8GdvWWLvCJ43xP+8P2cidMstbdZ+gDpYVUDhDTLnLlsTg+iOdKyDXNEiAivvPehlNMtBS4Q== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(376014)(82310400026)(1800799024)(36860700016)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: gVWg1bqVn/dD/eufo+YLxaD1YoTJzBtlrunYUkkWXw4M3h6KiY8fhCm2qFX6mSBagElEUPsNNk9XD0HoQtqCcZvE9so1dF9KHgbQFo4ZDLIaI4UiaWCcD4R2dcxBVzIINuLtugpk0SAtzYz9FS2CUG/l9ZXByJObZ9x6NTqRXBPuFDiAU+rYSOKsLSe8Sad4kMclVI0Ii+uBP3lkAIYelM99Ae/aZFibT7WwfZPxOmoT4tNxz4yiMutdsOhr8bj3WNFdtmEhCx03rNgKW9ClEZ0kuliC58WFXIuWWcQE+6FTVebl3G/YqaY7Wamb8QmMBBJdWDx/jY3IJJK1CfvOZ34oNWtRvtQgRQqrLjiIgDc/gvOlkDz2ZxDhOk23mOVx1IR1WORj9Fq0W07pjStbV2sQmJ5mDj86rjbkzYg4gbJcnZIJ+CRczc/X32Paqh3L X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2026 11:24:24.9355 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0e1d557e-a307-45bd-5eb3-08de90aa6535 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636C.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF75EAF8F39 Content-Type: text/plain; charset="utf-8" Futex hash computation requires a mask operation with read-only after init data that will be converted to a runtime constant in the subsequent commit. Introduce runtime_const_mask_32 to further optimize the mask operation in the futex hash computation hot path. GCC generates a: movz w1, #lo16, lsl #0 // w1 =3D bits [15:0] movk w1, #hi16, lsl #16 // w1 =3D full 32-bit value and w0, w0, w1 // w0 =3D w0 & w1 pattern to tackle arbitrary 32-bit masks and the same was also suggested by Claude which is implemented here. The (__mask & value) operation is intentiaonally placed outside of asm block to allow compilers to further optimize it if possible. __runtime_fixup_ptr() already patches a "movz, + movk lsl #16" sequence which has been reused to patch the same sequence for __runtime_fixup_mask(). Assisted-by: Claude:claude-sonnet-4-5 Signed-off-by: K Prateek Nayak --- Changelog v2..v3: o Reordered this to come after the text patching fixes for ARM64. (David) o Moved the "&" operation outside the inline asm block to allow for compilers to further optimize it if possible. (David) --- arch/arm64/include/asm/runtime-const.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/arch/arm64/include/asm/runtime-const.h b/arch/arm64/include/as= m/runtime-const.h index a3106f80912b..21f817eb5951 100644 --- a/arch/arm64/include/asm/runtime-const.h +++ b/arch/arm64/include/asm/runtime-const.h @@ -36,6 +36,17 @@ :"r" (0u+(val))); \ __ret; }) =20 +#define runtime_const_mask_32(val, sym) ({ \ + unsigned long __mask; \ + asm_inline("1:\t" \ + "movz %w0, #0xcdef\n\t" \ + "movk %w0, #0x89ab, lsl #16\n\t" \ + ".pushsection runtime_mask_" #sym ",\"a\"\n\t" \ + ".long 1b - .\n\t" \ + ".popsection" \ + :"=3Dr" (__mask)); \ + (__mask & val); }) + #define runtime_const_init(type, sym) do { \ extern s32 __start_runtime_##type##_##sym[]; \ extern s32 __stop_runtime_##type##_##sym[]; \ @@ -73,6 +84,14 @@ static inline void __runtime_fixup_shift(void *where, un= signed long val) aarch64_insn_patch_text_nosync(p, insn); } =20 +/* Immediate value is 6 bits starting at bit #16 */ +static inline void __runtime_fixup_mask(void *where, unsigned long val) +{ + __le32 *p =3D lm_alias(where); + __runtime_fixup_16(p, val); + __runtime_fixup_16(p+1, val >> 16); +} + static inline void runtime_const_fixup(void (*fn)(void *, unsigned long), unsigned long val, s32 *start, s32 *end) { --=20 2.34.1 From nobody Tue Apr 7 04:34:56 2026 Received: from PH7PR06CU001.outbound.protection.outlook.com (mail-westus3azon11010014.outbound.protection.outlook.com [52.101.201.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28A673DB62E; Thu, 2 Apr 2026 11:25:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.201.14 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129112; cv=fail; b=rWDirXwqgIwg4gCMq5W+YMh70MRbjfNzH266Yv8ejzLozhukFFQDj7cH8exskdJnxw5LAVKdHK8tT2X1VuXLSURPWHBO810aIWcLk9RKg6hGsSFJPj+BFATox2XyCwgy+XMTNmBMDc6nr3BUKpDXxZncyN07/zHVicskL4Zp4aM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129112; c=relaxed/simple; bh=aa72g+kjmtrkaOUdelUMgIkbkvLcyaYu2EkrafpBqnk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=I7bdIRF31Za6eZ/6x2vfSRz/7+LQlN9h041033DPNZaJVR/GEj0rROzURXg2BxYGtVa+pPXiaAVASptUjzZGPFeg+YI6NaE+o//5Dw3Sj7RmFf/PsV9h6mXMUvvSxwRa9KfE/ppJ3EZDAOBbYUmyFWEj27NPh2F3adaehiP+7Xw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=oAGg7lkA; arc=fail smtp.client-ip=52.101.201.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="oAGg7lkA" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=R2OalziUPqlOmTU4ufr2SJ6pnXmZJ97o9g+b1pVl8+/7C2znnVZwQu9HEz1qnYWt60a/dRd5KrTgDz8F05N32yIsLR7xHFyw34TI/NyIamN775jocD6zjW9aynZOERNj16apzt0CqZ123QTjDijZ/N+G2SVpfN50/He7fdK3dJwGOV0MGLrKkPZNs2qDq1+VCZCU6APARK3oGzs4U1nQ3PW32V/66T+AmxeOUFFVw5ye+ZR3ddVtsLlr5UJT+fyJLRRhVyR+jUgXcORytpUFa4Hga+lr6vSw+Ez8Cweq0FrAcvKMZzsh8pERtIVnbre2HA3MxRiAIeyBHIKd8gAiew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RHSa+xo/iTViJb7JDvDPIatbIaKCf8lHcG1QohC93pA=; b=GpsN/7VPlFvjhEhooiJYcbCadHd4c0Iue+G5hXnkLiF7SHJjXuGClx6ifyxqC3uPxLQ3Kod6lQnyL6x8ffCw5ent+jswuAQNvv8lj+daJBqevSKbzXnoMt0vkQHilHIN2Q3fsaSTfQaBzriNq20F05Yo4NdGyU2MGx+bsT7/gfN9WeBaXnY6L/+2ZDgX30nAMh42V3DTvUqNlOknn2wQ0G5PBD8Kqs2FNTLBubkQwc38595J6g2zbGrPzm2/7iFND49uFqe5J/351UnqM9tMSyKLOMWu29E1KPl+9BPE9FDLlbFm/OZgMsaF/upWAOtb52GpYFT5KpmqBroOX/A4Rg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RHSa+xo/iTViJb7JDvDPIatbIaKCf8lHcG1QohC93pA=; b=oAGg7lkAaeBy5aAWHPDrNT8ymzVfdkc/oRo4eYCUj9QPg7U1BHBQzRH/Kp/HBkzVcdzn5CN0FsHR5Md+6c0YeMr6z5XrmIVv8RiP84oJ82EtlzPgfqkN6ZIFuLny7QKnnlkTL3+T7TXTDcJtOjB/yT1qQsdywEf4Pb8BxWFL5AA= Received: from BY1P220CA0017.NAMP220.PROD.OUTLOOK.COM (2603:10b6:a03:5c3::6) by MW4PR12MB5602.namprd12.prod.outlook.com (2603:10b6:303:169::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17; Thu, 2 Apr 2026 11:24:56 +0000 Received: from BY1PEPF0001AE17.namprd04.prod.outlook.com (2603:10b6:a03:5c3:cafe::c6) by BY1P220CA0017.outlook.office365.com (2603:10b6:a03:5c3::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9769.20 via Frontend Transport; Thu, 2 Apr 2026 11:24:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by BY1PEPF0001AE17.mail.protection.outlook.com (10.167.242.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Thu, 2 Apr 2026 11:24:56 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Thu, 2 Apr 2026 06:24:37 -0500 From: K Prateek Nayak To: Thomas Gleixner , Ingo Molnar , "Peter Zijlstra" , Sebastian Andrzej Siewior , Paul Walmsley , Palmer Dabbelt , Albert Ou , Samuel Holland , David Laight CC: Darren Hart , Davidlohr Bueso , =?UTF-8?q?Andr=C3=A9=20Almeida?= , , , , , , K Prateek Nayak , Alexandre Ghiti , Charlie Jenkins , Charles Mirabile Subject: [PATCH v3 4/7] riscv/runtime-const: Introduce runtime_const_mask_32() Date: Thu, 2 Apr 2026 11:22:47 +0000 Message-ID: <20260402112250.2138-5-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260402112250.2138-1-kprateek.nayak@amd.com> References: <20260402112250.2138-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb08.amd.com (10.181.42.217) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY1PEPF0001AE17:EE_|MW4PR12MB5602:EE_ X-MS-Office365-Filtering-Correlation-Id: 9b0bab9c-1928-4b5f-9d81-08de90aa780a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|376014|1800799024|82310400026|7416014|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: mYliQQmrJ0DDcAY29ITT+0LdcXRBob4luygG382bGZorqfCuDPnd9ihz2Y9W3ZwS+5wCRq8hPihix6qul8PpR2OmYsMelsOwZdIWZs8yHozCpfiW3J23CUCBWnjsoBYalpcFwcwKvkDPrlVNKn4ckEwDdlxj198ulkriL7dSFj/khGr9wDjMejsIgSb0CYXoM7HDivDc4U0uKQx5mc+eH0/QtNlDIvfXwYYcwNS4TcgBtccRlyFwWGIJ51lXLJ7hDMbMFD+aPqi+BUFBPtbx6PdXgi/D9+DR6Fl3tBNkMVjxARkX0TEFGJeiLilC61kt96IVVbZblQT+rDtg+ZHVj+wVopMRPOget7qYaIiNBz2ZbEBLwo9j5PN8nMikFqRV3r81JJmQ/GlQ+YAPOzTeEm3m2QSyDuxBgoBWoKL+Xeo6Pt5+khwMT5wj80YRLT7dumn5Onvn/6uTNQcxzCy8V07+oVJT2FZPZubu0p0TwKtXRTBmkXqFzG1IolJjVsbPke3WSi4P1p3niBIAKvy9LxfRmhOSrxun1cUlkR7UTLVmHG6EV9kaFJhduuXsRymOHlHhVZpwTmH9BfeneNcUBdcglgE9Y2AhNM83JSw0RtYzToI3Rlsl4IVjUJcFUQRV47eAcVAfGtLXyOY8P9DZCItNv+CvqCEf9BNY10O0mjJJuuxWZJQ2IQYmZgeD9QBKux14hlPuP7/pICuJMU8xJ/6NXrhN4DoS7DtWCadKrWnBoxkdBXRLvHPA1GlxboBZeitnxUue5IUEEe2rUBnCbg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700016)(376014)(1800799024)(82310400026)(7416014)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: aKLCGsXlrPY0ulIXhwYdqgSAbafUs5KqMVtton1feddH2iliF6orIzjQRlx9+wM3G7fMOvQebn3qY8ocgEphomQRmKUFdxYqx/sJGZb2i4XoLAf2AqQit3ZYk1RV2+eYZ0PsszWd3RlmLmik839jiEg/yB0Ht9j8N0yld28ItOgPX/qqfaKZhcZCVS98WoeA0HdXOI6SXrB7fjm0RqKVAwptyfs9Up3svwsszgyuB+8Do6CzlGVk86n8wbH2NWRXGoGJcTn2RhWvs543zS8icTux4oTip6LfI4bhUdYoGywxyIGWT39u6nvyoz9cWiezEddf4XLQz+fF0PbC7/J4WGK0LbFDuzzQ++uvXBOVCS4Yl1f7gwMYlaZ/LzGEZrw+J4Om+Ux3D0l++17hKOYNyFrZfdRESLrhHOVkp7E+/YKPwHpXb14HHtTW3iTFjMS2 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2026 11:24:56.4585 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9b0bab9c-1928-4b5f-9d81-08de90aa780a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BY1PEPF0001AE17.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB5602 Content-Type: text/plain; charset="utf-8" Futex hash computation requires a mask operation with read-only after init data that will be converted to a runtime constant in the subsequent commit. Introduce runtime_const_mask_32 to further optimize the mask operation in the futex hash computation hot path. GCC generates a: lui a0, 0x12346 # upper; +0x800 then >>12 for correct rounding addi a0, a0, 0x678 # lower 12 bits and a1, a1, a0 # a1 =3D a1 & a0 pattern to tackle arbitrary 32-bit masks and the same was also suggested by Claude which is implemented here. The (__mask & val) operation is intentionally placed outside of asm block to allow compilers to further optimize it if possible. __runtime_fixup_ptr() already patches a "lui + addi" sequence which has been reused to patch the same sequence for __runtime_fixup_mask(). Assisted-by: Claude:claude-sonnet-4-5 Signed-off-by: K Prateek Nayak --- Changelog v2..v3: o Moved the "&" operation outside the inline asm block to allow for compilers to further optimize it if possible. (Based on David's comment on ARM64 bits). --- arch/riscv/include/asm/runtime-const.h | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/arch/riscv/include/asm/runtime-const.h b/arch/riscv/include/as= m/runtime-const.h index d766e2b9e6df..85efba8ecf12 100644 --- a/arch/riscv/include/asm/runtime-const.h +++ b/arch/riscv/include/asm/runtime-const.h @@ -153,6 +153,22 @@ __ret; \ }) =20 +#define runtime_const_mask_32(val, sym) \ +({ \ + u32 __mask; \ + asm_inline(".option push\n\t" \ + ".option norvc\n\t" \ + "1:\t" \ + "lui %[__mask],0x89abd\n\t" \ + "addi %[__mask],%[__mask],-0x211\n\t" \ + ".option pop\n\t" \ + ".pushsection runtime_mask_" #sym ",\"a\"\n\t" \ + ".long 1b - .\n\t" \ + ".popsection" \ + : [__mask] "=3Dr" (__mask)); \ + (__mask & val); \ +}) + #define runtime_const_init(type, sym) do { \ extern s32 __start_runtime_##type##_##sym[]; \ extern s32 __stop_runtime_##type##_##sym[]; \ @@ -256,6 +272,12 @@ static inline void __runtime_fixup_shift(void *where, = unsigned long val) mutex_unlock(&text_mutex); } =20 +static inline void __runtime_fixup_mask(void *where, unsigned long val) +{ + __runtime_fixup_32(where, where + 4, val); + __runtime_fixup_caches(where, 2); +} + static inline void runtime_const_fixup(void (*fn)(void *, unsigned long), unsigned long val, s32 *start, s32 *end) { --=20 2.34.1 From nobody Tue Apr 7 04:34:56 2026 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013023.outbound.protection.outlook.com [40.93.201.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 421343C1992; Thu, 2 Apr 2026 11:25:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.23 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129120; cv=fail; b=c78RQTNobHr6bZbM9/RQe5Qob/kGP2a7TfglQS+NXjkPQd8cRUzv5ZD49FBBFVz7F/BF4TumwVRS5WaH+68LAiAYnR56EEZYNq7TyiRfbzfjKlvowooiPHAi27tS0h79UEgMrLVqxEtQbHyDBjqRDfQBSXu/jtrBvy3UcB/7JDQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129120; c=relaxed/simple; bh=u4Pm+rhnDGkXq5chjgU54nOWObWJe0SA0j2GOuZExxo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GCCpwVvR02YixBUJj1X05KUndt3IEKuNDYrCSozgQCAGyNUMwqmNz8AbuHhJ6IG0+D/ECn7+zTFtuKThWbZEi2Pcu2eDlAfrt8uWivvWyN7PP/WTC8856hKYAsbyvtp1sBgIQdXYW98jSMONb2CnqwTLMSuVToroa8b7eWBnuyQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=f4/X6LIq; arc=fail smtp.client-ip=40.93.201.23 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="f4/X6LIq" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Gsx2qf4YUQ0n+uSvz8CBURF6ngDm4I0NYKfmlCDa+zCLeqlIav/+qSZR6XiMQJQzQX8e9v5/qnd8p9v0hMVscX0LAqC/BdSvDqHAcgwYc2MVis5VBQ7PSJOwY0GaMnJjk7RbFXRSDDq4gEqYEa/Q0XHBqkQC+3vNZs9Od1nCsPrAfsYCvvpWxRMhMuyabr0+rum2iJRpqC6Q/PYNgdvZavtCTXojiVi4+LWbfP89Q7yJBOaNbaich6dzpKkbPNbKcz/7AWhWp/5163zkHmEvIf8SvikEytlGaNkK7qg/vf+4xclE0exb4kPyjQR7GAmXF/o5i5kNCE1W9NgjvM13Qg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2g8iV9LAM6qau6Y/Bj5XnlgP3GqkVABvY4LMWp9IStk=; b=BuNgINBrThAoPSL6JuvqqQm/FzY7mLz6Ijuiyaaldcp6flV1PX10Tz4nDUu8LGyLpksJA9AtQ1vhW22GubUYRq+eWUkACZLthRcrl6aZA+kGgpNZMJs6AxKPTxTVVlCQVvNAcR01qu551KCmSXKYZOa6tYGlQDtNvSJ7CEfV0dZHfWXpx9IvmG1pJBm5G9171JVEO2piHSqxzH7c+unRyztLOKE4ZUZZZANuki7rplEMjZ4vtLgN2zkeRUe+C90L6ayGI1LADtrE78rm15xwqojySubNJX745tHCrZXrhorK26q9DmJRMQ2JQRnUquwJbDeGfJfon1VgWI/g1NQ6hQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2g8iV9LAM6qau6Y/Bj5XnlgP3GqkVABvY4LMWp9IStk=; b=f4/X6LIqeAFTNnZO5E0gCrNWOFhhDcKnR+/fKkYahkdtqzKRGPUFsjeyGfa+mMYQWAhcLs4ExY7ZZI0CJjStOCKApTg7FkDasNo+wfoLRh1bBEmEm5EyVa2PMUClzQkguduKVp4HxVT4aYA+mbdv1P6ERLMpmQC5TGOvG2RvLso= Received: from SN7P220CA0009.NAMP220.PROD.OUTLOOK.COM (2603:10b6:806:123::14) by SAVPR12MB999143.namprd12.prod.outlook.com (2603:10b6:806:4e5::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.16; Thu, 2 Apr 2026 11:25:05 +0000 Received: from SN1PEPF0002636C.namprd02.prod.outlook.com (2603:10b6:806:123:cafe::74) by SN7P220CA0009.outlook.office365.com (2603:10b6:806:123::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9769.18 via Frontend Transport; Thu, 2 Apr 2026 11:25:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SN1PEPF0002636C.mail.protection.outlook.com (10.167.241.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Thu, 2 Apr 2026 11:25:04 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Thu, 2 Apr 2026 06:24:56 -0500 From: K Prateek Nayak To: Thomas Gleixner , Ingo Molnar , "Peter Zijlstra" , Sebastian Andrzej Siewior , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , "Christian Borntraeger" CC: Darren Hart , Davidlohr Bueso , =?UTF-8?q?Andr=C3=A9=20Almeida?= , , , , , , K Prateek Nayak , Sven Schnelle Subject: [PATCH v3 5/7] s390/runtime-const: Introduce runtime_const_mask_32() Date: Thu, 2 Apr 2026 11:22:48 +0000 Message-ID: <20260402112250.2138-6-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260402112250.2138-1-kprateek.nayak@amd.com> References: <20260402112250.2138-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb08.amd.com (10.181.42.217) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636C:EE_|SAVPR12MB999143:EE_ X-MS-Office365-Filtering-Correlation-Id: 915c3c05-c1af-4830-19db-08de90aa7d14 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|1800799024|7416014|376014|82310400026|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: uzK+rKcm9Au4cQJUgs5WmQQL4yd57I96V3eLoqKW8ouUM955a3c/lWVnPdjr/CO2Tz9K946IPj5c5OJpO7DrOv5JwNWJlaiZD1K6qdDeIvv6O7pcxJ/lQ6DiHgiQQbWCQp7wUQqUt9Ubp7VmKJDErbTsWcWW6Kp7yYno+BAIstXb+JcdRtfm2WVK1Now0Cvmkuw0izaWYEtA8noIhuyb+89qaQ01iILpJRjKLavlKe/1SKYX6uYfFoC4ApGfzQL4zZGbzXq9VNv6ImgQCVsjvTUFrTODO3sNlDU+kMQOvfTAh4TMPNmM1dNDKn428u6UGzEaAeXF9l1+zIMU2xms3Bz1NlpjUfSGbDwotn+96uy7skgo9d5jmVB2N37UJbaRvPpUv3BGoGblr0t3gAiQvMM6+xCVhVBfKBihsyj5wAP4C8hQU7zYQ1UUa4MgEOYbHiYMyu8nsXmD6TJdvLEntiari7SvHDOMgQcBGHIzC8KGFQtYCno0XODeRr0Wu63OcO1fa8ePhl47gLBEvTGt5HT6DuQ/8pm1uSkwhMXAbYPFoijNzqyD3jTNyS1800H61mi2satOH1WbwggiMV/2UeDOBsAO95ZuJp8kmjIDy1he7JLHtVZvSycLO1hhf55y9roH7LT5d5iQZhNwhwjAi89Y0bZLKdLR4ZSoj5o2cC6ty16g2k6/+q3ZSAFivxpOveJJjwLOBaM8X/wmROvKMTDhRe+JrOVpHEquqTwqjWd03KkbaaokxTKLhXixDG2TzMY+MTRVp/6BD7eHHhTKFA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700016)(1800799024)(7416014)(376014)(82310400026)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ioXQ4y4yHhbk6gz4Mz+/SH2pX+GTFAO+MQRYxHIsvCZx9E6Gln5i0rs0Ar3GLwDUG88uOn3sm78phW9cIjYDSaPZULGvsvCTWVomuM1PytRvZp2+Mbi6qaZBxX6OmuhCm8to5lf0dAA3cQJf9Vs0zrp779ibzvkKsmqkN2IeCoZDeRzzDAh7dp/X89z/Q475PK2U+PCDUF/ADh6iT5JlW1pARccc960lmon2taDg06Gbz+HUbTg6MkmPhfWK4fq8o71be2r8yfzcJhzymmWlvtZJn6E+1nq9neTFlsxKsBebIAQ0EXp2aPo3saVPwEFMZ5LONl1avRkPGUVS/LvwcFbdUNoYNASexYrHzOPeaMfbXHZsC6XS2cD8yt/mVrBLrwSesTHgrFSTr9wW4/hKozlGEPI+wkJ1Mp1DCuSXnPK/dawWxw2lyCkKG8Cm65uL X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2026 11:25:04.9840 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 915c3c05-c1af-4830-19db-08de90aa7d14 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636C.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SAVPR12MB999143 Content-Type: text/plain; charset="utf-8" Futex hash computation requires a mask operation with read-only after init data that will be converted to a runtime constant in the subsequent commit. Introduce runtime_const_mask_32 to further optimize the mask operation in the futex hash computation hot path. GCC generates a: nilf %r1, to tackle arbitrary 32-bit masks and the same is implemented here. Immediate patching pattern for __runtime_fixup_mask() has been adopted from __runtime_fixup_ptr(). Acked-by: Heiko Carstens Signed-off-by: K Prateek Nayak --- Changelog v2..v3: o Collected Ack from Heiko after folding in the suggested diff. (Thanks a ton!) --- arch/s390/include/asm/runtime-const.h | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/s390/include/asm/runtime-const.h b/arch/s390/include/asm/= runtime-const.h index 17878b1d048c..7b71156031ec 100644 --- a/arch/s390/include/asm/runtime-const.h +++ b/arch/s390/include/asm/runtime-const.h @@ -33,6 +33,20 @@ __ret; \ }) =20 +#define runtime_const_mask_32(val, sym) \ +({ \ + unsigned int __ret =3D (val); \ + \ + asm_inline( \ + "0: nilf %[__ret],12\n" \ + ".pushsection runtime_mask_" #sym ",\"a\"\n" \ + ".long 0b - .\n" \ + ".popsection" \ + : [__ret] "+d" (__ret) \ + : : "cc"); \ + __ret; \ +}) + #define runtime_const_init(type, sym) do { \ extern s32 __start_runtime_##type##_##sym[]; \ extern s32 __stop_runtime_##type##_##sym[]; \ @@ -43,12 +57,12 @@ __stop_runtime_##type##_##sym); \ } while (0) =20 -/* 32-bit immediate for iihf and iilf in bits in I2 field */ static inline void __runtime_fixup_32(u32 *p, unsigned int val) { s390_kernel_write(p, &val, sizeof(val)); } =20 +/* 32-bit immediate for iihf and iilf in bits in I2 field */ static inline void __runtime_fixup_ptr(void *where, unsigned long val) { __runtime_fixup_32(where + 2, val >> 32); @@ -65,6 +79,12 @@ static inline void __runtime_fixup_shift(void *where, un= signed long val) s390_kernel_write(where, &insn, sizeof(insn)); } =20 +/* 32-bit immediate for nilf in bits in I2 field */ +static inline void __runtime_fixup_mask(void *where, unsigned long val) +{ + __runtime_fixup_32(where + 2, val); +} + static inline void runtime_const_fixup(void (*fn)(void *, unsigned long), unsigned long val, s32 *start, s32 *end) { --=20 2.34.1 From nobody Tue Apr 7 04:34:56 2026 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013056.outbound.protection.outlook.com [40.93.201.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F8BF3B52EF; Thu, 2 Apr 2026 11:25:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.56 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129142; cv=fail; b=g0g7m24AHEkrhSukno3AhQYzsk5tqf48J9gH3gmDlu9XMFGz+nSf4oZC95zLSfu65DzhBr3IFUI7y6qkoCR4NaKlXVrZDCikxrE54uNrsmzw2W2sfnpydoF9u5sweGMaaf4U1FFZ1qTPyXnnkOC4cZ4Z6pmil4HvWQnjFSX2ZXY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129142; c=relaxed/simple; bh=7uLdhrwqEP5cyieH1s3lHw6Gj94g6/MJ+TU9/HHm4ZU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=I793a988gzZF9PtNri+FIdd7WSYM2gQTpXQPTTZGrRRT5e9OGaax9sGyTe5oE71Y8negw1aa1rWc7BjXLJz84P4QoPlSYegbGilTaSB6jUnistEtuvV0cxZOHwlEyVO2KRRuz52b5/gH0tpDLo0XJ1eMJq0C72Sc3KN2kHeqASg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Zxl8Fo6V; arc=fail smtp.client-ip=40.93.201.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Zxl8Fo6V" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wgLlivLg+Y4WGwlQNo1O2W0Cjch3fPPWVjdKstdUMS0QArEFQYtVfhFJHHjRhjSKeNwDLXuDFq3toXoQLs7i3W52yiztKZJjH0RVVHCbbPhcip3oav5E14VWH+uiIvucfeLa4laWFXLQIl8aGUNqa5WVQroJ2bNMPfwcjRJp+srwl9oaJFgESE4zINxyU0V7B7Jt70geijOGbdYv5JOyfs2HEpQhYaCoFgKFhgckMaT+f9jwOiMdK9n1lTOdSP8TyHnjuJ3ru9+z8BRAeqjYGEXeB62fFYjqeoNX0PwsAm5zxBhumnxPd0ugb+mv+FauRRuOI58p6ZFjhslm6esPjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=W9yVkfs+D03aaONZyxxaxc0QmtRrPMgVegcfzsL6+Xo=; b=D0IGddRj8XabXbsP7pC9q44wAGUkPeWPrsprmZt/Wniy7VNJbQD4FWNu1bL0GCQbgY7Yf3Iw72QQoraHkS6R1MbY1p0Z5VJ6HHvWmVX935ObbCxosK5GT2ghx6Ctvb9MJBV409eQI1BgOAAgid24dBeIgWUYhI1zp/UWmjN/XYrYKke2SOwX4jYpIyJZz/sW7YawnwyRu03HLekFHqzokqgQh87nxIAa+sGIvRNS5Zri9X2M1X8RcDJpGj9m+PSdZG5Omsrg/JpaipGT8pA8lGOqTtxWOZtbRZOe1uSUwbY0L+JsAFjSzSZCZ0tCjNqqCVaSF34k08v5QYI25NJ3aA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W9yVkfs+D03aaONZyxxaxc0QmtRrPMgVegcfzsL6+Xo=; b=Zxl8Fo6VCr83VjpglJTrcXlhdh4nw8x7sUhq6/0EDGGMvrAFflDRcT3j4VfEYEilMF28VDzKRou13XiMTdmD/fbiojwRglyhXjMsXeLCKiBypGoHPH9h3K5HpYoaNFBZUAElTv+eIVche1amOrmZ2SmCZLRTJoyb3z9Rd1MPbmQ= Received: from SA1P222CA0002.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:22c::9) by DS0PR12MB6584.namprd12.prod.outlook.com (2603:10b6:8:d0::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.20; Thu, 2 Apr 2026 11:25:25 +0000 Received: from SN1PEPF0002636B.namprd02.prod.outlook.com (2603:10b6:806:22c:cafe::4a) by SA1P222CA0002.outlook.office365.com (2603:10b6:806:22c::9) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9769.18 via Frontend Transport; Thu, 2 Apr 2026 11:25:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SN1PEPF0002636B.mail.protection.outlook.com (10.167.241.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Thu, 2 Apr 2026 11:25:25 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Thu, 2 Apr 2026 06:25:20 -0500 From: K Prateek Nayak To: Thomas Gleixner , Ingo Molnar , "Peter Zijlstra" , Sebastian Andrzej Siewior , Arnd Bergmann CC: Darren Hart , Davidlohr Bueso , =?UTF-8?q?Andr=C3=A9=20Almeida?= , , , , , , K Prateek Nayak Subject: [PATCH v3 6/7] asm-generic/runtime-const: Add dummy runtime_const_mask_32() Date: Thu, 2 Apr 2026 11:22:49 +0000 Message-ID: <20260402112250.2138-7-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260402112250.2138-1-kprateek.nayak@amd.com> References: <20260402112250.2138-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb08.amd.com (10.181.42.217) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636B:EE_|DS0PR12MB6584:EE_ X-MS-Office365-Filtering-Correlation-Id: f799ba95-1bd5-4bf6-d63e-08de90aa895c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|36860700016|376014|82310400026|1800799024|13003099007|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: 6fh74lxNwIFNj4RQJoO7IaesS27K/FgiOYx3nHBeXokf4Gyspr1nDjKG/PT2D/ymbT1GnGzSZy95uUId7H06BCJ/hCFIu2x2LjhAA9FUPoFkGHra8Dei/OGvSfUOVmCPv/cW5rfQIOhvt6JJef2SErFgUNXgyPpy/R7ldtx1hN2rD3l7CO/yIHMr19RDyP8+GgLn0wyhC29wJU76Tt6DZKrN32xmq1/w4twBK1CBk94Fmh6NcTTAuLhLcLiZvnVT+aTOcGmQVCSw5wNrVvn0H3DeErvjmqSQQ3qEY0SSwLbqdfv3XUEB33rncCKz/ci8chU0jc2gCIGMN3SsKrbEfKGAjV1Zqw/p4FY0LPRPVQoc7Oa6HkB/hAYFgyLhvAxIesOYXBha/YKJBN5vuy89RZY+TKy+JHfzybaLocxwbZvfXgvLPgZMjkXVmK0Rveq9KuARkfn6isavvZHq/ty0n9qTPa3Ip8fj8Kvg08tLKKzBZYeYl+CXzSpjMSIBDZ2KKZZiVEpYCrS/K7L9moJ6ol4Qf0Ot8RMnDiccaRe5MD5XFYUjT+vLpWso98TL8IunVMeUziWs+sXzCkOAm5axBtt5uz6bgjcKm8bej5gbhcBcL8tLKBoasSId11bqpcVj6kh1fNYQ4Oqtm/EbvvajD2C/uHt7nLRmtkd7eMVCk7ap21HU/pXOhsV+kpbmjnijej0jKEKrVBmrNxrdJrnd0rpbN5O1qg3xGsd29VDOSyE= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(36860700016)(376014)(82310400026)(1800799024)(13003099007)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: WBfCpFsXxdU2gtIpC3ch0rVAgO7hbVQrAmhQ76nY20PWP1pTg3PYGdXrJLB7OOdnx11LeykW4hx7OopatURYcBZjxp1fZdWikOn/0YC4rBXQPqga5X3VdxddBVY6pMjaMoE4i4JmvZWQtuPNhjMGeYQi5k7ZnOthJUXGAFbAUencEHkewCbVNQOEDaqIR/RhLTEbkFZxyT2fVpPYJFpXpQz2qSyrfEi/7qzv4uWRFmPK8LqVuvEQEBfzmV33MuMGYNjIBhIF2UZA0efXh3cGSHeUFh2ac4a1r+68YWXJDpKG4SUJT+z/D5R6smIS2HexSaiCjE/ZbjkdhGbyeYQjuJYqJxjXz9RW3PQfsDH6uJx7vsyr9itrfDWfvLmDtp755oscYbX0zPAh19664u0PmQBxmu/Y3jGtgYGRwjCrQNrHUThrIcypUbPg4DPyTMvN X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2026 11:25:25.5969 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f799ba95-1bd5-4bf6-d63e-08de90aa895c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636B.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6584 Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra Add a dummy runtime_const_mask_32() for all the architectures that do not support runtime-const. Link: https://patch.msgid.link/20260227161841.GH606826@noisy.programming.ki= cks-ass.net Not-yet-signed-off-by: Peter Zijlstra Signed-off-by: K Prateek Nayak --- Changelog v2..v3: o No changes. --- include/asm-generic/runtime-const.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/asm-generic/runtime-const.h b/include/asm-generic/runt= ime-const.h index 670499459514..03e6e3e02401 100644 --- a/include/asm-generic/runtime-const.h +++ b/include/asm-generic/runtime-const.h @@ -10,6 +10,7 @@ */ #define runtime_const_ptr(sym) (sym) #define runtime_const_shift_right_32(val, sym) ((u32)(val)>>(sym)) +#define runtime_const_mask_32(val, sym) ((u32)(val)&(sym)) #define runtime_const_init(type,sym) do { } while (0) =20 #endif --=20 2.34.1 From nobody Tue Apr 7 04:34:56 2026 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013031.outbound.protection.outlook.com [40.93.201.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 968F63C13FC; Thu, 2 Apr 2026 11:25:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.31 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129166; cv=fail; b=aMJE48Hn/eDJlc1U7wUFJeFf/LXNSzkBRRMHg6yreChqRsQFQPmEnp640jkPirMP9BFbJ0DkPVGijDKXg/BHQbU18sprYn91xzXWHecCm0K7W3PuLfKgL4huNhWKUj6ugzOPOxCrlSl9km+RM1YgCRcNeiW6H2V8NIeKXwiEDYQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775129166; c=relaxed/simple; bh=GrCOisxPcaMAvgbfr3+/whh2DeWnX4UZQQDgFRHBjLE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=S8wDxlCyKLdX3EvMl8y3T2ZuaAxyEH8fpXnGxbekmutbhreRj/vjaf/M86aHLbnQRuENoEoFxDG3Nx5tEtqUVyZUXMF8AWv+QtYFfLojNkUWyo1As+lVGKNoBrFM0ds2J8kzXZl+okWQbXYW0tJVxd/Z07ApBVcsrh50h0/M6eo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=d/k1sR8g; arc=fail smtp.client-ip=40.93.201.31 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="d/k1sR8g" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Cz+V8SaDiVXk2YcQGZ8f4cNrDC35oFI0kuY3Cvti2rNXfkR3sSgg93Teu+1yafdQwXw0Vcaw7s2erRwPFGAT1YTqT8Du/Ma8sfRgh4+Pmh+vzrl5LXATYK3fxkJJHoEfgLDj+lA3ZptExq4ZO68sbbG7ArvMuPEa5CwPJwdGcQ9X1pdRH2LijmuuB8icgv+8o97Vdmd+YlW7hIT1RRXMZbOuhbxPQAPbUzATLUzORGsj/8f9BgH4Cd+2QtZaqpjoTkg4hDo1M+xb4oJHyQTlaRDQtWxunZw0jb4ZcQuWG0dkTSghIEe6JGif8TE46IPOUHID6c6AZFCMzizN/vvx6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2WMnMPmrxQrTbiw6MNrx6sOXgjqHWeuN90cGW0LyyWc=; b=r/ruXaxSA9ONUSRBtBucuqNw0Qx0/PgKvsttj06bfbslUDhqprzu8rW/RSvJ4sK2VvWCNAAEo9863gaRirHpKqEYIHlZUttrsUPRAfmH1a+XTQxnlRpref203i8IabHtpMEEYwyk3TjDk98RGTM8RmuMfuzIaLaTrmNO/OCOtA1IsjNwWtwu4DAp/8fM/6vDTpAWZdwHm9PpGmbp8wvb10YQvRcty5QGOeLAbpsLw1v3MNBxjGAVoAGDLj2U0DVrEVDCI7tscRsMg8xbPxOqiCvAB9d3fw3UfCqbs8DLA1Ct0kJIoLwHdSPHp5lEnEFOjhuGie3taE3Fstwt4e1I+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2WMnMPmrxQrTbiw6MNrx6sOXgjqHWeuN90cGW0LyyWc=; b=d/k1sR8gZH99Fl5O6kLBnK1upSqj/EoxsOnnzoIbLZp1QvXnYh5q9S5fAVDZJoPh7LLy1z0K+zh22uZU7sM45zXAGncLoHl6GJt680p8ebEwSNpwVuj2rNEjv2TzTXj41tIwnWMxYTRjeSusxC3RwfOFCzTvH+Nj4S0Nh9Gakk4= Received: from SN7PR04CA0067.namprd04.prod.outlook.com (2603:10b6:806:121::12) by IA1PR12MB7687.namprd12.prod.outlook.com (2603:10b6:208:421::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.15; Thu, 2 Apr 2026 11:25:46 +0000 Received: from SN1PEPF0002636E.namprd02.prod.outlook.com (2603:10b6:806:121:cafe::62) by SN7PR04CA0067.outlook.office365.com (2603:10b6:806:121::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9745.30 via Frontend Transport; Thu, 2 Apr 2026 11:25:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SN1PEPF0002636E.mail.protection.outlook.com (10.167.241.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Thu, 2 Apr 2026 11:25:46 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Thu, 2 Apr 2026 06:25:38 -0500 From: K Prateek Nayak To: Thomas Gleixner , Ingo Molnar , "Peter Zijlstra" , Sebastian Andrzej Siewior , Borislav Petkov , Dave Hansen , , Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , "Christian Borntraeger" , Arnd Bergmann , "David Laight" , Samuel Holland CC: Darren Hart , Davidlohr Bueso , =?UTF-8?q?Andr=C3=A9=20Almeida?= , , , , , , K Prateek Nayak Subject: [PATCH v3 7/7] futex: Use runtime constants for __futex_hash() hot path Date: Thu, 2 Apr 2026 11:22:50 +0000 Message-ID: <20260402112250.2138-8-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260402112250.2138-1-kprateek.nayak@amd.com> References: <20260402112250.2138-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb08.amd.com (10.181.42.217) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636E:EE_|IA1PR12MB7687:EE_ X-MS-Office365-Filtering-Correlation-Id: 4a247bbf-bdcd-44d4-660f-08de90aa95b7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|1800799024|7416014|376014|82310400026|921020|18002099003|13003099007|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: gIy1AprgtkbovkxLKdDwgawWDjpvMw+poVNfUUtmZYoySg6ylbxZvzTMoySMAT71tZAoQg5Vjo1pHVOG0fMpKwLq1BFiXo/7FffliHkqDFTztPVM2HdpIBLdUhR7hhoZedZKSfMxUNc0r674pGuRk6CaMiZqUyLofzNUYSGr2x6xToSHc/P480fvj/CUv7JZuPaO8WGQNWJrm2LJfpxX2PQoNsS73oTR+tSL9dvSLa2A+kuKvm1q9MfQejKdVPWliEFDa0qvjYXw5OMtr6vzVIa0K5Oi0ofPQWPuTJsUeiag9dnRbrXRmCf0IWuJHFA94nIa4s3JytV0fJ7mVwuM1UP6Og6NbX6m284h/jwW4Z9gWk+mbA4SLT4Qg5c77oarEw4CnTqK6i5ysZxBEMxkL+C5j/ClWFpIrzyN3jgn/unt0UWwl7GnTzsS8AzOl+hFrww070+16IuA3BfLovRt1w4WtQkSFveNwe5606xClM/sC0Ckx/N5yCZPrhlwdNpWsgjh/xLHiVeQXYyvfFrGPT8CXYnAtFEWdP5YJUfFUaTb+VAQEXTt2XwEEiQXyx2RCnOH4lhWujawZW6yCF9pLoVxtmTWZyvSm7ERN+T1rdy+uCQm+mdRJ6SNx8zo34BqdTbFpTRRmUU3KNKoIEmDNuAN5y6/YLAM8bR2TSShiqaD9z1n2DV3drxUfdG/A2/vub6Bu75vc+n1ZbsjzPmlrYrlivgDGKd6WLh07nY0WWYM2r4iu7B8OhjK4Yt+51Zae3HXDgDlQDhzX6zKQvx47Q== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700016)(1800799024)(7416014)(376014)(82310400026)(921020)(18002099003)(13003099007)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: vrOwWK4Bod3z1tHDppGAJ71xRhJvwJuES1T3icT4wtxmfwRwXsfP/bHEcKwQKpwhgnWVcNvNsvgWSgEVA1vZLyID6DvnD3ADDNOfE9sPVJmaMA1m2GoYSnoaC9jGXQCwGZN8cPjX6YmfSTG1OGchV7ApozALRC1igLp/8qu8gRzv5146eBjS6m69C3E5orFPHoRTvOmpV+VoaHjN9WCTVdWJldmS7L4HoB/wfVJEnXae0ykSYo5RubchZTHnqP69nZ7o6W2Os45K8jIS0AaQhc+/BISMEdOkCJi+uQ48kxcelUSaHCzEdL5FgvjJtCR7dMSGimsqA2yC1Wu5cpLuT8z0e+HsWgYeQjMFnClKMkjWKYuN93OnwFfv6nNQ99ixal0YdabhBmO27NDfQfLGHg2mar4fcdvi6BP4yJNHwVvYd0VmMfteCFXjhdimDOUy X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2026 11:25:46.3232 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4a247bbf-bdcd-44d4-660f-08de90aa95b7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636E.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7687 Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra Runtime constify the read-only after init data __futex_shift(shift_32), __futex_mask(mask_32), and __futex_queues(ptr) used in __futex_hash() hot path to avoid referencing global variable. This also allows __futex_queues to be allocated dynamically to "nr_node_ids" slots instead of reserving config dependent MAX_NUMNODES (1 << CONFIG_NODES_SHIFT) worth of slots upfront. No functional chages intended. [ prateek: Dynamically allocate __futex_queues, mark the global data __ro_after_init since they are constified after futex_init(). ] Link: https://patch.msgid.link/20260227161841.GH606826@noisy.programming.ki= cks-ass.net Reported-by: Sebastian Andrzej Siewior # MAX_NUMNOD= ES bloat Not-yet-signed-off-by: Peter Zijlstra Signed-off-by: K Prateek Nayak --- Changelog v2..v3: o No changes. --- include/asm-generic/vmlinux.lds.h | 5 +++- kernel/futex/core.c | 42 +++++++++++++++++-------------- 2 files changed, 27 insertions(+), 20 deletions(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinu= x.lds.h index 1e1580febe4b..86f99fa6ae24 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -975,7 +975,10 @@ RUNTIME_CONST(shift, d_hash_shift) \ RUNTIME_CONST(ptr, dentry_hashtable) \ RUNTIME_CONST(ptr, __dentry_cache) \ - RUNTIME_CONST(ptr, __names_cache) + RUNTIME_CONST(ptr, __names_cache) \ + RUNTIME_CONST(shift, __futex_shift) \ + RUNTIME_CONST(mask, __futex_mask) \ + RUNTIME_CONST(ptr, __futex_queues) =20 /* Alignment must be consistent with (kunit_suite *) in include/kunit/test= .h */ #define KUNIT_TABLE() \ diff --git a/kernel/futex/core.c b/kernel/futex/core.c index ff2a4fb2993f..73eade7184dc 100644 --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -45,23 +45,19 @@ #include #include =20 +#include + #include "futex.h" #include "../locking/rtmutex_common.h" =20 -/* - * The base of the bucket array and its size are always used together - * (after initialization only in futex_hash()), so ensure that they - * reside in the same cacheline. - */ -static struct { - unsigned long hashmask; - unsigned int hashshift; - struct futex_hash_bucket *queues[MAX_NUMNODES]; -} __futex_data __read_mostly __aligned(2*sizeof(long)); +static u32 __futex_mask __ro_after_init; +static u32 __futex_shift __ro_after_init; +static struct futex_hash_bucket **__futex_queues __ro_after_init; =20 -#define futex_hashmask (__futex_data.hashmask) -#define futex_hashshift (__futex_data.hashshift) -#define futex_queues (__futex_data.queues) +static __always_inline struct futex_hash_bucket **futex_queues(void) +{ + return runtime_const_ptr(__futex_queues); +} =20 struct futex_private_hash { int state; @@ -439,14 +435,14 @@ __futex_hash(union futex_key *key, struct futex_priva= te_hash *fph) * NOTE: this isn't perfectly uniform, but it is fast and * handles sparse node masks. */ - node =3D (hash >> futex_hashshift) % nr_node_ids; + node =3D runtime_const_shift_right_32(hash, __futex_shift) % nr_node_ids; if (!node_possible(node)) { node =3D find_next_bit_wrap(node_possible_map.bits, nr_node_ids, node); } } =20 - return &futex_queues[node][hash & futex_hashmask]; + return &futex_queues()[node][runtime_const_mask_32(hash, __futex_mask)]; } =20 /** @@ -1916,7 +1912,7 @@ int futex_hash_allocate_default(void) * 16 <=3D threads * 4 <=3D global hash size */ buckets =3D roundup_pow_of_two(4 * threads); - buckets =3D clamp(buckets, 16, futex_hashmask + 1); + buckets =3D clamp(buckets, 16, __futex_mask + 1); =20 if (current_buckets >=3D buckets) return 0; @@ -1986,10 +1982,19 @@ static int __init futex_init(void) hashsize =3D max(4, hashsize); hashsize =3D roundup_pow_of_two(hashsize); #endif - futex_hashshift =3D ilog2(hashsize); + __futex_mask =3D hashsize - 1; + __futex_shift =3D ilog2(hashsize); size =3D sizeof(struct futex_hash_bucket) * hashsize; order =3D get_order(size); =20 + __futex_queues =3D kcalloc(nr_node_ids, sizeof(*__futex_queues), GFP_KERN= EL); + + runtime_const_init(shift, __futex_shift); + runtime_const_init(mask, __futex_mask); + runtime_const_init(ptr, __futex_queues); + + BUG_ON(!futex_queues()); + for_each_node(n) { struct futex_hash_bucket *table; =20 @@ -2003,10 +2008,9 @@ static int __init futex_init(void) for (i =3D 0; i < hashsize; i++) futex_hash_bucket_init(&table[i], NULL); =20 - futex_queues[n] =3D table; + futex_queues()[n] =3D table; } =20 - futex_hashmask =3D hashsize - 1; pr_info("futex hash table entries: %lu (%lu bytes on %d NUMA nodes, total= %lu KiB, %s).\n", hashsize, size, num_possible_nodes(), size * num_possible_nodes() / 1024, order > MAX_PAGE_ORDER ? "vmalloc" : "linear"); --=20 2.34.1