From nobody Tue Jun 30 02:42:50 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 106B8C433EF for ; Thu, 27 Jan 2022 01:31:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234507AbiA0Bbw (ORCPT ); Wed, 26 Jan 2022 20:31:52 -0500 Received: from mail-eopbgr20055.outbound.protection.outlook.com ([40.107.2.55]:4416 "EHLO EUR02-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S231940AbiA0Bbv (ORCPT ); Wed, 26 Jan 2022 20:31:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZpYLEU5YhL5UvvWm3TLuC/DsV2wceNKpr3G5pQkFopk=; b=NM1vHAgK2g8PU8ZswTuv2Z4Q1Qb0qTtCayXaYKdgnrnoeb2woi4S98dz5G/7vb7hLFUmzj7CmXyJHTxReTRa6EGYdIdXriXrUMC6BlUgHVtLZfCrPHR9N4dDjOh3mzVhR4VWf5Ifcw88NAUegtQd5DYXHUYLnJhzjw2EOqz6a24= Received: from AS9PR06CA0172.eurprd06.prod.outlook.com (2603:10a6:20b:45c::35) by AM4PR08MB2659.eurprd08.prod.outlook.com (2603:10a6:205:e::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Thu, 27 Jan 2022 01:31:47 +0000 Received: from AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:45c:cafe::b) by AS9PR06CA0172.outlook.office365.com (2603:10a6:20b:45c::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15 via Frontend Transport; Thu, 27 Jan 2022 01:31:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT004.mail.protection.outlook.com (10.152.16.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15 via Frontend Transport; Thu, 27 Jan 2022 01:31:47 +0000 Received: ("Tessian outbound 826a6d8e58c3:v113"); Thu, 27 Jan 2022 01:31:47 +0000 X-CR-MTA-TID: 64aa7808 Received: from d510b130f701.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 67DA4F27-A979-4B13-B3B8-D7BC06ED40D4.1; Thu, 27 Jan 2022 01:31:37 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d510b130f701.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 27 Jan 2022 01:31:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qh/J/quyf6COxWo2i5Hb7UwJZPLafjFh7LK3j9BpnALN5BJCMVP796ad23nD/Z5t08GV9TXzyGpC5n1CaviBAiSK4bmHm9YMpestd9LDGsVG0QBCCF3wbFLhCbe8XkChQ0O95lnnGWyzCAmqSlYXC6ZcEzpHPJ3QYYBfl+cnsCwOAF2xqzuz1H+BqlcjMK4n2VVRrdTVemtctXgh51Avf/yS+9LEwiW8fn6kpM1497BS2BhPp6Vi6odQt8HJfDi0h/KdpeY8VxSNbBRMW7kS650N8Vu6LLdhid1GbUhGC2QU6zCCVSBsODWSafW6YfhSPCg7cIk6E52+E4EUwSqKxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZpYLEU5YhL5UvvWm3TLuC/DsV2wceNKpr3G5pQkFopk=; b=eTWEGHXlrROp/ci7zqpjWE/7vbk49qvfWbs+5jB9CzUHlX+iSpuPHwYeoA5OWAA9qBOI2Co8NV9oNDLYu8CqTpYwzEOtLtvxqqaHHoe4GvoKN1N7zJ91jRRBa6NAV9NEWXoNNiQGytAYCu1tVdMYIH7ZOV6srx4IFJ6sNfwNKwosEPmE0jPXPFyUF7BFNVLs44qMZPfnaO7CRhILVstlropuzDSgBM/dLaIeZGwI6+HMzt0nqWkStkCPnlbqsQY/cNbvvt8BNP3XcrzsV7q5cXO5Dc8QtXFs5UJ00me9QedRFGnvuBXhZ8RAYKjheWlFdqmd3P/0CzB8sUpg0pycWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZpYLEU5YhL5UvvWm3TLuC/DsV2wceNKpr3G5pQkFopk=; b=NM1vHAgK2g8PU8ZswTuv2Z4Q1Qb0qTtCayXaYKdgnrnoeb2woi4S98dz5G/7vb7hLFUmzj7CmXyJHTxReTRa6EGYdIdXriXrUMC6BlUgHVtLZfCrPHR9N4dDjOh3mzVhR4VWf5Ifcw88NAUegtQd5DYXHUYLnJhzjw2EOqz6a24= Received: from VI1PR08MB3742.eurprd08.prod.outlook.com (2603:10a6:803:c3::16) by AM6PR08MB3096.eurprd08.prod.outlook.com (2603:10a6:209:43::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Thu, 27 Jan 2022 01:31:34 +0000 Received: from VI1PR08MB3742.eurprd08.prod.outlook.com ([fe80::dc36:d5c0:73b9:4b13]) by VI1PR08MB3742.eurprd08.prod.outlook.com ([fe80::dc36:d5c0:73b9:4b13%5]) with mapi id 15.20.4909.019; Thu, 27 Jan 2022 01:31:34 +0000 From: Justin He To: Ard Biesheuvel CC: Catalin Marinas , Jianyong Wu , "will@kernel.org" , Anshuman Khandual , "akpm@linux-foundation.org" , "david@redhat.com" , "quic_qiancai@quicinc.com" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "gshan@redhat.com" , nd Subject: RE: [PATCH v3] arm64/mm: avoid fixmap race condition when create pud mapping Thread-Topic: [PATCH v3] arm64/mm: avoid fixmap race condition when create pud mapping Thread-Index: AQHX8lbtE25Jxomi6EyEj6gGYJIT7qxU2ImAgAEOxwCAAF/yAIABIQaAgAAZqICAHI870IABKecAgAEJrWA= Date: Thu, 27 Jan 2022 01:31:34 +0000 Message-ID: References: <20211216082812.165387-1-jianyong.wu@arm.com> In-Reply-To: Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 344CB0323FC0F14484FC579FA9724300.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-MS-Office365-Filtering-Correlation-Id: 884ecb34-3e53-4d6c-517a-08d9e134c8f5 x-ms-traffictypediagnostic: AM6PR08MB3096:EE_|AM5EUR03FT004:EE_|AM4PR08MB2659:EE_ X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: aE2fBGVkTGF5ESbB6EiZuJmMY43jUU5djPlO2S+M3HiGpysUHJ/UmQ6tSnvaXNh1urpW+GlONr+sj96uD0VY5/oUD24REn2e8JH/MgpO8q5lJIas0SvAIsroM+f0aKCzHSl1oPTqGH4Z9n5x30VWvOZHgmnzuJEHK5l/UXkcWbKlUN8utuZjcYwkZETIb0sCLhvYMD+OAwnETqRkiW6yrP7Ypi0gwQ7Q2aNnUq3Af0ruHoC6uBVwL9bvbJ7ueuxWUvseGBdPDEdtALfTA3A26t0uKZDwG9ulUWgEAmnVccGCsaG9FS+r8Fl2ChBIj6dsjYEnY4IkQd0mkyQLyLBtpxmN+88h60DBTmola4Z3CPCymJGeN5R3/XPDQrnRPLeKmsGzvpQyMxscetAAcHIf1E7SO3VnCcf8SYY7XTVviYrUHjMKh1+q85vGwAJg//svX4dccXWFo7loCbsSKY3lBpM2Ugn4v5nmLBclehTRhFU1X8jJdAEJXwXiFf4X5e13OLniQ9uFF9kIaEbwhjXPt3rA3NkNt03/NFtgI0JwvC/lKozyzTmtDqBFVcPWePD4DxNrG3b9uY7WHR6Sbok3C0hRlXuBbjLY2S7X/5RVRaMLkIQBI6tGmPt+BAZcD+Ae+qFKATR/k84BxfiTenvYl9xXLvG0QAJoD1HmyWLARUwyOJg5/UqPxwJ9dzXoy83irIe8jtrAFow3o2SBtTI3Gg== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR08MB3742.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(71200400001)(122000001)(7696005)(6506007)(38070700005)(38100700002)(4326008)(2906002)(76116006)(66946007)(9686003)(66476007)(66556008)(66446008)(53546011)(64756008)(83380400001)(86362001)(55016003)(508600001)(52536014)(186003)(26005)(5660300002)(8676002)(8936002)(33656002)(316002)(6916009)(54906003)(20210929001);DIR:OUT;SFP:1101; Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3096 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: e24054a7-19a0-47f8-318d-08d9e134c12d X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WHqFOmPn9h9HJDp2BZnuSTDwTtzezv780/RHsMoszLdstOyI8zX3jvy3vUhZNn8X6DtyViAuOeROz9rKYjR80q5FyQVB0C6rOZSYfH9BoZPh3/cMzbbj3LBFuc9vBlrJCIm/Hk7oHElwrHBbh0kZC9ulwrqGhFSalJgcai14cHSmPqMW76fgfQ7z9LTwmOcnp0MoAt5LZtcspdL9nSxYDQRKlUrQNXiOocGTFJ1pbixnkU5LqGRr/XB/r2DxBthHWNtkRvWWrLGwm0gWdRz8PBzW1h+YBbGKAZLOxyIaNoxyeXX+nU9r+z6QoPwCqbzpGJ4lVFQwfNTBIuEyfXOZR0aVv0/jhNDqD2jBrTCTMlubsI6x8LrArQKngS/WUJdaVk1OoEDRQstabdDAVq4klbPRwDY8isHnma+hLZ/ZsUu17f1fQTt9/xKIUMW11s7RKW27Mx6Y7gHSNGm4BeyNsW2cWH7Q2drwtdaNXhtZdybUwfriwQ5vEYbM3KCBev5X4cyFU+TekRu6kVp8aHx3KfIYLu0oQV+1Paj5fzXWb1EC9R+ezOTr7nSnml5wF7BFJ/rAr4D9HWJyry2WFUvd6Ms+M8L5uZnKVoClrxwjyHKr9Wht11ujBfd20bw0bbP3mbkKlCpc/PZdtpXfoF0VGrzwf/uJVSb3cL0Nd0meog7Zc35CLa34ruTJ/TAxw1Rx X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(336012)(356005)(36860700001)(9686003)(508600001)(47076005)(5660300002)(40460700003)(81166007)(52536014)(2906002)(55016003)(82310400004)(4326008)(86362001)(54906003)(70206006)(316002)(70586007)(83380400001)(6862004)(8936002)(8676002)(33656002)(186003)(26005)(53546011)(6506007)(7696005)(20210929001);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jan 2022 01:31:47.4924 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 884ecb34-3e53-4d6c-517a-08d9e134c8f5 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR08MB2659 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Ard > -----Original Message----- > From: Ard Biesheuvel > Sent: Wednesday, January 26, 2022 4:37 PM > To: Justin He > Cc: Catalin Marinas ; Jianyong Wu > ; will@kernel.org; Anshuman Khandual > ; akpm@linux-foundation.org; david@redhat.com; > quic_qiancai@quicinc.com; linux-kernel@vger.kernel.org; linux-arm- > kernel@lists.infradead.org; gshan@redhat.com; nd > Subject: Re: [PATCH v3] arm64/mm: avoid fixmap race condition when create > pud mapping >=20 > On Wed, 26 Jan 2022 at 05:21, Justin He wrote: > > > > Hi Catalin > > > > > -----Original Message----- > > > From: Catalin Marinas > > > Sent: Friday, January 7, 2022 6:43 PM > > > To: Jianyong Wu > > > Cc: will@kernel.org; Anshuman Khandual ; > > > akpm@linux-foundation.org; david@redhat.com; quic_qiancai@quicinc.com; > > > ardb@kernel.org; linux-kernel@vger.kernel.org; linux-arm- > > > kernel@lists.infradead.org; gshan@redhat.com; Justin He > > > ; nd > > > Subject: Re: [PATCH v3] arm64/mm: avoid fixmap race condition when > create > > > pud mapping > > > > > > On Fri, Jan 07, 2022 at 09:10:57AM +0000, Jianyong Wu wrote: > > > > Hi Catalin, > > > > > > > > I roughly find the root cause. > > > > alloc_init_pud will be called at the very beginning of kernel boot > in > > > create_mapping_noalloc where no memory allocator is initialized. But > > > lockdep check may need allocate memory. So, kernel take exception when > > > acquire lock.(I have not found the exact code that cause this issue) > > > that's say we may not be able to use a lock so early. > > > > > > > > I come up with 2 methods to address it. > > > > 1) skip dead lock check at the very beginning of kernel boot in > lockdep > > > code. > > > > 2) provided 2 two versions of __create_pgd_mapping, one with lock in > > > > it and the other without. There may be no possible of race for > memory > > > > mapping at the very beginning time of kernel boot, thus we can use > the > > > > no lock version of __create_pgd_mapping safely. > > > > In my test, this issue is gone if there is no lock held in > > > > create_mapping_noalloc. I think create_mapping_noalloc is called > early > > > > enough to avoid the race conditions of memory mapping, however, I > have > > > > not proved it. > > > > > > I think method 2 would work better but rather than implementing new > > > nolock functions I'd add a NO_LOCK flag and check it in > > > alloc_init_pud() before mutex_lock/unlock. Also add a comment when > > > passing the NO_LOCK flag on why it's needed and why there wouldn't be > > > any races at that stage (early boot etc.) > > > > > The problematic code path is: > > __primary_switched > > early_fdt_map->fixmap_remap_fdt > > create_mapping_noalloc->alloc_init_pud > > mutex_lock (with Jianyong's patch) > > > > The problem seems to be that we will clear BSS segment twice if kaslr > > is enabled. Hence, some of the static variables in lockdep init process > were > > messed up. That is to said, with kaslr enabled we might initialize > lockdep > > twice if we add mutex_lock/unlock in alloc_init_pud(). > > >=20 > Thanks for tracking that down. >=20 > Note that clearing the BSS twice is not the root problem here. The > root problem is that we set global state while the kernel runs at the > default link time address, and then refer to it again after the entire > kernel has been shifted in the kernel VA space. Such global state > could consist of mutable pointers to statically allocated data (which > would be reset to their default values after the relocation code runs > again), or global pointer variables in BSS. In either case, relying on > such a global variable after the second relocation performed by KASLR > would be risky, and so we should avoid manipulating global state at > all if it might involve pointer to statically allocated data > structures. >=20 Thanks for the explanation, which makes root cause clearer. I have a question off this thread: Should we avoid to invoke early_fdt_map and init_feature_override twice with kaslr enabled? In Commit f6f0c4362f07 ("arm64: Extract early FDT mapping from kaslr early_init() "), it implicitly invokes early_fdt_map first time before kaslr is enabled and 2nd time after it. What to you think of below changes (tested in both guest and host boot): diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 6a98f1a38c29..3758ac057a6a 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -450,12 +450,12 @@ SYM_FUNC_START_LOCAL(__primary_switched) #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) bl kasan_early_init #endif - mov x0, x21 // pass FDT address in x0 - bl early_fdt_map // Try mapping the FDT early - bl init_feature_override // Parse cpu feature overri= des #ifdef CONFIG_RANDOMIZE_BASE tst x23, ~(MIN_KIMG_ALIGN - 1) // already running randomiz= ed? b.ne 0f + mov x0, x21 // pass FDT address in x0 + bl early_fdt_map // Try mapping the FDT early + bl init_feature_override // Parse cpu feature overri= des bl kaslr_early_init // parse FDT for KASLR opti= ons cbz x0, 0f // KASLR disabled? just pro= ceed orr x23, x23, x0 // record KASLR offset -- Cheers, Justin (Jia He)