From nobody Fri Apr 19 19:11:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=2; a=rsa-sha256; t=1668768376; cv=pass; d=zohomail.com; s=zohoarc; b=FQy+bcOsCdDzcyjSJmF/m73xyzqCjfEb0Kd43ozE+auuh1oOTS1xksOOmL7afZh7f2xbTv5n3Uk+Rt5+da75uI9EH+kwgffzEUOaEWG0Vco6nYBlyiiIuD1X3drNmpeKkR70XcoByyB6OFn9bYwLeCj345FFsO2z6g4apLACouc= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1668768376; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=zPDBV9XWmcg+tmJ+qaggSEwwP7UYQrtfJrIfAn/ghm8=; b=H+4GpFpB3uZMkFhMfyID7D8KsMMRTY4vvj2g23d/hnpJomwl8IMwO9SDAcqxpCfqzsWGJW4ePMVHZyZs8lXQzdIU12PGxOflFpD8h2Gwv7i4b4gWUCmFOepZSLSe0kdUq4Ddksmpx/hSNXrmlnQWy/tMsRA/PlMWvaX4ctIzfvI= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1668768376521859.3531458704325; Fri, 18 Nov 2022 02:46:16 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.445562.700779 (Exim 4.92) (envelope-from ) id 1ovysR-0006MQ-Lp; Fri, 18 Nov 2022 10:45:43 +0000 Received: by outflank-mailman (output) from mailman id 445562.700779; Fri, 18 Nov 2022 10:45:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysR-0006Lq-HI; Fri, 18 Nov 2022 10:45:43 +0000 Received: by outflank-mailman (input) for mailman id 445562; Fri, 18 Nov 2022 10:45:42 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysQ-0006Il-OS for xen-devel@lists.xenproject.org; Fri, 18 Nov 2022 10:45:42 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2060d.outbound.protection.outlook.com [2a01:111:f400:7d00::60d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 2519d8d8-672e-11ed-91b6-6bf2151ebd3b; Fri, 18 Nov 2022 11:45:41 +0100 (CET) Received: from AS9PR05CA0005.eurprd05.prod.outlook.com (2603:10a6:20b:488::12) by GVXPR08MB7821.eurprd08.prod.outlook.com (2603:10a6:150:4::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.6; Fri, 18 Nov 2022 10:45:38 +0000 Received: from AM7EUR03FT006.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:488:cafe::84) by AS9PR05CA0005.outlook.office365.com (2603:10a6:20b:488::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.13 via Frontend Transport; Fri, 18 Nov 2022 10:45:38 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT006.mail.protection.outlook.com (100.127.141.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:38 +0000 Received: ("Tessian outbound 2ff13c8f2c05:v130"); Fri, 18 Nov 2022 10:45:37 +0000 Received: from 74e5848c6914.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 4DE39858-FB57-467E-9ABF-829731D5494E.1; Fri, 18 Nov 2022 10:45:30 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 74e5848c6914.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 18 Nov 2022 10:45:30 +0000 Received: from FR3P281CA0136.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:95::9) by DB4PR08MB9312.eurprd08.prod.outlook.com (2603:10a6:10:3f4::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.7; Fri, 18 Nov 2022 10:45:28 +0000 Received: from VI1EUR03FT015.eop-EUR03.prod.protection.outlook.com (2603:10a6:d10:95:cafe::eb) by FR3P281CA0136.outlook.office365.com (2603:10a6:d10:95::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.17 via Frontend Transport; Fri, 18 Nov 2022 10:45:28 +0000 Received: from nebula.arm.com (40.67.248.234) by VI1EUR03FT015.mail.protection.outlook.com (100.127.144.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:27 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Fri, 18 Nov 2022 10:45:16 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.16 via Frontend Transport; Fri, 18 Nov 2022 10:45:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2519d8d8-672e-11ed-91b6-6bf2151ebd3b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zPDBV9XWmcg+tmJ+qaggSEwwP7UYQrtfJrIfAn/ghm8=; b=gYpMeoi0cAV8S5MEdr2i0zEw2ONGil2iKvajFMMX8o1Ty9L7aWYU+GmCGKffP0cKd6+WL5dJHeIzZMeFYQi+zNWnYZWyj1JzIYxvuFuAtUJQqNen0X4H5hRhhSonVtCAEZ5DrBTooD5yl11aB9u6GPpMZQgH/zMFHE6ohhUYs1k= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 761e9207070253d9 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cbsOpHSk60d4zwAv4sREGs6gsp5u516P6BDfuTSrVbygwoDbpclj87i1aBYo8glKU3k8X6oL/Om8WPzel+Gg/3EQMmWk2VYoYQwmdKJwqziIyZ7VTnDX6qqpOmeJa1Wj1rsR0GUeGvSIZgrIMDOWMa+Tk8yFbZSOQLZLCuIo8wsJBhKB74kMaNYgjULIVvo8+llPoopnZFCvkTu/8euqHV0oVRs/QwiNF93YXZx4Qv7jMVHolsU/M2giwV6nrgFp5CCkx/1NXLxgaQb2Anr/2fRJFnfXiyL7zGb3oyb2sYk3XOpQ30pQjH9jQGbUIy20WSuuOeEWNyd6/Y3pMxOOmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zPDBV9XWmcg+tmJ+qaggSEwwP7UYQrtfJrIfAn/ghm8=; b=dtH4bsC2F7By9VmmRqd0p0GMNFDgpnWVGepIVGA2li+Bc8ZzpVko7YPH2B+Q2CSkZVB5NZe6zo3UREC/zEaxPMtjOTl3/2pQI1xDswYEZt8JtBtsPeYVCdhjJRXRHBGCUsg3jFDgKpq05zuLme+6TSicJqEn6cX6yT+rT2mppoBc1yuxXPWNOKy28A+3Qu0Xwkknx4rWrr7Pe1AssT+mvcF0LNRGO2izUyoVfQB+OsoIQe1iUKnYvRJvlFPaHw1lO4bWxMe3CpduTFzIS7Go5oF7mXeA9+TCu951OkyADoHFZ5bNay2mo8LpVtA7wYIEi59zlqB7LgDNSPraPxR2ng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zPDBV9XWmcg+tmJ+qaggSEwwP7UYQrtfJrIfAn/ghm8=; b=gYpMeoi0cAV8S5MEdr2i0zEw2ONGil2iKvajFMMX8o1Ty9L7aWYU+GmCGKffP0cKd6+WL5dJHeIzZMeFYQi+zNWnYZWyj1JzIYxvuFuAtUJQqNen0X4H5hRhhSonVtCAEZ5DrBTooD5yl11aB9u6GPpMZQgH/zMFHE6ohhUYs1k= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu Subject: [PATCH v9 1/6] xen/x86: Provide helpers for common code to access acpi_numa Date: Fri, 18 Nov 2022 18:45:03 +0800 Message-ID: <20221118104508.768274-2-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221118104508.768274-1-wei.chen@arm.com> References: <20221118104508.768274-1-wei.chen@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: VI1EUR03FT015:EE_|DB4PR08MB9312:EE_|AM7EUR03FT006:EE_|GVXPR08MB7821:EE_ X-MS-Office365-Filtering-Correlation-Id: c1b50885-3619-4c03-a583-08dac95207c9 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: XmcoCAlg8GIEFiR1CKMlavwkryiNGS0Sbm1nUTR2DFdM3czxnAusa5jdbOwjJ/c0lbk+P8bYCjDktfQEtkp6Y/E7WL7LnrcoPsWJnp31ccYJzGTcg6orVe0jzotnQxtrasMPYD1uBjozFIOw7IYG751le6HHp7RYcMRBcsuETOpflBuuzwgW3+wfwOiS1eAxs73wuNjAr4DYF7eyfkshZCdU1FtZGDrP5I2uYqYZyVx+gdUWhh1kdHI41fKnPVgRophDN70uIajivfaU+W9xhSyF6au/1k4pdLx3eRFT19ieS9exhVQQHdZpR/5VrLPePSA0WWoQ2VA8s+FvDvtj/z0clHdZ/BD4Na4pr9F771FR3V2SGX5zCjOI/WaVV7SEIdsdFOAV4d2qfyYjF7MadmmEKp5gwBgUD1mzTi/o0W0Oap26S1IIrQK+HKHm6haz35VcZCnElS8SKPYseSnH29NaZk33+77NsYqVjaYO7P90tX06MUqIEqx3BZk7OdRIMPU/GFtsagijXCFqFrpfj0BEICChFSVIHcRCCVRl/efpzSBMX/0bXl0HKT1fl7f/2k4h+bxDKWg4HllGwUuXYsuN0pZ9dtPxgX11kRnVJY1xa9uL30sLdbuz1jWSRwsjl9sXj2YE6Pu8gFEGQikcGVD7kzzVPaeObZKpgjI+FIKDLWzTKdzrpzIGAntPBU4L1NpgHQkgv63O0TlDwNveBwi8JZaB3lYW1M0VgFL6hrA= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(39850400004)(136003)(396003)(376002)(346002)(451199015)(46966006)(40470700004)(36840700001)(36756003)(82740400003)(44832011)(356005)(81166007)(2906002)(36860700001)(426003)(83380400001)(47076005)(86362001)(41300700001)(40460700003)(70586007)(4326008)(26005)(8676002)(70206006)(186003)(316002)(6916009)(54906003)(336012)(40480700001)(8936002)(478600001)(5660300002)(7696005)(2616005)(6666004)(82310400005)(1076003)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR08MB9312 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT006.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 48f56b80-f14d-4b6a-056a-08dac9520170 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KfZnQCnawTvhzOhrzFwCsYQDmX++EdtTuwnOrtC7kZXsxhyo6j5hrZxGMrunk9iVFydWO+Il7qIy79pT19nrCTesfD20E3b/GAkj+S/6eyXsG0Y9QWQvOhdVt+BKXtC1Jmxad0TL/r3+V/sLjEZOW0m1YVPZkzVJLifRYSrl7xpRzXGVxrZp43gBkXOWym2h0rN6ssQCpa0HRGuELRVsF/2W8/X8kUS3wSAT2bjz6GO7ecUz4tS/E6UrLIlz6QPRX/rWafLfnhII6Jk02D73e2/TtR35j5/cvQ9A54l9xw+UR2r4eurB2x54kXCuzfpNTWqeMEFZSoPoBicocf/Bu5JiUZMCX+vBN3N26xoqjpS2xXWgWVkE6TrssuAn4bduKqjRgjTZtJZqwzONJjdKAz6d77GoVlOB/nItU9T9aEpjZqOSPYza8VGYlN77xNUufvBkoy7gJajeMAedWHi1enNUKbIdMPvOLQM3IbRJG4acPscfRyXKmzDN+L6MctoRbh6YPjND1UCgLWKHGlHnYGigpel/TtXHA7Gb92X4Azr2fSBa9n3GlcKwH+T22Rr3H1vldxDU3Xj9zbMGNAhoDEgds0JgBBV9sSu+zTNA7+BYNDEhMsdYozUiix5dt12VcL6MPA9NrjBsbTUe5RNSb+8RTedu8/yhSrFJPJCyGDoCflfz/GSjwNHw/E+JdKil0S0Xi7s96Mbe9VQ0aWNkaw== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(346002)(376002)(396003)(136003)(451199015)(46966006)(40470700004)(36840700001)(26005)(40480700001)(478600001)(6666004)(7696005)(6916009)(2616005)(86362001)(47076005)(54906003)(426003)(186003)(316002)(1076003)(336012)(82310400005)(2906002)(70586007)(82740400003)(36756003)(8936002)(36860700001)(81166007)(70206006)(83380400001)(5660300002)(41300700001)(4326008)(44832011)(40460700003)(8676002);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 10:45:38.1034 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c1b50885-3619-4c03-a583-08dac95207c9 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT006.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB7821 X-ZohoMail-DKIM: pass (identity @armh.onmicrosoft.com) X-ZM-MESSAGEID: 1668768377230100005 Content-Type: text/plain; charset="utf-8" acpi_numa is a specific NUMA switch for ACPI NUMA implementation. Other NUMA implementation may not need this switch. But this switch is not only used by ACPI code, it is also used directly in some general NUMA logic code. So far this hasn't caused any problem because Xen only has x86 implementing ACPI NUMA, but now Arm is implementing device tree based NUMA. Accesssing acpi_numa directly in some functions will be a block of reusing NUMA common code. It is also difficult for us to replace it with a new generic switch, because it is hard to prove that the new switch states can guarantee the original code will work correctly. So in this patch, we provide two helpers for common code to update and get states of acpi_numa. And other new NUMA implementations just need to provide the same helpers for common code. In this case, the generic NUMA logic code can be reused by all NUMA implementations. Signed-off-by: Wei Chen Reviewed-by: Jan Beulich --- v8 -> v9: 1. No change. v7 -> v8: 1. No change. v6 -> v7: 1. Add Rb. v5 -> v6: 1. Revert arch_numa_broken to arch_numa_disabled, as acpi_numa can be set to -1 by users. So acpi_numa < 0 does not mean a broken firmware. v4 -> v5: 1. Use arch_numa_broken instead of arch_numa_disabled for acpi_numa < 0 check. Because arch_numa_disabled might include acpi_numa < 0 (init failed) and acpi_numa =3D=3D 0 (no data or data no init) cases. v3 -> v4: 1. Drop parameter from arch_numa_disabled, the parameter will be introduced in later patch where use it. 2. Drop unnecessary "else" from arch_numa_setup, and fix its indentation. v2 -> v3: 1. Drop enumeration of numa status. 2. Use helpers to get/update acpi_numa. 3. Insert spaces among parameters of strncmp in numa_setup. v1 -> v2: 1. Remove fw_numa. 2. Use enumeration to replace numa_off and acpi_numa. 3. Correct return value of srat_disabled. 4. Introduce numa_enabled_with_firmware. --- xen/arch/x86/include/asm/numa.h | 5 +++-- xen/arch/x86/numa.c | 38 ++++++++++++++++++++++----------- 2 files changed, 28 insertions(+), 15 deletions(-) diff --git a/xen/arch/x86/include/asm/numa.h b/xen/arch/x86/include/asm/num= a.h index c32ccffde3..237f2c6dbf 100644 --- a/xen/arch/x86/include/asm/numa.h +++ b/xen/arch/x86/include/asm/numa.h @@ -32,8 +32,9 @@ extern void numa_add_cpu(int cpu); extern void numa_init_array(void); extern bool numa_off; =20 - -extern int srat_disabled(void); +extern int arch_numa_setup(const char *opt); +extern bool arch_numa_disabled(void); +extern bool srat_disabled(void); extern void numa_set_node(int cpu, nodeid_t node); extern nodeid_t setup_node(unsigned int pxm); extern void srat_detect_node(int cpu); diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c index 322157fab7..1c3198445d 100644 --- a/xen/arch/x86/numa.c +++ b/xen/arch/x86/numa.c @@ -50,9 +50,28 @@ nodemask_t __read_mostly node_online_map =3D { { [0] =3D= 1UL } }; bool numa_off; s8 acpi_numa =3D 0; =20 -int srat_disabled(void) +int __init arch_numa_setup(const char *opt) { - return numa_off || acpi_numa < 0; +#ifdef CONFIG_ACPI_NUMA + if ( !strncmp(opt, "noacpi", 6) ) + { + numa_off =3D false; + acpi_numa =3D -1; + return 0; + } +#endif + + return -EINVAL; +} + +bool arch_numa_disabled(void) +{ + return acpi_numa < 0; +} + +bool srat_disabled(void) +{ + return numa_off || arch_numa_disabled(); } =20 /* @@ -294,28 +313,21 @@ void numa_set_node(int cpu, nodeid_t node) /* [numa=3Doff] */ static int __init cf_check numa_setup(const char *opt) { - if ( !strncmp(opt,"off",3) ) + if ( !strncmp(opt, "off", 3) ) numa_off =3D true; - else if ( !strncmp(opt,"on",2) ) + else if ( !strncmp(opt, "on", 2) ) numa_off =3D false; #ifdef CONFIG_NUMA_EMU else if ( !strncmp(opt, "fake=3D", 5) ) { numa_off =3D false; - numa_fake =3D simple_strtoul(opt+5,NULL,0); + numa_fake =3D simple_strtoul(opt + 5, NULL, 0); if ( numa_fake >=3D MAX_NUMNODES ) numa_fake =3D MAX_NUMNODES; } -#endif -#ifdef CONFIG_ACPI_NUMA - else if ( !strncmp(opt,"noacpi",6) ) - { - numa_off =3D false; - acpi_numa =3D -1; - } #endif else - return -EINVAL; + return arch_numa_setup(opt); =20 return 0; }=20 --=20 2.25.1 From nobody Fri Apr 19 19:11:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=2; a=rsa-sha256; t=1668768384; cv=pass; d=zohomail.com; s=zohoarc; b=kino44Mnuww/t1TYzsNsESuFHa7D7xr0G/MMkOnUFQ2haVnpZiTQZXT+5ri5lSOs/89rSWL3YKXaYt+EPJqSOaDXwWs+V1/h2v7b+NtiY/efWPz9tPUNkkB6GXMFIC6aVIZ3jTgy5xGbeOx+aEUH9Kl3tU8PIC3Q9wMZ4iHK41I= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1668768384; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=TvHCs77zTKjeJCRttCK5jTM1WRW48h5RjGDiyWGaZqM=; b=VQSh0NcZ/lHt4aaIHDYCBLUHtWby2nuezYNEqrO8ppqmUFeu57B9xQVOUfXUxeUFkcmro6/LC8BFEHQxNaXY7JzGAp0Ne8Wx7fGA6+08dDVkaQi0v23hIhm0mbIjU385tZr5BzIQGL9TT+TWM9WdO6+dBlQcCV/io7IqhFMRvf0= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1668768384140811.6347037729853; Fri, 18 Nov 2022 02:46:24 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.445568.700840 (Exim 4.92) (envelope-from ) id 1ovysh-0008BN-LT; Fri, 18 Nov 2022 10:45:59 +0000 Received: by outflank-mailman (output) from mailman id 445568.700840; Fri, 18 Nov 2022 10:45:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysh-0008B5-Gr; Fri, 18 Nov 2022 10:45:59 +0000 Received: by outflank-mailman (input) for mailman id 445568; Fri, 18 Nov 2022 10:45:58 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysf-0006R2-H0 for xen-devel@lists.xenproject.org; Fri, 18 Nov 2022 10:45:58 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on20602.outbound.protection.outlook.com [2a01:111:f400:7e1a::602]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 28845b50-672e-11ed-8fd2-01056ac49cbb; Fri, 18 Nov 2022 11:45:46 +0100 (CET) Received: from AM6P191CA0034.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8b::47) by AM9PR08MB6164.eurprd08.prod.outlook.com (2603:10a6:20b:287::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.8; Fri, 18 Nov 2022 10:45:49 +0000 Received: from AM7EUR03FT041.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:8b:cafe::89) by AM6P191CA0034.outlook.office365.com (2603:10a6:209:8b::47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.20 via Frontend Transport; Fri, 18 Nov 2022 10:45:49 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT041.mail.protection.outlook.com (100.127.140.233) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:49 +0000 Received: ("Tessian outbound f394866f3f2b:v130"); Fri, 18 Nov 2022 10:45:48 +0000 Received: from a36fd8b09b5b.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 78C014BE-4453-40A7-974F-8BAE98B7C0D3.1; Fri, 18 Nov 2022 10:45:41 +0000 Received: from EUR01-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a36fd8b09b5b.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 18 Nov 2022 10:45:41 +0000 Received: from FR3P281CA0146.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:95::18) by AM8PR08MB6449.eurprd08.prod.outlook.com (2603:10a6:20b:364::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8; Fri, 18 Nov 2022 10:45:30 +0000 Received: from VI1EUR03FT015.eop-EUR03.prod.protection.outlook.com (2603:10a6:d10:95:cafe::a5) by FR3P281CA0146.outlook.office365.com (2603:10a6:d10:95::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.18 via Frontend Transport; Fri, 18 Nov 2022 10:45:30 +0000 Received: from nebula.arm.com (40.67.248.234) by VI1EUR03FT015.mail.protection.outlook.com (100.127.144.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:30 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Fri, 18 Nov 2022 10:45:20 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.16 via Frontend Transport; Fri, 18 Nov 2022 10:45:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 28845b50-672e-11ed-8fd2-01056ac49cbb DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TvHCs77zTKjeJCRttCK5jTM1WRW48h5RjGDiyWGaZqM=; b=D8WHmA9BcFu+Tlt5sILvC2wDqGxDUrNfNr6b+Dy+z2WckjKsyZlTzaZK6arz3PjGlz6mvWF1FNerc63VShxOpBWdq3zykzDJ8+MoQRb7kal+TD/6IZgh5qWf3jiB/aC/APlOdVMdBN/E+EgyKEhX03RVpjVysyOwyVbsm4Dtl2Q= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 55a5b162c61f6222 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=inHTS0E1ROt4ix3wImrljxvDQpsHWO4cugzuKHxm8ABjL9roXhcpua5FtsoJ9508BCa8wIkSLLzRAj93rS2C4SiWqM6DMi8LTTQewEdvrMARXwkfS17UwtQsdQ+UmL6N9mwDW1tDkR7CIYNEh19IlWQbALRKWXpqhay8u8IQUWhpphdqSTSW8W8HwEasii+vFXGOXqHLyc8zKaeAbBkje8ct8BsKLfuVpNnb1wbLIFMdl0JF2+wXjOn+pzMTWpXSL26v0qDCR7rc13rL3MrbIPwDbEbaeT8hthXQlgLVmAvA9G7llj5iFI/qKU6a4VBgagHogidla6ZdefrzBO7SDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TvHCs77zTKjeJCRttCK5jTM1WRW48h5RjGDiyWGaZqM=; b=LQJ4YBSqnsDU8HCg/lVfQwF2axAd9F9s+FXL1mUBb3ypbmICKR4P03eOx13uwTRUSzXZJQexoZTm8+zleKni6GFKhALf2jTG47ZE/Yk6TugN9oAQKt2eaM3FO6gluTHxdXF9qd3aq2/Ljd95gsDCqN5GYh8BImGr/5/ffEeusuL0dW3eLx79LvEG39yfqhnEMXEGoK0tlV/axjxf+KpBYXl4jHQqN9ZYJm6ASjQBfyCkHRod7g0X7s93IpfjSr4gWC/0QyCbdVPjyTyYk5Yq9Rscj2oPDGK8GYm95p/rSJqOlFuK176sAswvic6h3VqDUOk5qkiuah1drNjJDx/FMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TvHCs77zTKjeJCRttCK5jTM1WRW48h5RjGDiyWGaZqM=; b=D8WHmA9BcFu+Tlt5sILvC2wDqGxDUrNfNr6b+Dy+z2WckjKsyZlTzaZK6arz3PjGlz6mvWF1FNerc63VShxOpBWdq3zykzDJ8+MoQRb7kal+TD/6IZgh5qWf3jiB/aC/APlOdVMdBN/E+EgyKEhX03RVpjVysyOwyVbsm4Dtl2Q= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v9 2/6] xen/x86: move generically usable NUMA code from x86 to common Date: Fri, 18 Nov 2022 18:45:04 +0800 Message-ID: <20221118104508.768274-3-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221118104508.768274-1-wei.chen@arm.com> References: <20221118104508.768274-1-wei.chen@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: VI1EUR03FT015:EE_|AM8PR08MB6449:EE_|AM7EUR03FT041:EE_|AM9PR08MB6164:EE_ X-MS-Office365-Filtering-Correlation-Id: 3472fa8b-54ee-4ec2-2a04-08dac9520e5c x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: orCV2S6pIt1+SlDo1gxet6kHJGCoMfSflOGKJS/M9b6wVqL9wYTOeKDQOiXzaA3pQES9++Vho5zMh2q5gH/DUJgM26yHHTdgQSU0Lfmv6kP+lzorVMo27C4DI+4696OjnSkqnJG6w6Ruv5TMZRpbhddpvjnzo+NrSDxBn3FNb4nYxMBHoETnXsl9NPTSi0pxNMT1B6Sy4NbgEXX20QCCNcKYG1J0RGpUOah1mAIHo2N6KN1XFlfcY2MmLn6IdpSqIgBsKb8loHk5p2ruBnQa/k5gK8CbWYWRW8t7URbgwVgrSNv1vTrECXYglbH9KHhXWFHyZFcdkDbIZAU4ZAkL0cm4na6ECUdTM05WqJlGXv0DbfM+qPbdJ8w6GVk0damTmpmp4Smp6SVHUhFi37Rc5Fr8Ed0eQ7cy6lLfm/NAdEJg+F20s+n2ipbVw5MTo1abFytF3XXdG3iYR4Vol0F+RJfe3g6s25pGvqF5N+CuIqQPwSD39aAE6daU3tx9CSKSOwS8fR2cI1vABXjiJzvKmYnK3SogP3+02TnsZL+RnhxydhH2Yf5uDiLQ2Ur+0GJuyaha5ENj0it9yJcCAdXHqg/sqwlyFJkFr14kbOMXD6KueXHeVFAjeBQC4NCVpNH1kkSfTJXvFdFxNjodLJbEKSJ3HwAHjRSBhAMIysp7PDn5m21jeiSpxh31Bv8i1t68/YP89iIAP8MV2Wj4+3cQFx51QSUDGMTV8gjCLZJnQbpTrMnZRu1mOVM8V+ogv78S X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(136003)(39860400002)(346002)(376002)(396003)(451199015)(40470700004)(46966006)(36840700001)(40460700003)(36756003)(2616005)(81166007)(6916009)(356005)(316002)(54906003)(40480700001)(6666004)(41300700001)(7696005)(82310400005)(26005)(70206006)(8676002)(4326008)(426003)(86362001)(70586007)(47076005)(83380400001)(2906002)(30864003)(186003)(5660300002)(44832011)(336012)(82740400003)(8936002)(1076003)(36860700001)(478600001)(21314003)(36900700001)(579004);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB6449 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT041.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 05dc9b1d-d8f2-4f75-c127-08dac952031a X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2GcICMKnxLqeI47rCADDgF+OSuBJ9J2RNaS4ogZ5Uwh5HbMRP+6KtBIlHVFF/VgvSXJwWESK47EBpsQHcxH+sqHUwDLjHiOPE6X+MyvwQYsQ27FZQfoKOk/YApzkI1bwe7OqH0L5UuHH0c3UfgCI8g1W7/tx6FLw1LnC5+Z/DEkdSieXnurBtCOZlYI2hal+4nJyYBCLtOxy55cGBH8vfoMrFspglctDYOcV3Gn81bS9m8aAyq7/3Q6GP6M6IiTdWG2TbCxaIC47FD+75fhaXCtsbDnpPXATbPh7n4cIe/+uoPZrQ9uIH7V75W+xJy7L8hI3cqkhulEm67J3ZfL8R0A8Q8bCJS0SWzd/Cm+iriE0sjGk7YMdXRzGO/A71I84sdW3BgMjpHCNhmFLQ1ePjYonWeUxIWvG53EnrH0ei3gs2wU+A8adQlXVR04dAwCxduZV4K1LZp0K+Nft4Hk7gY+8fIP1dGhO5sUVFauh91aveNbHdV8otAX8a6T3H3CfQ0LX0NKPXLdJzV1e3ubBn7IiAW7dp2Q00A0nCCMHNSvosXNf7Z4b3Dlf41R928twzA0j5Zr12AIGs7BHwardORP+k7er+k06YeDv9+5165eeF1DaCmFfttSXVf36C8KRyzVZ4WxmKHK994iH1eaulK0eM8GtPpLn3Mx63Ax7Se/8JSb+88yKN1NoroJvXVbY9pzXK+53vO6R4x4n0g658YWcr9yw0rvm/xLr4CWg37DCrKt9Xy7SU9+9+YoeHrJA+1+p3lW8N6AIjQbijcAq9g== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230022)(4636009)(346002)(396003)(39860400002)(136003)(376002)(451199015)(40470700004)(46966006)(36840700001)(8936002)(44832011)(30864003)(5660300002)(4326008)(8676002)(70586007)(41300700001)(54906003)(70206006)(316002)(6916009)(26005)(2906002)(478600001)(36756003)(186003)(1076003)(2616005)(47076005)(6666004)(7696005)(336012)(426003)(107886003)(36860700001)(82310400005)(40480700001)(40460700003)(83380400001)(81166007)(82740400003)(86362001)(21314003)(579004);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 10:45:49.0987 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3472fa8b-54ee-4ec2-2a04-08dac9520e5c X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT041.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6164 X-ZohoMail-DKIM: pass (identity @armh.onmicrosoft.com) X-ZM-MESSAGEID: 1668768385268100001 Content-Type: text/plain; charset="utf-8" There are some codes in x86/numa.c can be shared by common architectures to implememnt NUMA support. Just like some variables and functions to check and store NUMA memory map. And some variables and functions to do NUMA initialization. In this patch, we move them to common/numa.c and xen/numa.h and use the CONFIG_NUMA to gate them for non-NUMA supported architectures. As the target header file is Xen-style, so we trim some spaces and replace tabs for the codes that has been moved to xen/numa.h at the same time. As acpi_scan_nodes has been used in a common function, it doesn't make sense to use acpi_xxx in common code, so we rename it to numa_process_nodes in this patch too. After that if we still use CONFIG_ACPI_NUMA in to gate numa_process_nodes in numa_initmem_init, that doesn't make sense. As CONFIG_NUMA will be selected by CONFIG_ACPI_NUMA for x86. So, we replace CONFIG_ACPI_NUMA by CONFIG_NUMA to gate numa_process_nodes. As arch_numa_disabled has been implememnted for ACPI NUMA, we can rename srat_disabled to numa_disabled and move it to common code as well. The macro node_to_first_cpu(node) hasn't been used anywhere, so we drop it in this patch too. Because some architectures allow to use all 64 physical address bits, but some architectures are not (like Arm64 allows 52, 48 bits). In this case, we use min(PADDR_BITS, BITS_PER_LONG - 1) to calculate the shift when only one node is in the system in this patch too. Signed-off-by: Wei Chen Reviewed-by: Jan Beulich --- v8 -> v9: 1. No change. v7 -> v8: 1. Add Rb. 2. Change "of shift too small" to "or shift too small". v6 -> v7: 1. Restore %d for nodeid_t in dump_numa. 2. Use sizeof(page_num_node) for page_num_node size in memset. 3. Add description for using min(PADDR_BITS, BITS_PER_LONG - 1) to calculate the shift when only one node is in the system. 4. Use %pd for domain print. v5 -> v6: 1. Replace numa_scan_node to numa_process_nodes in commit log. 2. Limit the scope of page_num_node, vnuma and page of numa_setup function. 3. Use memset to init page_num_node instead of for_each_online_node. 4. Use %u instead of %d for nodeid_t and j in numa_setup print messages. 5. Use min(PADDR_BITS, BITS_PER_LONG - 1) to calculate the shift when only one node is in the system. 6. Drop the marco: node_to_first_cpu(node) v4 -> v5: 1. Use nodeid_t instead of uint8_t for memnodemap. 2. Restore to use typeof(*memnodemap) for _memnodemap, this will avoid the further adjustments for _memnodemap's type. 3. Use __ro_after_init for numa_off. 4. Use pointer-to-const for proper function parameters. 5. Use unsigned int for variables that are not realy used for node ID. 6. Fix code comments code-style and adjust the length. 7. Fix code-styles. 8. Rename numa_scan_nodes to numa_process_nodes. 9. Use a plain "int ret" to record compute_hash_shift return value. v3 -> v4: 1. Restore compute_hash_shift's return value to int. 2. Remove unnecessary parentheses for macros. 3. Use unsigned int for proper variables. 4. Fix some code-style. v2 -> v3: 1. Remove acpi.h from common/numa.c. 2. Rename acpi_scan_nodes to numa_scan_nodes. 3. Replace u8 by uint8_t for memnodemap. 4. Use unsigned int for memnode_shift and adjust related functions (compute_hash_shift, populate_memnodemap) to use correct types for return values or parameters. 5. Use nodeid_t for nodeid and node numbers. 6. Use __read_mostly and __ro_after_init for appropriate variables. 7. Adjust the __read_mostly and __initdata location for some variables. 8. convert from plain int to unsigned for cpuid and other proper variables. 9. Use __attribute_pure__ instead of __attribute__((pure)). 10. Replace CONFIG_ACPI_NUMA by CONFIG_NUMA in numa_initmem_init. 11. Add const for some functions' parameters. 12. Move srat_disabled to common code with new name numa_disabled. 13. Fix some spaces code-style for numa_emulation. 14. Change from int to unsigned int for numa_fake. v1 -> v2: 1. New patch in v2. --- xen/arch/x86/include/asm/acpi.h | 1 - xen/arch/x86/include/asm/numa.h | 57 +--- xen/arch/x86/include/asm/setup.h | 1 - xen/arch/x86/numa.c | 433 +--------------------------- xen/arch/x86/smpboot.c | 2 +- xen/arch/x86/srat.c | 10 +- xen/common/Makefile | 1 + xen/common/numa.c | 464 +++++++++++++++++++++++++++++++ xen/include/xen/numa.h | 66 +++++ 9 files changed, 539 insertions(+), 496 deletions(-) create mode 100644 xen/common/numa.c diff --git a/xen/arch/x86/include/asm/acpi.h b/xen/arch/x86/include/asm/acp= i.h index 9a9cc4c240..5c2dd5da2d 100644 --- a/xen/arch/x86/include/asm/acpi.h +++ b/xen/arch/x86/include/asm/acpi.h @@ -102,7 +102,6 @@ extern unsigned long acpi_wakeup_address; #define ARCH_HAS_POWER_INIT 1 =20 extern s8 acpi_numa; -extern int acpi_scan_nodes(u64 start, u64 end); #define NR_NODE_MEMBLKS (MAX_NUMNODES*2) =20 extern struct acpi_sleep_info acpi_sinfo; diff --git a/xen/arch/x86/include/asm/numa.h b/xen/arch/x86/include/asm/num= a.h index 237f2c6dbf..6c87942d43 100644 --- a/xen/arch/x86/include/asm/numa.h +++ b/xen/arch/x86/include/asm/numa.h @@ -9,72 +9,17 @@ typedef u8 nodeid_t; =20 extern int srat_rev; =20 -extern nodeid_t cpu_to_node[NR_CPUS]; -extern cpumask_t node_to_cpumask[]; - -#define cpu_to_node(cpu) (cpu_to_node[cpu]) -#define parent_node(node) (node) -#define node_to_first_cpu(node) (__ffs(node_to_cpumask[node])) -#define node_to_cpumask(node) (node_to_cpumask[node]) - -struct node {=20 - paddr_t start, end; -}; - -extern int compute_hash_shift(struct node *nodes, int numnodes, - nodeid_t *nodeids); extern nodeid_t pxm_to_node(unsigned int pxm); =20 #define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT)) -#define VIRTUAL_BUG_ON(x)=20 =20 -extern void numa_add_cpu(int cpu); -extern void numa_init_array(void); -extern bool numa_off; - -extern int arch_numa_setup(const char *opt); -extern bool arch_numa_disabled(void); -extern bool srat_disabled(void); -extern void numa_set_node(int cpu, nodeid_t node); +extern bool numa_disabled(void); extern nodeid_t setup_node(unsigned int pxm); extern void srat_detect_node(int cpu); =20 -extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end= ); extern nodeid_t apicid_to_node[]; extern void init_cpu_to_node(void); =20 -static inline void clear_node_cpumask(int cpu) -{ - cpumask_clear_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]); -} - -/* Simple perfect hash to map pdx to node numbers */ -extern int memnode_shift;=20 -extern unsigned long memnodemapsize; -extern u8 *memnodemap; - -struct node_data { - unsigned long node_start_pfn; - unsigned long node_spanned_pages; -}; - -extern struct node_data node_data[]; - -static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr) -{=20 - nodeid_t nid; - VIRTUAL_BUG_ON((paddr_to_pdx(addr) >> memnode_shift) >=3D memnodemapsize); - nid =3D memnodemap[paddr_to_pdx(addr) >> memnode_shift];=20 - VIRTUAL_BUG_ON(nid >=3D MAX_NUMNODES || !node_data[nid]);=20 - return nid;=20 -}=20 - -#define NODE_DATA(nid) (&(node_data[nid])) - -#define node_start_pfn(nid) (NODE_DATA(nid)->node_start_pfn) -#define node_spanned_pages(nid) (NODE_DATA(nid)->node_spanned_pages) -#define node_end_pfn(nid) (NODE_DATA(nid)->node_start_pfn + \ - NODE_DATA(nid)->node_spanned_pages) #define arch_want_default_dmazone() (num_online_nodes() > 1) =20 extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node); diff --git a/xen/arch/x86/include/asm/setup.h b/xen/arch/x86/include/asm/se= tup.h index 21037b7f31..ae470ea12f 100644 --- a/xen/arch/x86/include/asm/setup.h +++ b/xen/arch/x86/include/asm/setup.h @@ -20,7 +20,6 @@ void early_time_init(void); =20 void set_nr_cpu_ids(unsigned int max_cpus); =20 -void numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn); void arch_init_memory(void); void subarch_init_memory(void); =20 diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c index 1c3198445d..90b2a22591 100644 --- a/xen/arch/x86/numa.c +++ b/xen/arch/x86/numa.c @@ -4,20 +4,11 @@ * Adapted for Xen: Ryan Harper */=20 =20 -#include -#include #include -#include +#include #include #include -#include -#include -#include -#include -#include #include -#include -#include =20 #ifndef Dprintk #define Dprintk(x...) @@ -26,28 +17,13 @@ /* from proto.h */ #define round_up(x,y) ((((x)+(y))-1) & (~((y)-1))) =20 -struct node_data node_data[MAX_NUMNODES]; - -/* Mapping from pdx to node id */ -int memnode_shift; -static typeof(*memnodemap) _memnodemap[64]; -unsigned long memnodemapsize; -u8 *memnodemap; - -nodeid_t cpu_to_node[NR_CPUS] __read_mostly =3D { - [0 ... NR_CPUS-1] =3D NUMA_NO_NODE -}; /* * Keep BIOS's CPU2node information, should not be used for memory allocai= on */ nodeid_t apicid_to_node[MAX_LOCAL_APIC] =3D { [0 ... MAX_LOCAL_APIC-1] =3D NUMA_NO_NODE }; -cpumask_t node_to_cpumask[MAX_NUMNODES] __read_mostly; =20 -nodemask_t __read_mostly node_online_map =3D { { [0] =3D 1UL } }; - -bool numa_off; s8 acpi_numa =3D 0; =20 int __init arch_numa_setup(const char *opt) @@ -69,270 +45,6 @@ bool arch_numa_disabled(void) return acpi_numa < 0; } =20 -bool srat_disabled(void) -{ - return numa_off || arch_numa_disabled(); -} - -/* - * Given a shift value, try to populate memnodemap[] - * Returns : - * 1 if OK - * 0 if memnodmap[] too small (of shift too small) - * -1 if node overlap or lost ram (shift too big) - */ -static int __init populate_memnodemap(const struct node *nodes, - int numnodes, int shift, nodeid_t *n= odeids) -{ - int i, res =3D -1; - - memset(memnodemap, NUMA_NO_NODE, memnodemapsize * sizeof(*memnodemap)); - for ( i =3D 0; i < numnodes; i++ ) - { - unsigned long spdx =3D paddr_to_pdx(nodes[i].start); - unsigned long epdx =3D paddr_to_pdx(nodes[i].end - 1); - - if ( spdx > epdx ) - continue; - if ( (epdx >> shift) >=3D memnodemapsize ) - return 0; - do { - if ( memnodemap[spdx >> shift] !=3D NUMA_NO_NODE && - (!nodeids || memnodemap[spdx >> shift] !=3D nodeids[i]) ) - return -1; - - if ( !nodeids ) - memnodemap[spdx >> shift] =3D i; - else - memnodemap[spdx >> shift] =3D nodeids[i]; - - spdx +=3D (1UL << shift); - } while ( spdx <=3D epdx ); - res =3D 1; - } - - return res; -} - -static int __init allocate_cachealigned_memnodemap(void) -{ - unsigned long size =3D PFN_UP(memnodemapsize * sizeof(*memnodemap)); - unsigned long mfn =3D mfn_x(alloc_boot_pages(size, 1)); - - memnodemap =3D mfn_to_virt(mfn); - mfn <<=3D PAGE_SHIFT; - size <<=3D PAGE_SHIFT; - printk(KERN_DEBUG "NUMA: Allocated memnodemap from %lx - %lx\n", - mfn, mfn + size); - memnodemapsize =3D size / sizeof(*memnodemap); - - return 0; -} - -/* - * The LSB of all start addresses in the node map is the value of the - * maximum possible shift. - */ -static int __init extract_lsb_from_nodes(const struct node *nodes, - int numnodes, const nodeid_t *nod= eids) -{ - int i, nodes_used =3D 0; - unsigned long spdx, epdx; - unsigned long bitfield =3D 0, memtop =3D 0; - - for ( i =3D 0; i < numnodes; i++ ) - { - spdx =3D paddr_to_pdx(nodes[i].start); - epdx =3D paddr_to_pdx(nodes[i].end - 1) + 1; - if ( spdx >=3D epdx ) - continue; - if ( i && (!nodeids || nodeids[i - 1] !=3D nodeids[i]) ) - bitfield |=3D spdx; - if ( !i || !nodeids || nodeids[i - 1] !=3D nodeids[i] ) - nodes_used++; - if ( epdx > memtop ) - memtop =3D epdx; - } - if ( nodes_used <=3D 1 ) - i =3D BITS_PER_LONG - 1; - else - i =3D find_first_bit(&bitfield, sizeof(unsigned long)*8); - memnodemapsize =3D ((memtop - 1) >> i) + 1; - return i; -} - -int __init compute_hash_shift(struct node *nodes, int numnodes, - nodeid_t *nodeids) -{ - int shift; - - shift =3D extract_lsb_from_nodes(nodes, numnodes, nodeids); - if ( memnodemapsize <=3D ARRAY_SIZE(_memnodemap) ) - memnodemap =3D _memnodemap; - else if ( allocate_cachealigned_memnodemap() ) - return -1; - printk(KERN_DEBUG "NUMA: Using %d for the hash shift.\n", shift); - - if ( populate_memnodemap(nodes, numnodes, shift, nodeids) !=3D 1 ) - { - printk(KERN_INFO "Your memory is not aligned you need to " - "rebuild your hypervisor with a bigger NODEMAPSIZE " - "shift=3D%d\n", shift); - return -1; - } - - return shift; -} -/* initialize NODE_DATA given nodeid and start/end */ -void __init setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end) -{ - unsigned long start_pfn =3D paddr_to_pfn(start); - unsigned long end_pfn =3D paddr_to_pfn(end); - - NODE_DATA(nodeid)->node_start_pfn =3D start_pfn; - NODE_DATA(nodeid)->node_spanned_pages =3D end_pfn - start_pfn; - - node_set_online(nodeid); -}=20 - -void __init numa_init_array(void) -{ - int rr, i; - - /* There are unfortunately some poorly designed mainboards around - that only connect memory to a single CPU. This breaks the 1:1 cpu->= node - mapping. To avoid this fill in the mapping for all possible - CPUs, as the number of CPUs is not known yet. - We round robin the existing nodes. */ - rr =3D first_node(node_online_map); - for ( i =3D 0; i < nr_cpu_ids; i++ ) - { - if ( cpu_to_node[i] !=3D NUMA_NO_NODE ) - continue; - numa_set_node(i, rr); - rr =3D cycle_node(rr, node_online_map); - } -} - -#ifdef CONFIG_NUMA_EMU -static int numa_fake __initdata =3D 0; - -/* Numa emulation */ -static int __init numa_emulation(unsigned long start_pfn, - unsigned long end_pfn) -{ - int i; - struct node nodes[MAX_NUMNODES]; - uint64_t sz =3D pfn_to_paddr(end_pfn - start_pfn) / numa_fake; - - /* Kludge needed for the hash function */ - if ( hweight64(sz) > 1 ) - { - u64 x =3D 1; - while ( (x << 1) < sz ) - x <<=3D 1; - if ( x < sz/2 ) - printk(KERN_ERR "Numa emulation unbalanced. Complain to mainta= iner\n"); - sz =3D x; - } - - memset(&nodes,0,sizeof(nodes)); - for ( i =3D 0; i < numa_fake; i++ ) - { - nodes[i].start =3D pfn_to_paddr(start_pfn) + i * sz; - if ( i =3D=3D numa_fake - 1 ) - sz =3D pfn_to_paddr(end_pfn) - nodes[i].start; - nodes[i].end =3D nodes[i].start + sz; - printk(KERN_INFO "Faking node %d at %"PRIx64"-%"PRIx64" (%"PRIu64"= MB)\n", - i, - nodes[i].start, nodes[i].end, - (nodes[i].end - nodes[i].start) >> 20); - node_set_online(i); - } - memnode_shift =3D compute_hash_shift(nodes, numa_fake, NULL); - if ( memnode_shift < 0 ) - { - memnode_shift =3D 0; - printk(KERN_ERR "No NUMA hash function found. Emulation disabled.\= n"); - return -1; - } - for_each_online_node ( i ) - setup_node_bootmem(i, nodes[i].start, nodes[i].end); - numa_init_array(); - - return 0; -} -#endif - -void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_p= fn) -{=20 - int i; - paddr_t start =3D pfn_to_paddr(start_pfn); - paddr_t end =3D pfn_to_paddr(end_pfn); - -#ifdef CONFIG_NUMA_EMU - if ( numa_fake && !numa_emulation(start_pfn, end_pfn) ) - return; -#endif - -#ifdef CONFIG_ACPI_NUMA - if ( !numa_off && !acpi_scan_nodes(start, end) ) - return; -#endif - - printk(KERN_INFO "%s\n", - numa_off ? "NUMA turned off" : "No NUMA configuration found"); - - printk(KERN_INFO "Faking a node at %"PRIpaddr"-%"PRIpaddr"\n", - start, end); - /* setup dummy node covering all memory */ - memnode_shift =3D BITS_PER_LONG - 1; - memnodemap =3D _memnodemap; - /* Dummy node only uses 1 slot in reality */ - memnodemap[0] =3D 0; - memnodemapsize =3D 1; - - nodes_clear(node_online_map); - node_set_online(0); - for ( i =3D 0; i < nr_cpu_ids; i++ ) - numa_set_node(i, 0); - cpumask_copy(&node_to_cpumask[0], cpumask_of(0)); - setup_node_bootmem(0, start, end); -} - -void numa_add_cpu(int cpu) -{ - cpumask_set_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]); -}=20 - -void numa_set_node(int cpu, nodeid_t node) -{ - cpu_to_node[cpu] =3D node; -} - -/* [numa=3Doff] */ -static int __init cf_check numa_setup(const char *opt) -{ - if ( !strncmp(opt, "off", 3) ) - numa_off =3D true; - else if ( !strncmp(opt, "on", 2) ) - numa_off =3D false; -#ifdef CONFIG_NUMA_EMU - else if ( !strncmp(opt, "fake=3D", 5) ) - { - numa_off =3D false; - numa_fake =3D simple_strtoul(opt + 5, NULL, 0); - if ( numa_fake >=3D MAX_NUMNODES ) - numa_fake =3D MAX_NUMNODES; - } -#endif - else - return arch_numa_setup(opt); - - return 0; -}=20 -custom_param("numa", numa_setup); - /* * Setup early cpu_to_node. * @@ -381,146 +93,3 @@ unsigned int __init arch_get_dma_bitsize(void) flsl(node_start_pfn(node) + node_spanned_pages(node) / 4 = - 1) + PAGE_SHIFT, 32); } - -static void cf_check dump_numa(unsigned char key) -{ - s_time_t now =3D NOW(); - unsigned int i, j, n; - struct domain *d; - struct page_info *page; - unsigned int page_num_node[MAX_NUMNODES]; - const struct vnuma_info *vnuma; - - printk("'%c' pressed -> dumping numa info (now =3D %"PRI_stime")\n", k= ey, - now); - - for_each_online_node ( i ) - { - paddr_t pa =3D pfn_to_paddr(node_start_pfn(i) + 1); - - printk("NODE%u start->%lu size->%lu free->%lu\n", - i, node_start_pfn(i), node_spanned_pages(i), - avail_node_heap_pages(i)); - /* sanity check phys_to_nid() */ - if ( phys_to_nid(pa) !=3D i ) - printk("phys_to_nid(%"PRIpaddr") -> %d should be %u\n", - pa, phys_to_nid(pa), i); - } - - j =3D cpumask_first(&cpu_online_map); - n =3D 0; - for_each_online_cpu ( i ) - { - if ( i !=3D j + n || cpu_to_node[j] !=3D cpu_to_node[i] ) - { - if ( n > 1 ) - printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node= [j]); - else - printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]); - j =3D i; - n =3D 1; - } - else - ++n; - } - if ( n > 1 ) - printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]); - else - printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]); - - rcu_read_lock(&domlist_read_lock); - - printk("Memory location of each domain:\n"); - for_each_domain ( d ) - { - process_pending_softirqs(); - - printk("Domain %u (total: %u):\n", d->domain_id, domain_tot_pages(= d)); - - for_each_online_node ( i ) - page_num_node[i] =3D 0; - - spin_lock(&d->page_alloc_lock); - page_list_for_each(page, &d->page_list) - { - i =3D phys_to_nid(page_to_maddr(page)); - page_num_node[i]++; - } - spin_unlock(&d->page_alloc_lock); - - for_each_online_node ( i ) - printk(" Node %u: %u\n", i, page_num_node[i]); - - if ( !read_trylock(&d->vnuma_rwlock) ) - continue; - - if ( !d->vnuma ) - { - read_unlock(&d->vnuma_rwlock); - continue; - } - - vnuma =3D d->vnuma; - printk(" %u vnodes, %u vcpus, guest physical layout:\n", - vnuma->nr_vnodes, d->max_vcpus); - for ( i =3D 0; i < vnuma->nr_vnodes; i++ ) - { - unsigned int start_cpu =3D ~0U; - - if ( vnuma->vnode_to_pnode[i] =3D=3D NUMA_NO_NODE ) - printk(" %3u: pnode ???,", i); - else - printk(" %3u: pnode %3u,", i, vnuma->vnode_to_pnode[= i]); - - printk(" vcpus "); - - for ( j =3D 0; j < d->max_vcpus; j++ ) - { - if ( !(j & 0x3f) ) - process_pending_softirqs(); - - if ( vnuma->vcpu_to_vnode[j] =3D=3D i ) - { - if ( start_cpu =3D=3D ~0U ) - { - printk("%d", j); - start_cpu =3D j; - } - } - else if ( start_cpu !=3D ~0U ) - { - if ( j - 1 !=3D start_cpu ) - printk("-%d ", j - 1); - else - printk(" "); - start_cpu =3D ~0U; - } - } - - if ( start_cpu !=3D ~0U && start_cpu !=3D j - 1 ) - printk("-%d", j - 1); - - printk("\n"); - - for ( j =3D 0; j < vnuma->nr_vmemranges; j++ ) - { - if ( vnuma->vmemrange[j].nid =3D=3D i ) - printk(" %016"PRIx64" - %016"PRIx64"\n", - vnuma->vmemrange[j].start, - vnuma->vmemrange[j].end); - } - } - - read_unlock(&d->vnuma_rwlock); - } - - rcu_read_unlock(&domlist_read_lock); -} - -static int __init cf_check register_numa_trigger(void) -{ - register_keyhandler('u', dump_numa, "dump NUMA info", 1); - return 0; -} -__initcall(register_numa_trigger); - diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index b46fd9ab18..9df08e9366 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -1350,7 +1350,7 @@ int cpu_add(uint32_t apic_id, uint32_t acpi_id, uint3= 2_t pxm) =20 x86_acpiid_to_apicid[acpi_id] =3D apic_id; =20 - if ( !srat_disabled() ) + if ( !numa_disabled() ) { nodeid_t node =3D setup_node(pxm); =20 diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index fbcd8749c4..ce507dac9e 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -238,7 +238,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x= 2apic_cpu_affinity *pa) unsigned pxm; nodeid_t node; =20 - if (srat_disabled()) + if (numa_disabled()) return; if (pa->header.length < sizeof(struct acpi_srat_x2apic_cpu_affinity)) { bad_srat(); @@ -274,7 +274,7 @@ acpi_numa_processor_affinity_init(const struct acpi_sra= t_cpu_affinity *pa) unsigned pxm; nodeid_t node; =20 - if (srat_disabled()) + if (numa_disabled()) return; if (pa->header.length !=3D sizeof(struct acpi_srat_cpu_affinity)) { bad_srat(); @@ -314,7 +314,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_m= em_affinity *ma) unsigned int i; bool next =3D false; =20 - if (srat_disabled()) + if (numa_disabled()) return; if (ma->header.length !=3D sizeof(struct acpi_srat_mem_affinity)) { bad_srat(); @@ -543,8 +543,8 @@ void __init srat_parse_regions(paddr_t addr) pfn_pdx_hole_setup(mask >> PAGE_SHIFT); } =20 -/* Use the information discovered above to actually set up the nodes. */ -int __init acpi_scan_nodes(paddr_t start, paddr_t end) +/* Use discovered information to actually set up the nodes. */ +int __init numa_process_nodes(paddr_t start, paddr_t end) { int i; nodemask_t all_nodes_parsed; diff --git a/xen/common/Makefile b/xen/common/Makefile index 3baf83d527..9a3a12b12d 100644 --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -26,6 +26,7 @@ obj-$(CONFIG_MEM_ACCESS) +=3D mem_access.o obj-y +=3D memory.o obj-y +=3D multicall.o obj-y +=3D notifier.o +obj-$(CONFIG_NUMA) +=3D numa.o obj-y +=3D page_alloc.o obj-$(CONFIG_HAS_PDX) +=3D pdx.o obj-$(CONFIG_PERF_COUNTERS) +=3D perfc.o diff --git a/xen/common/numa.c b/xen/common/numa.c new file mode 100644 index 0000000000..a135154c3e --- /dev/null +++ b/xen/common/numa.c @@ -0,0 +1,464 @@ +/* + * Generic VM initialization for NUMA setups. + * Copyright 2002,2003 Andi Kleen, SuSE Labs. + * Adapted for Xen: Ryan Harper + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +struct node_data __ro_after_init node_data[MAX_NUMNODES]; + +/* Mapping from pdx to node id */ +unsigned int __ro_after_init memnode_shift; +unsigned long __ro_after_init memnodemapsize; +nodeid_t *__ro_after_init memnodemap; +static typeof(*memnodemap) __ro_after_init _memnodemap[64]; + +nodeid_t __read_mostly cpu_to_node[NR_CPUS] =3D { + [0 ... NR_CPUS-1] =3D NUMA_NO_NODE +}; + +cpumask_t __read_mostly node_to_cpumask[MAX_NUMNODES]; + +nodemask_t __read_mostly node_online_map =3D { { [0] =3D 1UL } }; + +bool __ro_after_init numa_off; + +bool numa_disabled(void) +{ + return numa_off || arch_numa_disabled(); +} + +/* + * Given a shift value, try to populate memnodemap[] + * Returns : + * 1 if OK + * 0 if memnodmap[] too small (or shift too small) + * -1 if node overlap or lost ram (shift too big) + */ +static int __init populate_memnodemap(const struct node *nodes, + unsigned int numnodes, unsigned int = shift, + const nodeid_t *nodeids) +{ + unsigned int i; + int res =3D -1; + + memset(memnodemap, NUMA_NO_NODE, memnodemapsize * sizeof(*memnodemap)); + + for ( i =3D 0; i < numnodes; i++ ) + { + unsigned long spdx =3D paddr_to_pdx(nodes[i].start); + unsigned long epdx =3D paddr_to_pdx(nodes[i].end - 1); + + if ( spdx > epdx ) + continue; + + if ( (epdx >> shift) >=3D memnodemapsize ) + return 0; + + do { + if ( memnodemap[spdx >> shift] !=3D NUMA_NO_NODE && + (!nodeids || memnodemap[spdx >> shift] !=3D nodeids[i]) ) + return -1; + + if ( !nodeids ) + memnodemap[spdx >> shift] =3D i; + else + memnodemap[spdx >> shift] =3D nodeids[i]; + + spdx +=3D (1UL << shift); + } while ( spdx <=3D epdx ); + + res =3D 1; + } + + return res; +} + +static int __init allocate_cachealigned_memnodemap(void) +{ + unsigned long size =3D PFN_UP(memnodemapsize * sizeof(*memnodemap)); + unsigned long mfn =3D mfn_x(alloc_boot_pages(size, 1)); + + memnodemap =3D mfn_to_virt(mfn); + mfn <<=3D PAGE_SHIFT; + size <<=3D PAGE_SHIFT; + printk(KERN_DEBUG "NUMA: Allocated memnodemap from %lx - %lx\n", + mfn, mfn + size); + memnodemapsize =3D size / sizeof(*memnodemap); + + return 0; +} + +/* + * The LSB of all start addresses in the node map is the value of the + * maximum possible shift. + */ +static unsigned int __init extract_lsb_from_nodes(const struct node *nodes, + nodeid_t numnodes, + const nodeid_t *nodeids) +{ + unsigned int i, nodes_used =3D 0; + unsigned long bitfield =3D 0, memtop =3D 0; + + for ( i =3D 0; i < numnodes; i++ ) + { + unsigned long spdx =3D paddr_to_pdx(nodes[i].start); + unsigned long epdx =3D paddr_to_pdx(nodes[i].end - 1) + 1; + + if ( spdx >=3D epdx ) + continue; + + if ( i && (!nodeids || nodeids[i - 1] !=3D nodeids[i]) ) + bitfield |=3D spdx; + + if ( !i || !nodeids || nodeids[i - 1] !=3D nodeids[i] ) + nodes_used++; + + if ( epdx > memtop ) + memtop =3D epdx; + } + + if ( nodes_used <=3D 1 ) + i =3D min(PADDR_BITS, BITS_PER_LONG - 1); + else + i =3D find_first_bit(&bitfield, sizeof(unsigned long) * 8); + + memnodemapsize =3D ((memtop - 1) >> i) + 1; + + return i; +} + +int __init compute_hash_shift(const struct node *nodes, + unsigned int numnodes, const nodeid_t *nodei= ds) +{ + unsigned int shift =3D extract_lsb_from_nodes(nodes, numnodes, nodeids= ); + + if ( memnodemapsize <=3D ARRAY_SIZE(_memnodemap) ) + memnodemap =3D _memnodemap; + else if ( allocate_cachealigned_memnodemap() ) + return -1; + + printk(KERN_DEBUG "NUMA: Using %u for the hash shift\n", shift); + + if ( populate_memnodemap(nodes, numnodes, shift, nodeids) !=3D 1 ) + { + printk(KERN_INFO "Your memory is not aligned you need to " + "rebuild your hypervisor with a bigger NODEMAPSIZE " + "shift=3D%u\n", shift); + return -1; + } + + return shift; +} + +/* Initialize NODE_DATA given nodeid and start/end */ +void __init setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end) +{ + unsigned long start_pfn =3D paddr_to_pfn(start); + unsigned long end_pfn =3D paddr_to_pfn(end); + + NODE_DATA(nodeid)->node_start_pfn =3D start_pfn; + NODE_DATA(nodeid)->node_spanned_pages =3D end_pfn - start_pfn; + + node_set_online(nodeid); +} + +void __init numa_init_array(void) +{ + unsigned int i; + nodeid_t rr; + + /* + * There are unfortunately some poorly designed mainboards + * around that only connect memory to a single CPU. This + * breaks the 1:1 cpu->node mapping. To avoid this fill in + * the mapping for all possible CPUs, as the number of CPUs + * is not known yet. We round robin the existing nodes. + */ + rr =3D first_node(node_online_map); + for ( i =3D 0; i < nr_cpu_ids; i++ ) + { + if ( cpu_to_node[i] !=3D NUMA_NO_NODE ) + continue; + numa_set_node(i, rr); + rr =3D cycle_node(rr, node_online_map); + } +} + +#ifdef CONFIG_NUMA_EMU +static unsigned int __initdata numa_fake; + +/* Numa emulation */ +static int __init numa_emulation(unsigned long start_pfn, + unsigned long end_pfn) +{ + int ret; + unsigned int i; + struct node nodes[MAX_NUMNODES]; + uint64_t sz =3D pfn_to_paddr(end_pfn - start_pfn) / numa_fake; + + /* Kludge needed for the hash function */ + if ( hweight64(sz) > 1 ) + { + uint64_t x =3D 1; + + while ( (x << 1) < sz ) + x <<=3D 1; + if ( x < sz / 2 ) + printk(KERN_ERR "Numa emulation unbalanced. Complain to mainta= iner\n"); + sz =3D x; + } + + memset(&nodes, 0, sizeof(nodes)); + for ( i =3D 0; i < numa_fake; i++ ) + { + nodes[i].start =3D pfn_to_paddr(start_pfn) + i * sz; + + if ( i =3D=3D numa_fake - 1 ) + sz =3D pfn_to_paddr(end_pfn) - nodes[i].start; + + nodes[i].end =3D nodes[i].start + sz; + printk(KERN_INFO "Faking node %u at %"PRIx64"-%"PRIx64" (%"PRIu64"= MB)\n", + i, nodes[i].start, nodes[i].end, + (nodes[i].end - nodes[i].start) >> 20); + node_set_online(i); + } + + ret =3D compute_hash_shift(nodes, numa_fake, NULL); + if ( ret < 0 ) + { + printk(KERN_ERR "No NUMA hash function found. Emulation disabled.\= n"); + return -1; + } + memnode_shift =3D ret; + + for_each_online_node ( i ) + setup_node_bootmem(i, nodes[i].start, nodes[i].end); + + numa_init_array(); + + return 0; +} +#endif + +void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_p= fn) +{ + unsigned int i; + paddr_t start =3D pfn_to_paddr(start_pfn); + paddr_t end =3D pfn_to_paddr(end_pfn); + +#ifdef CONFIG_NUMA_EMU + if ( numa_fake && !numa_emulation(start_pfn, end_pfn) ) + return; +#endif + +#ifdef CONFIG_NUMA + if ( !numa_off && !numa_process_nodes(start, end) ) + return; +#endif + + printk(KERN_INFO "%s\n", + numa_off ? "NUMA turned off" : "No NUMA configuration found"); + + printk(KERN_INFO "Faking a node at %"PRIpaddr"-%"PRIpaddr"\n", + start, end); + + /* Setup dummy node covering all memory */ + memnode_shift =3D BITS_PER_LONG - 1; + memnodemap =3D _memnodemap; + + /* Dummy node only uses 1 slot in reality */ + memnodemap[0] =3D 0; + memnodemapsize =3D 1; + + nodes_clear(node_online_map); + node_set_online(0); + for ( i =3D 0; i < nr_cpu_ids; i++ ) + numa_set_node(i, 0); + + cpumask_copy(&node_to_cpumask[0], cpumask_of(0)); + setup_node_bootmem(0, start, end); +} + +void numa_add_cpu(unsigned int cpu) +{ + cpumask_set_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]); +} + +void numa_set_node(unsigned int cpu, nodeid_t node) +{ + cpu_to_node[cpu] =3D node; +} + +/* [numa=3Doff] */ +static int __init cf_check numa_setup(const char *opt) +{ + if ( !strncmp(opt, "off", 3) ) + numa_off =3D true; + else if ( !strncmp(opt, "on", 2) ) + numa_off =3D false; +#ifdef CONFIG_NUMA_EMU + else if ( !strncmp(opt, "fake=3D", 5) ) + { + numa_off =3D false; + numa_fake =3D simple_strtoul(opt + 5, NULL, 0); + if ( numa_fake >=3D MAX_NUMNODES ) + numa_fake =3D MAX_NUMNODES; + } +#endif + else + return arch_numa_setup(opt); + + return 0; +} +custom_param("numa", numa_setup); + +static void cf_check dump_numa(unsigned char key) +{ + s_time_t now =3D NOW(); + unsigned int i, j, n; + struct domain *d; + + printk("'%c' pressed -> dumping numa info (now =3D %"PRI_stime")\n", k= ey, + now); + + for_each_online_node ( i ) + { + paddr_t pa =3D pfn_to_paddr(node_start_pfn(i) + 1); + + printk("NODE%u start->%lu size->%lu free->%lu\n", + i, node_start_pfn(i), node_spanned_pages(i), + avail_node_heap_pages(i)); + /* Sanity check phys_to_nid() */ + if ( phys_to_nid(pa) !=3D i ) + printk("phys_to_nid(%"PRIpaddr") -> %d should be %u\n", + pa, phys_to_nid(pa), i); + } + + j =3D cpumask_first(&cpu_online_map); + n =3D 0; + for_each_online_cpu ( i ) + { + if ( i !=3D j + n || cpu_to_node[j] !=3D cpu_to_node[i] ) + { + if ( n > 1 ) + printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node= [j]); + else + printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]); + j =3D i; + n =3D 1; + } + else + ++n; + } + if ( n > 1 ) + printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]); + else + printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]); + + rcu_read_lock(&domlist_read_lock); + + printk("Memory location of each domain:\n"); + for_each_domain ( d ) + { + const struct page_info *page; + unsigned int page_num_node[MAX_NUMNODES]; + const struct vnuma_info *vnuma; + + process_pending_softirqs(); + + printk("%pd (total: %u):\n", d, domain_tot_pages(d)); + + memset(page_num_node, 0, sizeof(page_num_node)); + + spin_lock(&d->page_alloc_lock); + page_list_for_each ( page, &d->page_list ) + { + i =3D phys_to_nid(page_to_maddr(page)); + page_num_node[i]++; + } + spin_unlock(&d->page_alloc_lock); + + for_each_online_node ( i ) + printk(" Node %u: %u\n", i, page_num_node[i]); + + if ( !read_trylock(&d->vnuma_rwlock) ) + continue; + + if ( !d->vnuma ) + { + read_unlock(&d->vnuma_rwlock); + continue; + } + + vnuma =3D d->vnuma; + printk(" %u vnodes, %u vcpus, guest physical layout:\n", + vnuma->nr_vnodes, d->max_vcpus); + for ( i =3D 0; i < vnuma->nr_vnodes; i++ ) + { + unsigned int start_cpu =3D ~0U; + + if ( vnuma->vnode_to_pnode[i] =3D=3D NUMA_NO_NODE ) + printk(" %3u: pnode ???,", i); + else + printk(" %3u: pnode %3u,", i, vnuma->vnode_to_pnode[= i]); + + printk(" vcpus "); + + for ( j =3D 0; j < d->max_vcpus; j++ ) + { + if ( !(j & 0x3f) ) + process_pending_softirqs(); + + if ( vnuma->vcpu_to_vnode[j] =3D=3D i ) + { + if ( start_cpu =3D=3D ~0U ) + { + printk("%u", j); + start_cpu =3D j; + } + } + else if ( start_cpu !=3D ~0U ) + { + if ( j - 1 !=3D start_cpu ) + printk("-%u ", j - 1); + else + printk(" "); + start_cpu =3D ~0U; + } + } + + if ( start_cpu !=3D ~0U && start_cpu !=3D j - 1 ) + printk("-%u", j - 1); + + printk("\n"); + + for ( j =3D 0; j < vnuma->nr_vmemranges; j++ ) + { + if ( vnuma->vmemrange[j].nid =3D=3D i ) + printk(" %016"PRIx64" - %016"PRIx64"\n", + vnuma->vmemrange[j].start, + vnuma->vmemrange[j].end); + } + } + + read_unlock(&d->vnuma_rwlock); + } + + rcu_read_unlock(&domlist_read_lock); +} + +static int __init cf_check register_numa_trigger(void) +{ + register_keyhandler('u', dump_numa, "dump NUMA info", 1); + return 0; +} +__initcall(register_numa_trigger); diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 7aef1a88dc..5b3877344b 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -18,4 +18,70 @@ (((d)->vcpu !=3D NULL && (d)->vcpu[0] !=3D NULL) \ ? vcpu_to_node((d)->vcpu[0]) : NUMA_NO_NODE) =20 +/* The following content can be used when NUMA feature is enabled */ +#ifdef CONFIG_NUMA + +extern nodeid_t cpu_to_node[NR_CPUS]; +extern cpumask_t node_to_cpumask[]; + +#define cpu_to_node(cpu) cpu_to_node[cpu] +#define parent_node(node) (node) +#define node_to_cpumask(node) node_to_cpumask[node] + +struct node { + paddr_t start, end; +}; + +extern int compute_hash_shift(const struct node *nodes, + unsigned int numnodes, const nodeid_t *nodei= ds); + +#define VIRTUAL_BUG_ON(x) + +extern bool numa_off; + +extern void numa_add_cpu(unsigned int cpu); +extern void numa_init_array(void); +extern void numa_set_node(unsigned int cpu, nodeid_t node); +extern void numa_initmem_init(unsigned long start_pfn, unsigned long end_p= fn); +extern int numa_process_nodes(paddr_t start, paddr_t end); + +extern int arch_numa_setup(const char *opt); +extern bool arch_numa_disabled(void); +extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end= ); + +static inline void clear_node_cpumask(unsigned int cpu) +{ + cpumask_clear_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]); +} + +/* Simple perfect hash to map pdx to node numbers */ +extern unsigned int memnode_shift; +extern unsigned long memnodemapsize; +extern uint8_t *memnodemap; + +struct node_data { + unsigned long node_start_pfn; + unsigned long node_spanned_pages; +}; + +extern struct node_data node_data[]; + +static inline nodeid_t __attribute_pure__ phys_to_nid(paddr_t addr) +{ + nodeid_t nid; + VIRTUAL_BUG_ON((paddr_to_pdx(addr) >> memnode_shift) >=3D memnodemapsi= ze); + nid =3D memnodemap[paddr_to_pdx(addr) >> memnode_shift]; + VIRTUAL_BUG_ON(nid >=3D MAX_NUMNODES || !node_data[nid]); + return nid; +} + +#define NODE_DATA(nid) (&node_data[nid]) + +#define node_start_pfn(nid) (NODE_DATA(nid)->node_start_pfn) +#define node_spanned_pages(nid) (NODE_DATA(nid)->node_spanned_pages) +#define node_end_pfn(nid) (NODE_DATA(nid)->node_start_pfn + \ + NODE_DATA(nid)->node_spanned_pages) + +#endif + #endif /* _XEN_NUMA_H */ --=20 2.25.1 From nobody Fri Apr 19 19:11:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=2; a=rsa-sha256; t=1668768377; cv=pass; d=zohomail.com; s=zohoarc; b=hd2L/LaoSH4qwOJh6TD6M8/qAOtTI+sQRTmk0aorTXF+gtjGGJhoYwQbXlN023nCpkZrqFi2Xzb5X9FzmdyJCOFoVWBFA89bY1CIuJtW+RJW7KyuN+33v6CRfXEwGASpiDKrKl5WcegseNxz+qkSrp+taS++5qYnPt0FFAHunCI= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1668768377; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cZdzCTYPCRdEijtzlPbxujpvQi5i4DcA5OYYtuNHtTA=; b=hXxzLe6PqsCbnUx15FPrWeMYWM45N1gb5+uK01qQRKysYF0JAJPzQAL4fz1RYPfKgYf9u2fR92OVLLBWr617P04ZcZXu2u+8q2ucLMqfuzMS+3cAtCaGoGefEMwcjYNKXiwbhRCCyvvxh+32nLbvXgdSjPs+578DTiGae56k4fE= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1668768377630263.3125288243464; Fri, 18 Nov 2022 02:46:17 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.445564.700806 (Exim 4.92) (envelope-from ) id 1ovysX-00076T-Hz; Fri, 18 Nov 2022 10:45:49 +0000 Received: by outflank-mailman (output) from mailman id 445564.700806; Fri, 18 Nov 2022 10:45:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysX-00076M-Eu; Fri, 18 Nov 2022 10:45:49 +0000 Received: by outflank-mailman (input) for mailman id 445564; Fri, 18 Nov 2022 10:45:48 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysW-0006Il-1F for xen-devel@lists.xenproject.org; Fri, 18 Nov 2022 10:45:48 +0000 Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on20628.outbound.protection.outlook.com [2a01:111:f400:fe1a::628]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 28bb13a9-672e-11ed-91b6-6bf2151ebd3b; Fri, 18 Nov 2022 11:45:47 +0100 (CET) Received: from DB6PR0301CA0088.eurprd03.prod.outlook.com (2603:10a6:6:30::35) by AM7PR08MB5429.eurprd08.prod.outlook.com (2603:10a6:20b:107::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9; Fri, 18 Nov 2022 10:45:40 +0000 Received: from DBAEUR03FT040.eop-EUR03.prod.protection.outlook.com (2603:10a6:6:30:cafe::b) by DB6PR0301CA0088.outlook.office365.com (2603:10a6:6:30::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9 via Frontend Transport; Fri, 18 Nov 2022 10:45:40 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT040.mail.protection.outlook.com (100.127.142.157) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:40 +0000 Received: ("Tessian outbound aeae1c7b66fd:v130"); Fri, 18 Nov 2022 10:45:40 +0000 Received: from e3a20634adf0.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id DA6E12D7-70B0-4910-B3C3-49F863636645.1; Fri, 18 Nov 2022 10:45:33 +0000 Received: from EUR01-VE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e3a20634adf0.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 18 Nov 2022 10:45:33 +0000 Received: from FR3P281CA0145.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:95::8) by AM9PR08MB6258.eurprd08.prod.outlook.com (2603:10a6:20b:287::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9; Fri, 18 Nov 2022 10:45:31 +0000 Received: from VI1EUR03FT015.eop-EUR03.prod.protection.outlook.com (2603:10a6:d10:95:cafe::e2) by FR3P281CA0145.outlook.office365.com (2603:10a6:d10:95::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9 via Frontend Transport; Fri, 18 Nov 2022 10:45:31 +0000 Received: from nebula.arm.com (40.67.248.234) by VI1EUR03FT015.mail.protection.outlook.com (100.127.144.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:31 +0000 Received: from AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Fri, 18 Nov 2022 10:45:24 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Fri, 18 Nov 2022 10:45:24 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.16 via Frontend Transport; Fri, 18 Nov 2022 10:45:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 28bb13a9-672e-11ed-91b6-6bf2151ebd3b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cZdzCTYPCRdEijtzlPbxujpvQi5i4DcA5OYYtuNHtTA=; b=36KCODnxdQIUd6vS/M5g5EvmQUhnOkHBhWp1gz5em/Qt2fJZGdA4bIG44xf8Do7kQCYQdsLeuwXRdQ10JSqjliTFUaHQw2PNG7nr32imrIIsOf6JTeTE6FszvsxRqjKoRiCtp2BXYloEmHRt6jSagS24L7mpttz1df6+VcS/Bxw= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 68940c93ce4c54ac X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TOhPf3MiTjcAS4jEqV0YfupikNa2uTomaapC7QayCvCirsBR/VK7TqpLGweO26fAEOybIrMr1sQCGktkqgsQhTS7IDxILNU03K8HYLhusUIjZiLrKczznOfw9x7pATGZ/voP2OabsvtW1NR8xYoudvam2Exi34G3aYeunbnS4ZFwu+2BGw+CVHTc1plAakUqFZq0tnV6uTgVs40ESjQ34qCbh2jtwGQlObFTIkSV483oDxiZAd9xi1wqhYEDWll4ttuCWls2xMC7sSvMkez3Wo4efZB7o++nv68y0uk3zt4uWnD+vC+bjqrQ9lCr59InQIsk9p9i32wmKbR7WLI9oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cZdzCTYPCRdEijtzlPbxujpvQi5i4DcA5OYYtuNHtTA=; b=Xi5oZyIgVWjGLqPYYPUUsM9FCXf1qVZ8HUWVaXVv592NTCDpxgyGjtIqu2RF+aCKwqLQaZIQiEyePNo680JZv3eYueSA+6fSTB3DJl4mUsfczrAlUgzakjVbEyRtf/AsQ10WJQr6MQyMbB5wqW9omw+1gb99tFxhM4rk+0Liy5MLydwgmyB1FsKQ8OvRlXFQSW754uji4tOeRdqoWvR/hAxTt5Vhx6zu42uzYh5fDhqhj/c2ukGez63r5v+D7kmINXdZZ1zvjcy9j6vK7Y8Q8jem5juf6X9XmAUHvEYPfxVpDPOLn7DH7DvGmXKIXxH3l7yHRDLDO8Zit/YwrvpOhw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cZdzCTYPCRdEijtzlPbxujpvQi5i4DcA5OYYtuNHtTA=; b=36KCODnxdQIUd6vS/M5g5EvmQUhnOkHBhWp1gz5em/Qt2fJZGdA4bIG44xf8Do7kQCYQdsLeuwXRdQ10JSqjliTFUaHQw2PNG7nr32imrIIsOf6JTeTE6FszvsxRqjKoRiCtp2BXYloEmHRt6jSagS24L7mpttz1df6+VcS/Bxw= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Andrew Cooper , George Dunlap , "Jan Beulich" , Julien Grall , "Stefano Stabellini" , Wei Liu , Jiamei Xie Subject: [PATCH v9 3/6] xen/x86: Use ASSERT instead of VIRTUAL_BUG_ON for phys_to_nid Date: Fri, 18 Nov 2022 18:45:05 +0800 Message-ID: <20221118104508.768274-4-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221118104508.768274-1-wei.chen@arm.com> References: <20221118104508.768274-1-wei.chen@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: VI1EUR03FT015:EE_|AM9PR08MB6258:EE_|DBAEUR03FT040:EE_|AM7PR08MB5429:EE_ X-MS-Office365-Filtering-Correlation-Id: 33e28034-a03a-4168-02a9-08dac9520915 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: E5xqWbLvAnn3Vd2pP0lTxuXaIuhCakLKm1XGAHYGj8VFbaQkh/c01f9SxL9zqO8rCpRWQP1pChaLHXuVWOW7Duh/33iOlqHEn7QjO2Ph8e0WQtNfjmybYMLbzfJwF1YJ1kuu4bH4orCAPbPIoYoxkL2ts3euoIwUI198VI1/n13IJf3rphIDInq8cTdT4fh4AF/Xw2o8fX/MN1xxGEOVPGLwBa1JX6q1f9I1v4vYR2XF6gYM4zXYDFap65KuHM0/SvxxPhPNokOoCf8PDUM6sGi/uLjFsi8KfS5BLF1RPwwuHvJvhB9DWGJ4OnqZXaWiOMqosFZOGowFvoeGa3RSEbCNs6gLGsLLcR0JKlNCvuOnebTsQXYTlO6YedhPzXaYxV/9Z0Bw6c4vk0KKwu06p+h45NN0TDTX9Q6tr+BH7O8ABZmbzupKfug7osoA8FbAFDwucdiHBrcuOVPAKZmjPGqh4Ax1Jti0dpSnK5MDPpGeNDP0NKgb0J35xciQPH2AsTnTGhYKSVyMI1O+R1gnIPyVtu4rnE2Cldfuo75KyPCW1n3jGWO3fMsrq4c9Rog3g2vkNEOIKiBhVIAkk141rNIrCoyyU8k+eLTAIhi8l1+y8QQld8IAxg/zInfrD26vY3RYG+Ik+sCNj2RTkfUqwrRmcOYqedIpoZn4voPDxx+Fuyoo7YKn06L/ZISbBBYz2KBkqCREBS/CxngSyII84hFGFkAJbLuW5XMb67VFawg= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(136003)(346002)(376002)(396003)(39860400002)(451199015)(36840700001)(46966006)(40470700004)(6916009)(356005)(82310400005)(44832011)(316002)(82740400003)(47076005)(426003)(36756003)(40480700001)(54906003)(186003)(36860700001)(8936002)(2906002)(1076003)(8676002)(83380400001)(70206006)(5660300002)(41300700001)(70586007)(4326008)(336012)(86362001)(2616005)(40460700003)(6666004)(7696005)(26005)(81166007)(478600001)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6258 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT040.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 8721e258-1031-40d2-8ba0-08dac95203ec X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pAbFxshUfxsJ2ofJjy7KQhwxrBPChiInqhsL3B33EnhicnpHzcRgskMGRiLmKcagvhdHzliZf62z0FCBZTFDVxQ/5CZr9i/E+RSFX0wwQq1YQVVjN2B3Xc9kF0i4kNyP3barzj/2LaiT7dcXrHsZf3OgLFHvspIW+qu5FkyEyjea/07RxE9YJkRHbIQmUdTe0L5p9FceCu725GrOWa4CDrR/t3irqUS2dwUv1G2nPgxFp1l7xQBVybROWW4tpNJbK3OpHkwSvrDEevN+s9drBvBSqUllNanTWCzsAAuN8Di/d1u6ed5cEHd60rxofzAnou8oe5I9PSptiNBw/tA+RmYVsvYd3DNTWVftCqiVu59RrG9V5apUd0Z2w53WHRgadcVcN+cb2gr1vyxeqdwO+zCDmoU9i04v65h1G6QCupjC4medjhXTD/VIXtiJjdJZ7PE9GMSqcGDcsHW1shF+1K2kJq9wXky3WjUKTpy4zUxXglDCMIbhtzVij2sYPZH8cHy4ov4YBXoJFibUrxjDvnYdrDFG1eNWWLGAdkmhjv+7x6o3Muj3nh50Lpx2jEi+cR+i4qrzgdxhtxcz474TpIDxEcvWcmiFoow8U9uoiGjboygH+h3dg7u5BQqshr/vLzkF/EAFpOL6ABKF8YJ9x0vuE6vId650XRd1gqCLcA9DhazalJlXUA3Kh86Xc8LwKojVAA7WFFO/97ZkLFfO4A== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230022)(4636009)(396003)(136003)(376002)(346002)(39850400004)(451199015)(40470700004)(36840700001)(46966006)(81166007)(41300700001)(82310400005)(4326008)(40480700001)(8676002)(70586007)(44832011)(36860700001)(83380400001)(8936002)(316002)(7696005)(6666004)(6916009)(478600001)(86362001)(26005)(54906003)(47076005)(186003)(336012)(2906002)(2616005)(426003)(1076003)(5660300002)(36756003)(82740400003)(70206006)(40460700003);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 10:45:40.3219 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 33e28034-a03a-4168-02a9-08dac9520915 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT040.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5429 X-ZohoMail-DKIM: pass (identity @armh.onmicrosoft.com) X-ZM-MESSAGEID: 1668768379298100009 Content-Type: text/plain; charset="utf-8" VIRTUAL_BUG_ON is an empty macro used in phys_to_nid. This results in two lines of error-checking code in phys_to_nid that is not actually working and causing two compilation errors: 1. error: "MAX_NUMNODES" undeclared (first use in this function). This is because in the common header file, "MAX_NUMNODES" is defined after the common header file includes the ARCH header file, where phys_to_nid has attempted to use "MAX_NUMNODES". This error was resolved after we moved the phys_to_nid from x86 ARCH header file to common header file. 2. error: wrong type argument to unary exclamation mark. This is because, the error-checking code contains !node_data[nid]. But node_data is a data structure variable, it's not a pointer. So, in this patch, we use ASSERT instead of VIRTUAL_BUG_ON to enable the two lines of error-checking code. And fix the left compilation errors by replacing !node_data[nid] to !node_data[nid].node_spanned_pages. Although NUMA allows one node can only have CPUs but without any memory. And node with 0 bytes of memory might have an entry in memnodemap[] theoretically. But that doesn't mean phys_to_nid can find any valid address from a node with 0 bytes memory. Signed-off-by: Wei Chen Tested-by: Jiamei Xie Acked-by: Jan Beulich --- v8 -> v9: 1. No change. v7 -> v8: 1. No change. v6 -> v7: 1. No change. v5 -> v6: 1. No change. v4 -> v5: 1. No change. v3 -> v4: 1. No change. v2 -> v3: 1. Remove unnecessary change items in history. 2. Add Acked-by. v1 -> v2: 1. Use ASSERT to replace VIRTUAL_BUG_ON in phys_to_nid. 2. Adjust the conditional express for ASSERT. 3. Refine the justification of using !node_data[nid].node_spanned_pages. --- xen/include/xen/numa.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 5b3877344b..04556f3a6f 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -35,8 +35,6 @@ struct node { extern int compute_hash_shift(const struct node *nodes, unsigned int numnodes, const nodeid_t *nodei= ds); =20 -#define VIRTUAL_BUG_ON(x) - extern bool numa_off; =20 extern void numa_add_cpu(unsigned int cpu); @@ -69,9 +67,9 @@ extern struct node_data node_data[]; static inline nodeid_t __attribute_pure__ phys_to_nid(paddr_t addr) { nodeid_t nid; - VIRTUAL_BUG_ON((paddr_to_pdx(addr) >> memnode_shift) >=3D memnodemapsi= ze); + ASSERT((paddr_to_pdx(addr) >> memnode_shift) < memnodemapsize); nid =3D memnodemap[paddr_to_pdx(addr) >> memnode_shift]; - VIRTUAL_BUG_ON(nid >=3D MAX_NUMNODES || !node_data[nid]); + ASSERT(nid < MAX_NUMNODES && node_data[nid].node_spanned_pages); return nid; } =20 --=20 2.25.1 From nobody Fri Apr 19 19:11:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=2; a=rsa-sha256; t=1668768375; cv=pass; d=zohomail.com; s=zohoarc; b=hhtzZGdndeUWNyIk6uht2/MhVZvvFUyl2FKTQoVZhtJIxy2zL2OL+ZBIRffWCHVrQBKRCE+BzKeCZ2cF+aBlJcl4BMsJB2jkiLcm52iLL/r5AfR0ucH1ToWqfVbdEDSSqshL7WpEC+esHAqKv9atMaim+Ia8u2kIfRHnu4NXtVw= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1668768375; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2wQ8/FZBsYtn+E4uyHcWiQeWkJ4kxRPSI3oqXiHsNW8=; b=L8IzAQyYULWGwlpbiZlxSwGOHyjUj/X7SVWRY6GXQMDTN85qGK9NWvdBWKnHbHQhnU/uDq+RTvxSdW3RFVuCNGNhA1mpc1AGlHGJqnonPi3JISypUE7b42ESwML4khbk0/eIGuClO9u7iAzBbWcpYPh9M0yCfeIFn+N0VjwJGRE= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1668768375654657.4532941660813; Fri, 18 Nov 2022 02:46:15 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.445563.700796 (Exim 4.92) (envelope-from ) id 1ovysV-0006oc-5R; Fri, 18 Nov 2022 10:45:47 +0000 Received: by outflank-mailman (output) from mailman id 445563.700796; Fri, 18 Nov 2022 10:45:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysV-0006oV-1b; Fri, 18 Nov 2022 10:45:47 +0000 Received: by outflank-mailman (input) for mailman id 445563; Fri, 18 Nov 2022 10:45:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysT-0006R2-04 for xen-devel@lists.xenproject.org; Fri, 18 Nov 2022 10:45:45 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on20622.outbound.protection.outlook.com [2a01:111:f400:7e1b::622]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 21399c53-672e-11ed-8fd2-01056ac49cbb; Fri, 18 Nov 2022 11:45:34 +0100 (CET) Received: from AM6PR10CA0069.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:80::46) by VI1PR08MB9984.eurprd08.prod.outlook.com (2603:10a6:800:1c7::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.6; Fri, 18 Nov 2022 10:45:38 +0000 Received: from AM7EUR03FT024.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:80:cafe::5a) by AM6PR10CA0069.outlook.office365.com (2603:10a6:209:80::46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.20 via Frontend Transport; Fri, 18 Nov 2022 10:45:38 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT024.mail.protection.outlook.com (100.127.140.238) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:37 +0000 Received: ("Tessian outbound f394866f3f2b:v130"); Fri, 18 Nov 2022 10:45:37 +0000 Received: from 12f38b6ab8ce.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 142453EE-88AA-4242-A3D0-BE195FC8BD56.1; Fri, 18 Nov 2022 10:45:30 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 12f38b6ab8ce.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 18 Nov 2022 10:45:30 +0000 Received: from AM6P194CA0044.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:84::21) by AM9PR08MB6020.eurprd08.prod.outlook.com (2603:10a6:20b:2d6::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5857.8; Fri, 18 Nov 2022 10:45:28 +0000 Received: from AM7EUR03FT051.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:84:cafe::83) by AM6P194CA0044.outlook.office365.com (2603:10a6:209:84::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.20 via Frontend Transport; Fri, 18 Nov 2022 10:45:28 +0000 Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT051.mail.protection.outlook.com (100.127.140.64) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:28 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Fri, 18 Nov 2022 10:45:27 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.16 via Frontend Transport; Fri, 18 Nov 2022 10:45:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 21399c53-672e-11ed-8fd2-01056ac49cbb DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2wQ8/FZBsYtn+E4uyHcWiQeWkJ4kxRPSI3oqXiHsNW8=; b=h3GAD3I/ZzAFvDzcrb68L2+B6vKGtmEo9Y7ywnqmLHXIv42qhNKsJBPc3pWqNKNjn4bHJFJ44AHuTUQy+2Sxfb/sQq6LCEM7UjkAtYfdER0iAX4tNqlbWg1Ljuir2XY+1ygnGBscXPplAtvhDtsU/L5bKQ4zGfT6NWeCRLCgvC0= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: ce6e84f24473665c X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m7hJM/s2p8/2b4e6bGUl1QVvwH4A1qQac8ew2EOfQMrCaEtWXeMsmnIpoimPhPun5GvSeczT+m1NE9C2q/rJ/i+mobhm1Ye7YsHxkmqwkEJ3JJfvR5lHJfnqXQytYNlzzXiVSaepPCYwdUMfCCc4WD37V0qmrVfn3ope19LkYKzbtqOSpm3ownYDnHLb3jc3yeArCgSQr0xRZchiTzq0Bph54rmUAwqKxyB3plFyrwi61IrQXNiECNQCnXZ//qW69FyS2QG7SQpfLZ2E3HabAgNCQZuXMnI5pqtu06FcHG4GJpSRoAaVQGKJhuMVylALKz1ldVsd4LNbBQMi6+x4zQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2wQ8/FZBsYtn+E4uyHcWiQeWkJ4kxRPSI3oqXiHsNW8=; b=FbiXuRPBscn1NIaAeyT6z0J/8lCJ0W5UutTGTUln2tzOK7qcXJ1yP8RjG03bf8M4sc5q49/0cX+dTZgU3nJ50nt3LJTZpPHInVHPSYLTU/FlHaF/txVdUsVvy1xPzFPGKgTBCcNTA6sgv7Q7boCQf1S7tL0lAe7W0QpmyXuMdmLUEYaMftACkZ06QJLg97ph/QW1VlhhXr1xM7v47TASWtG6iZQ32RweL7WoQTdU4oeIyHjiEjtF92Psf6DB6D9ujlWf/FcUGZAgpbLBapKo+Ww/639a/ChT5iJSCFf7l+yuzNg15tTdrcoxxkfUtmtSTFXCWmcLr2RQXFI2bD//EA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2wQ8/FZBsYtn+E4uyHcWiQeWkJ4kxRPSI3oqXiHsNW8=; b=h3GAD3I/ZzAFvDzcrb68L2+B6vKGtmEo9Y7ywnqmLHXIv42qhNKsJBPc3pWqNKNjn4bHJFJ44AHuTUQy+2Sxfb/sQq6LCEM7UjkAtYfdER0iAX4tNqlbWg1Ljuir2XY+1ygnGBscXPplAtvhDtsU/L5bKQ4zGfT6NWeCRLCgvC0= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v9 4/6] xen/x86: use arch_get_ram_range to get information from E820 map Date: Fri, 18 Nov 2022 18:45:06 +0800 Message-ID: <20221118104508.768274-5-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221118104508.768274-1-wei.chen@arm.com> References: <20221118104508.768274-1-wei.chen@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT051:EE_|AM9PR08MB6020:EE_|AM7EUR03FT024:EE_|VI1PR08MB9984:EE_ X-MS-Office365-Filtering-Correlation-Id: 52de9e07-dcee-4dde-30dc-08dac95207b9 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: Q5feJxiPUYUxysp6wFC8qVKhHHK/s6qWlkQda+v7udtPMnaYnFOcqYCSWbKdY3ES8+UUMJd035MGv7GfqpmsFPtmMNbK3hYDNF6W5ACn/rWP+76miTwjyGcd4YXvBtYAqbXvev6GKMDiw4FYpv6+Gnl3IBzDLdNqIVGdOws7bxVXZTULMK05nDPx/CGVcRIFfaYOO1KlsFd+Lig3LYZCFQhgxUqfjuXhuZGP4u3VfTUMhjLqGKGElaijC3yUroaaB87y9UvsCTkyegEN4urBCm6BWKcNbiQDZVV4ElH8u6bHSnrtLH4FKRUyHKusvp1iAPybOxa0KJ9KeY6Du9BpJanZ1cSySmbM1H8gHwjw2wysb2DXCC/fsL9eYiDi7Ph4JLB23MAh//BoJW1Zz3iTioav+Vha3dm7/Q7QIwsM8qXkZQFeV8rdnEs2M/d62y5x9Mru88HYPYz0UZtfJp4bkU1Dt4sa6PnySgDArkrOp5IJyGjYFJVEshs+n4wwNXiYknT6SMscQfYZQZuU4+wEef5hCpJLLMjErSNi/jUTzjhIoQQk8UdmGGa2ZSYpu9ViJrVRCRIexOGYmM7/PuwYv255AVy8itgD1pqZNkUlUbp2jvSLm9LaYaTyWcWLld2C5vN/AJ/zpX/fyFVeXd5Vpj3LfrbEFnyUDSkOjZ5DSFpnj1cnXkdNYAlU+UJSQ6I/4ZIGCJU24Mpm0IAKQeYdNOeqWOLDNwBmLsNQTioiCuM= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(376002)(396003)(346002)(136003)(39860400002)(451199015)(46966006)(40470700004)(36840700001)(83380400001)(86362001)(40460700003)(7696005)(6666004)(26005)(478600001)(356005)(81166007)(36756003)(40480700001)(70586007)(82310400005)(36860700001)(336012)(186003)(2616005)(1076003)(426003)(47076005)(8936002)(70206006)(44832011)(41300700001)(82740400003)(8676002)(316002)(5660300002)(4326008)(54906003)(6916009)(2906002)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6020 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT024.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 4ce3f9f0-9fa5-4ed6-28f9-08dac952021e X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1yizkIegv7EIu1eebYVFE3nX8Fm/nJF8nWrEyBhdqUlKvjCYzGH3J23mhHiyU3LrSZFkR+r4wYqVC8xbkTWslrXEWQhc3UTexuSZsGaxtcdGd4UeTxj0kTH6NRkRWaIX7wuoaMy+96hjQNtKaeBXekELBhEZm8aosZN5STJ5ADEHYVSDlkux5Wso8r8lCqxRvyJnpID81Y7Eetdup0IVq1l8GuBxDvoVWQu6CLMS/GimszR9u/lL/zjlht42K8MU4cOHo1sDHB9/eOawdFEfHZuHpIGOqmJu6ZkhWIBA5mLWOUDCyrLnHfxDrmFl3RNz/z78Bel62stQTBNIYU+h4jd4m+lDEGAg0c39yzpYOF0OjBowp8a5dRbqgamUPdoSjnQhi8xHx5gauF1APIRKvx2pm13aSbbmx98SrX9yYQI/uEkHZc2rw4Csgw5MVp2ql4J5LhrBLaDFYDIagPvwu/PJtvbhA9Vatnpy4eVgoeGIT9thNs3saOk0Uh3DwEq6L7zx3QxtmQjZkf1X0s8ivLrxIDwBkPE5YydoUd+Cspgtxpn0gutID6AoucxHTeKeuVRWZvik5UTDdGIwxQT3lRyXwy+sbgUlOBvOtjrc6my7yNI2j4Ot94W7z4bxMIGMRiVLpY1HWDDiSKDEw88qua/p/PkcGp6ncDPhdgTLxLIxnl9tEyw3Hwv8+rFaIx2eor6HtA4l1phy8Yz9abYOUg== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230022)(4636009)(346002)(39850400004)(136003)(376002)(396003)(451199015)(40470700004)(46966006)(36840700001)(36756003)(8936002)(36860700001)(7696005)(40480700001)(86362001)(81166007)(82740400003)(40460700003)(8676002)(6666004)(47076005)(2906002)(426003)(5660300002)(41300700001)(83380400001)(2616005)(82310400005)(107886003)(44832011)(478600001)(1076003)(26005)(186003)(4326008)(336012)(70206006)(70586007)(54906003)(6916009)(316002);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 10:45:37.9799 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 52de9e07-dcee-4dde-30dc-08dac95207b9 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT024.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB9984 X-ZohoMail-DKIM: pass (identity @armh.onmicrosoft.com) X-ZM-MESSAGEID: 1668768377134100003 Content-Type: text/plain; charset="utf-8" The sanity check of nodes_cover_memory is also a requirement of other architectures that support NUMA. But now, the code of nodes_cover_memory is tied to the x86 E820. In this case, we introduce arch_get_ram_range to decouple architecture specific memory map from this function. This means, other architectures like Arm can also use it to check its node and memory coverage from bootmem info. Depends arch_get_ram_range, we make nodes_cover_memory become architecture independent. We also use neutral words to replace SRAT and E820 in the print message of this function. This will to make the massage seems more common. As arch_get_ram_range use unsigned int for index, we also adjust the index in nodes_cover_memory from int to unsigned int. Signed-off-by: Wei Chen Reviewed-by: Jan Beulich --- v8 -> v9: 1. No change. v7 -> v8: 1. No change. v6 -> v7: 1. No change. v5 -> v6: 1. No change. v4 -> v5: 1. Add Rb. 2. Adjust the code comments. v3 -> v4: 1. Move function comment to header file. 2. Use bool for found, and add a new "err" for the return value of arch_get_ram_range. 3. Use -ENODATA instead of -EINVAL for non-RAM type ranges. v2 -> v3: 1. Rename arch_get_memory_map to arch_get_ram_range. 2. Use -ENOENT instead of -ENODEV to indicate end of memory map. 3. Add description to code comment that arch_get_ram_range returns RAM range in [start, end) format. v1 -> v2: 1. Use arch_get_memory_map to replace arch_get_memory_bank_range and arch_get_memory_bank_number. 2. Remove the !start || !end check, because caller guarantee these two pointers will not be NULL. --- xen/arch/x86/numa.c | 15 +++++++++++++++ xen/arch/x86/srat.c | 30 ++++++++++++++++++------------ xen/include/xen/numa.h | 13 +++++++++++++ 3 files changed, 46 insertions(+), 12 deletions(-) diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c index 90b2a22591..fa8caaa084 100644 --- a/xen/arch/x86/numa.c +++ b/xen/arch/x86/numa.c @@ -9,6 +9,7 @@ #include #include #include +#include =20 #ifndef Dprintk #define Dprintk(x...) @@ -93,3 +94,17 @@ unsigned int __init arch_get_dma_bitsize(void) flsl(node_start_pfn(node) + node_spanned_pages(node) / 4 = - 1) + PAGE_SHIFT, 32); } + +int __init arch_get_ram_range(unsigned int idx, paddr_t *start, paddr_t *e= nd) +{ + if ( idx >=3D e820.nr_map ) + return -ENOENT; + + if ( e820.map[idx].type !=3D E820_RAM ) + return -ENODATA; + + *start =3D e820.map[idx].addr; + *end =3D *start + e820.map[idx].size; + + return 0; +} diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index ce507dac9e..1a108a34c6 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -452,37 +452,43 @@ acpi_numa_memory_affinity_init(const struct acpi_srat= _mem_affinity *ma) Make sure the PXMs cover all memory. */ static int __init nodes_cover_memory(void) { - int i; + unsigned int i; =20 - for (i =3D 0; i < e820.nr_map; i++) { - int j, found; + for (i =3D 0; ; i++) { + int err; + unsigned int j; + bool found; paddr_t start, end; =20 - if (e820.map[i].type !=3D E820_RAM) { - continue; - } + /* Try to loop memory map from index 0 to end to get RAM ranges. */ + err =3D arch_get_ram_range(i, &start, &end); =20 - start =3D e820.map[i].addr; - end =3D e820.map[i].addr + e820.map[i].size; + /* Reached the end of the memory map? */ + if (err =3D=3D -ENOENT) + break; + + /* Skip non-RAM entries. */ + if (err) + continue; =20 do { - found =3D 0; + found =3D false; for_each_node_mask(j, memory_nodes_parsed) if (start < nodes[j].end && end > nodes[j].start) { if (start >=3D nodes[j].start) { start =3D nodes[j].end; - found =3D 1; + found =3D true; } if (end <=3D nodes[j].end) { end =3D nodes[j].start; - found =3D 1; + found =3D true; } } } while (found && start < end); =20 if (start < end) { - printk(KERN_ERR "SRAT: No PXM for e820 range: " + printk(KERN_ERR "NUMA: No NODE for RAM range: " "[%"PRIpaddr", %"PRIpaddr"]\n", start, end - 1); return 0; } diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 04556f3a6f..9da0e7d555 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -80,6 +80,19 @@ static inline nodeid_t __attribute_pure__ phys_to_nid(pa= ddr_t addr) #define node_end_pfn(nid) (NODE_DATA(nid)->node_start_pfn + \ NODE_DATA(nid)->node_spanned_pages) =20 +/* + * This function provides the ability for caller to get one RAM entry + * from architectural memory map by index. + * + * This function will return zero if it can return a proper RAM entry. + * Otherwise it will return -ENOENT for out of scope index, or other + * error codes, e.g. return -ENODATA for non-RAM type memory entry. + * + * Note: the range is exclusive at the end, e.g. [*start, *end). + */ +extern int arch_get_ram_range(unsigned int idx, + paddr_t *start, paddr_t *end); + #endif =20 #endif /* _XEN_NUMA_H */ --=20 2.25.1 From nobody Fri Apr 19 19:11:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=2; a=rsa-sha256; t=1668768377; cv=pass; d=zohomail.com; s=zohoarc; b=SK6d5+uEQ+VdCkuBHxio4GHzJtI+xd1/ktaZL1i/lNJohq+Qrr3QaKOW7ltwi/2F41i2HOVqTWxtwDzpFEqt9oWNNMORrvLTtRAzY1GjHLE18QjsphFEacicOfiqLX5QcAnvAy1qK6UVo5+tWggEVwIEfFlFcl0l40HrXaxjWpw= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1668768377; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=6DCak0YJtenRHku7K8L/gHsRORPfDEtl/CMn+1huSvk=; b=OALQw8eMiOzOZntE0eunJq6il/9+gTXFCkbjA47FvTK4YNuY1/fEusY69NPi0hMIAJHvvv2Pbq2LrZaF7b7o1Qu51YqcK0O3zoQKYU2y8iYrEagBWPv9r2rSQVEB0ovJvxMs7hMxUV3yNHcEeRqesgh6udm4/GwiKFL8yJqvYWs= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1668768377531729.8009028076248; Fri, 18 Nov 2022 02:46:17 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.445566.700829 (Exim 4.92) (envelope-from ) id 1ovysc-0007il-5S; Fri, 18 Nov 2022 10:45:54 +0000 Received: by outflank-mailman (output) from mailman id 445566.700829; Fri, 18 Nov 2022 10:45:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysc-0007hf-0n; Fri, 18 Nov 2022 10:45:54 +0000 Received: by outflank-mailman (input) for mailman id 445566; Fri, 18 Nov 2022 10:45:52 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysa-0006Il-9P for xen-devel@lists.xenproject.org; Fri, 18 Nov 2022 10:45:52 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04on0617.outbound.protection.outlook.com [2a01:111:f400:fe0c::617]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 2a8cc51b-672e-11ed-91b6-6bf2151ebd3b; Fri, 18 Nov 2022 11:45:50 +0100 (CET) Received: from DU2PR04CA0216.eurprd04.prod.outlook.com (2603:10a6:10:2b1::11) by DB9PR08MB6505.eurprd08.prod.outlook.com (2603:10a6:10:23e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9; Fri, 18 Nov 2022 10:45:45 +0000 Received: from DBAEUR03FT023.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:2b1:cafe::2b) by DU2PR04CA0216.outlook.office365.com (2603:10a6:10:2b1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.13 via Frontend Transport; Fri, 18 Nov 2022 10:45:45 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT023.mail.protection.outlook.com (100.127.142.253) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:44 +0000 Received: ("Tessian outbound 2ff13c8f2c05:v130"); Fri, 18 Nov 2022 10:45:44 +0000 Received: from 165182cd3568.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 93DA0F33-148C-48EA-872E-E52E8153D64D.1; Fri, 18 Nov 2022 10:45:37 +0000 Received: from EUR02-AM0-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 165182cd3568.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 18 Nov 2022 10:45:37 +0000 Received: from AM6P194CA0036.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:90::49) by PAXPR08MB6557.eurprd08.prod.outlook.com (2603:10a6:102:de::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9; Fri, 18 Nov 2022 10:45:34 +0000 Received: from AM7EUR03FT005.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:90:cafe::c5) by AM6P194CA0036.outlook.office365.com (2603:10a6:209:90::49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.20 via Frontend Transport; Fri, 18 Nov 2022 10:45:34 +0000 Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT005.mail.protection.outlook.com (100.127.140.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:33 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Fri, 18 Nov 2022 10:45:31 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.16 via Frontend Transport; Fri, 18 Nov 2022 10:45:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2a8cc51b-672e-11ed-91b6-6bf2151ebd3b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6DCak0YJtenRHku7K8L/gHsRORPfDEtl/CMn+1huSvk=; b=bgTHYGxPBjA7swnttxbhY8ZXp+ah9uxm6N9OPetzcUld5IMfeUbY6svEzcWdhyXpoIe02Ndvx7R07sT9JOJTnRMnVGOLrozcWEwjDC46A9sNcEJBh1CPXAX0nNFdA4uJQ4gjfQn2lhQ+TB2wBFooikEHWy7Vevh2GuXvZeuvDCs= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 53401273d3e89ba8 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eJMtcncqWlnjHNUfw98IlHW8p/8VSDSJYEw5d4P/TOqTKCvxq2YLTDKPFhuOYsUTR2qEhfqG5erk5Ey+Wjtugen6Uv5MXtiEL5Dg/tnoU/s9lDMLc/nESngEYnxUY97Rk1nY/Cbi3Yq6aBL9fswx8KaluAeEidtVDM2ZjlTJtx3i9iPQSqO7Pr8cOWxHAwYdfVw15O2we5JfPBl9gemCD6uaJ3sQwFg8tzzQOptjiNtf6SM0B8x7B5nlozZgH9bxkKfTxo+fPsxXfa9FPYhn5qgOcOBtX69WZ2rSJurgPoqXoe4qleNhRwifIFz9NOG1KwbMrAWAMjUTjp3xknbZDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6DCak0YJtenRHku7K8L/gHsRORPfDEtl/CMn+1huSvk=; b=bGnS3q7cl7EHCcSUJXQRa8KLX82BChHPHMEMuFHcvMSuS/Lc5VuRujVfuNSH0Mc4FRAiZt8DbVB1t6MVvrWVDistZ6XhemYkdMxTvfnahEybt9IYtNKK58jmhNnTuYHy8SN9BNwfRsWOvUfdOUxJCTv1nYfYmgk8rDMrQZO7c2S0SROGmAEOQRTKxGW7nHII0h/R+M/swRxCWHC83aWmrbR3FhvK1BBODVa46BiU/zIRuWdyT5ZJ5644odaB8+V3HcQA3dot037wAnH5NboQjbWhiv4odE1iGOS9XsEcoCyU45M3lkBaqi10jsajVcBQ5amyXvNOYJpSo9dV2Cyy0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6DCak0YJtenRHku7K8L/gHsRORPfDEtl/CMn+1huSvk=; b=bgTHYGxPBjA7swnttxbhY8ZXp+ah9uxm6N9OPetzcUld5IMfeUbY6svEzcWdhyXpoIe02Ndvx7R07sT9JOJTnRMnVGOLrozcWEwjDC46A9sNcEJBh1CPXAX0nNFdA4uJQ4gjfQn2lhQ+TB2wBFooikEHWy7Vevh2GuXvZeuvDCs= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v9 5/6] xen/x86: move NUMA process nodes nodes code from x86 to common Date: Fri, 18 Nov 2022 18:45:07 +0800 Message-ID: <20221118104508.768274-6-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221118104508.768274-1-wei.chen@arm.com> References: <20221118104508.768274-1-wei.chen@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT005:EE_|PAXPR08MB6557:EE_|DBAEUR03FT023:EE_|DB9PR08MB6505:EE_ X-MS-Office365-Filtering-Correlation-Id: f9793fb4-c197-41ee-7f2a-08dac9520bc8 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: mr6SKWk0souSxL2jv8fg2E25Rjq81WqI3m7uOMaDVZ5yVrUfL2eNfUigA2ci+8kiKlht7sUee2VoQ/D6M6ajJAwVMI2itGeHYQoA8ASgpzg9MzbrM/p9NpLiFaOXvrArSi6cD6MMtMp0ZD7xYwzM2Njcg/Uk+2JeaJekG3Y+vUED1/N6e11wJ0fonAuWZ3q6yezqSaukFf+dODMkN0fB0C/1ZSkhFYAdVdLY42maIJOVKtPkvIOJa9gd5wUWYUTBQydtdf4WTRMgzTfR/yXCUu7/4RyfbMIeQpZ3mDKkFnQHefKoDYmkLdj7mfyLYH6W17KwHif9IgfDiEgD2tQVLMszFHVzc+jK8MzZ62/UNFjW73AHtyOZtDlWsnO4AdR/xPwEQzOe5tV+PgIurkQEFUjr+97tscARPY7pChHpZx1C3pBonHCf4KzKUmCusX/hzOsIBz9vLB5B88psy28kPm9WCiALmU7k2HZ//KkwbBKgpVntlGNEPGa96EKxyILhRBGafUN9cnLYRRHqdM/Vw0UU7J7mEQ66tUAkALgGpV/ahSgc9juDAOVv90QS9UYRsTxVRLIeVWbcvwqeZp0jxnxSGTPfXTXW3pyUgQzIG4ZMJgMIr1lVsQ8lr/29tcTU2rw4Ll8qGhojRACc7ldBsOwuPKYtuN62DUBbjGVaemTK3SVM0RI2nAhVt9TBpF3rZcnNU1pgwwM9vlvLAf6NRBrWHUCrld9PJUrbcXrFH4A= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(346002)(396003)(136003)(376002)(451199015)(46966006)(40470700004)(36840700001)(86362001)(83380400001)(36860700001)(70586007)(356005)(82740400003)(81166007)(8936002)(40480700001)(2906002)(4326008)(41300700001)(40460700003)(82310400005)(6916009)(44832011)(26005)(30864003)(7696005)(70206006)(6666004)(186003)(336012)(36756003)(2616005)(8676002)(1076003)(316002)(426003)(54906003)(47076005)(478600001)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6557 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT023.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 1861b5e6-a87a-4d06-0abe-08dac9520512 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5JbfMcTpdGw41gTZ4KPba+Dc9DsWzHBMqDv19jb8cJGR9/PfqtiPQvNnI5/J1U15GSkPMTdhWEGhelCYCNWlKfnvjdm/UxcBQpTciUy+3hLhny19xeHp1kEIgFFaGh8KywV+N+PL7h4JGNfP6PzvyMqljvUlMx2qAxpE5hn1uZvYtvIzbb9eFj70YyJoZGKYnKnLLXTVcn0rB8C2TamyfCq9KzdHWWgttBRZpukwpKTrE64drLnYWFTQ4L1l97fKlwf7PTB3mbDm2RaXDYJRlTs+93dGMI7r3LFe4rpjJbKtlk2Sy2cxR7lHV0lWgL2FPHFv/BYEETmICTFbYgRXB0X35yemaPx7cFfSud4e877zq4Jz7o1cRznnobRtwsMUTYfsPwTxcuse3Z3Bf+R8R1zmUTC3VGJdN18AjrN/bUB1WavWKpW72XbmG5nkOI1oK5JQJClkUcmcGc0rB7p0NvA5GdzWbpj4z8wGWNioczi0TDHXQWjB3CfCsEx+YrZ5b/o2IZUzx0oFn3ubLeXy6ywklj1PP3Ypu1x7XmjuoM/5Jny6qMfs3gavMJ5onif5fyzcNKe9uKl8W3uW7hjTF/XTMWbiiIuaffIkvNIHOyVLY34FxFmhuNgAfaY4fMjvgZIsssj2TsfQ/zvkx17SUuv1O7VyNBgp5zpHMeeXVMPEXLIlpd0bCMcv536MCFHAPkDbZwMGRNBs2RlnGSOSwA== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230022)(4636009)(346002)(136003)(376002)(39860400002)(396003)(451199015)(46966006)(36840700001)(40470700004)(40480700001)(82310400005)(36756003)(2616005)(186003)(336012)(1076003)(47076005)(54906003)(426003)(107886003)(83380400001)(26005)(7696005)(2906002)(86362001)(478600001)(6666004)(70206006)(6916009)(8676002)(5660300002)(70586007)(4326008)(36860700001)(30864003)(40460700003)(44832011)(82740400003)(316002)(8936002)(81166007)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 10:45:44.8380 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f9793fb4-c197-41ee-7f2a-08dac9520bc8 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT023.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6505 X-ZohoMail-DKIM: pass (identity @armh.onmicrosoft.com) X-ZM-MESSAGEID: 1668768379366100011 x86 has implemented a set of codes to process NUMA nodes. These codes will parse NUMA memory and processor information from ACPI SRAT table. But except some ACPI specific codes, most of the process code like memory blocks validation, node memory range updates and some sanity check can be reused by other NUMA implementation. So in this patch, we move some variables and related functions for NUMA memory and processor to common as library. At the same time, numa_set_processor_nodes_parsed has been introduced for ACPI specific code to update processor parsing results. With this helper, we can reuse most of NUMA memory affinity init code from ACPI. As bad_srat and node_to_pxm functions have been used in common code to do architectural fallback and node to architectural node info translation. But it doesn't make sense to reuse the functions names in common code, we have rename them to neutral names as well. PXM is an ACPI specific item, we can't use it in common code directly. So we introduced an numa_fw_nid_name for each NUMA implementation to set their specific firmware NUMA node name. In this case, we do not need to retain a lot of per-arch code but still can print architectural log messages for different NUMA implementations. A default value "???" will be set to indicate an unset numa_fw_nid_name. mem_hotplug is accessed by common code if memory hotplug is activated. Even if this is only supported by x86, export the variable so that other architectures could support it in the future. As asm/acpi.h has been removed from common/numa.c, we have to move NR_NODE_MEMBLKS from asm/acpi.h to xen/numa.h in this patch as well. Signed-off-by: Wei Chen Reviewed-by: Jan Beulich --- v8 -> v9: 1. Remove incrrect description from commit log. 2. Add code comments for the places to set numa_fw_nid_name. v7 -> v8: 1. Add numa_fw_nid_name setting in srat_parse_regions after acpi_table_parse succeeded. v6 -> v7: 1. Add __init for arch_numa_unavailable. 2. Use __ro_after_init for mem_hotplug. 3. Use "???" instead of "NONAME" for unset numa_fw_nid_name. 4. Fix code-style. v5 -> v6: 1. Fix code-style. 2. Use arch_numa_unavailable to replace arch_numa_disabled for acpi_numa <=3D 0. 3. Remove Kconfig for HAS_NUMA_NODE_FWID. 4. Use numa_fw_nid_name for NUMA implementation to set their fw NUMA node name for print messages. v4 -> v5: 1. Introduce arch_numa_disabled for acpi_numa <=3D 0 in this patch. 2. Remove the paramter init_as_disable of arch_numa_disabled. 3. Fix typo "expandsion". 4. Add const to proper varibales. 5. Fix Indentation for l1tf_safe_maddr. 6. Remove double blank lines. 7. Add a space between for_each_node_mask and '('. Add a space page_list_for_each and '('. 8. Use bool for nodes_cover_memory return value. 9. Use a plain "int ret" to record compute_hash_shift return value. 10. Add a blank line before the function's main "return". 11. Add new Kconfig option HAS_NUMA_NODE_FWID to common/Kconfig. v3 -> v4: 1. Use bool as return value for functions that only return 0/1 or 0/-EINVAL. 2. Move mem_hotplug to a proper place in mm.h 3. Remove useless "size" in numa_scan_nodes. 4. Use unsigned int or const for proper variables. 5. Fix code-style. 6. Add init_as_disable as arch_numa_disabled parameter. 7. Add CONFIG_HAS_NUMA_NODE_FWID to gate print the mapping between node id and architectural node id (fw node id). v2 -> v3: 1. Add __ro_after_init to proper variables. 2. Rename bad_srat to numa_fw_bad. 3. Rename node_to_pxm to numa_node_to_arch_nid. 4. Merge patch#7 and #8 into this patch. 5. Correct int to unsigned int in proper places. 6. Move NR_NODE_MEMBLKS from x86/acpi.h to common/numa.h 7. Drop helpers to access mem_hotplug, we export mem_hotplug from x86/mm.c to common/page_alloc.c v1 -> v2: 1. Add code comment for numa_update_node_memblks to explain: Assumes all memory regions belonging to a single node are in one chunk. Holes between them will be included in the node. 2. Merge this single patch instead of serval patches to move x86 SRAT code to common. 3. Export node_to_pxm to keep pxm information in NUMA scan nodes error messages. 4. Change the code style to target file's Xen code-style. 5. Adjust some __init and __initdata for some functions and variables. 6. Merge two patches into this patch: 1. replace CONFIG_ACPI_NUMA by CONFIG_NUMA. 2. replace "SRAT" texts. 7. Turn numa_scan_nodes to static. --- xen/arch/x86/include/asm/acpi.h | 1 - xen/arch/x86/include/asm/mm.h | 2 - xen/arch/x86/include/asm/numa.h | 3 +- xen/arch/x86/mm.c | 2 - xen/arch/x86/numa.c | 5 + xen/arch/x86/srat.c | 337 +++---------------------------- xen/common/numa.c | 341 +++++++++++++++++++++++++++++++- xen/common/page_alloc.c | 2 + xen/include/xen/mm.h | 2 + xen/include/xen/numa.h | 10 +- 10 files changed, 386 insertions(+), 319 deletions(-) diff --git a/xen/arch/x86/include/asm/acpi.h b/xen/arch/x86/include/asm/acp= i.h index 5c2dd5da2d..c453450a74 100644 --- a/xen/arch/x86/include/asm/acpi.h +++ b/xen/arch/x86/include/asm/acpi.h @@ -102,7 +102,6 @@ extern unsigned long acpi_wakeup_address; #define ARCH_HAS_POWER_INIT 1 =20 extern s8 acpi_numa; -#define NR_NODE_MEMBLKS (MAX_NUMNODES*2) =20 extern struct acpi_sleep_info acpi_sinfo; #define acpi_video_flags bootsym(video_flags) diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index d723c7c38f..2c5f856067 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -456,8 +456,6 @@ static inline int get_page_and_type(struct page_info *p= age, ASSERT(((_p)->count_info & PGC_count_mask) !=3D 0); \ ASSERT(page_get_owner(_p) =3D=3D (_d)) =20 -extern paddr_t mem_hotplug; - /*************************************************************************= ***** * With shadow pagetables, the different kinds of address start * to get get confusing. diff --git a/xen/arch/x86/include/asm/numa.h b/xen/arch/x86/include/asm/num= a.h index 6c87942d43..2ca3475271 100644 --- a/xen/arch/x86/include/asm/numa.h +++ b/xen/arch/x86/include/asm/numa.h @@ -10,6 +10,7 @@ typedef u8 nodeid_t; extern int srat_rev; =20 extern nodeid_t pxm_to_node(unsigned int pxm); +extern unsigned int numa_node_to_arch_nid(nodeid_t n); =20 #define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT)) =20 @@ -22,8 +23,6 @@ extern void init_cpu_to_node(void); =20 #define arch_want_default_dmazone() (num_online_nodes() > 1) =20 -extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node); - void srat_parse_regions(paddr_t addr); extern u8 __node_distance(nodeid_t a, nodeid_t b); unsigned int arch_get_dma_bitsize(void); diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index db1817b691..68f9989e1f 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -157,8 +157,6 @@ l1_pgentry_t __section(".bss.page_aligned") __aligned(P= AGE_SIZE) l1_pgentry_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) l1_fixmap_x[L1_PAGETABLE_ENTRIES]; =20 -paddr_t __read_mostly mem_hotplug; - /* Frame table size in pages. */ unsigned long max_page; unsigned long total_pages; diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c index fa8caaa084..4b0b297c7e 100644 --- a/xen/arch/x86/numa.c +++ b/xen/arch/x86/numa.c @@ -46,6 +46,11 @@ bool arch_numa_disabled(void) return acpi_numa < 0; } =20 +bool __init arch_numa_unavailable(void) +{ + return acpi_numa <=3D 0; +} + /* * Setup early cpu_to_node. * diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index 1a108a34c6..56749ddca5 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -24,10 +24,6 @@ =20 static struct acpi_table_slit *__read_mostly acpi_slit; =20 -static nodemask_t memory_nodes_parsed __initdata; -static nodemask_t processor_nodes_parsed __initdata; -static struct node nodes[MAX_NUMNODES] __initdata; - struct pxm2node { unsigned pxm; nodeid_t node; @@ -35,19 +31,6 @@ struct pxm2node { static struct pxm2node __read_mostly pxm2node[MAX_NUMNODES] =3D { [0 ... MAX_NUMNODES - 1] =3D {.node =3D NUMA_NO_NODE} }; =20 -static unsigned node_to_pxm(nodeid_t n); - -static int num_node_memblks; -static struct node node_memblk_range[NR_NODE_MEMBLKS]; -static nodeid_t memblk_nodeid[NR_NODE_MEMBLKS]; -static __initdata DECLARE_BITMAP(memblk_hotplug, NR_NODE_MEMBLKS); - -enum conflicts { - NO_CONFLICT, - OVERLAP, - INTERLEAVE, -}; - static inline bool node_found(unsigned idx, unsigned pxm) { return ((pxm2node[idx].pxm =3D=3D pxm) && @@ -110,78 +93,7 @@ nodeid_t setup_node(unsigned pxm) return node; } =20 -int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node) -{ - int i; - - for (i =3D 0; i < num_node_memblks; i++) { - struct node *nd =3D &node_memblk_range[i]; - - if (nd->start <=3D start && nd->end >=3D end && - memblk_nodeid[i] =3D=3D node) - return 1; - } - - return 0; -} - -static -enum conflicts __init conflicting_memblks(nodeid_t nid, paddr_t start, - paddr_t end, paddr_t nd_start, - paddr_t nd_end, unsigned int *mblkid) -{ - unsigned int i; - - /* - * Scan all recorded nodes' memory blocks to check conflicts: - * Overlap or interleave. - */ - for (i =3D 0; i < num_node_memblks; i++) { - struct node *nd =3D &node_memblk_range[i]; - - *mblkid =3D i; - - /* Skip 0 bytes node memory block. */ - if (nd->start =3D=3D nd->end) - continue; - /* - * Use memblk range to check memblk overlaps, include the - * self-overlap case. As nd's range is non-empty, the special - * case "nd->end =3D=3D end && nd->start =3D=3D start" also can be cover= ed. - */ - if (nd->end > start && nd->start < end) - return OVERLAP; - - /* - * Use node memory range to check whether new range contains - * memory from other nodes - interleave check. We just need - * to check full contains situation. Because overlaps have - * been checked above. - */ - if (nid !=3D memblk_nodeid[i] && - nd->start >=3D nd_start && nd->end <=3D nd_end) - return INTERLEAVE; - } - - return NO_CONFLICT; -} - -static __init void cutoff_node(int i, paddr_t start, paddr_t end) -{ - struct node *nd =3D &nodes[i]; - if (nd->start < start) { - nd->start =3D start; - if (nd->end < nd->start) - nd->start =3D nd->end; - } - if (nd->end > end) { - nd->end =3D end; - if (nd->start > nd->end) - nd->start =3D nd->end; - } -} - -static __init void bad_srat(void) +void __init numa_fw_bad(void) { int i; printk(KERN_ERR "SRAT: SRAT not used.\n"); @@ -241,7 +153,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x= 2apic_cpu_affinity *pa) if (numa_disabled()) return; if (pa->header.length < sizeof(struct acpi_srat_x2apic_cpu_affinity)) { - bad_srat(); + numa_fw_bad(); return; } if (!(pa->flags & ACPI_SRAT_CPU_ENABLED)) @@ -254,12 +166,12 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat= _x2apic_cpu_affinity *pa) pxm =3D pa->proximity_domain; node =3D setup_node(pxm); if (node =3D=3D NUMA_NO_NODE) { - bad_srat(); + numa_fw_bad(); return; } =20 apicid_to_node[pa->apic_id] =3D node; - node_set(node, processor_nodes_parsed); + numa_set_processor_nodes_parsed(node); acpi_numa =3D 1; =20 if (opt_acpi_verbose) @@ -277,7 +189,7 @@ acpi_numa_processor_affinity_init(const struct acpi_sra= t_cpu_affinity *pa) if (numa_disabled()) return; if (pa->header.length !=3D sizeof(struct acpi_srat_cpu_affinity)) { - bad_srat(); + numa_fw_bad(); return; } if (!(pa->flags & ACPI_SRAT_CPU_ENABLED)) @@ -290,11 +202,11 @@ acpi_numa_processor_affinity_init(const struct acpi_s= rat_cpu_affinity *pa) } node =3D setup_node(pxm); if (node =3D=3D NUMA_NO_NODE) { - bad_srat(); + numa_fw_bad(); return; } apicid_to_node[pa->apic_id] =3D node; - node_set(node, processor_nodes_parsed); + numa_set_processor_nodes_parsed(node); acpi_numa =3D 1; =20 if (opt_acpi_verbose) @@ -306,33 +218,27 @@ acpi_numa_processor_affinity_init(const struct acpi_s= rat_cpu_affinity *pa) void __init acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma) { - struct node *nd; - paddr_t nd_start, nd_end; - paddr_t start, end; unsigned pxm; nodeid_t node; - unsigned int i; - bool next =3D false; =20 if (numa_disabled()) return; if (ma->header.length !=3D sizeof(struct acpi_srat_mem_affinity)) { - bad_srat(); + numa_fw_bad(); return; } if (!(ma->flags & ACPI_SRAT_MEM_ENABLED)) return; =20 - start =3D ma->base_address; - end =3D start + ma->length; /* Supplement the heuristics in l1tf_calculations(). */ - l1tf_safe_maddr =3D max(l1tf_safe_maddr, ROUNDUP(end, PAGE_SIZE)); + l1tf_safe_maddr =3D max(l1tf_safe_maddr, + ROUNDUP(ma->base_address + ma->length, + PAGE_SIZE)); =20 - if (num_node_memblks >=3D NR_NODE_MEMBLKS) - { + if (!numa_memblks_available()) { dprintk(XENLOG_WARNING, - "Too many numa entry, try bigger NR_NODE_MEMBLKS \n"); - bad_srat(); + "Too many numa entries, try bigger NR_NODE_MEMBLKS!\n"); + numa_fw_bad(); return; } =20 @@ -341,159 +247,18 @@ acpi_numa_memory_affinity_init(const struct acpi_sra= t_mem_affinity *ma) pxm &=3D 0xff; node =3D setup_node(pxm); if (node =3D=3D NUMA_NO_NODE) { - bad_srat(); + numa_fw_bad(); return; } =20 /* - * For the node that already has some memory blocks, we will - * expand the node memory range temporarily to check memory - * interleaves with other nodes. We will not use this node - * temp memory range to check overlaps, because it will mask - * the overlaps in same node. - * - * Node with 0 bytes memory doesn't need this expandsion. + * In an extremely unlikely case, srat_parse_regions might not + * be called. So set the variable here just in case. */ - nd_start =3D start; - nd_end =3D end; - nd =3D &nodes[node]; - if (nd->start !=3D nd->end) { - if (nd_start > nd->start) - nd_start =3D nd->start; - - if (nd_end < nd->end) - nd_end =3D nd->end; - } - - /* It is fine to add this area to the nodes data it will be used later*/ - switch (conflicting_memblks(node, start, end, nd_start, nd_end, &i)) { - case OVERLAP: - if (memblk_nodeid[i] =3D=3D node) { - bool mismatch =3D !(ma->flags & - ACPI_SRAT_MEM_HOT_PLUGGABLE) !=3D - !test_bit(i, memblk_hotplug); - - printk("%sSRAT: PXM %u [%"PRIpaddr", %"PRIpaddr"] overlaps with itself = [%"PRIpaddr", %"PRIpaddr"]\n", - mismatch ? KERN_ERR : KERN_WARNING, pxm, start, - end - 1, node_memblk_range[i].start, - node_memblk_range[i].end - 1); - if (mismatch) { - bad_srat(); - return; - } - break; - } - - printk(KERN_ERR - "SRAT: PXM %u [%"PRIpaddr", %"PRIpaddr"] overlaps with PXM %u [%"= PRIpaddr", %"PRIpaddr"]\n", - pxm, start, end - 1, node_to_pxm(memblk_nodeid[i]), - node_memblk_range[i].start, - node_memblk_range[i].end - 1); - bad_srat(); - return; - - case INTERLEAVE: - printk(KERN_ERR - "SRAT=EF=BC=9A PXM %u: [%"PRIpaddr", %"PRIpaddr"] interleaves wit= h PXM %u memblk [%"PRIpaddr", %"PRIpaddr"]\n", - pxm, nd_start, nd_end - 1, node_to_pxm(memblk_nodeid[i]), - node_memblk_range[i].start, node_memblk_range[i].end - 1); - bad_srat(); - return; - - case NO_CONFLICT: - break; - } - - if (!(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE)) { - node_set(node, memory_nodes_parsed); - nd->start =3D nd_start; - nd->end =3D nd_end; - } - - printk(KERN_INFO "SRAT: Node %u PXM %u [%"PRIpaddr", %"PRIpaddr"]%s\n", - node, pxm, start, end - 1, - ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE ? " (hotplug)" : ""); - - /* Keep node_memblk_range[] sorted by address. */ - for (i =3D 0; i < num_node_memblks; ++i) - if (node_memblk_range[i].start > start || - (node_memblk_range[i].start =3D=3D start && - node_memblk_range[i].end > end)) - break; - - memmove(&node_memblk_range[i + 1], &node_memblk_range[i], - (num_node_memblks - i) * sizeof(*node_memblk_range)); - node_memblk_range[i].start =3D start; - node_memblk_range[i].end =3D end; - - memmove(&memblk_nodeid[i + 1], &memblk_nodeid[i], - (num_node_memblks - i) * sizeof(*memblk_nodeid)); - memblk_nodeid[i] =3D node; - - if (ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) { - next =3D true; - if (end > mem_hotplug) - mem_hotplug =3D end; - } - for (; i <=3D num_node_memblks; ++i) { - bool prev =3D next; - - next =3D test_bit(i, memblk_hotplug); - if (prev) - __set_bit(i, memblk_hotplug); - else - __clear_bit(i, memblk_hotplug); - } - - num_node_memblks++; -} - -/* Sanity check to catch more bad SRATs (they are amazingly common). - Make sure the PXMs cover all memory. */ -static int __init nodes_cover_memory(void) -{ - unsigned int i; - - for (i =3D 0; ; i++) { - int err; - unsigned int j; - bool found; - paddr_t start, end; - - /* Try to loop memory map from index 0 to end to get RAM ranges. */ - err =3D arch_get_ram_range(i, &start, &end); - - /* Reached the end of the memory map? */ - if (err =3D=3D -ENOENT) - break; - - /* Skip non-RAM entries. */ - if (err) - continue; - - do { - found =3D false; - for_each_node_mask(j, memory_nodes_parsed) - if (start < nodes[j].end - && end > nodes[j].start) { - if (start >=3D nodes[j].start) { - start =3D nodes[j].end; - found =3D true; - } - if (end <=3D nodes[j].end) { - end =3D nodes[j].start; - found =3D true; - } - } - } while (found && start < end); - - if (start < end) { - printk(KERN_ERR "NUMA: No NODE for RAM range: " - "[%"PRIpaddr", %"PRIpaddr"]\n", start, end - 1); - return 0; - } - } - return 1; + numa_fw_nid_name =3D "PXM"; + if (!numa_update_node_memblks(node, pxm, ma->base_address, ma->length, + ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE)) + numa_fw_bad(); } =20 void __init acpi_numa_arch_fixup(void) {} @@ -534,6 +299,8 @@ void __init srat_parse_regions(paddr_t addr) acpi_table_parse(ACPI_SIG_SRAT, acpi_parse_srat)) return; =20 + /* Set "PXM" as early as feasible. */ + numa_fw_nid_name =3D "PXM"; srat_region_mask =3D pdx_init_mask(addr); acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY, srat_parse_region, 0); @@ -549,59 +316,9 @@ void __init srat_parse_regions(paddr_t addr) pfn_pdx_hole_setup(mask >> PAGE_SHIFT); } =20 -/* Use discovered information to actually set up the nodes. */ -int __init numa_process_nodes(paddr_t start, paddr_t end) -{ - int i; - nodemask_t all_nodes_parsed; - - /* First clean up the node list */ - for (i =3D 0; i < MAX_NUMNODES; i++) - cutoff_node(i, start, end); - - if (acpi_numa <=3D 0) - return -1; - - if (!nodes_cover_memory()) { - bad_srat(); - return -1; - } - - memnode_shift =3D compute_hash_shift(node_memblk_range, num_node_memblks, - memblk_nodeid); - - if (memnode_shift < 0) { - printk(KERN_ERR - "SRAT: No NUMA node hash function found. Contact maintainer\n"); - bad_srat(); - return -1; - } - - nodes_or(all_nodes_parsed, memory_nodes_parsed, processor_nodes_parsed); - - /* Finally register nodes */ - for_each_node_mask(i, all_nodes_parsed) - { - uint64_t size =3D nodes[i].end - nodes[i].start; - - if ( size =3D=3D 0 ) - printk(KERN_INFO "SRAT: node %u has no memory\n", i); - - setup_node_bootmem(i, nodes[i].start, nodes[i].end); - } - for (i =3D 0; i < nr_cpu_ids; i++) { - if (cpu_to_node[i] =3D=3D NUMA_NO_NODE) - continue; - if (!nodemask_test(cpu_to_node[i], &processor_nodes_parsed)) - numa_set_node(i, NUMA_NO_NODE); - } - numa_init_array(); - return 0; -} - -static unsigned node_to_pxm(nodeid_t n) +unsigned int numa_node_to_arch_nid(nodeid_t n) { - unsigned i; + unsigned int i; =20 if ((n < ARRAY_SIZE(pxm2node)) && (pxm2node[n].node =3D=3D n)) return pxm2node[n].pxm; @@ -618,8 +335,8 @@ u8 __node_distance(nodeid_t a, nodeid_t b) =20 if (!acpi_slit) return a =3D=3D b ? 10 : 20; - index =3D acpi_slit->locality_count * node_to_pxm(a); - slit_val =3D acpi_slit->entry[index + node_to_pxm(b)]; + index =3D acpi_slit->locality_count * numa_node_to_arch_nid(a); + slit_val =3D acpi_slit->entry[index + numa_node_to_arch_nid(b)]; =20 /* ACPI defines 0xff as an unreachable node and 0-9 are undefined */ if ((slit_val =3D=3D 0xff) || (slit_val <=3D 9)) diff --git a/xen/common/numa.c b/xen/common/numa.c index a135154c3e..4948b21fbe 100644 --- a/xen/common/numa.c +++ b/xen/common/numa.c @@ -13,6 +13,21 @@ #include #include =20 +static nodemask_t __initdata processor_nodes_parsed; +static nodemask_t __initdata memory_nodes_parsed; +static struct node __initdata nodes[MAX_NUMNODES]; + +static unsigned int __ro_after_init num_node_memblks; +static struct node __ro_after_init node_memblk_range[NR_NODE_MEMBLKS]; +static nodeid_t __ro_after_init memblk_nodeid[NR_NODE_MEMBLKS]; +static __initdata DECLARE_BITMAP(memblk_hotplug, NR_NODE_MEMBLKS); + +enum conflicts { + NO_CONFLICT, + OVERLAP, + INTERLEAVE, +}; + struct node_data __ro_after_init node_data[MAX_NUMNODES]; =20 /* Mapping from pdx to node id */ @@ -31,11 +46,335 @@ nodemask_t __read_mostly node_online_map =3D { { [0] = =3D 1UL } }; =20 bool __ro_after_init numa_off; =20 +const char *__ro_after_init numa_fw_nid_name =3D "???"; + bool numa_disabled(void) { return numa_off || arch_numa_disabled(); } =20 +void __init numa_set_processor_nodes_parsed(nodeid_t node) +{ + node_set(node, processor_nodes_parsed); +} + +bool valid_numa_range(paddr_t start, paddr_t end, nodeid_t node) +{ + unsigned int i; + + for ( i =3D 0; i < num_node_memblks; i++ ) + { + const struct node *nd =3D &node_memblk_range[i]; + + if ( nd->start <=3D start && nd->end >=3D end && + memblk_nodeid[i] =3D=3D node ) + return true; + } + + return false; +} + +static enum conflicts __init conflicting_memblks( + nodeid_t nid, paddr_t start, paddr_t end, paddr_t nd_start, + paddr_t nd_end, unsigned int *mblkid) +{ + unsigned int i; + + /* + * Scan all recorded nodes' memory blocks to check conflicts: + * Overlap or interleave. + */ + for ( i =3D 0; i < num_node_memblks; i++ ) + { + const struct node *nd =3D &node_memblk_range[i]; + + *mblkid =3D i; + + /* Skip 0 bytes node memory block. */ + if ( nd->start =3D=3D nd->end ) + continue; + /* + * Use memblk range to check memblk overlaps, include the + * self-overlap case. As nd's range is non-empty, the special + * case "nd->end =3D=3D end && nd->start =3D=3D start" also can be= covered. + */ + if ( nd->end > start && nd->start < end ) + return OVERLAP; + + /* + * Use node memory range to check whether new range contains + * memory from other nodes - interleave check. We just need + * to check full contains situation. Because overlaps have + * been checked above. + */ + if ( nid !=3D memblk_nodeid[i] && + nd->start >=3D nd_start && nd->end <=3D nd_end ) + return INTERLEAVE; + } + + return NO_CONFLICT; +} + +static void __init cutoff_node(nodeid_t i, paddr_t start, paddr_t end) +{ + struct node *nd =3D &nodes[i]; + + if ( nd->start < start ) + { + nd->start =3D start; + if ( nd->end < nd->start ) + nd->start =3D nd->end; + } + + if ( nd->end > end ) + { + nd->end =3D end; + if ( nd->start > nd->end ) + nd->start =3D nd->end; + } +} + +bool __init numa_memblks_available(void) +{ + return num_node_memblks < NR_NODE_MEMBLKS; +} + +/* + * This function will be called by NUMA memory affinity initialization to + * update NUMA node's memory range. In this function, we assume all memory + * regions belonging to a single node are in one chunk. Holes (or MMIO + * ranges) between them will be included in the node. + * + * So in numa_update_node_memblks, if there are multiple banks for each + * node, start and end are stretched to cover the holes between them, and + * it works as long as memory banks of different NUMA nodes don't interlea= ve. + */ +bool __init numa_update_node_memblks(nodeid_t node, unsigned int arch_nid, + paddr_t start, paddr_t size, bool hot= plug) +{ + unsigned int i; + bool next =3D false; + paddr_t end =3D start + size; + paddr_t nd_start =3D start; + paddr_t nd_end =3D end; + struct node *nd =3D &nodes[node]; + + /* + * For the node that already has some memory blocks, we will + * expand the node memory range temporarily to check memory + * interleaves with other nodes. We will not use this node + * temp memory range to check overlaps, because it will mask + * the overlaps in same node. + * + * Node with 0 bytes memory doesn't need this expansion. + */ + if ( nd->start !=3D nd->end ) + { + if ( nd_start > nd->start ) + nd_start =3D nd->start; + + if ( nd_end < nd->end ) + nd_end =3D nd->end; + } + + /* It is fine to add this area to the nodes data it will be used later= */ + switch ( conflicting_memblks(node, start, end, nd_start, nd_end, &i) ) + { + case OVERLAP: + if ( memblk_nodeid[i] =3D=3D node ) + { + bool mismatch =3D !hotplug !=3D !test_bit(i, memblk_hotplug); + + printk("%sNUMA: %s %u [%"PRIpaddr", %"PRIpaddr"] overlaps with= itself [%"PRIpaddr", %"PRIpaddr"]\n", + mismatch ? KERN_ERR : KERN_WARNING, numa_fw_nid_name, + arch_nid, start, end - 1, + node_memblk_range[i].start, node_memblk_range[i].end - = 1); + if ( mismatch ) + return false; + break; + } + + printk(KERN_ERR + "NUMA: %s %u [%"PRIpaddr", %"PRIpaddr"] overlaps with %s %u= [%"PRIpaddr", %"PRIpaddr"]\n", + numa_fw_nid_name, arch_nid, start, end - 1, numa_fw_nid_nam= e, + numa_node_to_arch_nid(memblk_nodeid[i]), + node_memblk_range[i].start, node_memblk_range[i].end - 1); + return false; + + case INTERLEAVE: + printk(KERN_ERR + "NUMA=EF=BC=9A %s %u: [%"PRIpaddr", %"PRIpaddr"] interleave= s with %s %u memblk [%"PRIpaddr", %"PRIpaddr"]\n", + numa_fw_nid_name, arch_nid, nd_start, nd_end - 1, + numa_fw_nid_name, numa_node_to_arch_nid(memblk_nodeid[i]), + node_memblk_range[i].start, node_memblk_range[i].end - 1); + return false; + + case NO_CONFLICT: + break; + } + + if ( !hotplug ) + { + node_set(node, memory_nodes_parsed); + nd->start =3D nd_start; + nd->end =3D nd_end; + } + + printk(KERN_INFO "NUMA: Node %u %s %u [%"PRIpaddr", %"PRIpaddr"]%s\n", + node, numa_fw_nid_name, arch_nid, start, end - 1, + hotplug ? " (hotplug)" : ""); + + /* Keep node_memblk_range[] sorted by address. */ + for ( i =3D 0; i < num_node_memblks; ++i ) + if ( node_memblk_range[i].start > start || + (node_memblk_range[i].start =3D=3D start && + node_memblk_range[i].end > end) ) + break; + + memmove(&node_memblk_range[i + 1], &node_memblk_range[i], + (num_node_memblks - i) * sizeof(*node_memblk_range)); + node_memblk_range[i].start =3D start; + node_memblk_range[i].end =3D end; + + memmove(&memblk_nodeid[i + 1], &memblk_nodeid[i], + (num_node_memblks - i) * sizeof(*memblk_nodeid)); + memblk_nodeid[i] =3D node; + + if ( hotplug ) + { + next =3D true; + if ( end > mem_hotplug ) + mem_hotplug =3D end; + } + + for ( ; i <=3D num_node_memblks; ++i ) + { + bool prev =3D next; + + next =3D test_bit(i, memblk_hotplug); + if ( prev ) + __set_bit(i, memblk_hotplug); + else + __clear_bit(i, memblk_hotplug); + } + + num_node_memblks++; + + return true; +} + +/* + * Sanity check to catch more bad SRATs (they are amazingly common). + * Make sure the PXMs cover all memory. + */ +static bool __init nodes_cover_memory(void) +{ + unsigned int i; + + for ( i =3D 0; ; i++ ) + { + int err; + unsigned int j; + bool found; + paddr_t start, end; + + /* Try to loop memory map from index 0 to end to get RAM ranges. */ + err =3D arch_get_ram_range(i, &start, &end); + + /* Reached the end of the memory map? */ + if ( err =3D=3D -ENOENT ) + break; + + /* Skip non-RAM entries. */ + if ( err ) + continue; + + do { + found =3D false; + for_each_node_mask ( j, memory_nodes_parsed ) + if ( start < nodes[j].end && end > nodes[j].start ) + { + if ( start >=3D nodes[j].start ) + { + start =3D nodes[j].end; + found =3D true; + } + + if ( end <=3D nodes[j].end ) + { + end =3D nodes[j].start; + found =3D true; + } + } + } while ( found && start < end ); + + if ( start < end ) + { + printk(KERN_ERR "NUMA: No node for RAM range: " + "[%"PRIpaddr", %"PRIpaddr"]\n", start, end - 1); + return false; + } + } + + return true; +} + +/* Use discovered information to actually set up the nodes. */ +static bool __init numa_process_nodes(paddr_t start, paddr_t end) +{ + int ret; + unsigned int i; + nodemask_t all_nodes_parsed; + + /* First clean up the node list */ + for ( i =3D 0; i < MAX_NUMNODES; i++ ) + cutoff_node(i, start, end); + + /* When numa is on and has data, we can start to process numa nodes. */ + if ( arch_numa_unavailable() ) + return false; + + if ( !nodes_cover_memory() ) + { + numa_fw_bad(); + return false; + } + + ret =3D compute_hash_shift(node_memblk_range, num_node_memblks, + memblk_nodeid); + if ( ret < 0 ) + { + printk(KERN_ERR + "NUMA: No NUMA node hash function found. Contact maintainer= \n"); + numa_fw_bad(); + return false; + } + memnode_shift =3D ret; + + nodes_or(all_nodes_parsed, memory_nodes_parsed, processor_nodes_parsed= ); + + /* Finally register nodes */ + for_each_node_mask ( i, all_nodes_parsed ) + { + if ( nodes[i].end =3D=3D nodes[i].start ) + printk(KERN_INFO "NUMA: node %u has no memory\n", i); + + setup_node_bootmem(i, nodes[i].start, nodes[i].end); + } + + for ( i =3D 0; i < nr_cpu_ids; i++ ) + { + if ( cpu_to_node[i] =3D=3D NUMA_NO_NODE ) + continue; + if ( !nodemask_test(cpu_to_node[i], &processor_nodes_parsed) ) + numa_set_node(i, NUMA_NO_NODE); + } + + numa_init_array(); + + return true; +} + /* * Given a shift value, try to populate memnodemap[] * Returns : @@ -261,7 +600,7 @@ void __init numa_initmem_init(unsigned long start_pfn, = unsigned long end_pfn) #endif =20 #ifdef CONFIG_NUMA - if ( !numa_off && !numa_process_nodes(start, end) ) + if ( !numa_off && numa_process_nodes(start, end) ) return; #endif =20 diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 62afb07bc6..07df36d475 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -159,6 +159,8 @@ #define PGT_TYPE_INFO_INITIALIZER 0 #endif =20 +paddr_t __ro_after_init mem_hotplug; + /* * Comma-separated list of hexadecimal page numbers containing bad bytes. * e.g. 'badpage=3D0x3f45,0x8a321'. diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index a925028ab3..9d14aed74b 100644 --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -147,6 +147,8 @@ int assign_page( /* Dump info to serial console */ void arch_dump_shared_mem_info(void); =20 +extern paddr_t mem_hotplug; + /* * Extra fault info types which are used to further describe * the source of an access violation. diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 9da0e7d555..04ecaf7769 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -11,6 +11,7 @@ #define NUMA_NO_DISTANCE 0xFF =20 #define MAX_NUMNODES (1 << NODES_SHIFT) +#define NR_NODE_MEMBLKS (MAX_NUMNODES * 2) =20 #define vcpu_to_node(v) (cpu_to_node((v)->processor)) =20 @@ -36,14 +37,16 @@ extern int compute_hash_shift(const struct node *nodes, unsigned int numnodes, const nodeid_t *nodei= ds); =20 extern bool numa_off; +extern const char *numa_fw_nid_name; =20 extern void numa_add_cpu(unsigned int cpu); extern void numa_init_array(void); extern void numa_set_node(unsigned int cpu, nodeid_t node); extern void numa_initmem_init(unsigned long start_pfn, unsigned long end_p= fn); -extern int numa_process_nodes(paddr_t start, paddr_t end); +extern void numa_fw_bad(void); =20 extern int arch_numa_setup(const char *opt); +extern bool arch_numa_unavailable(void); extern bool arch_numa_disabled(void); extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end= ); =20 @@ -92,6 +95,11 @@ static inline nodeid_t __attribute_pure__ phys_to_nid(pa= ddr_t addr) */ extern int arch_get_ram_range(unsigned int idx, paddr_t *start, paddr_t *end); +extern bool valid_numa_range(paddr_t start, paddr_t end, nodeid_t node); +extern bool numa_memblks_available(void); +extern bool numa_update_node_memblks(nodeid_t node, unsigned int arch_nid, + paddr_t start, paddr_t size, bool hot= plug); +extern void numa_set_processor_nodes_parsed(nodeid_t node); =20 #endif =20 --=20 2.25.1 From nobody Fri Apr 19 19:11:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=2; a=rsa-sha256; t=1668768374; cv=pass; d=zohomail.com; s=zohoarc; b=K77zeUHyXXVrtQnI+fMPZqTOGL4bD09lCo40R4StD4Rk8jxGpBxjVmPQMd1GLDVNa/aS1p8ROXH3ZHzS3ZxK7x163DzxeDgEcis7WAEnhx7cKtl62j3U0DP03eUZQ2XRG1qFbHlTlJR9iTzLPOz78h7pDV1qB/QQUi+MgWXASj4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1668768374; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=a0uINGRHJ9MGqTaLt9vxK7vqJLpO9hspJ13faxPvkX8=; b=Fst75QCxZiBw0B/7xX42Z3O5AJiSPibZVa7exnd2it9yqBb97Zcpe9S2gnyTv95l+2rFtViwHHkv3JnpL7sBC46LILZOxjugaU3WKrAlItaLql2F7tOeLO7f5RV9AZ8A1Tsv3CRekyVcoKWqeHekgY7lodHPeqltQe1iPo3ZGgQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=arm.com); dmarc=fail header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 166876837435231.292581266685943; Fri, 18 Nov 2022 02:46:14 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.445565.700818 (Exim 4.92) (envelope-from ) id 1ovysa-0007Qm-Rb; Fri, 18 Nov 2022 10:45:52 +0000 Received: by outflank-mailman (output) from mailman id 445565.700818; Fri, 18 Nov 2022 10:45:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysa-0007Qd-O9; Fri, 18 Nov 2022 10:45:52 +0000 Received: by outflank-mailman (input) for mailman id 445565; Fri, 18 Nov 2022 10:45:51 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovysZ-0006R2-6s for xen-devel@lists.xenproject.org; Fri, 18 Nov 2022 10:45:51 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2060c.outbound.protection.outlook.com [2a01:111:f400:7d00::60c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 25bd905d-672e-11ed-8fd2-01056ac49cbb; Fri, 18 Nov 2022 11:45:42 +0100 (CET) Received: from AS8PR04CA0183.eurprd04.prod.outlook.com (2603:10a6:20b:2f3::8) by AS2PR08MB9545.eurprd08.prod.outlook.com (2603:10a6:20b:60e::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.7; Fri, 18 Nov 2022 10:45:45 +0000 Received: from AM7EUR03FT004.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:2f3:cafe::88) by AS8PR04CA0183.outlook.office365.com (2603:10a6:20b:2f3::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9 via Frontend Transport; Fri, 18 Nov 2022 10:45:45 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT004.mail.protection.outlook.com (100.127.140.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:45 +0000 Received: ("Tessian outbound 58faf9791229:v130"); Fri, 18 Nov 2022 10:45:45 +0000 Received: from 1aa20f94f073.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 5FE67FEF-AEF8-4105-8EA3-FB23F0FCEF6D.1; Fri, 18 Nov 2022 10:45:38 +0000 Received: from EUR03-DBA-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 1aa20f94f073.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 18 Nov 2022 10:45:38 +0000 Received: from AS9PR01CA0029.eurprd01.prod.exchangelabs.com (2603:10a6:20b:542::10) by AS8PR08MB8086.eurprd08.prod.outlook.com (2603:10a6:20b:54b::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.5; Fri, 18 Nov 2022 10:45:37 +0000 Received: from AM7EUR03FT027.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:542:cafe::5f) by AS9PR01CA0029.outlook.office365.com (2603:10a6:20b:542::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.20 via Frontend Transport; Fri, 18 Nov 2022 10:45:37 +0000 Received: from nebula.arm.com (40.67.248.234) by AM7EUR03FT027.mail.protection.outlook.com (100.127.140.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 10:45:35 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX03.Arm.com (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Fri, 18 Nov 2022 10:45:34 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.16 via Frontend Transport; Fri, 18 Nov 2022 10:45:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 25bd905d-672e-11ed-8fd2-01056ac49cbb DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a0uINGRHJ9MGqTaLt9vxK7vqJLpO9hspJ13faxPvkX8=; b=KZB+rO6qL2vIA1yK4i9ScdXQT4P62w8QE94PwLgPzJswBXwHpRGz5PXDdlZvRyo6RjvPp/rF31uRTl8XR7jzKHZG78FberPVJiSmVMd75U4jp9OQb/lQE5yDBFSZ948WSBIy9sM+7Z+KQrn7xvjuhfL5hj5GsWVrw8k2zT7GJJc= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 538d1a37e0c8e672 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dNgfrgVlccsY/UpgCDbPW3Y24JX6qTRII3hJ/e+7uj1EPMgbSaVVxyQlhQ3fOjL/BeEJT62sciQjhG27AgI3OVHbd1jBUNHv/nbWly+wn9TxXxjwmaXP1xipZcYMPTF55xz3bwYeoh+y3z7hAwGaQsVhdubL4fEk8GlXbr+oojaMvpH4drS4u9zShnxVSAeD2TCvVmFRgbFdnthmGBQ2j7B9iPSPekDvBbGFmIN2Oe1jr1zwBjCv7s80IFcPtSUKU7ck3+fT47RQnl2USi1Ye+hYHaawo4vbcGjepaG/Ydu3L/RHn552AXH5mmp9UKnddnfbIfIljCpaWO5H2I7qaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a0uINGRHJ9MGqTaLt9vxK7vqJLpO9hspJ13faxPvkX8=; b=jVJEkBztxYU9iJEdvoeHKWzbQy75U8YOTAXkH2vAVq9ah1210U1vi/hXd76/7CgiNPWAJOJ6Zs3S64o6B3vu0NbMxdcLfLrK5yWDSsEz1VhgoVc99MwmL3TNicDcMsQK8CD7c17FhIjy4qdFQViI4NvJC/jHX9LJQyA2N8CYG73hLoQkVghEPjW+/eoFLbgb7AdP4t/gqqsu+nPllqqEsxl5dXdgCSwxM8EDvdyQvAEp3d1Rmr1Y4SoqCDmH+CKLXMOz9uQNJbEnPFTCQVZBZmD8TC0Ov15Pbbhvnfk5agfs7xhmmE0rlwKVQzVPuKIKtQ4GB1XwAE8Ci5zrE2fsZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a0uINGRHJ9MGqTaLt9vxK7vqJLpO9hspJ13faxPvkX8=; b=KZB+rO6qL2vIA1yK4i9ScdXQT4P62w8QE94PwLgPzJswBXwHpRGz5PXDdlZvRyo6RjvPp/rF31uRTl8XR7jzKHZG78FberPVJiSmVMd75U4jp9OQb/lQE5yDBFSZ948WSBIy9sM+7Z+KQrn7xvjuhfL5hj5GsWVrw8k2zT7GJJc= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Andrew Cooper , George Dunlap , "Jan Beulich" , Julien Grall , "Stefano Stabellini" , Wei Liu , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [PATCH v9 6/6] xen: introduce a Kconfig option to configure NUMA nodes number Date: Fri, 18 Nov 2022 18:45:08 +0800 Message-ID: <20221118104508.768274-7-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221118104508.768274-1-wei.chen@arm.com> References: <20221118104508.768274-1-wei.chen@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: AM7EUR03FT027:EE_|AS8PR08MB8086:EE_|AM7EUR03FT004:EE_|AS2PR08MB9545:EE_ X-MS-Office365-Filtering-Correlation-Id: db18db0f-67eb-4c59-61a5-08dac9520c3d x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: eoNakGpJXW9D4YYAXEMwJW4+Afx624h/WS6ylokZrAnx0wQZea1PUiQkNJC5VA+bnnQbJywLxV59DQO74TaC3PWyACrmYxBF3164EFpuUomaRkQ5HEIX97zm71j0Og46U8u9ei9d3c68lnKgN3G4LDn9zBcsqAEsbUg1k6aNzk8vG3xs3REjKtOu3Zwa6iXCYENVjElHc6y5uangSyNk/uiqLVaZ1+RRhBpnqpk4DkmQSSSwLNy66oMmNhFlphIV0pVacuogUS1AVc0j3GP5pesngPUnJ7TFnu5ZUet/bQxeifOduiz/61wpLyUB7Sfw0FqNLBTFUfVbPsgkDNtWmayJ6f5q6qaG9v1vEowNdHngfiuKtYieGXFM7tRIpzAbRFCu4cWN5FHiGZsa8u+OMJMjZcbdmU7bYONK64FmbGKST6k+Zod8TkEGcBd9uPA9Qg1t5Jxw42L1F82n28Vof7twl3FxiwAxmJZr7DY5xjuuUw70kXYRpXeyzWGT0vliMXSwjiCVPU0oOsz+Iz4sCBKJ1LkUVXNo17R/pc05zcxSbKypKe/wTqCxdR4GudgjCYP7RBdEEkDe6MOVmnelzcxmd251GKHMciPfCdR74eOf9RDB8bwED3wwaxPgPZOxUCtlnX4weFLXqaKq+igNHcdUdJVZ/0OTmYE84zL5Lxfx5WuxmyIff/grDHBGbgHSEPsWyAS2+YWBcCfznxcpNF5HpVgJBfw1hHKVK2VCLtw= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(396003)(136003)(376002)(346002)(451199015)(46966006)(40470700004)(36840700001)(36860700001)(54906003)(8936002)(5660300002)(6916009)(86362001)(316002)(2906002)(81166007)(82740400003)(83380400001)(41300700001)(426003)(82310400005)(6666004)(7696005)(26005)(356005)(44832011)(70586007)(47076005)(4326008)(1076003)(478600001)(186003)(40480700001)(336012)(2616005)(70206006)(36756003)(40460700003)(8676002)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8086 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT004.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: f9fa8ad5-0fc8-48b8-e817-08dac9520672 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0w5jLCUy/XwmD2ILznR4gjFvUh9w6RdS7cHwpg4vq7IA1m1IqBaZ4QMECePn2pdj1OhJtGunJNgpGac+z6w9e7iEOnbh03nhe6fh+DMy952gQSeyV8Uzp20bXSAICcu5UZFD/ISKmAq90opNjY3haGSlYR8LDktabLytGdTgHaBKMPBz1RYg35SAhacu+MkkJcUYVYG6QHR96lXyyyj1AHlemZiF+YUJW205i3DS+or36LuVRMG+ozYjnjezSSHk7QKaZNiMR50zhdbyqye6ksv9OdnWIpdKL1IqdwcSVCOKs+/6348QH5MkPYjE6f/jggm2jWAlOTQPPZ2FP6hK+zAj08T98s2FQkbSntHh2uC/AdqlRkK8mrEEFPzfXRMDEohuh1PBdz0zgsCMFtn8gPcIQslqwCsr065Qct7Ae38xfnTFCl7fDsvGHeQa3ahSL/VWHORWjItIKVB0bB2j1BbVTCtUaxYzChaV2y5mRP47C2FK1cqIwtca0VknuW+ReP8AhUWkUioJoFv1xoXZW6puUVqTh+u1fW7o5a2Esk0xb/9rvJcH9GAvZpn7bL19vOCbUw0SU7H8IAsRte3SdDi6CgHLkG7eiTqZ9h9iRQWgzaDXhe4wlxO7qkoT5QjoWpVo4xtQUpbUnof8XjJxwd29rYq0n+fjdVVPLVfnMYDzvLwpX3pzV7aJZ30UBLrLY6oj54t3257W5x5YsZem/A== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230022)(4636009)(376002)(396003)(346002)(136003)(39860400002)(451199015)(40470700004)(36840700001)(46966006)(2906002)(8936002)(5660300002)(8676002)(44832011)(4326008)(70206006)(70586007)(41300700001)(40480700001)(6916009)(54906003)(36860700001)(6666004)(7696005)(107886003)(26005)(478600001)(86362001)(47076005)(81166007)(316002)(1076003)(336012)(2616005)(83380400001)(82310400005)(82740400003)(426003)(186003)(36756003)(40460700003);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 10:45:45.5728 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: db18db0f-67eb-4c59-61a5-08dac9520c3d X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT004.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB9545 X-ZohoMail-DKIM: pass (identity @armh.onmicrosoft.com) X-ZM-MESSAGEID: 1668768375737100001 Content-Type: text/plain; charset="utf-8" Currently the maximum number of NUMA nodes is a hardcoded value. This provides little flexibility unless changing the code. Introduce a new Kconfig option to change the maximum number of NUMA nodes conveniently. Also considering that not all architectures support NUMA, this Kconfig option is only visible on NUMA enabled architectures. Architectures not supporting NUMA still use 1 for MAX_NUMNODES. As NODES_SHIFT is currently unused, we're taking this opportunity to remove it. Signed-off-by: Wei Chen Acked-by: Jan Beulich --- v8 -> v9: 1. No change. v7 -> v8: 1. No change. v6 -> v7: 1. No change. v5 -> v6: 1. No change. v4 -> v5: 1. No change. v3 -> v4: 1. Update the commit log to follow Jan's suggestion. 2. Add Ack-by. v2 -> v3: 1. Fix indent. 2. Use 2-64 for node range. v1 -> v2: 1. Add NODES_SHIFT remove message in commit log 2. Change NR_NUMA_NODES upper bound from 4095 to 255. --- xen/arch/Kconfig | 11 +++++++++++ xen/arch/x86/include/asm/numa.h | 2 -- xen/include/xen/numa.h | 11 ++++++----- 3 files changed, 17 insertions(+), 7 deletions(-) diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig index f16eb0df43..7028f7b74f 100644 --- a/xen/arch/Kconfig +++ b/xen/arch/Kconfig @@ -17,3 +17,14 @@ config NR_CPUS For CPU cores which support Simultaneous Multi-Threading or similar technologies, this the number of logical threads which Xen will support. + +config NR_NUMA_NODES + int "Maximum number of NUMA nodes supported" + range 2 64 + default "64" + depends on NUMA + help + Controls the build-time size of various arrays and bitmaps + associated with multiple-nodes management. It is the upper bound of + the number of NUMA nodes that the scheduler, memory allocation and + other NUMA-aware components can handle. diff --git a/xen/arch/x86/include/asm/numa.h b/xen/arch/x86/include/asm/num= a.h index 2ca3475271..7866afa408 100644 --- a/xen/arch/x86/include/asm/numa.h +++ b/xen/arch/x86/include/asm/numa.h @@ -3,8 +3,6 @@ =20 #include =20 -#define NODES_SHIFT 6 - typedef u8 nodeid_t; =20 extern int srat_rev; diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 04ecaf7769..71a5f837b3 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -3,14 +3,15 @@ =20 #include =20 -#ifndef NODES_SHIFT -#define NODES_SHIFT 0 -#endif - #define NUMA_NO_NODE 0xFF #define NUMA_NO_DISTANCE 0xFF =20 -#define MAX_NUMNODES (1 << NODES_SHIFT) +#ifdef CONFIG_NR_NUMA_NODES +#define MAX_NUMNODES CONFIG_NR_NUMA_NODES +#else +#define MAX_NUMNODES 1 +#endif + #define NR_NODE_MEMBLKS (MAX_NUMNODES * 2) =20 #define vcpu_to_node(v) (cpu_to_node((v)->processor)) --=20 2.25.1