From nobody Wed Feb 11 16:10:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A913C77B7F for ; Mon, 8 May 2023 08:42:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233027AbjEHImP (ORCPT ); Mon, 8 May 2023 04:42:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60176 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232528AbjEHImM (ORCPT ); Mon, 8 May 2023 04:42:12 -0400 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2043.outbound.protection.outlook.com [40.107.212.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A449C106 for ; Mon, 8 May 2023 01:42:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A3dau9NZQA65e3ZXs1Pf8nmC4wrJMBg0ZeyKmaoTb4ONtJ5KHbPDT4fGW2Wji4Sc2iUeY6fzs+OwOPa0djdqq6P8k9desU9XzfhredxOIFKnezpJzS9aYSRH9Zjhlkd0Mpe/YR/L0QyzLa5HUAe4e6HQbwHz+HDNbXovqPHhKr9Okw22q+zVj6QwLRvczS36LoItY0Nkue8aQEmjeFUnJLNeudrDnsxAzwAwWxw/QmiPugFtR7d8vDDIv0B6YZBhPQ3Mrh0CrZbLRJHeXstK8LWOQ3hzGgRIFldM2vGliW2cTMc65d+nW0nq+WmLdZ1QxWR5Lse7VKldp8Wj/9RJRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Bxs9iZYoPs7rKu1hF4Uk7jh0uZRFQVJdzdvBdXvsJjA=; b=bixLojfHlMMLgr+cuLvvHsTNDZaklbYE5k7gSH9ZQoTBOfeb5jqBzWgL+VSp5a9NNpELY4eCdEHomYHCAAY0/YMJMx9VWWPWKAkrIOyM1LqIFzDBKyvTvOcm9CgE18ynNvwPoaZtPo8aVnkcjsPbaq4M+LRrFOrCO46sCCpSzfcAFUB/RRXYyf5DgQ/s2/O93Zsy0NB2x1YbmvjvZq744szH73woyN0kG95nYRPQisDHbqi67gsaMzuoT7EEIRP8oqIZR75X6FceTfgAmQK+Tq5Q+fkVxNzOygFnyVXiw1BTYAfd6JWm4PE2cPJw4d8EvRaicSxoVBd5Inb8J4Xokw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Bxs9iZYoPs7rKu1hF4Uk7jh0uZRFQVJdzdvBdXvsJjA=; b=HfJX41zHgL6rz6SSMAsH59v357FeAO9UsG8hLrPikOsjZeq1kDZ3V7dB8eHOOdeNRsT+nUlTBRCBrxp7yscawE+d5059iMGYP6rlG34MTeqcj/0oQEaTiIW5jY8RtTmItcfYkIAOuV3t0jgYOe5Rjajge6rLrrppHa1WgKlkUMM= Received: from DM6PR21CA0019.namprd21.prod.outlook.com (2603:10b6:5:174::29) by MN2PR12MB4469.namprd12.prod.outlook.com (2603:10b6:208:268::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32; Mon, 8 May 2023 08:42:08 +0000 Received: from DM6NAM11FT054.eop-nam11.prod.protection.outlook.com (2603:10b6:5:174:cafe::5f) by DM6PR21CA0019.outlook.office365.com (2603:10b6:5:174::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.4 via Frontend Transport; Mon, 8 May 2023 08:42:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT054.mail.protection.outlook.com (10.13.173.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.32 via Frontend Transport; Mon, 8 May 2023 08:42:08 +0000 Received: from BLR5CG134614W.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 8 May 2023 03:42:02 -0500 From: K Prateek Nayak To: CC: , , , , , , , , , , Subject: [PATCH 1/2] drivers: base: cacheinfo: Fix shared_cpu_map changes in event of CPU hotplug Date: Mon, 8 May 2023 14:11:14 +0530 Message-ID: <20230508084115.1157-2-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230508084115.1157-1-kprateek.nayak@amd.com> References: <20230508084115.1157-1-kprateek.nayak@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT054:EE_|MN2PR12MB4469:EE_ X-MS-Office365-Filtering-Correlation-Id: 649e920b-7ec1-43a0-df88-08db4fa01bdf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: o2+XnUYRWsonQWpbVhbAwonz05O9DwtLtEgwagHZmJe0jNWgsFeCYwkmnFupzvUzzld3whVssmTAJ8iSrWiiMam4Ckk1PlSC/tBb2UEnZ6mt/WiCjI/kmWBdSuEQZFeldg4RGt1qefe6BUPdULve5m4L+nYzq9qTerQvJQ3uCeS/XDJYdbEqNqq+QS0VADYhh3+H73VKnN7ZTSfn4eC+kYaq7gemt06v4HqK7AXcgUS8ba3c5yeeE51QvNyGPXpJrbmDORL/zwZYRfcq4uUWDPESjYMu4tq75J7HR32WLfiABonwA1nyEF9DG11InNB3VW5jsO7qqa20k1zCH/UzxGlUAmdukqdkFHUXoTHw4XFxyJsoqHKCPxyaIJkcQgCnwAfYELGM3ll6mChFStZ4+3Ai7L3SaRENYPUKqKc3Vd85ndt2s2veKexR/3hspeAfnaUOY49IT0qRLM+DgvibVyLsUxCXqtdRrbpLt1WOgJ1O+30dXjmVZ8fZ5G0+ODZs53otc3ZUvjY1pmxefF60YKFyh+qY66yttrKVBH054Jk0cijgWE/LlUQ7ocOQywOjqbAJltZTWnddaSBfYmQfv5GS7N1N2iR/XNuAvD/kK5gyl3sB766Frp3A/Mmj7mwUdVFrw8tbZ50W++CWsU5t3p4FpRaRaWbRktYs9RLXp15W0NZK0w/ErTjRAYZL7wcK9n0ExDE0nGmEaVOnn2L/vduC206w67oTQaZSYWlwJIw= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(39860400002)(396003)(376002)(136003)(451199021)(46966006)(40470700004)(36840700001)(86362001)(36756003)(6666004)(54906003)(316002)(70206006)(6916009)(4326008)(70586007)(7696005)(478600001)(82310400005)(40480700001)(8936002)(5660300002)(8676002)(2906002)(7416002)(41300700001)(356005)(82740400003)(81166007)(16526019)(186003)(107886003)(1076003)(26005)(36860700001)(47076005)(336012)(83380400001)(426003)(2616005)(40460700003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2023 08:42:08.3294 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 649e920b-7ec1-43a0-df88-08db4fa01bdf X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT054.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4469 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" While building the shared_cpu_map, check if the cache level and cache type matches. On certain systems that build the cache topology based on the instance ID, there are cases where the same ID may repeat across multiple cache levels, leading inaccurate topology. In event of CPU offlining, the cache_shared_cpu_map_remove() does not consider if IDs at same level are being compared. As a result, when same IDs repeat across different cache levels, the CPU going offline is not removed from all the shared_cpu_map. Below is the output of cache topology of CPU8 and it's SMT sibling after CPU8 is offlined on a dual socket 3rd Generation AMD EPYC processor (2 x 64C/128T) running kernel release v6.3: # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do = echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # echo 0 > /sys/devices/system/cpu/cpu8/online # for i in /sys/devices/system/cpu/cpu136/cache/index*/shared_cpu_list; d= o echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list: 136 /sys/devices/system/cpu/cpu136/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu136/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list: 9-15,136-1= 43 CPU8 is removed from index0 (L1i) but remains in the shared_cpu_list of index1 (L1d) and index2 (L2). Since L1i, L1d, and L2 are shared by the SMT siblings, and they have the same cache instance ID, CPU 2 is only removed from the first index with matching ID which is index1 (L1i) in this case. With this fix, the results are as expected when performing the same experiment on the same system: # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do = echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # echo 0 > /sys/devices/system/cpu/cpu8/online # for i in /sys/devices/system/cpu/cpu136/cache/index*/shared_cpu_list; d= o echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list: 136 /sys/devices/system/cpu/cpu136/cache/index1/shared_cpu_list: 136 /sys/devices/system/cpu/cpu136/cache/index2/shared_cpu_list: 136 /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list: 9-15,136-1= 43 When rebuilding topology, the same problem appears as cache_shared_cpu_map_setup() implements a similar logic. Consider the same 3rd Generation EPYC processor: CPUs in Core 1, that share the L1 and L2 caches, have L1 and L2 instance ID as 1. For all the CPUs on the second chiplet, the L3 ID is also 1 leading to grouping on CPUs from Core 1 (1, 17) and the entire second chiplet (8-15, 24-31) as CPUs sharing one cache domain. This went undetected since x86 processors depended on arch specific populate_cache_leaves() method to repopulate the shared_cpus_map when CPU came back online until kernel release v6.3-rc5. Fixes: 198102c9103f ("cacheinfo: Fix shared_cpu_map to handle shared caches= at different levels") Signed-off-by: K Prateek Nayak Reviewed-by: Sudeep Holla --- drivers/base/cacheinfo.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c index bba3482ddeb8..d1ae443fd7a0 100644 --- a/drivers/base/cacheinfo.c +++ b/drivers/base/cacheinfo.c @@ -388,6 +388,16 @@ static int cache_shared_cpu_map_setup(unsigned int cpu) continue;/* skip if itself or no cacheinfo */ for (sib_index =3D 0; sib_index < cache_leaves(i); sib_index++) { sib_leaf =3D per_cpu_cacheinfo_idx(i, sib_index); + + /* + * Comparing cache IDs only makes sense if the leaves + * belong to the same cache level of same type. Skip + * the check if level and type do not match. + */ + if (sib_leaf->level !=3D this_leaf->level || + sib_leaf->type !=3D this_leaf->type) + continue; + if (cache_leaves_are_shared(this_leaf, sib_leaf)) { cpumask_set_cpu(cpu, &sib_leaf->shared_cpu_map); cpumask_set_cpu(i, &this_leaf->shared_cpu_map); @@ -419,6 +429,16 @@ static void cache_shared_cpu_map_remove(unsigned int c= pu) =20 for (sib_index =3D 0; sib_index < cache_leaves(sibling); sib_index++) { sib_leaf =3D per_cpu_cacheinfo_idx(sibling, sib_index); + + /* + * Comparing cache IDs only makes sense if the leaves + * belong to the same cache level of same type. Skip + * the check if level and type do not match. + */ + if (sib_leaf->level !=3D this_leaf->level || + sib_leaf->type !=3D this_leaf->type) + continue; + if (cache_leaves_are_shared(this_leaf, sib_leaf)) { cpumask_clear_cpu(cpu, &sib_leaf->shared_cpu_map); cpumask_clear_cpu(sibling, &this_leaf->shared_cpu_map); --=20 2.34.1 From nobody Wed Feb 11 16:10:45 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F9FBC77B75 for ; Mon, 8 May 2023 08:42:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233163AbjEHImi (ORCPT ); Mon, 8 May 2023 04:42:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60492 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233059AbjEHIme (ORCPT ); Mon, 8 May 2023 04:42:34 -0400 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2082.outbound.protection.outlook.com [40.107.102.82]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9520749F3 for ; Mon, 8 May 2023 01:42:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WW2dD+fkzDawYaHsvB8vTHJeLUWfBoRsd+QO9g2xOlG4xZGzzGauSX4ZLSqXNyXiGSxpBOpVPic9DKJMQFfSng5uabaIK09tIN1EUfOOuteGZdLyuBfmXGvcMPewyyWg28SU6ESW0Q8CEP8VvJyJm6xz8Io1YdBXra14xlUfiWn11oIiN/PyB7bATo9VqybZQ4AiYnnQvCkO6xkjpSo4ql1dq0C/W6ILc8YA+HQ3JiGfXeeBWHUUPFqLwfGri0TNSjPFM9QRPOl3sVnp5XKbQSsmn+CiONA4McOGYeGvCjpOkLtHASrqLJ7L0nXx0xkrXOFL2w4gsUbBrrEgCqg5vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=h/4/depAAu7SHzKRfHzuG0A9ZYnJRjQ9+v3bBVZCieM=; b=GXkcu0Fmq2QQCBQ6xVj7aWVhybEhkRC7dKbsh+M8I/f61eP6CNacevU9KIGwuTVhkt0HaSm72nNQC71kuzadp4b6SmLYrA1VRHYFXqhsTyk635Zy5ShWtmTAxFO4uC5glIKty6XBKBCQY/t+/+3OlMCjfasIQyBET7+xeRiUY5DtQP4AZdwN449Zp8aCUf5mvW1DLSAKGCSeUIf8Q11+KWb/FxIjZOMjI4cjlyaF3uo6owWmg3kVJI6DiEHfcbzKBxEKq2jOJNWaD5C/8wyjm4NKS12wCIcywcXioCjY9lqMa1CE5dpDbuCgwoQCLh8mo8un8OSNBgRgNaZC82Pt4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h/4/depAAu7SHzKRfHzuG0A9ZYnJRjQ9+v3bBVZCieM=; b=IAxcWldF1nF3l6OuADxJYzoWaaq6BOFABg0y1EYuXFIChC2WkkF8YD6wNWgrlVA7RPcGM3Xo2PPZZAn3S6w1OgCugiW+ZQutXAaAnl194EOPp3ngZpyMkoG3clMxUC8hYf0RAKTltHsu3zH5eHeso+zMnUOWGT2GITk4R4nD0U4= Received: from SJ0PR03CA0300.namprd03.prod.outlook.com (2603:10b6:a03:39e::35) by BL1PR12MB5190.namprd12.prod.outlook.com (2603:10b6:208:31c::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32; Mon, 8 May 2023 08:42:31 +0000 Received: from DM6NAM11FT062.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:39e:cafe::79) by SJ0PR03CA0300.outlook.office365.com (2603:10b6:a03:39e::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32 via Frontend Transport; Mon, 8 May 2023 08:42:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT062.mail.protection.outlook.com (10.13.173.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.33 via Frontend Transport; Mon, 8 May 2023 08:42:30 +0000 Received: from BLR5CG134614W.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 8 May 2023 03:42:26 -0500 From: K Prateek Nayak To: CC: , , , , , , , , , , Subject: [PATCH 2/2] drivers: base: cacheinfo: Update cpu_map_populated during CPU Hotplug Date: Mon, 8 May 2023 14:11:15 +0530 Message-ID: <20230508084115.1157-3-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230508084115.1157-1-kprateek.nayak@amd.com> References: <20230508084115.1157-1-kprateek.nayak@amd.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT062:EE_|BL1PR12MB5190:EE_ X-MS-Office365-Filtering-Correlation-Id: 92a553d2-90cd-4baf-dab5-08db4fa02956 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /m5CTdQz0ALcca30zbbDUMn3kiMxSwk9S4YJA68Trl6o1Nz7zhGRwcf68TZ7wjNbkFnRL0BzS+MlEr66t4Xa5k1/Q3vEPeTan9QFagsoBmybvj9TlBtecsWK3Vy7QXXyp+w3FJxCJRvS5ZQba8hjyg9gwZteDJvazAcFJ+X6+eVll+dEHBiXs3QmzOujBnIgtE3ieyBS4sDLFvJYNG8hFGxYShshZzDqSeq44+LZdgwbAcMt5MQhm3kt2Q9ddq8woE2RY+L2/wjwr6nPIYykUcx6+HPSDdteMD18JuWG9bSoVngOrlJAtEtx8NS8d5oujGajqwNGsrk8bxBj9nvYl4KWvH5h+S3TZM248CXOabTEj2OjDnWlLssx9LT4d/qHsXMTZHNDeFX1iguYA4amGfXtndjncofoVFP85F4F3a+9k5VTUHYXIo+L870o31nPN8i6qXB3oisJIMZBF+xy9BrYiTASABNmkDBF4KgsmfY3F8ciZk4J9JcGyaVvWstg0K8QAbjgY+nPpe1bu8pp9LMVLJ123+WrWbZww/laLyZgPujJVRvbjbdYrNiBL3f53JOs3i4qkyG0ALLGQAUcBS6xAr0+mKBtVp7eIQ9LuiHtgSllR/T/x+Ki0YZgVolNVrEDSpM/9fC45bOOndRLzfp259OtSTd+WJYPelm4gD6ArJLcXsHZc/T3PYHIxRFKsuBC3DMTNYH2QNrAmRORp7ozBuEC+jjfVvdd2upmR94= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(39860400002)(396003)(376002)(136003)(451199021)(46966006)(40470700004)(36840700001)(86362001)(36756003)(6666004)(54906003)(316002)(70206006)(6916009)(4326008)(70586007)(7696005)(478600001)(82310400005)(40480700001)(8936002)(5660300002)(8676002)(2906002)(7416002)(41300700001)(356005)(82740400003)(81166007)(16526019)(186003)(107886003)(1076003)(26005)(36860700001)(47076005)(336012)(83380400001)(426003)(2616005)(40460700003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2023 08:42:30.9039 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 92a553d2-90cd-4baf-dab5-08db4fa02956 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT062.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5190 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Until commit 5c2712387d48 ("cacheinfo: Fix LLC is not exported through sysfs"), cacheinfo called populate_cache_leaves() for CPU coming online which let the arch specific functions handle (at least on x86) populating the shared_cpu_map. However, with the changes in the aforementioned commit, populate_cache_leaves() is not called when a CPU comes online as a result of hotplug since last_level_cache_is_valid() returns true as the cacheinfo data is not discarded. The CPU coming online is not present in shared_cpu_map, however, it will not be added since the cpu_cacheinfo->cpu_map_populated flag is set (it is set in populate_cache_leaves() when cacheinfo is first populated for x86) This can lead to inconsistencies in the shared_cpu_map when an offlined CPU comes online again. Example below depicts the inconsistency in the shared_cpu_list in cacheinfo when CPU8 is offlined and onlined again on a 3rd Generation EPYC processor: # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do = echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # echo 0 > /sys/devices/system/cpu/cpu8/online # echo 1 > /sys/devices/system/cpu/cpu8/online # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do = echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8 # cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list 136 # cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list 9-15,136-143 Clear the flag when the CPU is removed from shared_cpu_map when cache_shared_cpu_map_remove() is called during CPU hotplug. This will allow cache_shared_cpu_map_setup() to add the CPU coming back online in the shared_cpu_map. Set the flag again when the shared_cpu_map is setup. Following are results of performing the same test as described above with the changes: # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do = echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # echo 0 > /sys/devices/system/cpu/cpu8/online # echo 1 > /sys/devices/system/cpu/cpu8/online # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do = echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list 8,136 # cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list 8-15,136-143 Fixes: 5c2712387d48 ("cacheinfo: Fix LLC is not exported through sysfs") Signed-off-by: K Prateek Nayak Reviewed-by: Sudeep Holla Reviewed-by: Yicong Yang --- drivers/base/cacheinfo.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c index d1ae443fd7a0..cbae8be1fe52 100644 --- a/drivers/base/cacheinfo.c +++ b/drivers/base/cacheinfo.c @@ -410,11 +410,14 @@ static int cache_shared_cpu_map_setup(unsigned int cp= u) coherency_max_size =3D this_leaf->coherency_line_size; } =20 + /* shared_cpu_map is now populated for the cpu */ + this_cpu_ci->cpu_map_populated =3D true; return 0; } =20 static void cache_shared_cpu_map_remove(unsigned int cpu) { + struct cpu_cacheinfo *this_cpu_ci =3D get_cpu_cacheinfo(cpu); struct cacheinfo *this_leaf, *sib_leaf; unsigned int sibling, index, sib_index; =20 @@ -447,6 +450,9 @@ static void cache_shared_cpu_map_remove(unsigned int cp= u) } } } + + /* cpu is no longer populated in the shared map */ + this_cpu_ci->cpu_map_populated =3D false; } =20 static void free_cache_attributes(unsigned int cpu) --=20 2.34.1