From nobody Sun Feb 8 18:30:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1169CA0FE6 for ; Fri, 1 Sep 2023 08:56:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245271AbjIAI4U (ORCPT ); Fri, 1 Sep 2023 04:56:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229452AbjIAI4S (ORCPT ); Fri, 1 Sep 2023 04:56:18 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1672310D3 for ; Fri, 1 Sep 2023 01:56:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1693558574; x=1725094574; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nDVQFhqTixCtGXwWXAstRFhOuFQ3MPyV5OjPTLkEWEQ=; b=WT26XUG4/RvKE5CkgN4Md2wzaP0ctelYDAjJOEvTOr5rZ9capku3eHj5 l+rRolJ/9EjVHbcCZErtoyMNFjJcF6p5kEvK17+Ehvv+BfM6WH+WoEq0b ICC7UUpjf4O+fZTqr3ZmsYUrGe+TWY7ckzxb6Xw0x1lxovpW8RTm2/JMr MRQjd4RA82ae5Nl/P23E3P3DKc1frUxUa8egCqD/9EyAJrZT2oisMJfl5 QW1ZaHmR6PhAQazjJODln21CyC+YAdPwXHAnb3/s3Z8E25sbaLbHrQy/V sBhw3rzq2Vc8yPFldFgK/5KfWElFYcWS//LtX3QVLcicLFKh3OS2475RD g==; X-IronPort-AV: E=McAfee;i="6600,9927,10819"; a="440151652" X-IronPort-AV: E=Sophos;i="6.02,219,1688454000"; d="scan'208";a="440151652" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2023 01:56:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10819"; a="769133437" X-IronPort-AV: E=Sophos;i="6.02,219,1688454000"; d="scan'208";a="769133437" Received: from akoczor-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.213.14.236]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2023 01:56:09 -0700 From: "Wieczor-Retman, Maciej" To: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" Cc: linux-kernel@vger.kernel.org Subject: [PATCH 1/3] x86/resctrl: Enable non-contiguous bits in Intel CAT Date: Fri, 1 Sep 2023 10:55:37 +0200 Message-ID: <1bd3aab725a4e2948530095eb48b11bcc4028f8e.1693557919.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The setting for non-contiguous 1s support in Intel CAT is hardcoded to false. On these systems, writing non-contiguous 1s into the schemata file will fail before resctrl passes the value to the hardware. In Intel CAT CPUID.0x10.1:ECX[3] and CPUID.0x10.2:ECX[3] stopped being reserved and now carry information about non-contiguous 1s value support for L3 and L2 cache respectively. The CAT capacity bitmask (CBM) supports a non-contiguous 1s value if the bit is set. Replace the hardcoded non-contiguous support value with the support learned from the hardware. Add hardcoded non-contiguous support value to Haswell probe since it can't make use of CPUID for Cache allocation. Originally-by: Fenghua Yu Signed-off-by: Wieczor-Retman, Maciej --- arch/x86/kernel/cpu/resctrl/core.c | 9 ++++++--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 11 +++++++---- arch/x86/kernel/cpu/resctrl/internal.h | 9 +++++++++ 3 files changed, 22 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 030d3b409768..c783a873147c 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -152,6 +152,7 @@ static inline void cache_alloc_hsw_probe(void) r->cache.cbm_len =3D 20; r->cache.shareable_bits =3D 0xc0000; r->cache.min_cbm_bits =3D 2; + r->cache.arch_has_sparse_bitmaps =3D false; r->alloc_capable =3D true; =20 rdt_alloc_capable =3D true; @@ -267,15 +268,18 @@ static void rdt_get_cache_alloc_cfg(int idx, struct r= dt_resource *r) { struct rdt_hw_resource *hw_res =3D resctrl_to_arch_res(r); union cpuid_0x10_1_eax eax; + union cpuid_0x10_x_ecx ecx; union cpuid_0x10_x_edx edx; - u32 ebx, ecx; + u32 ebx; =20 - cpuid_count(0x00000010, idx, &eax.full, &ebx, &ecx, &edx.full); + cpuid_count(0x00000010, idx, &eax.full, &ebx, &ecx.full, &edx.full); hw_res->num_closid =3D edx.split.cos_max + 1; r->cache.cbm_len =3D eax.split.cbm_len + 1; r->default_ctrl =3D BIT_MASK(eax.split.cbm_len + 1) - 1; r->cache.shareable_bits =3D ebx & r->default_ctrl; r->data_width =3D (r->cache.cbm_len + 3) / 4; + if (boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_INTEL) + r->cache.arch_has_sparse_bitmaps =3D ecx.split.noncont; r->alloc_capable =3D true; } =20 @@ -872,7 +876,6 @@ static __init void rdt_init_res_defs_intel(void) =20 if (r->rid =3D=3D RDT_RESOURCE_L3 || r->rid =3D=3D RDT_RESOURCE_L2) { - r->cache.arch_has_sparse_bitmaps =3D false; r->cache.arch_has_per_cpu_cfg =3D false; r->cache.min_cbm_bits =3D 1; } else if (r->rid =3D=3D RDT_RESOURCE_MBA) { diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index b44c487727d4..782e2700290b 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -87,10 +87,13 @@ int parse_bw(struct rdt_parse_data *data, struct resctr= l_schema *s, =20 /* * Check whether a cache bit mask is valid. - * For Intel the SDM says: - * Please note that all (and only) contiguous '1' combinations - * are allowed (e.g. FFFFH, 0FF0H, 003CH, etc.). - * Additionally Haswell requires at least two bits set. + * On Intel CPUs, non-contiguous 1s value support is indicated by CPUID: + * - CPUID.0x10.1:ECX[3]: L3 non-contiguous 1s value supported if 1 + * - CPUID.0x10.2:ECX[3]: L2 non-contiguous 1s value supported if 1 + * + * Additionally Haswell requires at least two bits set. Since it does not + * have CPUID enumeration support for Cache allocation the non-contiguous = 1s + * value support is disabled. * AMD allows non-contiguous bitmasks. */ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 85ceaf9a31ac..c47ef2f13e8e 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -492,6 +492,15 @@ union cpuid_0x10_3_eax { unsigned int full; }; =20 +/* CPUID.(EAX=3D10H, ECX=3DResID).ECX */ +union cpuid_0x10_x_ecx { + struct { + unsigned int reserved:3; + unsigned int noncont:1; + } split; + unsigned int full; +}; + /* CPUID.(EAX=3D10H, ECX=3DResID).EDX */ union cpuid_0x10_x_edx { struct { --=20 2.42.0 From nobody Sun Feb 8 18:30:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21EB7CA0FE6 for ; Fri, 1 Sep 2023 08:56:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348671AbjIAI4s (ORCPT ); Fri, 1 Sep 2023 04:56:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348664AbjIAI4o (ORCPT ); Fri, 1 Sep 2023 04:56:44 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26AF910D7 for ; Fri, 1 Sep 2023 01:56:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1693558602; x=1725094602; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=c/D8+l4Mqu2fxn77jMbYJEecLUZHRiY1giSI4dGMx40=; b=hkMnpF1bxm+6CDykFT4GknxZ7yXJ3mGSPS3v/1X2Gjrk7JNESAmTUw5S BN9jXqdZuIirz8GytIybI78jEIE3wyrnL97NvFI1ffUIc/+0OSPlQoYoe LkVkHwYID1KDfwy8p9mHLXykaubSziKVsFBAAC+IStbr3Poe++Ua4dQ9g fnGBHXjYXM2r2qdPGKZcYfuvWeAf6txQi1Y/H1XWHzGrnTThzra8dNyPu /NKbvYlsb3nus+TUb+jQJTUKACeCxKnlIIsuuiyHrMj/F2jJ3pv3WLq4L XC4lgf/SuCfG7bGBQcGaAoQD00R6v5d8ukIa1mWUIDVdUphlSWkA8HEsz A==; X-IronPort-AV: E=McAfee;i="6600,9927,10819"; a="440151908" X-IronPort-AV: E=Sophos;i="6.02,219,1688454000"; d="scan'208";a="440151908" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2023 01:56:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10819"; a="769133618" X-IronPort-AV: E=Sophos;i="6.02,219,1688454000"; d="scan'208";a="769133618" Received: from akoczor-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.213.14.236]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2023 01:56:38 -0700 From: "Wieczor-Retman, Maciej" To: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" Cc: linux-kernel@vger.kernel.org Subject: [PATCH 2/3] x86/resctrl: Add sparse_bitmaps file in info Date: Fri, 1 Sep 2023 10:55:38 +0200 Message-ID: <27fc083358bc215676354818ad5a7abebd00e332.1693557919.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Fenghua Yu Add the interface in resctrl FS to show if sparse CAT bitmaps are supported on the platform. Reading the file returns either a "1" if non-contiguous 1s are supported and "0" otherwise. The file path is /sys/fs/resctrl/info/{resource}/sparse_bitmaps, where {resource} can be either "L2" or "L3" depending on their support in the CAT feature. Signed-off-by: Fenghua Yu Signed-off-by: Wieczor-Retman, Maciej --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 725344048f85..4d27354f3f30 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -895,6 +895,17 @@ static int rdt_shareable_bits_show(struct kernfs_open_= file *of, return 0; } =20 +static int rdt_has_sparse_bitmaps_show(struct kernfs_open_file *of, + struct seq_file *seq, void *v) +{ + struct resctrl_schema *s =3D of->kn->parent->priv; + struct rdt_resource *r =3D s->res; + + seq_printf(seq, "%u\n", r->cache.arch_has_sparse_bitmaps); + + return 0; +} + /** * rdt_bit_usage_show - Display current usage of resources * @@ -1839,6 +1850,13 @@ static struct rftype res_common_files[] =3D { .seq_show =3D rdtgroup_size_show, .fflags =3D RF_CTRL_BASE, }, + { + .name =3D "sparse_bitmaps", + .mode =3D 0444, + .kf_ops =3D &rdtgroup_kf_single_ops, + .seq_show =3D rdt_has_sparse_bitmaps_show, + .fflags =3D RF_CTRL_INFO | RFTYPE_RES_CACHE, + }, =20 }; =20 --=20 2.42.0 From nobody Sun Feb 8 18:30:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23042CA0FE6 for ; Fri, 1 Sep 2023 08:57:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348692AbjIAI5N (ORCPT ); Fri, 1 Sep 2023 04:57:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57262 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348674AbjIAI5L (ORCPT ); Fri, 1 Sep 2023 04:57:11 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 275FF10F2; Fri, 1 Sep 2023 01:57:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1693558622; x=1725094622; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=psJh6vY3E+3/B3OgEgSlz9IZ/M/u0a/9heXK3c+Qo8Q=; b=VoM3k9ttp/vEBhTifx350RKyZWLmuvssOO1zLjfqatlLDgIgdxiKDTk+ a5t/v4gVmsOVSVMgoTO4oxnxFIWFqvaJUAMkS76JwGvHS/ZexSdsPZCsm YAXjeZrCoHX3SqQ+W3ADZIGUDiSjO2BFYt26Bt6TEU0yU+o/6AGPvuZWg RTHYb2Jo8F+HCQ58lTGikVzxVb90dc5dDzN0sj5r8aNLa0cywUaSVYgP1 /Md7NNbkG86FnKVq+ko02+X5jmt9xYeDY95JY01dFaZSHaYenZ3OTRw5y LdvD09d13vgpW0UGcVmZ3HAHje7qd2QmtAVu8ZdP7gMy4WcocReIosQnk w==; X-IronPort-AV: E=McAfee;i="6600,9927,10819"; a="440151945" X-IronPort-AV: E=Sophos;i="6.02,219,1688454000"; d="scan'208";a="440151945" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2023 01:56:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10819"; a="769133703" X-IronPort-AV: E=Sophos;i="6.02,219,1688454000"; d="scan'208";a="769133703" Received: from akoczor-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.213.14.236]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2023 01:56:52 -0700 From: "Wieczor-Retman, Maciej" To: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Subject: [PATCH 3/3] Documentation/x86: Document resctrl's new sparse_bitmaps Date: Fri, 1 Sep 2023 10:55:39 +0200 Message-ID: <89435c36b43651d63bd034adc142e932062aab21.1693557919.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Fenghua Yu The documentation mentions that non-contiguous bitmasks are not supported in Intel Cache Allocation Technology (CAT). Update the documentation on how to determine if sparse bitmasks are allowed in L2 and L3 CAT. Mention the file with feature support information is located in the /sys/fs/resctrl/info/{resource}/ directories and enumerate what are the possible outputs on file read operation. Signed-off-by: Fenghua Yu Signed-off-by: Wieczor-Retman, Maciej --- Documentation/arch/x86/resctrl.rst | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/Documentation/arch/x86/resctrl.rst b/Documentation/arch/x86/re= sctrl.rst index cb05d90111b4..38694fc5800c 100644 --- a/Documentation/arch/x86/resctrl.rst +++ b/Documentation/arch/x86/resctrl.rst @@ -124,6 +124,13 @@ related to allocation: "P": Corresponding region is pseudo-locked. No sharing allowed. +"sparse_bitmaps": + Indicates if non-contiguous 1s value in CBM is supported. + + "0": + Only contiguous 1s value in CBM is supported. + "1": + Non-contiguous 1s value in CBM is supported. =20 Memory bandwidth(MB) subdirectory contains the following files with respect to allocation: @@ -445,12 +452,13 @@ For cache resources we describe the portion of the ca= che that is available for allocation using a bitmask. The maximum value of the mask is defined by each cpu model (and may be different for different cache levels). It is found using CPUID, but is also provided in the "info" directory of -the resctrl file system in "info/{resource}/cbm_mask". Intel hardware +the resctrl file system in "info/{resource}/cbm_mask". Some Intel hardware requires that these masks have all the '1' bits in a contiguous block. So 0x3, 0x6 and 0xC are legal 4-bit masks with two bits set, but 0x5, 0x9 -and 0xA are not. On a system with a 20-bit mask each bit represents 5% -of the capacity of the cache. You could partition the cache into four -equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +and 0xA are not. Check /sys/fs/resctrl/info/{resource}/sparse_bitmaps +if non-contiguous 1s value is supported. On a system with a 20-bit mask +each bit represents 5% of the capacity of the cache. You could partition +the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. =20 Memory bandwidth Allocation and monitoring =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --=20 2.42.0