From nobody Fri Dec 19 21:11:50 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0264914388F for ; Tue, 28 May 2024 22:20:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716934830; cv=none; b=eeIQzYzbBeGVLQ2gOW2dD4D7TJTRlCahiofmPGq+oD2f7L4640aWpYBKyQy4QYxZsuKHXzjc7OmyIzZPzTPEPADa1L4wFrCzlUtiI8yqm0ivFaE3hKgUqMfuzuQNY65ihW1tmK+tMyXdxThc+Tr58LO6fs8tA2j0aUs+3O8t108= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716934830; c=relaxed/simple; bh=5GfAF4d64QU3XOAsHg5jauTIBzHoAkFgwTZurK76Vu8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=d3jdzBD+B53ad2uFZLxtVd0p5YbpSHbCOTDUY3L6s9uB2mA7UAfKlKlk2cxEueblVyao72TTh9ycN38bsHwTUNv61qUXAHurSsgXvtftVP2f99+cFk7cdQRfEhsx33nzw0okUjIN6/q06YPAcPLA0jhlBghZY/j4uuFNpoZvc7I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=U7en//bj; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="U7en//bj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716934829; x=1748470829; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5GfAF4d64QU3XOAsHg5jauTIBzHoAkFgwTZurK76Vu8=; b=U7en//bjhoRqeqhePPfQMCNXrRzIXDqmY6uBKov26YsjiBfDAyeOHNgZ ncubh0bEXzjM15CJGApRnh1pamB9qOJmInhpbQ3b2QCXnwNZq2ralOCnf rP8J5ihjiTQmbwuoQG0yBYDJfPdbh53tvvKqkpydvOKEBnV3AAXt8wmFI xc3x7Nr8yeedWeIrBcK4YKWF1zy8UWV6jqT6d015WafL6Rym7EtgKb3yN cVlKMlgIhzbyczrK3c4KJcmm1DB4AM0keRX4PtLNKEU18V36euJx8151M b3lJTtjPiXL9obmObjmbqD0sgsoKh2v4XKPEc5D2FTkeJEAGfD1JVOHtg g==; X-CSE-ConnectionGUID: LiC+DGyiQh+cgvrw0YFYkg== X-CSE-MsgGUID: oChf0C6TRouw761fOuhL8g== X-IronPort-AV: E=McAfee;i="6600,9927,11085"; a="30812261" X-IronPort-AV: E=Sophos;i="6.08,196,1712646000"; d="scan'208";a="30812261" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 May 2024 15:20:20 -0700 X-CSE-ConnectionGUID: Gdv81BkWQd+PmSNCI+Vv2g== X-CSE-MsgGUID: LKu/UuMkQSaPr3K9+a86Wg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,196,1712646000"; d="scan'208";a="40090780" Received: from agluck-desk3.sc.intel.com ([172.25.222.70]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 May 2024 15:20:19 -0700 From: Tony Luck To: Fenghua Yu , Reinette Chatre , Maciej Wieczor-Retman , Peter Newman , James Morse , Babu Moger , Drew Fustini , Dave Martin Cc: x86@kernel.org, linux-kernel@vger.kernel.org, patches@lists.linux.dev, Tony Luck Subject: [PATCH v19 18/20] x86/resctrl: Enable RMID shared RMID mode on Sub-NUMA Cluster (SNC) systems Date: Tue, 28 May 2024 15:20:03 -0700 Message-ID: <20240528222006.58283-19-tony.luck@intel.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240528222006.58283-1-tony.luck@intel.com> References: <20240528222006.58283-1-tony.luck@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" There is an MSR which configures how RMIDs are distributed across SNC nodes. When SNC is enabled bit 0 of this MSR must be cleared. Add an architecture specific hook into domain_add_cpu_mon() to call a function to set the MSR. Signed-off-by: Tony Luck --- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/core.c | 2 ++ arch/x86/kernel/cpu/resctrl/monitor.c | 26 ++++++++++++++++++++++++++ 4 files changed, 31 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-in= dex.h index e022e6eb766c..3cb8dd6311c3 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -1164,6 +1164,7 @@ #define MSR_IA32_QM_CTR 0xc8e #define MSR_IA32_PQR_ASSOC 0xc8f #define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0 #define MSR_IA32_L2_CBM_BASE 0xd10 #define MSR_IA32_MBA_THRTL_BASE 0xd50 =20 diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 7957fc38b71c..08520321f5d0 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -532,6 +532,8 @@ static inline bool resctrl_arch_get_cdp_enabled(enum re= sctrl_res_level l) =20 int resctrl_arch_set_cdp_enabled(enum resctrl_res_level l, bool enable); =20 +void arch_mon_domain_online(struct rdt_resource *r, struct rdt_mon_domain = *d); + /* * Get the cacheinfo structure of the cache associated with @cpu at level = @level. * cpuhp lock must be held. diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 95ef8fe3cb50..1930fce9dfe9 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -615,6 +615,8 @@ static void domain_add_cpu_mon(int cpu, struct rdt_reso= urce *r) } cpumask_set_cpu(cpu, &d->hdr.cpu_mask); =20 + arch_mon_domain_online(r, d); + if (arch_domain_mbm_alloc(r->num_rmid, hw_dom)) { mon_domain_free(hw_dom); return; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index e7a8e96821e5..c7559735e33a 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -1069,6 +1069,32 @@ static void l3_mon_evt_init(struct rdt_resource *r) list_add_tail(&mbm_local_event.list, &r->evt_list); } =20 +/* + * The power-on reset value of MSR_RMID_SNC_CONFIG is 0x1 + * which indicates that RMIDs are configured in legacy mode. + * This mode is incompatible with Linux resctrl semantics + * as RMIDs are partitioned between SNC nodes, which requires + * a user to know which RMID is allocated to a task. + * Clearing bit 0 reconfigures the RMID counters for use + * in Sub-NUMA Cluster mode. This mode is better for Linux. + * The RMID space is divided between all SNC nodes with the + * RMIDs renumbered to start from zero in each node when + * counting operations from tasks. Code to read the counters + * must adjust RMID counter numbers based on SNC node. See + * logical_rmid_to_physical_rmid() for code that does this. + */ +void arch_mon_domain_online(struct rdt_resource *r, struct rdt_mon_domain = *d) +{ + u64 val; + + if (snc_nodes_per_l3_cache =3D=3D 1) + return; + + rdmsrl(MSR_RMID_SNC_CONFIG, val); + val &=3D ~BIT_ULL(0); + wrmsrl(MSR_RMID_SNC_CONFIG, val); +} + int __init rdt_get_mon_l3_config(struct rdt_resource *r) { unsigned int mbm_offset =3D boot_cpu_data.x86_cache_mbm_width_offset; --=20 2.45.0