From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6E9CD2620E5 for ; Tue, 11 Mar 2025 18:40:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718414; cv=none; b=c95/aM9OWqNADG1vI/wDtZ4XhRcFGVXkQpHQst2wbnhzSgRuVDUD91yiK4T13WqarKDvO6u27RRIuKESMRDuIDSI95S+cdxEtLivSbPv5mapkL70FmKWLUL/yR/0uL7Zl8VjBibhne/qZv07NAP6ccGsNkWAB1YgbrcQy/gnqZA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718414; c=relaxed/simple; bh=pwTegAret97LxZZwPrfVmtV0rYonQX3fVU6UcvLs95U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rLLNMjINyWXZG9CNPWXuWR2oLEwLvkN6hqQ/lN+rbEWVt6+JNSSyvpyrTuniQKJWhZxjukhUGvvwR9Q/XZYuvtgPAw7kmQeXQczLctJwtAdM5SbNnniN1RwW4CRgyBNcq+/+QbPI7sd80YAgl8Zx+LC+hLBftCkfGQ8wKf56Mgo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BB4521515; Tue, 11 Mar 2025 11:40:22 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3F5993F694; Tue, 11 Mar 2025 11:40:08 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 01/30] x86/resctrl: Fix allocation of cleanest CLOSID on platforms with no monitors Date: Tue, 11 Mar 2025 18:36:46 +0000 Message-Id: <20250311183715.16445-2-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" commit 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by searching closid_num_dirty_rmid") added logic that causes resctrl to search for the CLOSID with the fewest dirty cache lines when creating a new control group, if requested by the arch code. This depends on the values read from the llc_occupancy counters. The logic is applicable to architectures where the CLOSID effectively forms part of the monitoring identifier and so do not allow complete freedom to choose an unused monitoring identifier for a given CLOSID. This support missed that some platforms may not have these counters. This causes a NULL pointer dereference when creating a new control group as the array was not allocated by dom_data_init(). As this feature isn't necessary on platforms that don't have cache occupancy monitors, add this to the check that occurs when a new control group is allocated. Fixes: 6eac36bb9eb0 ("x86/resctrl: Allocate the cleanest CLOSID by searchin= g closid_num_dirty_rmid") Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: David Hildenbrand Reviewed-by: Reinette Chatre Reviewed-by: Tony Luck Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- The existing code is not selected by any upstream platform, it makes no sense to backport this patch to stable. Changes since v1: * [Commit message only] Reword the first paragraph to make it clear that the issue being fixed wasn't directly associated with addition of a Kconfig option. (Actually, the option is not in Kconfig yet, and gets added later in this series.) --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 6419e04d8a7b..04b653d613e8 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -157,7 +157,8 @@ static int closid_alloc(void) =20 lockdep_assert_held(&rdtgroup_mutex); =20 - if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) && + is_llc_occupancy_enabled()) { cleanest_closid =3D resctrl_find_cleanest_closid(); if (cleanest_closid < 0) return cleanest_closid; --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C9E85263C77 for ; Tue, 11 Mar 2025 18:40:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718420; cv=none; b=Qa4zU8kRw+mYJknjkf8v5OnwFHJbnVGiBboEZUnrQS0ZWfLZQSPNf1NlnradWr3BlX6Ve+bNef5EzEgFLbyAOh7bubhP/7R8CMUfN1uyaoTqrDRUxNYL6MmvRL/a7A2Se0Va/ZA0mkTBJjILPFZoVmbThW7PhHwmulyFS4AbGiU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718420; c=relaxed/simple; bh=pOFGyCeiGpFAFOm6M+o0rFIaws8/T6KRakqXymEmEfU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LMiunSh0M+FOEWcWa3LVj4y8mtc38Ey8dMF3IE9FSQ2doO1RlLgeCPlearecZ7aJiffieDUg8BHXFVN8+U3drafJDHTyzCzPs+bCeTXtfa7XFI36+Fl1Ivk208lwFYLffiBpnOdnuiIuKGIqNK3gQEHJqFb/M35kWI+WMzGj2z4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B53C71688; Tue, 11 Mar 2025 11:40:26 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F35BF3F694; Tue, 11 Mar 2025 11:40:11 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Dave Martin , Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 02/30] x86/resctrl: Add a helper to avoid reaching into the arch code resource list Date: Tue, 11 Mar 2025 18:36:47 +0000 Message-Id: <20250311183715.16445-3-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Resctrl occasionally wants to know something about a specific resource, in these cases it reaches into the arch code's rdt_resources_all[] array. Once the filesystem parts of resctrl are moved to /fs/, this means it will need visibility of the architecture specific struct rdt_hw_resource definition, and the array of all resources. All architectures would also need a r_resctrl member in this struct. Instead, abstract this via a helper to allow architectures to do different things here. Move the level enum to the resctrl header and add a helper to retrieve the struct rdt_resource by 'rid'. resctrl_arch_get_resource() should not return NULL for any value in the enum, it may instead return a dummy resource that is !alloc_enabled && !mon_enabled. Co-developed-by: Dave Martin Signed-off-by: Dave Martin Signed-off-by: James Morse Tested-by: Peter Newman Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Tony Luck Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Change since v5: * Ensure rdt_resources_all[] is always padded to RDT_NUM_RESOURCES Changes since v1: * Backed out non-functional renaming of "r" to "l3" in rdt_get_tree(), and unhoisted the assignment of r (as now is) back into the if () where it started out. There seem to be no uses of this variable outside this if(). * [Commit message only] Typo fix: s/resctrl_hw_resource/rdt_hw_resource/g --- arch/x86/kernel/cpu/resctrl/core.c | 12 ++++++++++-- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- arch/x86/kernel/cpu/resctrl/internal.h | 10 ---------- arch/x86/kernel/cpu/resctrl/monitor.c | 8 ++++---- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 14 +++++++------- include/linux/resctrl.h | 17 +++++++++++++++++ 6 files changed, 39 insertions(+), 24 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 3d1735ed8d1f..12b41316d8f7 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -62,7 +62,7 @@ static void mba_wrmsr_amd(struct msr_param *m); #define ctrl_domain_init(id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctr= l.ctrl_domains) #define mon_domain_init(id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl= .mon_domains) =20 -struct rdt_hw_resource rdt_resources_all[] =3D { +struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCES] =3D { [RDT_RESOURCE_L3] =3D { .r_resctrl =3D { @@ -127,6 +127,14 @@ u32 resctrl_arch_system_num_rmid_idx(void) return r->num_rmid; } =20 +struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l) +{ + if (l >=3D RDT_NUM_RESOURCES) + return NULL; + + return &rdt_resources_all[l].r_resctrl; +} + /* * cache_alloc_hsw_probe() - Have to probe for Intel haswell server CPUs * as they do not have CPUID enumeration support for Cache allocation. @@ -174,7 +182,7 @@ static inline void cache_alloc_hsw_probe(void) bool is_mba_sc(struct rdt_resource *r) { if (!r) - return rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl.membw.mba_sc; + r =3D resctrl_arch_get_resource(RDT_RESOURCE_MBA); =20 /* * The software controller support is only applicable to MBA resource. diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index 536351159cc2..4af27ef5a8a1 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -649,7 +649,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) resid =3D md.u.rid; domid =3D md.u.domid; evtid =3D md.u.evtid; - r =3D &rdt_resources_all[resid].r_resctrl; + r =3D resctrl_arch_get_resource(resid); =20 if (md.u.sum) { /* diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 20c898f09b7e..75252a7e1ebc 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -512,16 +512,6 @@ extern struct rdtgroup rdtgroup_default; extern struct dentry *debugfs_resctrl; extern enum resctrl_event_id mba_mbps_default_event; =20 -enum resctrl_res_level { - RDT_RESOURCE_L3, - RDT_RESOURCE_L2, - RDT_RESOURCE_MBA, - RDT_RESOURCE_SMBA, - - /* Must be the last */ - RDT_NUM_RESOURCES, -}; - static inline struct rdt_resource *resctrl_inc(struct rdt_resource *res) { struct rdt_hw_resource *hw_res =3D resctrl_to_arch_res(res); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index 94a1d9780461..58b5b21349a8 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -365,7 +365,7 @@ static void limbo_release_entry(struct rmid_entry *entr= y) */ void __check_limbo(struct rdt_mon_domain *d, bool force_free) { - struct rdt_resource *r =3D &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + struct rdt_resource *r =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); u32 idx_limit =3D resctrl_arch_system_num_rmid_idx(); struct rmid_entry *entry; u32 idx, cur_idx =3D 1; @@ -521,7 +521,7 @@ int alloc_rmid(u32 closid) =20 static void add_rmid_to_limbo(struct rmid_entry *entry) { - struct rdt_resource *r =3D &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + struct rdt_resource *r =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); struct rdt_mon_domain *d; u32 idx; =20 @@ -761,7 +761,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct= rdt_mon_domain *dom_mbm) struct rdtgroup *entry; u32 cur_bw, user_bw; =20 - r_mba =3D &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl; + r_mba =3D resctrl_arch_get_resource(RDT_RESOURCE_MBA); evt_id =3D rgrp->mba_mbps_event; =20 closid =3D rgrp->closid; @@ -925,7 +925,7 @@ void mbm_handle_overflow(struct work_struct *work) if (!resctrl_mounted || !resctrl_arch_mon_capable()) goto out_unlock; =20 - r =3D &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + r =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); d =3D container_of(work, struct rdt_mon_domain, mbm_over.work); =20 list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) { diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 04b653d613e8..45093b9e8e63 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2256,7 +2256,7 @@ static void l2_qos_cfg_update(void *arg) =20 static inline bool is_mba_linear(void) { - return rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl.membw.delay_linear; + return resctrl_arch_get_resource(RDT_RESOURCE_MBA)->membw.delay_linear; } =20 static int set_cache_qos_cfg(int level, bool enable) @@ -2346,8 +2346,8 @@ static void mba_sc_domain_destroy(struct rdt_resource= *r, */ static bool supports_mba_mbps(void) { - struct rdt_resource *rmbm =3D &rdt_resources_all[RDT_RESOURCE_L3].r_resct= rl; - struct rdt_resource *r =3D &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl; + struct rdt_resource *rmbm =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); + struct rdt_resource *r =3D resctrl_arch_get_resource(RDT_RESOURCE_MBA); =20 return (is_mbm_enabled() && r->alloc_capable && is_mba_linear() && @@ -2360,7 +2360,7 @@ static bool supports_mba_mbps(void) */ static int set_mba_sc(bool mba_sc) { - struct rdt_resource *r =3D &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl; + struct rdt_resource *r =3D resctrl_arch_get_resource(RDT_RESOURCE_MBA); u32 num_closid =3D resctrl_arch_get_num_closid(r); struct rdt_ctrl_domain *d; unsigned long fflags; @@ -2714,7 +2714,7 @@ static int rdt_get_tree(struct fs_context *fc) resctrl_mounted =3D true; =20 if (is_mbm_enabled()) { - r =3D &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + r =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); list_for_each_entry(dom, &r->mon_domains, hdr.list) mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL, RESCTRL_PICK_ANY_CPU); @@ -3951,7 +3951,7 @@ static int rdtgroup_show_options(struct seq_file *seq= , struct kernfs_root *kf) if (resctrl_arch_get_cdp_enabled(RDT_RESOURCE_L2)) seq_puts(seq, ",cdpl2"); =20 - if (is_mba_sc(&rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl)) + if (is_mba_sc(resctrl_arch_get_resource(RDT_RESOURCE_MBA))) seq_puts(seq, ",mba_MBps"); =20 if (resctrl_debug) @@ -4151,7 +4151,7 @@ static void clear_childcpus(struct rdtgroup *r, unsig= ned int cpu) =20 void resctrl_offline_cpu(unsigned int cpu) { - struct rdt_resource *l3 =3D &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + struct rdt_resource *l3 =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); struct rdt_mon_domain *d; struct rdtgroup *rdtgrp; =20 diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index d94abba1c716..37279e2a89da 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -37,6 +37,16 @@ enum resctrl_conf_type { CDP_DATA, }; =20 +enum resctrl_res_level { + RDT_RESOURCE_L3, + RDT_RESOURCE_L2, + RDT_RESOURCE_MBA, + RDT_RESOURCE_SMBA, + + /* Must be the last */ + RDT_NUM_RESOURCES, +}; + #define CDP_NUM_TYPES (CDP_DATA + 1) =20 /* @@ -226,6 +236,13 @@ struct rdt_resource { bool cdp_capable; }; =20 +/* + * Get the resource that exists at this level. If the level is not support= ed + * a dummy/not-capable resource can be returned. Levels >=3D RDT_NUM_RESOU= RCES + * will return NULL. + */ +struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l); + /** * struct resctrl_schema - configuration abilities of a resource presented= to * user-space --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1A127263C85 for ; Tue, 11 Mar 2025 18:40:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718422; cv=none; b=aBf7jF/NTSXGFU0aB4bn+hUn2hvQ16XEetAVio2jEqzBKGHXv7I3g9Bls5h5NZHNnqtDv28lCsGwcartBY+wXbcIClQG1ho3OUGcbxbcaY5toe8GoFBWKMRRP75yoWICt6RZMBTpss3jsGFYg+qlCkASwKOLg9daHJpIMU0NoLY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718422; c=relaxed/simple; bh=95fOEJIWIBmqDF5ADw/HT09xx/pu52pC9PmfMtK0xqI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=B/c9C7CiYtt5b7mKAkWzSqYPo7CwwgKlteGIRSCEdNPz03LbBiahBkWouQ+GAH6CIaMM2h+GzJ+vgKsxg4BsEU9Rfvdug49i0z7uINfIJjzcg0pUwb0y+wGhdJniAoV3n+G/PeynjXERNKfDjYcPDZTWdC6wORgwheUYlQGXd1Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 700911713; Tue, 11 Mar 2025 11:40:30 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EC7F93F694; Tue, 11 Mar 2025 11:40:15 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 03/30] x86/resctrl: Remove fflags from struct rdt_resource Date: Tue, 11 Mar 2025 18:36:48 +0000 Message-Id: <20250311183715.16445-4-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The resctrl arch code specifies whether a resource controls a cache or memory using the fflags field. This field is then used by resctrl to determine which files should be exposed in the filesystem. Allowing the architecture to pick this value means the RFTYPE_ flags have to be in a shared header, and allows an architecture to create a combination that resctrl does not support. Remove the fflags field, and pick the value based on the resource id. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v5: * Made fflags_from_resource() return an unsigned long. * Removed a space. Changes since v4: * Removed an extra space * Fixed a typo --- arch/x86/kernel/cpu/resctrl/core.c | 4 ---- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 ++++++++++++++++-- include/linux/resctrl.h | 2 -- 3 files changed, 16 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 12b41316d8f7..8ef2e449b465 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -74,7 +74,6 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCE= S] =3D { .mon_domains =3D mon_domain_init(RDT_RESOURCE_L3), .parse_ctrlval =3D parse_cbm, .format_str =3D "%d=3D%0*x", - .fflags =3D RFTYPE_RES_CACHE, }, .msr_base =3D MSR_IA32_L3_CBM_BASE, .msr_update =3D cat_wrmsr, @@ -88,7 +87,6 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCE= S] =3D { .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_L2), .parse_ctrlval =3D parse_cbm, .format_str =3D "%d=3D%0*x", - .fflags =3D RFTYPE_RES_CACHE, }, .msr_base =3D MSR_IA32_L2_CBM_BASE, .msr_update =3D cat_wrmsr, @@ -102,7 +100,6 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOUR= CES] =3D { .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_MBA), .parse_ctrlval =3D parse_bw, .format_str =3D "%d=3D%*u", - .fflags =3D RFTYPE_RES_MB, }, }, [RDT_RESOURCE_SMBA] =3D @@ -114,7 +111,6 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOUR= CES] =3D { .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_SMBA), .parse_ctrlval =3D parse_bw, .format_str =3D "%d=3D%*u", - .fflags =3D RFTYPE_RES_MB, }, }, }; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 45093b9e8e63..3391ac8ecb2d 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2165,6 +2165,20 @@ static int rdtgroup_mkdir_info_resdir(void *priv, ch= ar *name, return ret; } =20 +static unsigned long fflags_from_resource(struct rdt_resource *r) +{ + switch (r->rid) { + case RDT_RESOURCE_L3: + case RDT_RESOURCE_L2: + return RFTYPE_RES_CACHE; + case RDT_RESOURCE_MBA: + case RDT_RESOURCE_SMBA: + return RFTYPE_RES_MB; + } + + return WARN_ON_ONCE(1); +} + static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn) { struct resctrl_schema *s; @@ -2185,14 +2199,14 @@ static int rdtgroup_create_info_dir(struct kernfs_n= ode *parent_kn) /* loop over enabled controls, these are all alloc_capable */ list_for_each_entry(s, &resctrl_schema_all, list) { r =3D s->res; - fflags =3D r->fflags | RFTYPE_CTRL_INFO; + fflags =3D fflags_from_resource(r) | RFTYPE_CTRL_INFO; ret =3D rdtgroup_mkdir_info_resdir(s, s->name, fflags); if (ret) goto out_destroy; } =20 for_each_mon_capable_rdt_resource(r) { - fflags =3D r->fflags | RFTYPE_MON_INFO; + fflags =3D fflags_from_resource(r) | RFTYPE_MON_INFO; sprintf(name, "%s_MON", r->name); ret =3D rdtgroup_mkdir_info_resdir(r, name, fflags); if (ret) diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 37279e2a89da..496ddcaa4ecf 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -210,7 +210,6 @@ enum resctrl_scope { * @format_str: Per resource format string to show domain value * @parse_ctrlval: Per resource function pointer to parse control values * @evt_list: List of monitoring events - * @fflags: flags to choose base and info files * @cdp_capable: Is the CDP feature available on this resource */ struct rdt_resource { @@ -232,7 +231,6 @@ struct rdt_resource { struct resctrl_schema *s, struct rdt_ctrl_domain *d); struct list_head evt_list; - unsigned long fflags; bool cdp_capable; }; =20 --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E1FAE2641D7 for ; Tue, 11 Mar 2025 18:40:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718425; cv=none; b=MJX9WHXdlZJkSuoVY13xvfu0HDf8SDGS2vOgeEj5DopqcfBxVFZTo/M7SnpsiX4HdVq+Jrrz2xsvPfslE0jkLorRoARZhEnBWLj9yGqQZ5XLjggUTkZYO0NPjGy1b4AtN53Dxerw9fd3TRt4nLKYA8OFxrSv2PdTiQlZGgSpDXI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718425; c=relaxed/simple; bh=pRCHB3f9zYWqSQ+idg4nbQqRc+LxR2Z7gVQLeEjuFUI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RRrFm/hGu5mXXfkMyfrWddTQovb6W8L1NmTHxuLMz397PNg+N9iVO21CAdbkqmexYHlAB8ifwYRvue5B3LsbaDga7yNzrJVv/akwQZIqBYo72gu/jLWr46n5kBM8K7l7SVaihFLCTJrVvCagGzJaYyUWKeZsDwbBI/D+AD5irYU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2C58C1515; Tue, 11 Mar 2025 11:40:34 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A75B43F694; Tue, 11 Mar 2025 11:40:19 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 04/30] x86/resctrl: Use schema type to determine how to parse schema values Date: Tue, 11 Mar 2025 18:36:49 +0000 Message-Id: <20250311183715.16445-5-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Resctrl's architecture code gets to specify a function pointer that is used when parsing schema entries. This is expected to be one of two helpers from the filesystem code. Setting this function pointer allows the architecture code to change the ABI resctrl presents to user-space, and forces resctrl to expose these helpers. Instead, add a schema format enum to choose which schema parser to use. This allows the helpers to be made static and the structs used for passing arguments moved out of shared headers. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Change since v6: * Added a full stop. Changes since v5: * Merged the contents of get_parser() with its only caller. * Removed the description of what the range schema is used for. * Waggled some whitespace. Changes since v4: * Creation of the enum moves into this patch - review tags not picked up. * Removed some whitespace. Changes since v3: * Removed a spurious semicolon Changes since v2: * This patch is new --- arch/x86/kernel/cpu/resctrl/core.c | 8 +++--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 32 +++++++++++++++++++---- arch/x86/kernel/cpu/resctrl/internal.h | 10 ------- include/linux/resctrl.h | 17 ++++++++---- 4 files changed, 43 insertions(+), 24 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 8ef2e449b465..e9fe129a02f8 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -72,7 +72,7 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCE= S] =3D { .mon_scope =3D RESCTRL_L3_CACHE, .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_L3), .mon_domains =3D mon_domain_init(RDT_RESOURCE_L3), - .parse_ctrlval =3D parse_cbm, + .schema_fmt =3D RESCTRL_SCHEMA_BITMAP, .format_str =3D "%d=3D%0*x", }, .msr_base =3D MSR_IA32_L3_CBM_BASE, @@ -85,7 +85,7 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCE= S] =3D { .name =3D "L2", .ctrl_scope =3D RESCTRL_L2_CACHE, .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_L2), - .parse_ctrlval =3D parse_cbm, + .schema_fmt =3D RESCTRL_SCHEMA_BITMAP, .format_str =3D "%d=3D%0*x", }, .msr_base =3D MSR_IA32_L2_CBM_BASE, @@ -98,7 +98,7 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCE= S] =3D { .name =3D "MB", .ctrl_scope =3D RESCTRL_L3_CACHE, .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_MBA), - .parse_ctrlval =3D parse_bw, + .schema_fmt =3D RESCTRL_SCHEMA_RANGE, .format_str =3D "%d=3D%*u", }, }, @@ -109,7 +109,7 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOUR= CES] =3D { .name =3D "SMBA", .ctrl_scope =3D RESCTRL_L3_CACHE, .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_SMBA), - .parse_ctrlval =3D parse_bw, + .schema_fmt =3D RESCTRL_SCHEMA_RANGE, .format_str =3D "%d=3D%*u", }, }, diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index 4af27ef5a8a1..f4334f437ffc 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -23,6 +23,15 @@ =20 #include "internal.h" =20 +struct rdt_parse_data { + struct rdtgroup *rdtgrp; + char *buf; +}; + +typedef int (ctrlval_parser_t)(struct rdt_parse_data *data, + struct resctrl_schema *s, + struct rdt_ctrl_domain *d); + /* * Check whether MBA bandwidth percentage value is correct. The value is * checked against the minimum and max bandwidth values specified by the @@ -64,8 +73,8 @@ static bool bw_validate(char *buf, u32 *data, struct rdt_= resource *r) return true; } =20 -int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s, - struct rdt_ctrl_domain *d) +static int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s, + struct rdt_ctrl_domain *d) { struct resctrl_staged_config *cfg; u32 closid =3D data->rdtgrp->closid; @@ -143,8 +152,8 @@ static bool cbm_validate(char *buf, u32 *data, struct r= dt_resource *r) * Read one cache bit mask (hex). Check that it is valid for the current * resource type. */ -int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s, - struct rdt_ctrl_domain *d) +static int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s, + struct rdt_ctrl_domain *d) { struct rdtgroup *rdtgrp =3D data->rdtgrp; struct resctrl_staged_config *cfg; @@ -210,6 +219,7 @@ static int parse_line(char *line, struct resctrl_schema= *s, struct rdtgroup *rdtgrp) { enum resctrl_conf_type t =3D s->conf_type; + ctrlval_parser_t *parse_ctrlval =3D NULL; struct resctrl_staged_config *cfg; struct rdt_resource *r =3D s->res; struct rdt_parse_data data; @@ -220,6 +230,18 @@ static int parse_line(char *line, struct resctrl_schem= a *s, /* Walking r->domains, ensure it can't race with cpuhp */ lockdep_assert_cpus_held(); =20 + switch (r->schema_fmt) { + case RESCTRL_SCHEMA_BITMAP: + parse_ctrlval =3D &parse_cbm; + break; + case RESCTRL_SCHEMA_RANGE: + parse_ctrlval =3D &parse_bw; + break; + } + + if (WARN_ON_ONCE(!parse_ctrlval)) + return -EINVAL; + if (rdtgrp->mode =3D=3D RDT_MODE_PSEUDO_LOCKSETUP && (r->rid =3D=3D RDT_RESOURCE_MBA || r->rid =3D=3D RDT_RESOURCE_SMBA)) { rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n"); @@ -240,7 +262,7 @@ static int parse_line(char *line, struct resctrl_schema= *s, if (d->hdr.id =3D=3D dom_id) { data.buf =3D dom; data.rdtgrp =3D rdtgrp; - if (r->parse_ctrlval(&data, s, d)) + if (parse_ctrlval(&data, s, d)) return -EINVAL; if (rdtgrp->mode =3D=3D RDT_MODE_PSEUDO_LOCKSETUP) { cfg =3D &d->staged_config[t]; diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 75252a7e1ebc..b5543bd506c3 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -459,11 +459,6 @@ static inline bool is_mbm_event(int e) e <=3D QOS_L3_MBM_LOCAL_EVENT_ID); } =20 -struct rdt_parse_data { - struct rdtgroup *rdtgrp; - char *buf; -}; - /** * struct rdt_hw_resource - arch private attributes of a resctrl resource * @r_resctrl: Attributes of the resource used directly by resctrl. @@ -500,11 +495,6 @@ static inline struct rdt_hw_resource *resctrl_to_arch_= res(struct rdt_resource *r return container_of(r, struct rdt_hw_resource, r_resctrl); } =20 -int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s, - struct rdt_ctrl_domain *d); -int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s, - struct rdt_ctrl_domain *d); - extern struct mutex rdtgroup_mutex; =20 extern struct rdt_hw_resource rdt_resources_all[]; diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 496ddcaa4ecf..3e1a41356b2a 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -183,7 +183,6 @@ struct resctrl_membw { u32 *mb_map; }; =20 -struct rdt_parse_data; struct resctrl_schema; =20 enum resctrl_scope { @@ -192,6 +191,16 @@ enum resctrl_scope { RESCTRL_L3_NODE, }; =20 +/** + * enum resctrl_schema_fmt - The format user-space provides for a schema. + * @RESCTRL_SCHEMA_BITMAP: The schema is a bitmap in hex. + * @RESCTRL_SCHEMA_RANGE: The schema is a decimal number. + */ +enum resctrl_schema_fmt { + RESCTRL_SCHEMA_BITMAP, + RESCTRL_SCHEMA_RANGE, +}; + /** * struct rdt_resource - attributes of a resctrl resource * @rid: The index of the resource @@ -208,7 +217,7 @@ enum resctrl_scope { * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent. * @format_str: Per resource format string to show domain value - * @parse_ctrlval: Per resource function pointer to parse control values + * @schema_fmt: Which format string and parser is used for this schema. * @evt_list: List of monitoring events * @cdp_capable: Is the CDP feature available on this resource */ @@ -227,9 +236,7 @@ struct rdt_resource { int data_width; u32 default_ctrl; const char *format_str; - int (*parse_ctrlval)(struct rdt_parse_data *data, - struct resctrl_schema *s, - struct rdt_ctrl_domain *d); + enum resctrl_schema_fmt schema_fmt; struct list_head evt_list; bool cdp_capable; }; --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C7F7D2641F5 for ; Tue, 11 Mar 2025 18:40:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718429; cv=none; b=Gcxf/iELaCXTr6wYo0tZZ39vSB1doy0yMo160Ds4k/WLQaR4DIIBdrEAU8BX0aNopQ3TKdEeONrL5a7mnMB/JSZeUgk11Bk8uEKm97MwCdBY5htcB8p/c0wHWKH037DIDeDx4cs2/jI1yk8l/jLEfNcjjXPVS1sCLGTXVgDDxFQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718429; c=relaxed/simple; bh=Qe77bJhCDYr1i/oD+Cdlt0KCIOTR3PruKv11ZFeO4nM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OgtBO9zXMCXKTwEhGyjhyktctF+SN1MpPans7eXcaZByN4/7Q7NKSPTGJQ+zttaetscyGaMC7rqO/UO69fNCVUiyQHKc7K+E3sQit0knEBEHp3rfYjzBzfjtDZe58jPHfbcqwjbwE4X5s6ITeWud0GnQ/SyvM4E1hwOkIlf5WMc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D9D131713; Tue, 11 Mar 2025 11:40:37 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 629353F694; Tue, 11 Mar 2025 11:40:23 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 05/30] x86/resctrl: Use schema type to determine the schema format string Date: Tue, 11 Mar 2025 18:36:50 +0000 Message-Id: <20250311183715.16445-6-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Resctrl's architecture code gets to specify a format string that is used when printing schema entries. This is expected to be one of two values that the filesystem code supports. Setting this format string allows the architecture code to change the ABI resctrl presents to user-space. Instead, use the schema format enum to choose which format string to use. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Tony Luck Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v5: * Added runtime check for resource schema_format being junk. Change since v4: * Added a stop to a struct comment. Changes since v2: * This patch is new. --- arch/x86/kernel/cpu/resctrl/core.c | 4 ---- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 14 ++++++++++++++ include/linux/resctrl.h | 4 ++-- 4 files changed, 17 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index e9fe129a02f8..542503a8c953 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -73,7 +73,6 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCE= S] =3D { .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_L3), .mon_domains =3D mon_domain_init(RDT_RESOURCE_L3), .schema_fmt =3D RESCTRL_SCHEMA_BITMAP, - .format_str =3D "%d=3D%0*x", }, .msr_base =3D MSR_IA32_L3_CBM_BASE, .msr_update =3D cat_wrmsr, @@ -86,7 +85,6 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCE= S] =3D { .ctrl_scope =3D RESCTRL_L2_CACHE, .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_L2), .schema_fmt =3D RESCTRL_SCHEMA_BITMAP, - .format_str =3D "%d=3D%0*x", }, .msr_base =3D MSR_IA32_L2_CBM_BASE, .msr_update =3D cat_wrmsr, @@ -99,7 +97,6 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCE= S] =3D { .ctrl_scope =3D RESCTRL_L3_CACHE, .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_MBA), .schema_fmt =3D RESCTRL_SCHEMA_RANGE, - .format_str =3D "%d=3D%*u", }, }, [RDT_RESOURCE_SMBA] =3D @@ -110,7 +107,6 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOUR= CES] =3D { .ctrl_scope =3D RESCTRL_L3_CACHE, .ctrl_domains =3D ctrl_domain_init(RDT_RESOURCE_SMBA), .schema_fmt =3D RESCTRL_SCHEMA_RANGE, - .format_str =3D "%d=3D%*u", }, }, }; diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index f4334f437ffc..c763cb4fb1a8 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -487,7 +487,7 @@ static void show_doms(struct seq_file *s, struct resctr= l_schema *schema, int clo ctrl_val =3D resctrl_arch_get_config(r, dom, closid, schema->conf_type); =20 - seq_printf(s, r->format_str, dom->hdr.id, max_data_width, + seq_printf(s, schema->fmt_str, dom->hdr.id, max_data_width, ctrl_val); sep =3D true; } diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 3391ac8ecb2d..e7862d0936c9 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2611,6 +2611,20 @@ static int schemata_list_add(struct rdt_resource *r,= enum resctrl_conf_type type if (cl > max_name_width) max_name_width =3D cl; =20 + switch (r->schema_fmt) { + case RESCTRL_SCHEMA_BITMAP: + s->fmt_str =3D "%d=3D%0*x"; + break; + case RESCTRL_SCHEMA_RANGE: + s->fmt_str =3D "%d=3D%0*u"; + break; + } + + if (WARN_ON_ONCE(!s->fmt_str)) { + kfree(s); + return -EINVAL; + } + INIT_LIST_HEAD(&s->list); list_add(&s->list, &resctrl_schema_all); =20 diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 3e1a41356b2a..736b9a9a9464 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -216,7 +216,6 @@ enum resctrl_schema_fmt { * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent. - * @format_str: Per resource format string to show domain value * @schema_fmt: Which format string and parser is used for this schema. * @evt_list: List of monitoring events * @cdp_capable: Is the CDP feature available on this resource @@ -235,7 +234,6 @@ struct rdt_resource { char *name; int data_width; u32 default_ctrl; - const char *format_str; enum resctrl_schema_fmt schema_fmt; struct list_head evt_list; bool cdp_capable; @@ -253,6 +251,7 @@ struct rdt_resource *resctrl_arch_get_resource(enum res= ctrl_res_level l); * user-space * @list: Member of resctrl_schema_all. * @name: The name to use in the "schemata" file. + * @fmt_str: Format string to show domain value. * @conf_type: Whether this schema is specific to code/data. * @res: The resource structure exported by the architecture to describe * the hardware that is configured by this schema. @@ -263,6 +262,7 @@ struct rdt_resource *resctrl_arch_get_resource(enum res= ctrl_res_level l); struct resctrl_schema { struct list_head list; char name[8]; + const char *fmt_str; enum resctrl_conf_type conf_type; struct rdt_resource *res; u32 num_closid; --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 462B426460B for ; Tue, 11 Mar 2025 18:40:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718433; cv=none; b=s2ByI8n6HzgLgKcj+VPoNu6t7as33Dr1cN3LRC/+EDKxVA092VOm+lR9i9+GgRR02CYTX2qgQCVRsiVWFt59hF3aqM0A3mM8lwbEvpbMxwUe/NnSCPepcEvarOfEzB4Ji5+J8yF2BlzYfCimesjp5HOqm82XL1y3du833DnKi2Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718433; c=relaxed/simple; bh=HSbVP+Ni2lfqgdhhtyAl+mGcj41xi1GRTOk2mILwqKo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pZoXxJooPXniNCzQgwZUBXSiDtN0SbojOX8HHMvYq6bC8DL9wOjif4ZZc5K+CyJ1kFmqF48OtjKFS2w72swYPvCmRO/SQxUMCu0CMfvvbkMo5DRgOclnICb43TXazRQtBiaiNkD4iTMAvs2YkvZnILHfJ9mortEZwAGsncRaAtw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 934A91515; Tue, 11 Mar 2025 11:40:41 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1C42F3F694; Tue, 11 Mar 2025 11:40:27 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 06/30] x86/resctrl: Remove data_width and the tabular format Date: Tue, 11 Mar 2025 18:36:51 +0000 Message-Id: <20250311183715.16445-7-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The resctrl architecture code provides a data_width for the controls of each resource. This is used to zero pad all control values in the schemata file so they appear in columns. The same is done with the resource names to complete the visual effect. e.g. | SMBA:0=3D2048 | L3:0=3D00ff AMD platforms discover their maximum bandwidth for the MB resource from firmware, but hard-code the data_width to 4. If the maximum bandwidth requires more digits - the tabular format is silently broken. This is also broken when the mba_MBps mount option is used as the field width isn't updated. If new schema are added resctrl will need to be able to determine the maximum width. The benefit of this pretty-printing is questionable. Instead of handling runtime discovery of the data_width for AMD platforms, remove the feature. These fields are always zero padded so should be harmless to remove if the whole field has been treated as a number. In the above example, this would now look like this: | SMBA:0=3D2048 | L3:0=3Dff Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- arch/x86/kernel/cpu/resctrl/core.c | 26 ----------------------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 3 +-- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 10 +++++++-- include/linux/resctrl.h | 2 -- 5 files changed, 10 insertions(+), 33 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 542503a8c953..754fb65565ec 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -43,12 +43,6 @@ static DEFINE_MUTEX(domain_list_lock); */ DEFINE_PER_CPU(struct resctrl_pqr_state, pqr_state); =20 -/* - * Used to store the max resource name width and max resource data width - * to display the schemata in a tabular format - */ -int max_name_width, max_data_width; - /* * Global boolean for rdt_alloc which is true if any * resource allocation is enabled. @@ -228,7 +222,6 @@ static __init bool __get_mem_config_intel(struct rdt_re= source *r) return false; r->membw.arch_needs_linear =3D false; } - r->data_width =3D 3; =20 if (boot_cpu_has(X86_FEATURE_PER_THREAD_MBA)) r->membw.throttle_mode =3D THREAD_THROTTLE_PER_THREAD; @@ -269,8 +262,6 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_= resource *r) r->membw.throttle_mode =3D THREAD_THROTTLE_UNDEFINED; r->membw.min_bw =3D 0; r->membw.bw_gran =3D 1; - /* Max value is 2048, Data width should be 4 in decimal */ - r->data_width =3D 4; =20 r->alloc_capable =3D true; =20 @@ -290,7 +281,6 @@ static void rdt_get_cache_alloc_cfg(int idx, struct rdt= _resource *r) r->cache.cbm_len =3D eax.split.cbm_len + 1; r->default_ctrl =3D BIT_MASK(eax.split.cbm_len + 1) - 1; r->cache.shareable_bits =3D ebx & r->default_ctrl; - r->data_width =3D (r->cache.cbm_len + 3) / 4; if (boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_INTEL) r->cache.arch_has_sparse_bitmasks =3D ecx.split.noncont; r->alloc_capable =3D true; @@ -786,20 +776,6 @@ static int resctrl_arch_offline_cpu(unsigned int cpu) return 0; } =20 -/* - * Choose a width for the resource name and resource data based on the - * resource that has widest name and cbm. - */ -static __init void rdt_init_padding(void) -{ - struct rdt_resource *r; - - for_each_alloc_capable_rdt_resource(r) { - if (r->data_width > max_data_width) - max_data_width =3D r->data_width; - } -} - enum { RDT_FLAG_CMT, RDT_FLAG_MBM_TOTAL, @@ -1102,8 +1078,6 @@ static int __init resctrl_late_init(void) if (!get_rdt_resources()) return -ENODEV; =20 - rdt_init_padding(); - state =3D cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/resctrl/cat:online:", resctrl_arch_online_cpu, diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index c763cb4fb1a8..59610b209b4e 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -487,8 +487,7 @@ static void show_doms(struct seq_file *s, struct resctr= l_schema *schema, int clo ctrl_val =3D resctrl_arch_get_config(r, dom, closid, schema->conf_type); =20 - seq_printf(s, schema->fmt_str, dom->hdr.id, max_data_width, - ctrl_val); + seq_printf(s, schema->fmt_str, dom->hdr.id, ctrl_val); sep =3D true; } seq_puts(s, "\n"); diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index b5543bd506c3..f975cd6cfe61 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -326,7 +326,7 @@ struct rdtgroup { /* List of all resource groups */ extern struct list_head rdt_all_groups; =20 -extern int max_name_width, max_data_width; +extern int max_name_width; =20 int __init rdtgroup_init(void); void __exit rdtgroup_exit(void); diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index e7862d0936c9..1e0bae1a9d95 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -57,6 +57,12 @@ static struct kernfs_node *kn_mongrp; /* Kernel fs node for "mon_data" directory under root */ static struct kernfs_node *kn_mondata; =20 +/* + * Used to store the max resource name width to display the schemata names= in + * a tabular format. + */ +int max_name_width; + static struct seq_buf last_cmd_status; static char last_cmd_status_buf[512]; =20 @@ -2613,10 +2619,10 @@ static int schemata_list_add(struct rdt_resource *r= , enum resctrl_conf_type type =20 switch (r->schema_fmt) { case RESCTRL_SCHEMA_BITMAP: - s->fmt_str =3D "%d=3D%0*x"; + s->fmt_str =3D "%d=3D%x"; break; case RESCTRL_SCHEMA_RANGE: - s->fmt_str =3D "%d=3D%0*u"; + s->fmt_str =3D "%d=3D%u"; break; } =20 diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 736b9a9a9464..e1a982adef45 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -214,7 +214,6 @@ enum resctrl_schema_fmt { * @ctrl_domains: RCU list of all control domains for this resource * @mon_domains: RCU list of all monitor domains for this resource * @name: Name to use in "schemata" file. - * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent. * @schema_fmt: Which format string and parser is used for this schema. * @evt_list: List of monitoring events @@ -232,7 +231,6 @@ struct rdt_resource { struct list_head ctrl_domains; struct list_head mon_domains; char *name; - int data_width; u32 default_ctrl; enum resctrl_schema_fmt schema_fmt; struct list_head evt_list; --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0269B26461F for ; Tue, 11 Mar 2025 18:40:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718436; cv=none; b=Rn9RGDiG+xCBzlAwwzuxrWYth7Dx7E7Du5Fz2vsSdvlkH8kbodBh9/x0bobDK4Gt2wKop2nsARCKR3Yxys4iAY3ZekXw827c3zs2fyN7VsezDKEfUw/txhtogzfjjyoQht2KUwgFU9IndSs+ULgY99DDbEDIcP45T+5G8/bORJk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718436; c=relaxed/simple; bh=d+XK1VkzcpSrU9CPsml/xRbtYcqbV5x6iGdvspUX/1U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LAEfuC7qpErVtplgYLeAlz4CzRcPeDzZ4p9KaAl6UplZRMslzwUvH+kRkGPQDM7w8tf1j6hr0pAfhjWpwqXV8RGq1/iO9hVs3m3Ma2pZ65FMO0uw3ZGAFC2ejnCBCThQ0A4uqTJzkoKV0gu3lQkj//R3608JbFi/hYU4mBb8AQs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4F9721713; Tue, 11 Mar 2025 11:40:45 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CA5C23F694; Tue, 11 Mar 2025 11:40:30 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 07/30] x86/resctrl: Add max_bw to struct resctrl_membw Date: Tue, 11 Mar 2025 18:36:52 +0000 Message-Id: <20250311183715.16445-8-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" __rdt_get_mem_config_amd() and __get_mem_config_intel() both use the default_ctrl property as a maximum value. This is because the MBA schema works differently between these platforms. Doing this complicates determining whether the default_ctrl property belongs to the arch code, or can be derived from the schema format. Deriving the maximum or default value from the schema format would avoid the architecture code having to tell resctrl such obvious things as the maximum percentage is 100, and the maximum bitmap is all ones. Maximum bandwidth is always going to vary per platform. Add max_bw as a special case. This is currently used for the maximum MBA percentage on Intel platforms, but can be removed from the architecture code if 'percentage' becomes a schema format resctrl supports directly. This value isn't needed for other schema formats. This will allow the default_ctrl to be generated from the schema properties when it is needed. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v5: * Removed redundant setting of schema_fmt on AMD platforms. * Fixed off by one in cbm_validate(). Changes since v2: * This patch is new. --- arch/x86/kernel/cpu/resctrl/core.c | 2 ++ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 4 ++-- include/linux/resctrl.h | 2 ++ 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 754fb65565ec..4504a12efc97 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -212,6 +212,7 @@ static __init bool __get_mem_config_intel(struct rdt_re= source *r) hw_res->num_closid =3D edx.split.cos_max + 1; max_delay =3D eax.split.max_delay + 1; r->default_ctrl =3D MAX_MBA_BW; + r->membw.max_bw =3D MAX_MBA_BW; r->membw.arch_needs_linear =3D true; if (ecx & MBA_IS_LINEAR) { r->membw.delay_linear =3D true; @@ -250,6 +251,7 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_= resource *r) cpuid_count(0x80000020, subleaf, &eax, &ebx, &ecx, &edx); hw_res->num_closid =3D edx + 1; r->default_ctrl =3D 1 << eax; + r->membw.max_bw =3D 1 << eax; =20 /* AMD does not use delay */ r->membw.delay_linear =3D false; diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index 59610b209b4e..23a01eaebd58 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -63,9 +63,9 @@ static bool bw_validate(char *buf, u32 *data, struct rdt_= resource *r) return true; } =20 - if (bw < r->membw.min_bw || bw > r->default_ctrl) { + if (bw < r->membw.min_bw || bw > r->membw.max_bw) { rdt_last_cmd_printf("MB value %u out of range [%d,%d]\n", - bw, r->membw.min_bw, r->default_ctrl); + bw, r->membw.min_bw, r->membw.max_bw); return false; } =20 diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index e1a982adef45..465f3cf8c4bc 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -165,6 +165,7 @@ enum membw_throttle_mode { /** * struct resctrl_membw - Memory bandwidth allocation related data * @min_bw: Minimum memory bandwidth percentage user can request + * @max_bw: Maximum memory bandwidth value, used as the reset value * @bw_gran: Granularity at which the memory bandwidth is allocated * @delay_linear: True if memory B/W delay is in linear scale * @arch_needs_linear: True if we can't configure non-linear resources @@ -175,6 +176,7 @@ enum membw_throttle_mode { */ struct resctrl_membw { u32 min_bw; + u32 max_bw; u32 bw_gran; u32 delay_linear; bool arch_needs_linear; --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E125926461F for ; Tue, 11 Mar 2025 18:40:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718440; cv=none; b=RanyGBWhezkEfy6RviWoKM1s5L6s/g4ZIpzBOroqfa8IhQ6RUhZqkrovOCKCemrs98DN6qq5Mt2d6dAwuyKcnpFQf/3H+231p/SpCPtdve+VN1ZF13nIpRfanp+PUTr+ENzGM2rXmB5TId9Z5kJQgSZLfZ89/TTyIRn3ggfIQT8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718440; c=relaxed/simple; bh=WIMTabFq5rtpRGSKg9xj7oyKCmBUSfdS3qZ0H430xHo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lb3ZRbv1DZgC09sWP9Y1V05TA+sprFyqN5hjGmKHQAYzre5H9Lt+HeJ6Lp2idJXrWdJn+0IbgM/NItGuZDydOG/6u2KcrJCi0uCFlfD1/We6P+CDPEchatg9N+Jh0fmCZzRanSNTRb5DcRd3vMWJuxuzFtZMGPk6cbhwKNGGqiQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 31AED1762; Tue, 11 Mar 2025 11:40:49 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 982213F694; Tue, 11 Mar 2025 11:40:34 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 08/30] x86/resctrl: Generate default_ctrl instead of sharing it Date: Tue, 11 Mar 2025 18:36:53 +0000 Message-Id: <20250311183715.16445-9-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The struct rdt_resource default_ctrl is used by both the architecture code for resetting the hardware controls, and sometimes by the filesystem code as the default value for the schema, unless the bandwidth software controller is in use. Having the default exposed by the architecture code causes unnecessary duplication for each architecture as the default value must be specified, but can be derived from other schema properties. Now that the maximum bandwidth is explicitly described, resctrl can derive the default value from the schema format and the other resource properties. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger Reviewed-by: Reinette Chatre --- Changes since v7: * Dropped the hunk in cache_alloc_hsw_probe() and left it open-coded for readability. Changes since v6: * Return MBA_BW_MAX from delay_bw_map() for improved readability. * Generate the bitmap directly in rdt_get_cache_alloc_cfg() to make it clear shareable_bits is being generated correctly. Changes since v5: * Rewrote commit message. Changes since v2: * This patch is new. --- arch/x86/kernel/cpu/resctrl/core.c | 13 +++++-------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 5 +++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 6 +++--- include/linux/resctrl.h | 19 +++++++++++++++++-- 4 files changed, 28 insertions(+), 15 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 4504a12efc97..d001ca43b53d 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -155,7 +155,6 @@ static inline void cache_alloc_hsw_probe(void) return; =20 hw_res->num_closid =3D 4; - r->default_ctrl =3D max_cbm; r->cache.cbm_len =3D 20; r->cache.shareable_bits =3D 0xc0000; r->cache.min_cbm_bits =3D 2; @@ -211,7 +210,6 @@ static __init bool __get_mem_config_intel(struct rdt_re= source *r) cpuid_count(0x00000010, 3, &eax.full, &ebx, &ecx, &edx.full); hw_res->num_closid =3D edx.split.cos_max + 1; max_delay =3D eax.split.max_delay + 1; - r->default_ctrl =3D MAX_MBA_BW; r->membw.max_bw =3D MAX_MBA_BW; r->membw.arch_needs_linear =3D true; if (ecx & MBA_IS_LINEAR) { @@ -250,7 +248,6 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_= resource *r) =20 cpuid_count(0x80000020, subleaf, &eax, &ebx, &ecx, &edx); hw_res->num_closid =3D edx + 1; - r->default_ctrl =3D 1 << eax; r->membw.max_bw =3D 1 << eax; =20 /* AMD does not use delay */ @@ -276,13 +273,13 @@ static void rdt_get_cache_alloc_cfg(int idx, struct r= dt_resource *r) union cpuid_0x10_1_eax eax; union cpuid_0x10_x_ecx ecx; union cpuid_0x10_x_edx edx; - u32 ebx; + u32 ebx, default_ctrl; =20 cpuid_count(0x00000010, idx, &eax.full, &ebx, &ecx.full, &edx.full); hw_res->num_closid =3D edx.split.cos_max + 1; r->cache.cbm_len =3D eax.split.cbm_len + 1; - r->default_ctrl =3D BIT_MASK(eax.split.cbm_len + 1) - 1; - r->cache.shareable_bits =3D ebx & r->default_ctrl; + default_ctrl =3D BIT_MASK(eax.split.cbm_len + 1) - 1; + r->cache.shareable_bits =3D ebx & default_ctrl; if (boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_INTEL) r->cache.arch_has_sparse_bitmasks =3D ecx.split.noncont; r->alloc_capable =3D true; @@ -329,7 +326,7 @@ static u32 delay_bw_map(unsigned long bw, struct rdt_re= source *r) return MAX_MBA_BW - bw; =20 pr_warn_once("Non Linear delay-bw map not supported but queried\n"); - return r->default_ctrl; + return MAX_MBA_BW; } =20 static void mba_wrmsr_intel(struct msr_param *m) @@ -438,7 +435,7 @@ static void setup_default_ctrlval(struct rdt_resource *= r, u32 *dc) * For Memory Allocation: Set b/w requested to 100% */ for (i =3D 0; i < hw_res->num_closid; i++, dc++) - *dc =3D r->default_ctrl; + *dc =3D resctrl_get_default_ctrl(r); } =20 static void ctrl_domain_free(struct rdt_hw_ctrl_domain *hw_dom) diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index 23a01eaebd58..5d87f279085f 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -113,8 +113,9 @@ static int parse_bw(struct rdt_parse_data *data, struct= resctrl_schema *s, */ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r) { - unsigned long first_bit, zero_bit, val; + u32 supported_bits =3D BIT_MASK(r->cache.cbm_len) - 1; unsigned int cbm_len =3D r->cache.cbm_len; + unsigned long first_bit, zero_bit, val; int ret; =20 ret =3D kstrtoul(buf, 16, &val); @@ -123,7 +124,7 @@ static bool cbm_validate(char *buf, u32 *data, struct r= dt_resource *r) return false; } =20 - if ((r->cache.min_cbm_bits > 0 && val =3D=3D 0) || val > r->default_ctrl)= { + if ((r->cache.min_cbm_bits > 0 && val =3D=3D 0) || val > supported_bits) { rdt_last_cmd_puts("Mask out of range\n"); return false; } diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 1e0bae1a9d95..cd8f65c12124 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -978,7 +978,7 @@ static int rdt_default_ctrl_show(struct kernfs_open_fil= e *of, struct resctrl_schema *s =3D of->kn->parent->priv; struct rdt_resource *r =3D s->res; =20 - seq_printf(seq, "%x\n", r->default_ctrl); + seq_printf(seq, "%x\n", resctrl_get_default_ctrl(r)); return 0; } =20 @@ -2882,7 +2882,7 @@ static int reset_all_ctrls(struct rdt_resource *r) hw_dom =3D resctrl_to_arch_ctrl_dom(d); =20 for (i =3D 0; i < hw_res->num_closid; i++) - hw_dom->ctrl_val[i] =3D r->default_ctrl; + hw_dom->ctrl_val[i] =3D resctrl_get_default_ctrl(r); msr_param.dom =3D d; smp_call_function_any(&d->hdr.cpu_mask, rdt_ctrl_update, &msr_param, 1); } @@ -3417,7 +3417,7 @@ static void rdtgroup_init_mba(struct rdt_resource *r,= u32 closid) } =20 cfg =3D &d->staged_config[CDP_NONE]; - cfg->new_ctrl =3D r->default_ctrl; + cfg->new_ctrl =3D resctrl_get_default_ctrl(r); cfg->have_new_ctrl =3D true; } } diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 465f3cf8c4bc..d16dc960f1fc 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -216,7 +216,6 @@ enum resctrl_schema_fmt { * @ctrl_domains: RCU list of all control domains for this resource * @mon_domains: RCU list of all monitor domains for this resource * @name: Name to use in "schemata" file. - * @default_ctrl: Specifies default cache cbm or memory B/W percent. * @schema_fmt: Which format string and parser is used for this schema. * @evt_list: List of monitoring events * @cdp_capable: Is the CDP feature available on this resource @@ -233,7 +232,6 @@ struct rdt_resource { struct list_head ctrl_domains; struct list_head mon_domains; char *name; - u32 default_ctrl; enum resctrl_schema_fmt schema_fmt; struct list_head evt_list; bool cdp_capable; @@ -268,6 +266,23 @@ struct resctrl_schema { u32 num_closid; }; =20 +/** + * resctrl_get_default_ctrl() - Return the default control value for this + * resource. + * @r: The resource whose default control type is queried. + */ +static inline u32 resctrl_get_default_ctrl(struct rdt_resource *r) +{ + switch (r->schema_fmt) { + case RESCTRL_SCHEMA_BITMAP: + return BIT_MASK(r->cache.cbm_len) - 1; + case RESCTRL_SCHEMA_RANGE: + return r->membw.max_bw; + } + + return WARN_ON_ONCE(1); +} + /* The number of closid supported by this resource regardless of CDP */ u32 resctrl_arch_get_num_closid(struct rdt_resource *r); u32 resctrl_arch_system_num_rmid_idx(void); --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BA65826461F for ; Tue, 11 Mar 2025 18:40:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718445; cv=none; b=YLZsyKwvbfCezs9zpujoJctNhPr+z5zMvEBl/sjZatJxextOHGb5XbqlqOdfrYeG0awWXvkCJEuPvD49cDSFm68H5D3nZz0ihsmCYoNz+osEx7nAV5uzWA6En3Km8XsDsQO9oUwFkKUN5EUH15XLbf2x1vqxEQVdNUkKisV2Dak= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718445; c=relaxed/simple; bh=Jh41fY980JaH64LJxqAWQCuuK+qdnu0szatxbJBEOJ0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=B7HGAae+3iqDqygbadf9LZTdMIrnH0uYjzOxYiDNxjHVMh+FxIU/FKo87j8lR4eWQD0//imn92nM9jt8jjPhosfxDNTrZWz5IjSQSjhEBQw8tred1PzLM0iETXRyNdY9j21GUeGstx1/05gSyQYvZpW6biNk210K/eom/4xAS90= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0D6381515; Tue, 11 Mar 2025 11:40:53 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 69E593F694; Tue, 11 Mar 2025 11:40:38 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Dave Martin , Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 09/30] x86/resctrl: Add helper for setting CPU default properties Date: Tue, 11 Mar 2025 18:36:54 +0000 Message-Id: <20250311183715.16445-10-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" rdtgroup_rmdir_ctrl() and rdtgroup_rmdir_mon() set the per-CPU pqr_state for CPUs that were part of the rmdir()'d group. Another architecture might not have a 'pqr_state', its hardware may need the values in a different format. MPAM's equivalent of RMID values are not unique, and always need the CLOSID to be provided too. There is only one caller that modifies a single value, (rdtgroup_rmdir_mon()). MPAM always needs both CLOSID and RMID for the hardware value as these are written to the same system register. As rdtgroup_rmdir_mon() has the CLOSID on hand, only provide a helper to set both values. These values are read by __resctrl_sched_in(), but may be written by a different CPU without any locking, add READ/WRTE_ONCE() to avoid torn values. Co-developed-by: Dave Martin Signed-off-by: Dave Martin Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v1: * In rdtgroup_rmdir_mon(), (re)set CPU default closid based on the parent control group, to avoid the appearance of referencing something that we're in the process of destroying (even if it doesn't make a difference because the victim mon group necessarily has the same closid as the parent control group). Update comment to match. No (intentional) functional change. --- arch/x86/include/asm/resctrl.h | 14 +++++++++++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 20 ++++++++++++++------ 2 files changed, 25 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 8b1b6ce1e51b..6908cd0e6e40 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -4,8 +4,9 @@ =20 #ifdef CONFIG_X86_CPU_RESCTRL =20 -#include #include +#include +#include =20 /* * This value can never be a valid CLOSID, and is used when mapping a @@ -96,8 +97,8 @@ static inline void resctrl_arch_disable_mon(void) static inline void __resctrl_sched_in(struct task_struct *tsk) { struct resctrl_pqr_state *state =3D this_cpu_ptr(&pqr_state); - u32 closid =3D state->default_closid; - u32 rmid =3D state->default_rmid; + u32 closid =3D READ_ONCE(state->default_closid); + u32 rmid =3D READ_ONCE(state->default_rmid); u32 tmp; =20 /* @@ -132,6 +133,13 @@ static inline unsigned int resctrl_arch_round_mon_val(= unsigned int val) return val * scale; } =20 +static inline void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 c= losid, + u32 rmid) +{ + WRITE_ONCE(per_cpu(pqr_state.default_closid, cpu), closid); + WRITE_ONCE(per_cpu(pqr_state.default_rmid, cpu), rmid); +} + static inline void resctrl_arch_set_closid_rmid(struct task_struct *tsk, u32 closid, u32 rmid) { diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index cd8f65c12124..f706e5a288b1 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -3731,14 +3731,21 @@ static int rdtgroup_mkdir(struct kernfs_node *paren= t_kn, const char *name, static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpma= sk) { struct rdtgroup *prdtgrp =3D rdtgrp->mon.parent; + u32 closid, rmid; int cpu; =20 /* Give any tasks back to the parent group */ rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask); =20 - /* Update per cpu rmid of the moved CPUs first */ + /* + * Update per cpu closid/rmid of the moved CPUs first. + * Note: the closid will not change, but the arch code still needs it. + */ + closid =3D prdtgrp->closid; + rmid =3D prdtgrp->mon.rmid; for_each_cpu(cpu, &rdtgrp->cpu_mask) - per_cpu(pqr_state.default_rmid, cpu) =3D prdtgrp->mon.rmid; + resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid); + /* * Update the MSR on moved CPUs and CPUs which have moved * task running on them. @@ -3771,6 +3778,7 @@ static int rdtgroup_ctrl_remove(struct rdtgroup *rdtg= rp) =20 static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpm= ask) { + u32 closid, rmid; int cpu; =20 /* Give any tasks back to the default group */ @@ -3781,10 +3789,10 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdt= grp, cpumask_var_t tmpmask) &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask); =20 /* Update per cpu closid and rmid of the moved CPUs first */ - for_each_cpu(cpu, &rdtgrp->cpu_mask) { - per_cpu(pqr_state.default_closid, cpu) =3D rdtgroup_default.closid; - per_cpu(pqr_state.default_rmid, cpu) =3D rdtgroup_default.mon.rmid; - } + closid =3D rdtgroup_default.closid; + rmid =3D rdtgroup_default.mon.rmid; + for_each_cpu(cpu, &rdtgrp->cpu_mask) + resctrl_arch_set_cpu_default_closid_rmid(cpu, closid, rmid); =20 /* * Update the MSR on moved CPUs and CPUs which have moved --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7E58E264A74 for ; Tue, 11 Mar 2025 18:40:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718448; cv=none; b=RFjWx0lo2WrO1A1AwMf+8mQEk/smWDX/f4mIpxjWdYqQy+iehfbxAhw4d1Ez5olbHBRSQyCb0ZD0yI8H2ffaLWJh54m1Cr4Pd4hZWaqwwRlwFDs2JFy964HzBBTXbnlwFyDnmLhspH83I5hG5Rir+PV6TAO9atjuPNPU2uwo4s8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718448; c=relaxed/simple; bh=WIQme66ci33sDbcxxZgeKYMrkYE5Xa/hLcK+MnQWC1o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lIbjv4muPQNz1lY2lwy7wRn/cdgC+rao5Mq/DWhwS0JwoSUcF51/49WEfyPTe2O33tHE7+pZ2w6Td4QrO9NOKLNmjqyAAc77F3XfAaI/YCPx5XiSJWEI+I9PpoAiXPprSj1qxqMOYRBGsfp5CFKJknX4A6U3Yo6BNq8tX6xCcZk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D98191713; Tue, 11 Mar 2025 11:40:56 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 440D83F694; Tue, 11 Mar 2025 11:40:42 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Dave Martin , Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 10/30] x86/resctrl: Remove rdtgroup from update_cpu_closid_rmid() Date: Tue, 11 Mar 2025 18:36:55 +0000 Message-Id: <20250311183715.16445-11-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" update_cpu_closid_rmid() takes a struct rdtgroup as an argument, which it uses to update the local CPUs default pqr values. This is a problem once the resctrl parts move out to /fs/, as the arch code cannot poke around inside struct rdtgroup. Rename update_cpu_closid_rmid() as resctrl_arch_sync_cpus_defaults() to be used as the target of an IPI, and pass the effective CLOSID and RMID in a new struct. Co-developed-by: Dave Martin Signed-off-by: Dave Martin Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v1: * To clarify the meanings of the new helper and struct: Rename resctrl_arch_sync_cpu_default() to resctrl_arch_sync_cpu_closid_rmid(); Rename struct resctrl_cpu_sync to struct resctrl_cpu_defaults; Flesh out the comment block in . No functional change. --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 17 +++++++++++++---- include/linux/resctrl.h | 22 ++++++++++++++++++++++ 2 files changed, 35 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index f706e5a288b1..62d9a50c7bba 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -355,13 +355,13 @@ static int rdtgroup_cpus_show(struct kernfs_open_file= *of, * from update_closid_rmid() is protected against __switch_to() because * preemption is disabled. */ -static void update_cpu_closid_rmid(void *info) +void resctrl_arch_sync_cpu_closid_rmid(void *info) { - struct rdtgroup *r =3D info; + struct resctrl_cpu_defaults *r =3D info; =20 if (r) { this_cpu_write(pqr_state.default_closid, r->closid); - this_cpu_write(pqr_state.default_rmid, r->mon.rmid); + this_cpu_write(pqr_state.default_rmid, r->rmid); } =20 /* @@ -376,11 +376,20 @@ static void update_cpu_closid_rmid(void *info) * Update the PGR_ASSOC MSR on all cpus in @cpu_mask, * * Per task closids/rmids must have been set up before calling this functi= on. + * @r may be NULL. */ static void update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r) { - on_each_cpu_mask(cpu_mask, update_cpu_closid_rmid, r, 1); + struct resctrl_cpu_defaults defaults, *p =3D NULL; + + if (r) { + defaults.closid =3D r->closid; + defaults.rmid =3D r->mon.rmid; + p =3D &defaults; + } + + on_each_cpu_mask(cpu_mask, resctrl_arch_sync_cpu_closid_rmid, p, 1); } =20 static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask, diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index d16dc960f1fc..31808b3ddecb 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -266,6 +266,28 @@ struct resctrl_schema { u32 num_closid; }; =20 +struct resctrl_cpu_defaults { + u32 closid; + u32 rmid; +}; + +/** + * resctrl_arch_sync_cpu_closid_rmid() - Refresh this CPU's CLOSID and RMI= D. + * Call via IPI. + * @info: If non-NULL, a pointer to a struct resctrl_cpu_defaults + * specifying the new CLOSID and RMID for tasks in the default + * resctrl ctrl and mon group when running on this CPU. If NULL, + * this CPU is not re-assigned to a different default group. + * + * Propagates reassignment of CPUs and/or tasks to different resctrl groups + * when requested by the resctrl core code. + * + * This function records the per-cpu defaults specified by @info (if any), + * and then reconfigures the CPU's hardware CLOSID and RMID for subsequent + * execution based on @current, in the same way as during a task switch. + */ +void resctrl_arch_sync_cpu_closid_rmid(void *info); + /** * resctrl_get_default_ctrl() - Return the default control value for this * resource. --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5D5E0264A97 for ; Tue, 11 Mar 2025 18:40:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718452; cv=none; b=dT/DLJy/cYKaGjoq7sHhnNZdnK4hpZmbSbRLXX40fpRTQVHb6L92EpfsndByoxzHuvwNCxicDy4hO2xGD8K0PlB/JIWQorTDAKBFsG3FW07glbLmAYu9sHZ573vWjnnOS/wsp4raNKS3WoMFlI2PypNhQ9ZZw+FVJaknbOPsdz0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718452; c=relaxed/simple; bh=YHxPYlsXKkKdVm9LZYNLYL5e/+NX8DFfUf6GJU6A5Vs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Yh7yBngdv9OC42eOferL45xNpPxBf5P48Hou4M9MJ6NgZ3syBekVbMh3FXSInThwgLRwXu2Qm3x9rj7PQOwkN3zDtXHXouhd0k5lNzcpYHQOaBme0Q5AyYYjK9Icd5XCXK1RZwfs1XYSVC+iWfIPOGoOsBErUgNxu961WN+nYWk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B1A491515; Tue, 11 Mar 2025 11:41:00 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1A6FD3F694; Tue, 11 Mar 2025 11:40:45 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Dave Martin , Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 11/30] x86/resctrl: Expose resctrl fs's init function to the rest of the kernel Date: Tue, 11 Mar 2025 18:36:56 +0000 Message-Id: <20250311183715.16445-12-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" rdtgroup_init() needs exposing to the rest of the kernel so that arch code can call it once it lives in core code. As this is one of the few functions exposed, rename it to have "resctrl" in the name. The same goes for the exit call. Rename x86's arch code init functions for RDT to have an arch prefix to make it clear these are part of the architecture code. Co-developed-by: Dave Martin Signed-off-by: Dave Martin Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v5: * Tweaked the word 'export' Changes since v4: * Changed the voice of some of the commit message. Changes since v1: * Rename stale rdtgroup_init() to resctrl_init() in arch/x86/kernel/cpu/resctrl/monitor.c comments. No functional change. * [Commit message only] Minor rewording to avoid "impersonating code". * [Commit message only] Typo fix: s/to have the resctrl/to have resctrl/ in commit message. --- arch/x86/kernel/cpu/resctrl/core.c | 12 ++++++------ arch/x86/kernel/cpu/resctrl/internal.h | 3 --- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++---- include/linux/resctrl.h | 3 +++ 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index d001ca43b53d..212995149eac 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -1061,7 +1061,7 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c) } } =20 -static int __init resctrl_late_init(void) +static int __init resctrl_arch_late_init(void) { struct rdt_resource *r; int state, ret; @@ -1084,7 +1084,7 @@ static int __init resctrl_late_init(void) if (state < 0) return state; =20 - ret =3D rdtgroup_init(); + ret =3D resctrl_init(); if (ret) { cpuhp_remove_state(state); return ret; @@ -1100,18 +1100,18 @@ static int __init resctrl_late_init(void) return 0; } =20 -late_initcall(resctrl_late_init); +late_initcall(resctrl_arch_late_init); =20 -static void __exit resctrl_exit(void) +static void __exit resctrl_arch_exit(void) { struct rdt_resource *r =3D &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; =20 cpuhp_remove_state(rdt_online); =20 - rdtgroup_exit(); + resctrl_exit(); =20 if (r->mon_capable) rdt_put_mon_l3_config(); } =20 -__exitcall(resctrl_exit); +__exitcall(resctrl_arch_exit); diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index f975cd6cfe61..8291f1b59981 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -328,9 +328,6 @@ extern struct list_head rdt_all_groups; =20 extern int max_name_width; =20 -int __init rdtgroup_init(void); -void __exit rdtgroup_exit(void); - /** * struct rftype - describe each file in the resctrl file system * @name: File name diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index 58b5b21349a8..e8388d19a579 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -1027,7 +1027,7 @@ static int dom_data_init(struct rdt_resource *r) /* * RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and * are always allocated. These are used for the rdtgroup_default - * control group, which will be setup later in rdtgroup_init(). + * control group, which will be setup later in resctrl_init(). */ idx =3D resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID, RESCTRL_RESERVED_RMID); diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 62d9a50c7bba..b2dad689e780 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -4235,14 +4235,14 @@ void resctrl_offline_cpu(unsigned int cpu) } =20 /* - * rdtgroup_init - rdtgroup initialization + * resctrl_init - resctrl filesystem initialization * * Setup resctrl file system including set up root, create mount point, - * register rdtgroup filesystem, and initialize files under root directory. + * register resctrl filesystem, and initialize files under root directory. * * Return: 0 on success or -errno */ -int __init rdtgroup_init(void) +int __init resctrl_init(void) { int ret =3D 0; =20 @@ -4290,7 +4290,7 @@ int __init rdtgroup_init(void) return ret; } =20 -void __exit rdtgroup_exit(void) +void __exit resctrl_exit(void) { debugfs_remove_recursive(debugfs_resctrl); unregister_filesystem(&rdt_fs_type); diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 31808b3ddecb..f1979e375da9 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -402,4 +402,7 @@ void resctrl_arch_reset_rmid_all(struct rdt_resource *r= , struct rdt_mon_domain * extern unsigned int resctrl_rmid_realloc_threshold; extern unsigned int resctrl_rmid_realloc_limit; =20 +int __init resctrl_init(void); +void __exit resctrl_exit(void); + #endif /* _RESCTRL_H */ --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AE9B6264F83 for ; Tue, 11 Mar 2025 18:40:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718455; cv=none; b=fE/3a25TtFmaayyIuscfuyX/3ngFfvaIJYbF1bhTYy2FEDVRNrDUS6tWBvLrqbh4/5g8uO3u0Htbk8Zqv37oygSOq3mpC+cBdi7pi5VWgaYH+JfNcRomwewp0VkqR7Snd0gTDW1HrGX8zjdGQSe8URh9Co+hXJahP0ea5j/fMjo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718455; c=relaxed/simple; bh=JPc8u02/qcXj5iOAULT0TRxIFltrzTXjIAfhuc345Ck=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=j/rhQ+NcHO5E0wIHqHFi/kSwWvMwznJxTtAiBrL5G1oLgwifCzP1w0C170RvYxfkBiTvARwQl3E5VmqWBq43Z0k6aeGiNVM3Z6a/y59btfYNGxEdQA2EVckYhsDIxAey3Ev3xd2hGrs4LD4P1+i4/Rgqk03xhN/4hnzrqjUo1+k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 57EE21688; Tue, 11 Mar 2025 11:41:04 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E8ED83F694; Tue, 11 Mar 2025 11:40:49 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger Subject: [PATCH v8:for-boris 12/30] x86/resctrl: Move rdt_find_domain() to be visible to arch and fs code Date: Tue, 11 Mar 2025 18:36:57 +0000 Message-Id: <20250311183715.16445-13-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" rdt_find_domain() finds a domain given a resource and a cache-id. This is used by both the architecture code and the filesystem code. After the filesystem code moves to live in /fs/, this helper is either duplicated by all architectures, or needs exposing by the filesystem code. Add the declaration to the global header file. As its now globally visible, and has only a handful of callers, swap the 'rdt' for 'resctrl'. Move the function to live with its caller in ctrlmondata.c as the filesystem code will not have anything corresponding to core.c. Signed-off-by: James Morse Tested-by: Peter Newman Tested-by: Shaopeng Tan Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Fenghua Yu Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v7: * "definition" -> "declaration" Changes since v6: * Don't inline this helper. * Moved some words from the discussion into the commit message. Changes since v5: * This patch replaced one that split off the 'new entry to insert' behaviour. --- arch/x86/kernel/cpu/resctrl/core.c | 38 +++-------------------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 24 +++++++++++++- arch/x86/kernel/cpu/resctrl/internal.h | 2 -- include/linux/resctrl.h | 14 +++++++++ 4 files changed, 41 insertions(+), 37 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 212995149eac..850cb8643c4a 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -394,36 +394,6 @@ void rdt_ctrl_update(void *arg) hw_res->msr_update(m); } =20 -/* - * rdt_find_domain - Search for a domain id in a resource domain list. - * - * Search the domain list to find the domain id. If the domain id is - * found, return the domain. NULL otherwise. If the domain id is not - * found (and NULL returned) then the first domain with id bigger than - * the input id can be returned to the caller via @pos. - */ -struct rdt_domain_hdr *rdt_find_domain(struct list_head *h, int id, - struct list_head **pos) -{ - struct rdt_domain_hdr *d; - struct list_head *l; - - list_for_each(l, h) { - d =3D list_entry(l, struct rdt_domain_hdr, list); - /* When id is found, return its domain. */ - if (id =3D=3D d->id) - return d; - /* Stop searching when finding id's position in sorted list. */ - if (id < d->id) - break; - } - - if (pos) - *pos =3D l; - - return NULL; -} - static void setup_default_ctrlval(struct rdt_resource *r, u32 *dc) { struct rdt_hw_resource *hw_res =3D resctrl_to_arch_res(r); @@ -534,7 +504,7 @@ static void domain_add_cpu_ctrl(int cpu, struct rdt_res= ource *r) return; } =20 - hdr =3D rdt_find_domain(&r->ctrl_domains, id, &add_pos); + hdr =3D resctrl_find_domain(&r->ctrl_domains, id, &add_pos); if (hdr) { if (WARN_ON_ONCE(hdr->type !=3D RESCTRL_CTRL_DOMAIN)) return; @@ -589,7 +559,7 @@ static void domain_add_cpu_mon(int cpu, struct rdt_reso= urce *r) return; } =20 - hdr =3D rdt_find_domain(&r->mon_domains, id, &add_pos); + hdr =3D resctrl_find_domain(&r->mon_domains, id, &add_pos); if (hdr) { if (WARN_ON_ONCE(hdr->type !=3D RESCTRL_MON_DOMAIN)) return; @@ -654,7 +624,7 @@ static void domain_remove_cpu_ctrl(int cpu, struct rdt_= resource *r) return; } =20 - hdr =3D rdt_find_domain(&r->ctrl_domains, id, NULL); + hdr =3D resctrl_find_domain(&r->ctrl_domains, id, NULL); if (!hdr) { pr_warn("Can't find control domain for id=3D%d for CPU %d for resource %= s\n", id, cpu, r->name); @@ -700,7 +670,7 @@ static void domain_remove_cpu_mon(int cpu, struct rdt_r= esource *r) return; } =20 - hdr =3D rdt_find_domain(&r->mon_domains, id, NULL); + hdr =3D resctrl_find_domain(&r->mon_domains, id, NULL); if (!hdr) { pr_warn("Can't find monitor domain for id=3D%d for CPU %d for resource %= s\n", id, cpu, r->name); diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index 5d87f279085f..763317ea2256 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -610,6 +610,28 @@ int rdtgroup_mba_mbps_event_show(struct kernfs_open_fi= le *of, return ret; } =20 +struct rdt_domain_hdr *resctrl_find_domain(struct list_head *h, int id, + struct list_head **pos) +{ + struct rdt_domain_hdr *d; + struct list_head *l; + + list_for_each(l, h) { + d =3D list_entry(l, struct rdt_domain_hdr, list); + /* When id is found, return its domain. */ + if (id =3D=3D d->id) + return d; + /* Stop searching when finding id's position in sorted list. */ + if (id < d->id) + break; + } + + if (pos) + *pos =3D l; + + return NULL; +} + void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_mon_domain *d, struct rdtgroup *rdtgrp, cpumask_t *cpumask, int evtid, int first) @@ -695,7 +717,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) * This file provides data from a single domain. Search * the resource to find the domain with "domid". */ - hdr =3D rdt_find_domain(&r->mon_domains, domid, NULL); + hdr =3D resctrl_find_domain(&r->mon_domains, domid, NULL); if (!hdr || WARN_ON_ONCE(hdr->type !=3D RESCTRL_MON_DOMAIN)) { ret =3D -ENOENT; goto out; diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 8291f1b59981..da73404183da 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -581,8 +581,6 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn); int rdtgroup_kn_mode_restrict(struct rdtgroup *r, const char *name); int rdtgroup_kn_mode_restore(struct rdtgroup *r, const char *name, umode_t mask); -struct rdt_domain_hdr *rdt_find_domain(struct list_head *h, int id, - struct list_head **pos); ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off); int rdtgroup_schemata_show(struct kernfs_open_file *of, diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index f1979e375da9..93d9a435f035 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -372,6 +372,20 @@ static inline void resctrl_arch_rmid_read_context_chec= k(void) might_sleep(); } =20 +/** + * resctrl_find_domain() - Search for a domain id in a resource domain lis= t. + * @h: The domain list to search. + * @id: The domain id to search for. + * @pos: A pointer to position in the list id should be inserted. + * + * Search the domain list to find the domain id. If the domain id is + * found, return the domain. NULL otherwise. If the domain id is not + * found (and NULL returned) then the first domain with id bigger than + * the input id can be returned to the caller via @pos. + */ +struct rdt_domain_hdr *resctrl_find_domain(struct list_head *h, int id, + struct list_head **pos); + /** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid * and eventid. --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B920E264F93 for ; Tue, 11 Mar 2025 18:40:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718459; cv=none; b=Mw+gOEuGtAOzj4Mf7+6Wr2kkr3lyTTO5IrnUzfHsnNDDM/XJOd6zjd/8RTLohqwjdYYIP2bL5j2O4TBGIZShrhYlbnu8qleVZZLSiJfn0zxZigs1JksLj1hk4EBra/kYsXqpbXyI6KnC0h9jXivptrpX4y5D1P865LcNQv60A1o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718459; c=relaxed/simple; bh=616hTUR6IFZ3lVm4uiCHm6fQ4taBBoJ3UxwHzfB59l8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=j6keD+PWRorn0goNm/gooGh4BrT0DxmUgCvWdm5yzz1Mugk7uzIMsqqUnNdg3OBBsfkX6vd3z6gL+YtIm8HTOs5Bxx93N5iIyBU7QMulr1IIz8FXgMviZIv6WCkEJfjICZWlDz7t4nJXkNkKj5rPAXPh8k51qb/fBNBW0JeTJ3w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1551C1515; Tue, 11 Mar 2025 11:41:08 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9183C3F694; Tue, 11 Mar 2025 11:40:53 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 13/30] x86/resctrl: Move resctrl types to a separate header Date: Tue, 11 Mar 2025 18:36:58 +0000 Message-Id: <20250311183715.16445-14-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When resctrl is fully factored into core and per-arch code, each arch will need to use some resctrl common definitions in order to define its own specializations and helpers. Following conventional practice, it would be desirable to put the dependent arch definitions in an header that is included by the common header. However, this can make it awkward to avoid a circular dependency between and the arch header. To avoid such dependencies, move the affected common types and constants into a new header that does not need to depend on or on the arch headers. The same logic applies to the monitor-configuration defines, move these too. Some kind of enumeration for events is needed between the filesystem and architecture code. Take the x86 definition as its convenient for x86. The definition of enum resctrl_event_id is needed to allow the architecture code to define resctrl_arch_mon_ctx_alloc() and resctrl_arch_mon_ctx_free(). The definition of enum resctrl_res_level is needed to allow the architecture code to define resctrl_arch_set_cdp_enabled() and resctrl_arch_get_cdp_enabled(). The bits for mbm_local_bytes_config et al are ABI, and must be the same on all architectures. These are documented in Documentation/arch/x86/resctrl.rst The maintainers entry for these headers was missed when resctrl.h was created. Add a wildcard entry to match both resctrl.h and resctrl_types.h. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v6: * Used a different year in the boiler-plate copyright notice. Change since v3: * Added header include. * Corrected lists in the commit message. Changes since v2: * Added to the commit message why each of these things is necessary. * Moved the enum resctrl_conf_type back to resctrl.h - this week arm's CDP emulation code gets away without this... Changes since v1: * [Commit message only] Rewrite commit message to clarify the the rationale for refactoring the headers in this way. --- MAINTAINERS | 1 + arch/x86/include/asm/resctrl.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 24 ------------ include/linux/resctrl.h | 21 +--------- include/linux/resctrl_types.h | 54 ++++++++++++++++++++++++++ 5 files changed, 57 insertions(+), 44 deletions(-) create mode 100644 include/linux/resctrl_types.h diff --git a/MAINTAINERS b/MAINTAINERS index efee40ea589f..3487ac429a9a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -19881,6 +19881,7 @@ S: Supported F: Documentation/arch/x86/resctrl* F: arch/x86/include/asm/resctrl.h F: arch/x86/kernel/cpu/resctrl/ +F: include/linux/resctrl*.h F: tools/testing/selftests/resctrl/ =20 READ-COPY UPDATE (RCU) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 6908cd0e6e40..52f2326e2b1e 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -6,6 +6,7 @@ =20 #include #include +#include #include =20 /* diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index da73404183da..5f3713fb2eaf 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -32,30 +32,6 @@ */ #define MBM_CNTR_WIDTH_OFFSET_MAX (62 - MBM_CNTR_WIDTH_BASE) =20 -/* Reads to Local DRAM Memory */ -#define READS_TO_LOCAL_MEM BIT(0) - -/* Reads to Remote DRAM Memory */ -#define READS_TO_REMOTE_MEM BIT(1) - -/* Non-Temporal Writes to Local Memory */ -#define NON_TEMP_WRITE_TO_LOCAL_MEM BIT(2) - -/* Non-Temporal Writes to Remote Memory */ -#define NON_TEMP_WRITE_TO_REMOTE_MEM BIT(3) - -/* Reads to Local Memory the system identifies as "Slow Memory" */ -#define READS_TO_LOCAL_S_MEM BIT(4) - -/* Reads to Remote Memory the system identifies as "Slow Memory" */ -#define READS_TO_REMOTE_S_MEM BIT(5) - -/* Dirty Victims to All Types of Memory */ -#define DIRTY_VICTIMS_TO_ALL_MEM BIT(6) - -/* Max event bits supported */ -#define MAX_EVT_CONFIG_BITS GENMASK(6, 0) - /** * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those = that * aren't marked nohz_full diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 93d9a435f035..326f7ba220e7 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -6,6 +6,7 @@ #include #include #include +#include =20 /* CLOSID, RMID value used by the default control group */ #define RESCTRL_RESERVED_CLOSID 0 @@ -37,28 +38,8 @@ enum resctrl_conf_type { CDP_DATA, }; =20 -enum resctrl_res_level { - RDT_RESOURCE_L3, - RDT_RESOURCE_L2, - RDT_RESOURCE_MBA, - RDT_RESOURCE_SMBA, - - /* Must be the last */ - RDT_NUM_RESOURCES, -}; - #define CDP_NUM_TYPES (CDP_DATA + 1) =20 -/* - * Event IDs, the values match those used to program IA32_QM_EVTSEL before - * reading IA32_QM_CTR on RDT systems. - */ -enum resctrl_event_id { - QOS_L3_OCCUP_EVENT_ID =3D 0x01, - QOS_L3_MBM_TOTAL_EVENT_ID =3D 0x02, - QOS_L3_MBM_LOCAL_EVENT_ID =3D 0x03, -}; - /** * struct resctrl_staged_config - parsed configuration to be applied * @new_ctrl: new ctrl value to be loaded diff --git a/include/linux/resctrl_types.h b/include/linux/resctrl_types.h new file mode 100644 index 000000000000..f26450b3326b --- /dev/null +++ b/include/linux/resctrl_types.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2025 Arm Ltd. + * Based on arch/x86/kernel/cpu/resctrl/internal.h + */ + +#ifndef __LINUX_RESCTRL_TYPES_H +#define __LINUX_RESCTRL_TYPES_H + +/* Reads to Local DRAM Memory */ +#define READS_TO_LOCAL_MEM BIT(0) + +/* Reads to Remote DRAM Memory */ +#define READS_TO_REMOTE_MEM BIT(1) + +/* Non-Temporal Writes to Local Memory */ +#define NON_TEMP_WRITE_TO_LOCAL_MEM BIT(2) + +/* Non-Temporal Writes to Remote Memory */ +#define NON_TEMP_WRITE_TO_REMOTE_MEM BIT(3) + +/* Reads to Local Memory the system identifies as "Slow Memory" */ +#define READS_TO_LOCAL_S_MEM BIT(4) + +/* Reads to Remote Memory the system identifies as "Slow Memory" */ +#define READS_TO_REMOTE_S_MEM BIT(5) + +/* Dirty Victims to All Types of Memory */ +#define DIRTY_VICTIMS_TO_ALL_MEM BIT(6) + +/* Max event bits supported */ +#define MAX_EVT_CONFIG_BITS GENMASK(6, 0) + +enum resctrl_res_level { + RDT_RESOURCE_L3, + RDT_RESOURCE_L2, + RDT_RESOURCE_MBA, + RDT_RESOURCE_SMBA, + + /* Must be the last */ + RDT_NUM_RESOURCES, +}; + +/* + * Event IDs, the values match those used to program IA32_QM_EVTSEL before + * reading IA32_QM_CTR on RDT systems. + */ +enum resctrl_event_id { + QOS_L3_OCCUP_EVENT_ID =3D 0x01, + QOS_L3_MBM_TOTAL_EVENT_ID =3D 0x02, + QOS_L3_MBM_LOCAL_EVENT_ID =3D 0x03, +}; + +#endif /* __LINUX_RESCTRL_TYPES_H */ --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3BC1B264FAC for ; Tue, 11 Mar 2025 18:41:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718463; cv=none; b=abEGlEPVnWmlO6TGnkvGgW58HhoMp57+GM++z0YcM9owQ/R3p8p0NvEf/E4WtJ3dSWc79WcozFKlpeYIJAa3oeVVdrJEW23NhtAcTxUripxqUlBw/zyvn38sExEdPEjnKdZEKJcK7oNCop9PWKZ59c75tAYafLvMEwN18BiPF9Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718463; c=relaxed/simple; bh=cn7IbwGIVru5bQpzJ+V2/pCEH9PomrzZzqxWji3R2dw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R0cdd3bRdBNZ+y8tqTe55NG9tvxOrjaT+iuWgt6pg7NFjMiGBoi1TSQn7DZZu23NXYAKPaQKGisxLItD3FrI5G/DaRIKHV/5FseN7POzCvytCgQKAEqc1SgwREmc+OOjWY5mSe/gO/Ys4E6R5/1l5Y8qduIP0nu2/ym12UYAVi8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C594E1713; Tue, 11 Mar 2025 11:41:11 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4CE673F694; Tue, 11 Mar 2025 11:40:57 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 14/30] x86/resctrl: Add an arch helper to reset one resource Date: Tue, 11 Mar 2025 18:36:59 +0000 Message-Id: <20250311183715.16445-15-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" On umount(), resctrl resets each resource back to its default configuration. It only ever does this for all resources in one go. reset_all_ctrls() is architecture specific as it works with struct rdt_hw_resource. Make reset_all_ctrls() an arch helper that resets one resource. Signed-off-by: James Morse Tested-by: Peter Newman Tested-by: Shaopeng Tan Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v5: * Move the arch/fs split into the for-each loop at Reinette's suggestion. * Dropped a bunch of tags and rewrote the commit message. Changes since v1: * Rename the for_each_capable_rdt_resource() introduced in the new function resctrl_arch_reset_resources(), back to for_each_alloc_capable_rdt_resource() as it was in the original code. The change looked unintentional; and presumably a resource that does not support resource allocation doesn't have any properties to reset... --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 9 +++++---- include/linux/resctrl.h | 9 +++++++++ 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index b2dad689e780..9eb57ebb36c6 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2867,7 +2867,7 @@ static int rdt_init_fs_context(struct fs_context *fc) return 0; } =20 -static int reset_all_ctrls(struct rdt_resource *r) +void resctrl_arch_reset_all_ctrls(struct rdt_resource *r) { struct rdt_hw_resource *hw_res =3D resctrl_to_arch_res(r); struct rdt_hw_ctrl_domain *hw_dom; @@ -2896,7 +2896,7 @@ static int reset_all_ctrls(struct rdt_resource *r) smp_call_function_any(&d->hdr.cpu_mask, rdt_ctrl_update, &msr_param, 1); } =20 - return 0; + return; } =20 /* @@ -3015,9 +3015,10 @@ static void rdt_kill_sb(struct super_block *sb) =20 rdt_disable_ctx(); =20 - /*Put everything back to default values. */ + /* Put everything back to default values. */ for_each_alloc_capable_rdt_resource(r) - reset_all_ctrls(r); + resctrl_arch_reset_all_ctrls(r); + rmdir_all_sub(); rdt_pseudo_lock_release(); rdtgroup_default.mode =3D RDT_MODE_SHAREABLE; diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 326f7ba220e7..487b9657a7b5 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -394,6 +394,15 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, s= truct rdt_mon_domain *d, */ void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_mon_do= main *d); =20 +/** + * resctrl_arch_reset_all_ctrls() - Reset the control for each CLOSID to i= ts + * default. + * @r: The resctrl resource to reset. + * + * This can be called from any CPU. + */ +void resctrl_arch_reset_all_ctrls(struct rdt_resource *r); + extern unsigned int resctrl_rmid_realloc_threshold; extern unsigned int resctrl_rmid_realloc_limit; =20 --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 315E6265607 for ; Tue, 11 Mar 2025 18:41:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718466; cv=none; b=Yb1qhvH7cxJVEujiMLy5IWjgSaa2pgDz7hjBuMrBjYVVsRfXlTOD+zzMFLCQxVXIO7sPKRRm2i6lQAUL18aWU6HFHlXLr17yQ1KGMuxjk7fMUqw8s7mDyQdnObKGwLX4dopfBeFei8vyzmrfDmxGazz4ZfIt4A0jc0JW5COO0j8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718466; c=relaxed/simple; bh=WRvNjBYZAwW+LCLMPw/GAZCNoSYgLJpWTWgNCfzCwK0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AurW3sLCrf8aHD7tLaz+ss2r8WR0gfg7RXrvwoDLvW9JRzuJ9GJjvaq7aw01q5PG9Ti8mmGIhkH4jjgFOZ80sKupZrwOsyYxOCr0FSfUvm+Xa5zsBt7RbYbwKPpdKUlDl6gK595DBi8lKRRTB12IqLwso+893gMyK8fxZ2xig14= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7F6BE1515; Tue, 11 Mar 2025 11:41:15 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 079253F694; Tue, 11 Mar 2025 11:41:00 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 15/30] x86/resctrl: Move monitor exit work to a resctrl exit call Date: Tue, 11 Mar 2025 18:37:00 +0000 Message-Id: <20250311183715.16445-16-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" rdt_put_mon_l3_config() is called via the architecture's resctrl_arch_exit() call, and appears to free the rmid_ptrs[] and closid_num_dirty_rmid[] arrays. In reality this code is marked __exit, and is removed by the linker as resctrl can't be built as a module. To separate the filesystem and architecture parts of resctrl, this free()ing work needs to be triggered by the filesystem, as these structures belong to the filesystem code. Rename rdt_put_mon_l3_config() resctrl_mon_resource_exit() and call it from resctrl_exit(). The kfree() is currently dependent on r->mon_capable. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v4: * Added __exit so it can be removed in the next patch. Changes since v3: * Moved r->mon_capable check under the lock. * Dropped references to resctrl_mon_resource_init() from the commit messag= e. * Fixed more resctrl typos, Changes since v2: * Dropped __exit as needed in the next patch. Change since v1: * [Commit message only] Typo fixes: s/restrl/resctrl/g s/resctl/resctrl/g * [Commit message only] Reword second paragraph to remove reference to the MPAM error interrupt, which provides background rationale for a later patch rather than for this patch, and so it is not really relevant here. --- arch/x86/kernel/cpu/resctrl/core.c | 5 ----- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 12 +++++++++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 2 ++ 4 files changed, 12 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 850cb8643c4a..a1577bdd3c90 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -1074,14 +1074,9 @@ late_initcall(resctrl_arch_late_init); =20 static void __exit resctrl_arch_exit(void) { - struct rdt_resource *r =3D &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; - cpuhp_remove_state(rdt_online); =20 resctrl_exit(); - - if (r->mon_capable) - rdt_put_mon_l3_config(); } =20 __exitcall(resctrl_arch_exit); diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 5f3713fb2eaf..73005ca2dda1 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -586,7 +586,7 @@ void closid_free(int closid); int alloc_rmid(u32 closid); void free_rmid(u32 closid, u32 rmid); int rdt_get_mon_l3_config(struct rdt_resource *r); -void __exit rdt_put_mon_l3_config(void); +void __exit resctrl_mon_resource_exit(void); bool __init rdt_cpu_has(int flag); void mon_event_count(void *info); int rdtgroup_mondata_show(struct seq_file *m, void *arg); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index e8388d19a579..15e8c0190bfc 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -1040,10 +1040,13 @@ static int dom_data_init(struct rdt_resource *r) return err; } =20 -static void __exit dom_data_exit(void) +static void __exit dom_data_exit(struct rdt_resource *r) { mutex_lock(&rdtgroup_mutex); =20 + if (!r->mon_capable) + goto out_unlock; + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { kfree(closid_num_dirty_rmid); closid_num_dirty_rmid =3D NULL; @@ -1052,6 +1055,7 @@ static void __exit dom_data_exit(void) kfree(rmid_ptrs); rmid_ptrs =3D NULL; =20 +out_unlock: mutex_unlock(&rdtgroup_mutex); } =20 @@ -1237,9 +1241,11 @@ int __init rdt_get_mon_l3_config(struct rdt_resource= *r) return 0; } =20 -void __exit rdt_put_mon_l3_config(void) +void __exit resctrl_mon_resource_exit(void) { - dom_data_exit(); + struct rdt_resource *r =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); + + dom_data_exit(r); } =20 void __init intel_rdt_mbm_apply_quirk(void) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 9eb57ebb36c6..42c48e79364d 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -4296,4 +4296,6 @@ void __exit resctrl_exit(void) debugfs_remove_recursive(debugfs_resctrl); unregister_filesystem(&rdt_fs_type); sysfs_remove_mount_point(fs_kobj, "resctrl"); + + resctrl_mon_resource_exit(); } --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DC7FA264FB6 for ; Tue, 11 Mar 2025 18:41:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718470; cv=none; b=cKgMd3wzelr7xvyvpArPVOBcHMNpen9UVh6/gdvR34TaT3zLy2Dvhkja3S/IUOnjmyGeQgseTPyJrz6qL347Z4JsBpaGpxrlGQIsxJGfmUG8GzId3c2gN0ZDKiq/8+MsbRbhx+dFrmOfdVtZYHIxO5XUsUHwze0F0Z81RmY/dPs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718470; c=relaxed/simple; bh=BxaOXiYTCpQ8K0AVvR3TF1n+dFv7Kl5C/tquLcrcK5I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QCTTYY1cqJZFgSlRATikbWl0LJXdiDBJ4GhzwKFwiMo4hNIkrOkgfaDyynbvSJduuyh/4rH4ApjxbwYD4pE5YnCy/lzXzsH8fTQbtYw83W5Xbn+1mlb73yihdjhMildqF1Op+1d/7HJ5ItlRbSzRr2Um4ox6XzqqjgLn75mwq3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 39E871688; Tue, 11 Mar 2025 11:41:19 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B63513F694; Tue, 11 Mar 2025 11:41:04 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 16/30] x86/resctrl: Move monitor init work to a resctrl init call Date: Tue, 11 Mar 2025 18:37:01 +0000 Message-Id: <20250311183715.16445-17-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" rdt_get_mon_l3_config() is called from the architecture's resctrl_arch_late_init(), and initialises both architecture specific fields, such as hw_res->mon_scale and resctrl filesystem fields by calling dom_data_init(). To separate the filesystem and architecture parts of resctrl, this function needs splitting up. Add resctrl_mon_resource_init() to do the filesystem specific work, and call it from resctrl_init(). This runs later, but is still before the filesystem is mounted and the rmid_ptrs[] array can be used. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v4: * Removed __exit markers Changes since v3: * Added a comment over resctrl_mon_resource_init(). * Added a comment over domain_setup_mon_state() to warn of cpuhp ordering. * Added __init to resctrl_mon_resource_init(). Changes since v2: * Added error handling for the case sysfs files can't be created. --- arch/x86/kernel/cpu/resctrl/internal.h | 3 +- arch/x86/kernel/cpu/resctrl/monitor.c | 40 ++++++++++++++++++++------ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 22 +++++++++++++- 3 files changed, 54 insertions(+), 11 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 73005ca2dda1..70fbb902e85e 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -586,13 +586,14 @@ void closid_free(int closid); int alloc_rmid(u32 closid); void free_rmid(u32 closid, u32 rmid); int rdt_get_mon_l3_config(struct rdt_resource *r); -void __exit resctrl_mon_resource_exit(void); +void resctrl_mon_resource_exit(void); bool __init rdt_cpu_has(int flag); void mon_event_count(void *info); int rdtgroup_mondata_show(struct seq_file *m, void *arg); void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_mon_domain *d, struct rdtgroup *rdtgrp, cpumask_t *cpumask, int evtid, int first); +int __init resctrl_mon_resource_init(void); void mbm_setup_overflow_handler(struct rdt_mon_domain *dom, unsigned long delay_ms, int exclude_cpu); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index 15e8c0190bfc..1730ba814834 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -1040,7 +1040,7 @@ static int dom_data_init(struct rdt_resource *r) return err; } =20 -static void __exit dom_data_exit(struct rdt_resource *r) +static void dom_data_exit(struct rdt_resource *r) { mutex_lock(&rdtgroup_mutex); =20 @@ -1176,12 +1176,40 @@ static __init int snc_get_config(void) return ret; } =20 +/** + * resctrl_mon_resource_init() - Initialise global monitoring structures. + * + * Allocate and initialise global monitor resources that do not belong to a + * specific domain. i.e. the rmid_ptrs[] used for the limbo and free lists. + * Called once during boot after the struct rdt_resource's have been confi= gured + * but before the filesystem is mounted. + * Resctrl's cpuhp callbacks may be called before this point to bring a do= main + * online. + * + * Returns 0 for success, or -ENOMEM. + */ +int __init resctrl_mon_resource_init(void) +{ + struct rdt_resource *r =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); + int ret; + + if (!r->mon_capable) + return 0; + + ret =3D dom_data_init(r); + if (ret) + return ret; + + l3_mon_evt_init(r); + + return 0; +} + int __init rdt_get_mon_l3_config(struct rdt_resource *r) { unsigned int mbm_offset =3D boot_cpu_data.x86_cache_mbm_width_offset; struct rdt_hw_resource *hw_res =3D resctrl_to_arch_res(r); unsigned int threshold; - int ret; =20 snc_nodes_per_l3_cache =3D snc_get_config(); =20 @@ -1211,10 +1239,6 @@ int __init rdt_get_mon_l3_config(struct rdt_resource= *r) */ resctrl_rmid_realloc_threshold =3D resctrl_arch_round_mon_val(threshold); =20 - ret =3D dom_data_init(r); - if (ret) - return ret; - if (rdt_cpu_has(X86_FEATURE_BMEC)) { u32 eax, ebx, ecx, edx; =20 @@ -1234,14 +1258,12 @@ int __init rdt_get_mon_l3_config(struct rdt_resourc= e *r) } } =20 - l3_mon_evt_init(r); - r->mon_capable =3D true; =20 return 0; } =20 -void __exit resctrl_mon_resource_exit(void) +void resctrl_mon_resource_exit(void) { struct rdt_resource *r =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); =20 diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 42c48e79364d..badac3f5da72 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -4102,6 +4102,19 @@ void resctrl_offline_mon_domain(struct rdt_resource = *r, struct rdt_mon_domain *d mutex_unlock(&rdtgroup_mutex); } =20 +/** + * domain_setup_mon_state() - Initialise domain monitoring structures. + * @r: The resource for the newly online domain. + * @d: The newly online domain. + * + * Allocate monitor resources that belong to this domain. + * Called when the first CPU of a domain comes online, regardless of wheth= er + * the filesystem is mounted. + * During boot this may be called before global allocations have been made= by + * resctrl_mon_resource_init(). + * + * Returns 0 for success, or -ENOMEM. + */ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_mon_d= omain *d) { u32 idx_limit =3D resctrl_arch_system_num_rmid_idx(); @@ -4252,9 +4265,15 @@ int __init resctrl_init(void) =20 rdtgroup_setup_default(); =20 + ret =3D resctrl_mon_resource_init(); + if (ret) + return ret; + ret =3D sysfs_create_mount_point(fs_kobj, "resctrl"); - if (ret) + if (ret) { + resctrl_mon_resource_exit(); return ret; + } =20 ret =3D register_filesystem(&rdt_fs_type); if (ret) @@ -4287,6 +4306,7 @@ int __init resctrl_init(void) =20 cleanup_mountpoint: sysfs_remove_mount_point(fs_kobj, "resctrl"); + resctrl_mon_resource_exit(); =20 return ret; } --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4CA6E2641FE for ; Tue, 11 Mar 2025 18:41:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718474; cv=none; b=aGbo8ImfVojgYROGmy9QPhV+pKAigeYEPoe4JOYoZfbPskpAK2WmfzoSqhI7k/o2Hs46QwrekE/lFHGvW35sVZ2jD75aW+hUDSd79NT1J1RmECfAosvAS7lNgoFmk4N/RBVIZr5QCeA4ockH3otVAi2LwrPn3xKD41Wursvh5sc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718474; c=relaxed/simple; bh=9EDGNR8q/Byw6s/pdROjytw0PMfwTkAveK7IuHkf4XA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tVa/Rds7FZMc9POShYjngOegGG7RJHT/MSxnymlq/8Eag9Xj8JX/Ar/KsrLPCOkvM3D5aqRRmnwr+XQQMrlHXb6CSd1AK7pwsBA42BkVN3ZmlBMB7ZqokKk02uDzDxZviRrKmCmbVeGg1gRmSwX7Qf9jvJXM4WU6SCYQhEU/OiM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E8F2B1515; Tue, 11 Mar 2025 11:41:22 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 713D23F694; Tue, 11 Mar 2025 11:41:08 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 17/30] x86/resctrl: Rewrite and move the for_each_*_rdt_resource() walkers Date: Tue, 11 Mar 2025 18:37:02 +0000 Message-Id: <20250311183715.16445-18-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The for_each_*_rdt_resource() helpers walk the architecture's array of structures, using the resctrl visible part as an iterator. These became over-complex when the structures were split into a filesystem and architecture-specific struct. This approach avoided the need to touch every call site, and was done before there was a helper to retrieve a resource by rid. Once the filesystem parts of resctrl are moved to /fs/, both the architecture's resource array, and the definition of those structures is no longer accessible. To support resctrl, each architecture would have to provide equally complex macros. Rewrite the macro to make use of resctrl_arch_get_resource(), and move these to include/linux/resctrl.h so existing x86 arch code continues to use them. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v5: * Fixed an off by one in for_each_rdt_resource(). * Added header file path to commit message. Changes since v3: * Restructure the existing macros instead of open-coding the for loop. Changes since v1: * [Whitespace only] Fix bogus whitespace introduced in rdtgroup_create_info_dir(). * [Commit message only] Typo fix: s/architectures/architecture's/g --- arch/x86/kernel/cpu/resctrl/internal.h | 29 -------------------------- include/linux/resctrl.h | 18 ++++++++++++++++ 2 files changed, 18 insertions(+), 29 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 70fbb902e85e..82dbc1606663 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -475,14 +475,6 @@ extern struct rdtgroup rdtgroup_default; extern struct dentry *debugfs_resctrl; extern enum resctrl_event_id mba_mbps_default_event; =20 -static inline struct rdt_resource *resctrl_inc(struct rdt_resource *res) -{ - struct rdt_hw_resource *hw_res =3D resctrl_to_arch_res(res); - - hw_res++; - return &hw_res->r_resctrl; -} - static inline bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level l) { return rdt_resources_all[l].cdp_enabled; @@ -492,27 +484,6 @@ int resctrl_arch_set_cdp_enabled(enum resctrl_res_leve= l l, bool enable); =20 void arch_mon_domain_online(struct rdt_resource *r, struct rdt_mon_domain = *d); =20 -/* - * To return the common struct rdt_resource, which is contained in struct - * rdt_hw_resource, walk the resctrl member of struct rdt_hw_resource. - */ -#define for_each_rdt_resource(r) \ - for (r =3D &rdt_resources_all[0].r_resctrl; \ - r <=3D &rdt_resources_all[RDT_NUM_RESOURCES - 1].r_resctrl; \ - r =3D resctrl_inc(r)) - -#define for_each_capable_rdt_resource(r) \ - for_each_rdt_resource(r) \ - if (r->alloc_capable || r->mon_capable) - -#define for_each_alloc_capable_rdt_resource(r) \ - for_each_rdt_resource(r) \ - if (r->alloc_capable) - -#define for_each_mon_capable_rdt_resource(r) \ - for_each_rdt_resource(r) \ - if (r->mon_capable) - /* CPUID.(EAX=3D10H, ECX=3DResID=3D1).EAX */ union cpuid_0x10_1_eax { struct { diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 487b9657a7b5..a392480dc4b6 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -26,6 +26,24 @@ int proc_resctrl_show(struct seq_file *m, /* max value for struct rdt_domain's mbps_val */ #define MBA_MAX_MBPS U32_MAX =20 +/* Walk all possible resources, with variants for only controls or monitor= s. */ +#define for_each_rdt_resource(_r) \ + for ((_r) =3D resctrl_arch_get_resource(0); \ + (_r) && (_r)->rid < RDT_NUM_RESOURCES; \ + (_r) =3D resctrl_arch_get_resource((_r)->rid + 1)) + +#define for_each_capable_rdt_resource(r) \ + for_each_rdt_resource((r)) \ + if ((r)->alloc_capable || (r)->mon_capable) + +#define for_each_alloc_capable_rdt_resource(r) \ + for_each_rdt_resource((r)) \ + if ((r)->alloc_capable) + +#define for_each_mon_capable_rdt_resource(r) \ + for_each_rdt_resource((r)) \ + if ((r)->mon_capable) + /** * enum resctrl_conf_type - The type of configuration. * @CDP_NONE: No prioritisation, both code and data are controlled or moni= tored. --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 84FBE264607 for ; Tue, 11 Mar 2025 18:41:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718478; cv=none; b=LyK4a0XhL6DmMebOCzGBo0j3ItpO/8T+e4BeLUOOC6UFeXBt7udo5/eqi+mY8CNzZw7qI+Y/cV7Qmn54CoKmIjiLoN6nJ6ZXOy07FInHU849fH8+aNWeiysLXj87Az0L6+g0DLZNmyc1H3hFDgSRrKxg9rlKS9eS5VhS3bhEuCg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718478; c=relaxed/simple; bh=vzAOU1+Uua1fAcqQVOnqVtgkq48bYqSpCRoD32sPdgk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GLGJVyVnVmH+i2Pzp3YqkWlDz1JR186HaVCEkCeW48MZWpm+DjjiND+2T2QEbL9Svdq0yvwB5BhGLGQzEMNyFkcLMAP52f8jCy+xMJN03mIlaFt015ah1uwZI1fji1IwCVx0Oo4ejLPUwVmR3mlabZXXCMq0pOyKC+xiKv/d8/E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BE80A1688; Tue, 11 Mar 2025 11:41:26 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2C1C63F694; Tue, 11 Mar 2025 11:41:12 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 18/30] x86/resctrl: Move the is_mbm_*_enabled() helpers to asm/resctrl.h Date: Tue, 11 Mar 2025 18:37:03 +0000 Message-Id: <20250311183715.16445-19-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The architecture specific parts of resctrl provide helpers like is_mbm_total_enabled() and is_mbm_local_enabled() to hide accesses to the rdt_mon_features bitmap. Exposing a group of helpers between the architecture and filesystem code is preferable to a single unsigned-long like rdt_mon_features. Helpers can be more readable and have a well defined behaviour, while allowing architectures to hide more complex behaviour. Once the filesystem parts of resctrl are moved, these existing helpers can no longer live in internal.h. Move them to include/linux/resctrl.h Once these are exposed to the wider kernel, they should have a 'resctrl_arch_' prefix, to fit the rest of the arch<->fs interface. Move and rename the helpers that touch rdt_mon_features directly. is_mbm_event() and is_mbm_enabled() are only called from rdtgroup.c, so can be moved into that file. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Tony Luck Reviewed-by: Fenghua Yu Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Chagnes since v6: * Dropped a spurious check re-introduced by a bad rebase conflict. Changes since v5: * Removed the word 'export' due to its kernel-specific meaning. * Reworded commit message. --- arch/x86/include/asm/resctrl.h | 16 +++++++++ arch/x86/kernel/cpu/resctrl/core.c | 8 ++--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 4 +-- arch/x86/kernel/cpu/resctrl/internal.h | 27 --------------- arch/x86/kernel/cpu/resctrl/monitor.c | 16 ++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 40 +++++++++++++++-------- 6 files changed, 56 insertions(+), 55 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 52f2326e2b1e..6d4c7ea2c9e3 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -42,6 +42,7 @@ DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state); =20 extern bool rdt_alloc_capable; extern bool rdt_mon_capable; +extern unsigned int rdt_mon_features; =20 DECLARE_STATIC_KEY_FALSE(rdt_enable_key); DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key); @@ -81,6 +82,21 @@ static inline void resctrl_arch_disable_mon(void) static_branch_dec_cpuslocked(&rdt_enable_key); } =20 +static inline bool resctrl_arch_is_llc_occupancy_enabled(void) +{ + return (rdt_mon_features & (1 << QOS_L3_OCCUP_EVENT_ID)); +} + +static inline bool resctrl_arch_is_mbm_total_enabled(void) +{ + return (rdt_mon_features & (1 << QOS_L3_MBM_TOTAL_EVENT_ID)); +} + +static inline bool resctrl_arch_is_mbm_local_enabled(void) +{ + return (rdt_mon_features & (1 << QOS_L3_MBM_LOCAL_EVENT_ID)); +} + /* * __resctrl_sched_in() - Writes the task's CLOSid/RMID to IA32_PQR_MSR * diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index a1577bdd3c90..eba210db4a7b 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -453,13 +453,13 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct= rdt_hw_mon_domain *hw_dom) { size_t tsize; =20 - if (is_mbm_total_enabled()) { + if (resctrl_arch_is_mbm_total_enabled()) { tsize =3D sizeof(*hw_dom->arch_mbm_total); hw_dom->arch_mbm_total =3D kcalloc(num_rmid, tsize, GFP_KERNEL); if (!hw_dom->arch_mbm_total) return -ENOMEM; } - if (is_mbm_local_enabled()) { + if (resctrl_arch_is_mbm_local_enabled()) { tsize =3D sizeof(*hw_dom->arch_mbm_local); hw_dom->arch_mbm_local =3D kcalloc(num_rmid, tsize, GFP_KERNEL); if (!hw_dom->arch_mbm_local) { @@ -908,9 +908,9 @@ static __init bool get_rdt_mon_resources(void) if (!rdt_mon_features) return false; =20 - if (is_mbm_local_enabled()) + if (resctrl_arch_is_mbm_local_enabled()) mba_mbps_default_event =3D QOS_L3_MBM_LOCAL_EVENT_ID; - else if (is_mbm_total_enabled()) + else if (resctrl_arch_is_mbm_total_enabled()) mba_mbps_default_event =3D QOS_L3_MBM_TOTAL_EVENT_ID; =20 return !rdt_get_mon_l3_config(r); diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index 763317ea2256..1ecc93282b7d 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -559,12 +559,12 @@ ssize_t rdtgroup_mba_mbps_event_write(struct kernfs_o= pen_file *of, rdt_last_cmd_clear(); =20 if (!strcmp(buf, "mbm_local_bytes")) { - if (is_mbm_local_enabled()) + if (resctrl_arch_is_mbm_local_enabled()) rdtgrp->mba_mbps_event =3D QOS_L3_MBM_LOCAL_EVENT_ID; else ret =3D -EINVAL; } else if (!strcmp(buf, "mbm_total_bytes")) { - if (is_mbm_total_enabled()) + if (resctrl_arch_is_mbm_total_enabled()) rdtgrp->mba_mbps_event =3D QOS_L3_MBM_TOTAL_EVENT_ID; else ret =3D -EINVAL; diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 82dbc1606663..4a5996d1e060 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -156,7 +156,6 @@ struct rmid_read { void *arch_mon_ctx; }; =20 -extern unsigned int rdt_mon_features; extern struct list_head resctrl_schema_all; extern bool resctrl_mounted; =20 @@ -406,32 +405,6 @@ struct msr_param { u32 high; }; =20 -static inline bool is_llc_occupancy_enabled(void) -{ - return (rdt_mon_features & (1 << QOS_L3_OCCUP_EVENT_ID)); -} - -static inline bool is_mbm_total_enabled(void) -{ - return (rdt_mon_features & (1 << QOS_L3_MBM_TOTAL_EVENT_ID)); -} - -static inline bool is_mbm_local_enabled(void) -{ - return (rdt_mon_features & (1 << QOS_L3_MBM_LOCAL_EVENT_ID)); -} - -static inline bool is_mbm_enabled(void) -{ - return (is_mbm_total_enabled() || is_mbm_local_enabled()); -} - -static inline bool is_mbm_event(int e) -{ - return (e >=3D QOS_L3_MBM_TOTAL_EVENT_ID && - e <=3D QOS_L3_MBM_LOCAL_EVENT_ID); -} - /** * struct rdt_hw_resource - arch private attributes of a resctrl resource * @r_resctrl: Attributes of the resource used directly by resctrl. diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index 1730ba814834..d883ed56ec90 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -295,11 +295,11 @@ void resctrl_arch_reset_rmid_all(struct rdt_resource = *r, struct rdt_mon_domain * { struct rdt_hw_mon_domain *hw_dom =3D resctrl_to_arch_mon_dom(d); =20 - if (is_mbm_total_enabled()) + if (resctrl_arch_is_mbm_total_enabled()) memset(hw_dom->arch_mbm_total, 0, sizeof(*hw_dom->arch_mbm_total) * r->num_rmid); =20 - if (is_mbm_local_enabled()) + if (resctrl_arch_is_mbm_local_enabled()) memset(hw_dom->arch_mbm_local, 0, sizeof(*hw_dom->arch_mbm_local) * r->num_rmid); } @@ -569,7 +569,7 @@ void free_rmid(u32 closid, u32 rmid) =20 entry =3D __rmid_entry(idx); =20 - if (is_llc_occupancy_enabled()) + if (resctrl_arch_is_llc_occupancy_enabled()) add_rmid_to_limbo(entry); else list_add_tail(&entry->list, &rmid_free_lru); @@ -852,10 +852,10 @@ static void mbm_update(struct rdt_resource *r, struct= rdt_mon_domain *d, * This is protected from concurrent reads from user as both * the user and overflow handler hold the global mutex. */ - if (is_mbm_total_enabled()) + if (resctrl_arch_is_mbm_total_enabled()) mbm_update_one_event(r, d, closid, rmid, QOS_L3_MBM_TOTAL_EVENT_ID); =20 - if (is_mbm_local_enabled()) + if (resctrl_arch_is_mbm_local_enabled()) mbm_update_one_event(r, d, closid, rmid, QOS_L3_MBM_LOCAL_EVENT_ID); } =20 @@ -1085,11 +1085,11 @@ static void l3_mon_evt_init(struct rdt_resource *r) { INIT_LIST_HEAD(&r->evt_list); =20 - if (is_llc_occupancy_enabled()) + if (resctrl_arch_is_llc_occupancy_enabled()) list_add_tail(&llc_occupancy_event.list, &r->evt_list); - if (is_mbm_total_enabled()) + if (resctrl_arch_is_mbm_total_enabled()) list_add_tail(&mbm_total_event.list, &r->evt_list); - if (is_mbm_local_enabled()) + if (resctrl_arch_is_mbm_local_enabled()) list_add_tail(&mbm_local_event.list, &r->evt_list); } =20 diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index badac3f5da72..eb32fbc3abea 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -117,6 +117,18 @@ void rdt_staged_configs_clear(void) } } =20 +static bool resctrl_is_mbm_enabled(void) +{ + return (resctrl_arch_is_mbm_total_enabled() || + resctrl_arch_is_mbm_local_enabled()); +} + +static bool resctrl_is_mbm_event(int e) +{ + return (e >=3D QOS_L3_MBM_TOTAL_EVENT_ID && + e <=3D QOS_L3_MBM_LOCAL_EVENT_ID); +} + /* * Trivial allocator for CLOSIDs. Since h/w only supports a small number, * we can keep a bitmap of free CLOSIDs in a single integer. @@ -164,7 +176,7 @@ static int closid_alloc(void) lockdep_assert_held(&rdtgroup_mutex); =20 if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) && - is_llc_occupancy_enabled()) { + resctrl_arch_is_llc_occupancy_enabled()) { cleanest_closid =3D resctrl_find_cleanest_closid(); if (cleanest_closid < 0) return cleanest_closid; @@ -2378,7 +2390,7 @@ static bool supports_mba_mbps(void) struct rdt_resource *rmbm =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); struct rdt_resource *r =3D resctrl_arch_get_resource(RDT_RESOURCE_MBA); =20 - return (is_mbm_enabled() && + return (resctrl_is_mbm_enabled() && r->alloc_capable && is_mba_linear() && r->ctrl_scope =3D=3D rmbm->mon_scope); } @@ -2756,7 +2768,7 @@ static int rdt_get_tree(struct fs_context *fc) if (resctrl_arch_alloc_capable() || resctrl_arch_mon_capable()) resctrl_mounted =3D true; =20 - if (is_mbm_enabled()) { + if (resctrl_is_mbm_enabled()) { r =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); list_for_each_entry(dom, &r->mon_domains, hdr.list) mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL, @@ -3125,7 +3137,7 @@ static int mon_add_all_files(struct kernfs_node *kn, = struct rdt_mon_domain *d, if (ret) return ret; =20 - if (!do_sum && is_mbm_event(mevt->evtid)) + if (!do_sum && resctrl_is_mbm_event(mevt->evtid)) mon_event_read(&rr, r, d, prgrp, &d->hdr.cpu_mask, mevt->evtid, true); } =20 @@ -4082,9 +4094,9 @@ void resctrl_offline_mon_domain(struct rdt_resource *= r, struct rdt_mon_domain *d if (resctrl_mounted && resctrl_arch_mon_capable()) rmdir_mondata_subdir_allrdtgrp(r, d); =20 - if (is_mbm_enabled()) + if (resctrl_is_mbm_enabled()) cancel_delayed_work(&d->mbm_over); - if (is_llc_occupancy_enabled() && has_busy_rmid(d)) { + if (resctrl_arch_is_llc_occupancy_enabled() && has_busy_rmid(d)) { /* * When a package is going down, forcefully * decrement rmid->ebusy. There is no way to know @@ -4120,12 +4132,12 @@ static int domain_setup_mon_state(struct rdt_resour= ce *r, struct rdt_mon_domain u32 idx_limit =3D resctrl_arch_system_num_rmid_idx(); size_t tsize; =20 - if (is_llc_occupancy_enabled()) { + if (resctrl_arch_is_llc_occupancy_enabled()) { d->rmid_busy_llc =3D bitmap_zalloc(idx_limit, GFP_KERNEL); if (!d->rmid_busy_llc) return -ENOMEM; } - if (is_mbm_total_enabled()) { + if (resctrl_arch_is_mbm_total_enabled()) { tsize =3D sizeof(*d->mbm_total); d->mbm_total =3D kcalloc(idx_limit, tsize, GFP_KERNEL); if (!d->mbm_total) { @@ -4133,7 +4145,7 @@ static int domain_setup_mon_state(struct rdt_resource= *r, struct rdt_mon_domain return -ENOMEM; } } - if (is_mbm_local_enabled()) { + if (resctrl_arch_is_mbm_local_enabled()) { tsize =3D sizeof(*d->mbm_local); d->mbm_local =3D kcalloc(idx_limit, tsize, GFP_KERNEL); if (!d->mbm_local) { @@ -4172,13 +4184,13 @@ int resctrl_online_mon_domain(struct rdt_resource *= r, struct rdt_mon_domain *d) if (err) goto out_unlock; =20 - if (is_mbm_enabled()) { + if (resctrl_is_mbm_enabled()) { INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow); mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL, RESCTRL_PICK_ANY_CPU); } =20 - if (is_llc_occupancy_enabled()) + if (resctrl_arch_is_llc_occupancy_enabled()) INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo); =20 /* @@ -4233,12 +4245,12 @@ void resctrl_offline_cpu(unsigned int cpu) =20 d =3D get_mon_domain_from_cpu(cpu, l3); if (d) { - if (is_mbm_enabled() && cpu =3D=3D d->mbm_work_cpu) { + if (resctrl_is_mbm_enabled() && cpu =3D=3D d->mbm_work_cpu) { cancel_delayed_work(&d->mbm_over); mbm_setup_overflow_handler(d, 0, cpu); } - if (is_llc_occupancy_enabled() && cpu =3D=3D d->cqm_work_cpu && - has_busy_rmid(d)) { + if (resctrl_arch_is_llc_occupancy_enabled() && + cpu =3D=3D d->cqm_work_cpu && has_busy_rmid(d)) { cancel_delayed_work(&d->cqm_limbo); cqm_setup_limbo_handler(d, 0, cpu); } --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D3EAE265CA6 for ; Tue, 11 Mar 2025 18:41:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718481; cv=none; b=KAUvj4deZLYEjp+bg6uT45xZtfpCU2npUI9bS1b4qJa/jatNwbB2kac+e0mfBLmrMpgujY5W3iQAnFkTprvGCNx8kQG5MWrD4CZ96HBxyjx99ks5Z6BhFs7uN8xBqMCadzd5YTpRkGkDNCJOeZcvlCFUe8xi6kb0KDE2m8sXelE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718481; c=relaxed/simple; bh=H8SAvZxBuiqxOW6IcUcZOM5EvY3KBSniQ95Fr9mVzWc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=B9X0fqgXsCNmbZoiTs3DhsoEsAsIGReHt28D2iaDyNEdKdxP09FG5TaRDqbgPL9wfy8bz2GQQF75w0J0GQf9aKs2wF/RlRQmYkw0zSg5keP7+pZ6puiHTeH70sdKFZW5Dw6J8BkI3gfSffZwj83X83sIZvGEPQjXErTXGMSqy2M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 78D421515; Tue, 11 Mar 2025 11:41:30 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0187B3F694; Tue, 11 Mar 2025 11:41:15 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 19/30] x86/resctrl: Add resctrl_arch_is_evt_configurable() to abstract BMEC Date: Tue, 11 Mar 2025 18:37:04 +0000 Message-Id: <20250311183715.16445-20-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When BMEC is supported the resctrl event can be configured in a number of ways. This depends on architecture support. rdt_get_mon_l3_config() modifies the struct mon_evt and calls resctrl_file_fflags_init() to create the files that allow the configuration. Splitting this into separate architecture and filesystem parts would require the struct mon_evt and resctrl_file_fflags_init() to be exposed. Instead, add resctrl_arch_is_evt_configurable(), and use this from resctrl_mon_resource_init() to initialise struct mon_evt and call resctrl_file_fflags_init() resctrl_arch_is_evt_configurable() calls rdt_cpu_has() so it doesn't obviously benefit from being inlined. Putting it in core.c will allow rdt_cpu_has() to eventually become static. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Fenghua Yu Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v7: * Fixed more instances of stale function name in the commit message. Changes since v6: * Fixed stale function name in the commit message. * Moved __init around in the function prototypes. Changes since v4: * Moved all the __init changes to a later patch now that the exit gubbins comes first. --- arch/x86/kernel/cpu/resctrl/core.c | 15 +++++++++++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 22 +++++++++++----------- include/linux/resctrl.h | 2 ++ 3 files changed, 28 insertions(+), 11 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index eba210db4a7b..6f0d1bbba901 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -830,6 +830,21 @@ bool __init rdt_cpu_has(int flag) return ret; } =20 +__init bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt) +{ + if (!rdt_cpu_has(X86_FEATURE_BMEC)) + return false; + + switch (evt) { + case QOS_L3_MBM_TOTAL_EVENT_ID: + return rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL); + case QOS_L3_MBM_LOCAL_EVENT_ID: + return rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL); + default: + return false; + } +} + static __init bool get_mem_config(void) { struct rdt_hw_resource *hw_res =3D &rdt_resources_all[RDT_RESOURCE_MBA]; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index d883ed56ec90..27d983121b0a 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -1202,6 +1202,17 @@ int __init resctrl_mon_resource_init(void) =20 l3_mon_evt_init(r); =20 + if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_TOTAL_EVENT_ID)) { + mbm_total_event.configurable =3D true; + resctrl_file_fflags_init("mbm_total_bytes_config", + RFTYPE_MON_INFO | RFTYPE_RES_CACHE); + } + if (resctrl_arch_is_evt_configurable(QOS_L3_MBM_LOCAL_EVENT_ID)) { + mbm_local_event.configurable =3D true; + resctrl_file_fflags_init("mbm_local_bytes_config", + RFTYPE_MON_INFO | RFTYPE_RES_CACHE); + } + return 0; } =20 @@ -1245,17 +1256,6 @@ int __init rdt_get_mon_l3_config(struct rdt_resource= *r) /* Detect list of bandwidth sources that can be tracked */ cpuid_count(0x80000020, 3, &eax, &ebx, &ecx, &edx); hw_res->mbm_cfg_mask =3D ecx & MAX_EVT_CONFIG_BITS; - - if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL)) { - mbm_total_event.configurable =3D true; - resctrl_file_fflags_init("mbm_total_bytes_config", - RFTYPE_MON_INFO | RFTYPE_RES_CACHE); - } - if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL)) { - mbm_local_event.configurable =3D true; - resctrl_file_fflags_init("mbm_local_bytes_config", - RFTYPE_MON_INFO | RFTYPE_RES_CACHE); - } } =20 r->mon_capable =3D true; diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index a392480dc4b6..cc76f308e6f3 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -309,6 +309,8 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *r); u32 resctrl_arch_system_num_rmid_idx(void); int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid); =20 +__init bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt); + /* * Update the ctrl_val and apply this config right now. * Must be called on one of the domain's CPUs. --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B8C13265CD2 for ; Tue, 11 Mar 2025 18:41:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718485; cv=none; b=Rb2xvjMc6pD5LizcRvWHWaryhKDTMnthE1WGTLcpqXahQcAqmoW8nckgNTMfj0X0DQH4ITW3sDfZ4CA6+ia5NWa/1fDux1WYVD1M9UkdrwG5P/kvLfzXwbCf63u51uPq3/mvHQ9hRwYLM/dc9G95/Tu08ioTU4FmJEOXJVpr9No= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718485; c=relaxed/simple; bh=pti2roIBDeLYI3qyr/Ts6yQwwCrevl9D38HtEydaQ/8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WZ4ZKUo/3p+jl3iHAnXJd8YyuGtWSRmSo8T/d3lIOT4dFw2OgXDb7nsmXZchTWJzhQ2wtwYvxL64EEG5xXsHJ7GJXHx+oxiG1lCmOUe2nzelRquY+r6HWaLGExTygoeZ9Bt8jn/nx9Rz54LseLjxj3wOPm9IpAn5ZVJYBo56UjA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5955E1713; Tue, 11 Mar 2025 11:41:34 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B15B33F694; Tue, 11 Mar 2025 11:41:19 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Dave Martin , Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 20/30] x86/resctrl: Change mon_event_config_{read,write}() to be arch helpers Date: Tue, 11 Mar 2025 18:37:05 +0000 Message-Id: <20250311183715.16445-21-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" mon_event_config_{read,write}() are called via IPI and access model specific registers to do their work. To support another architecture, this needs abstracting. Rename mon_event_config_{read,write}() to have a "resctrl_arch_" prefix, and move their struct mon_config_info parameter into . This allows another architecture to supply an implementation of these. As struct mon_config_info is now exposed globally, give it a 'resctrl_' prefix. MPAM systems need access to the domain to do this work, add the resource and domain to struct resctrl_mon_config_info. Co-developed-by: Dave Martin Signed-off-by: Dave Martin Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v5: * Renamed info to config_info. * Added description of which fields are read and written in the structure. * Clarified comment about which CPU this is called on for both kinds of reader. Changes since v3: * Added comments over the read/write helper to explain the type of the void pointer. Changes since v1: * [Whitespace only] Re-tabbed struct resctrl_mon_config_info in to fit the prevailing style. Non-functional change. * [Commit message only] Reword to align with the actual naming of the definitions and destination header file. --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 46 +++++++++++++------------- include/linux/resctrl.h | 31 +++++++++++++++++ 2 files changed, 54 insertions(+), 23 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index eb32fbc3abea..e7d1d8b6983d 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -1580,11 +1580,6 @@ static int rdtgroup_size_show(struct kernfs_open_fil= e *of, return ret; } =20 -struct mon_config_info { - u32 evtid; - u32 mon_config; -}; - #define INVALID_CONFIG_INDEX UINT_MAX =20 /** @@ -1609,31 +1604,32 @@ static inline unsigned int mon_event_config_index_g= et(u32 evtid) } } =20 -static void mon_event_config_read(void *info) +void resctrl_arch_mon_event_config_read(void *_config_info) { - struct mon_config_info *mon_info =3D info; + struct resctrl_mon_config_info *config_info =3D _config_info; unsigned int index; u64 msrval; =20 - index =3D mon_event_config_index_get(mon_info->evtid); + index =3D mon_event_config_index_get(config_info->evtid); if (index =3D=3D INVALID_CONFIG_INDEX) { - pr_warn_once("Invalid event id %d\n", mon_info->evtid); + pr_warn_once("Invalid event id %d\n", config_info->evtid); return; } rdmsrl(MSR_IA32_EVT_CFG_BASE + index, msrval); =20 /* Report only the valid event configuration bits */ - mon_info->mon_config =3D msrval & MAX_EVT_CONFIG_BITS; + config_info->mon_config =3D msrval & MAX_EVT_CONFIG_BITS; } =20 -static void mondata_config_read(struct rdt_mon_domain *d, struct mon_confi= g_info *mon_info) +static void mondata_config_read(struct resctrl_mon_config_info *mon_info) { - smp_call_function_any(&d->hdr.cpu_mask, mon_event_config_read, mon_info, = 1); + smp_call_function_any(&mon_info->d->hdr.cpu_mask, + resctrl_arch_mon_event_config_read, mon_info, 1); } =20 static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32= evtid) { - struct mon_config_info mon_info; + struct resctrl_mon_config_info mon_info; struct rdt_mon_domain *dom; bool sep =3D false; =20 @@ -1644,9 +1640,11 @@ static int mbm_config_show(struct seq_file *s, struc= t rdt_resource *r, u32 evtid if (sep) seq_puts(s, ";"); =20 - memset(&mon_info, 0, sizeof(struct mon_config_info)); + memset(&mon_info, 0, sizeof(struct resctrl_mon_config_info)); + mon_info.r =3D r; + mon_info.d =3D dom; mon_info.evtid =3D evtid; - mondata_config_read(dom, &mon_info); + mondata_config_read(&mon_info); =20 seq_printf(s, "%d=3D0x%02x", dom->hdr.id, mon_info.mon_config); sep =3D true; @@ -1679,30 +1677,32 @@ static int mbm_local_bytes_config_show(struct kernf= s_open_file *of, return 0; } =20 -static void mon_event_config_write(void *info) +void resctrl_arch_mon_event_config_write(void *_config_info) { - struct mon_config_info *mon_info =3D info; + struct resctrl_mon_config_info *config_info =3D _config_info; unsigned int index; =20 - index =3D mon_event_config_index_get(mon_info->evtid); + index =3D mon_event_config_index_get(config_info->evtid); if (index =3D=3D INVALID_CONFIG_INDEX) { - pr_warn_once("Invalid event id %d\n", mon_info->evtid); + pr_warn_once("Invalid event id %d\n", config_info->evtid); return; } - wrmsr(MSR_IA32_EVT_CFG_BASE + index, mon_info->mon_config, 0); + wrmsr(MSR_IA32_EVT_CFG_BASE + index, config_info->mon_config, 0); } =20 static void mbm_config_write_domain(struct rdt_resource *r, struct rdt_mon_domain *d, u32 evtid, u32 val) { - struct mon_config_info mon_info =3D {0}; + struct resctrl_mon_config_info mon_info =3D {0}; =20 /* * Read the current config value first. If both are the same then * no need to write it again. */ + mon_info.r =3D r; + mon_info.d =3D d; mon_info.evtid =3D evtid; - mondata_config_read(d, &mon_info); + mondata_config_read(&mon_info); if (mon_info.mon_config =3D=3D val) return; =20 @@ -1714,7 +1714,7 @@ static void mbm_config_write_domain(struct rdt_resour= ce *r, * are scoped at the domain level. Writing any of these MSRs * on one CPU is observed by all the CPUs in the domain. */ - smp_call_function_any(&d->hdr.cpu_mask, mon_event_config_write, + smp_call_function_any(&d->hdr.cpu_mask, resctrl_arch_mon_event_config_wri= te, &mon_info, 1); =20 /* diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index cc76f308e6f3..90b6563cf532 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -270,6 +270,13 @@ struct resctrl_cpu_defaults { u32 rmid; }; =20 +struct resctrl_mon_config_info { + struct rdt_resource *r; + struct rdt_mon_domain *d; + u32 evtid; + u32 mon_config; +}; + /** * resctrl_arch_sync_cpu_closid_rmid() - Refresh this CPU's CLOSID and RMI= D. * Call via IPI. @@ -311,6 +318,30 @@ int resctrl_arch_update_domains(struct rdt_resource *r= , u32 closid); =20 __init bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt); =20 +/** + * resctrl_arch_mon_event_config_write() - Write the config for an event. + * @config_info: struct resctrl_mon_config_info describing the resource, d= omain + * and event. + * + * Reads resource, domain and eventid from @config_info and writes the + * event config_info->mon_config into hardware. + * + * Called via IPI to reach a CPU that is a member of the specified domain. + */ +void resctrl_arch_mon_event_config_write(void *config_info); + +/** + * resctrl_arch_mon_event_config_read() - Read the config for an event. + * @config_info: struct resctrl_mon_config_info describing the resource, d= omain + * and event. + * + * Reads resource, domain and eventid from @config_info and reads the + * hardware config value into config_info->mon_config. + * + * Called via IPI to reach a CPU that is a member of the specified domain. + */ +void resctrl_arch_mon_event_config_read(void *config_info); + /* * Update the ctrl_val and apply this config right now. * Must be called on one of the domain's CPUs. --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 55C12265CD8 for ; Tue, 11 Mar 2025 18:41:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718488; cv=none; b=qMiI2/L0JUtWhpleInw6Lmhtu6z/+dpp2RXmujvmkPjYEhQEGwnL2CDwFwSSYs8Ams0v26Xy4iS1WetPtkkRN5ZYy7gULGgixx6I+IYQJXeTa47NIaJ/m1NWaRvikMsZFrgsGQUwoAAwB2MhYblD4uhI0MKDr+cx8IRcqZ2AXOY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718488; c=relaxed/simple; bh=Mas5yeNYoc/NwczvtKDhCbxECD0q1LfC24k4R8AfHg0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Tqf4D7O0yW+PfNTW3VnVnOyKzV7bQIGK0yEM550r/q5l8kBqpe2n+8aaRv4JTY79WsjhskTJW4es4cCIsyj3PnXOvDgh2cKzyy65nrvBfE0+NlLZ1fJoPkG9ywBQmJ4QXsDdX6aIBW6A0am8s5qfjFCN9lChMawKWByA54HVgRI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E7A581515; Tue, 11 Mar 2025 11:41:37 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 863623F694; Tue, 11 Mar 2025 11:41:23 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger Subject: [PATCH v8:for-boris 21/30] x86/resctrl: Move mba_mbps_default_event init to filesystem code Date: Tue, 11 Mar 2025 18:37:06 +0000 Message-Id: <20250311183715.16445-22-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" mba_mbps_default_event is initialised base on whether mbm_local or mbm_total is supported. In the case of both, it is initialised to mbm_local. mba_mbps_default_event is initialised in core.c's get_rdt_mon_resources(), while all the readers are in rdtgroup.c. After this code is split into architecture specific and filesystem code, get_rdt_mon_resources() remains part of the architecture code, which would mean mba_mbps_default_event has to be exposed by the filesystem code. Move the initialisation to the filesystem's resctrl_mon_resource_init() Signed-off-by: James Morse Tested-by: Peter Newman Tested-by: Shaopeng Tan Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- arch/x86/kernel/cpu/resctrl/core.c | 5 ----- arch/x86/kernel/cpu/resctrl/monitor.c | 5 +++++ 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 6f0d1bbba901..b9b74f53b8ee 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -923,11 +923,6 @@ static __init bool get_rdt_mon_resources(void) if (!rdt_mon_features) return false; =20 - if (resctrl_arch_is_mbm_local_enabled()) - mba_mbps_default_event =3D QOS_L3_MBM_LOCAL_EVENT_ID; - else if (resctrl_arch_is_mbm_total_enabled()) - mba_mbps_default_event =3D QOS_L3_MBM_TOTAL_EVENT_ID; - return !rdt_get_mon_l3_config(r); } =20 diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index 27d983121b0a..306b06ba32db 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -1213,6 +1213,11 @@ int __init resctrl_mon_resource_init(void) RFTYPE_MON_INFO | RFTYPE_RES_CACHE); } =20 + if (resctrl_arch_is_mbm_local_enabled()) + mba_mbps_default_event =3D QOS_L3_MBM_LOCAL_EVENT_ID; + else if (resctrl_arch_is_mbm_total_enabled()) + mba_mbps_default_event =3D QOS_L3_MBM_TOTAL_EVENT_ID; + return 0; } =20 --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 013AE265CD8 for ; Tue, 11 Mar 2025 18:41:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718496; cv=none; b=pX0oDcuP1baFUVWprpY2ORmhOh0cHW+bUCl3LAsPT857h3CFoEyHCNecSX9QtG55zGEJ9M5dqAXQI0t2RCB+eX0o6t+TDhz0CexVXfmS4uYglzXCIFY8UEqpZFBdNpFq1ZxvBHBv7fDS7TxdnY+8ZO3jql2uAoTVProcJ8SffVI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718496; c=relaxed/simple; bh=xuFwp+wVlNvvX6pMWaic4seoqFILwx8Tixryh/4/j98=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QT0SnkMqmYzwoSnipHdNIgwWPc8wY4iN/8I0exqtRwqN+36QO31kIFVfQAl8mlNZWoPbPL30coKLN2lX9/O4SH9ha+pN32cZVr/3e+iy815GxB4RIJm7DJNV/nzCJVukf0QeuiNjbTU8OADkHloji+G+Ncnx+pSZAXm0RLmVjxM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9EE2C1688; Tue, 11 Mar 2025 11:41:41 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 271053F694; Tue, 11 Mar 2025 11:41:27 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 22/30] x86/resctrl: Move mbm_cfg_mask to struct rdt_resource Date: Tue, 11 Mar 2025 18:37:07 +0000 Message-Id: <20250311183715.16445-23-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The mbm_cfg_mask field lists the bits that user-space can set when configuring an event. This value is output via the last_cmd_status file. Once the filesystem parts of resctrl are moved to live in /fs/, the struct rdt_hw_resource is inaccessible to the filesystem code. Because this value is output to user-space, it has to be accessible to the filesystem code. Move it to struct rdt_resource. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v5: * Rephrased a comment to remove one vendors marketing name for the feature. Change since v1: * Reword comments to avoid being overly arch-specific. --- arch/x86/kernel/cpu/resctrl/internal.h | 3 --- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 ++--- include/linux/resctrl.h | 3 +++ 4 files changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 4a5996d1e060..725f223ea07b 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -417,8 +417,6 @@ struct msr_param { * @msr_update: Function pointer to update QOS MSRs * @mon_scale: cqm counter * mon_scale =3D occupancy in bytes * @mbm_width: Monitor width, to detect and correct for overflow. - * @mbm_cfg_mask: Bandwidth sources that can be tracked when Bandwidth - * Monitoring Event Configuration (BMEC) is supported. * @cdp_enabled: CDP state of this resource * * Members of this structure are either private to the architecture @@ -432,7 +430,6 @@ struct rdt_hw_resource { void (*msr_update)(struct msr_param *m); unsigned int mon_scale; unsigned int mbm_width; - unsigned int mbm_cfg_mask; bool cdp_enabled; }; =20 diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index 306b06ba32db..83f90128d768 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -1260,7 +1260,7 @@ int __init rdt_get_mon_l3_config(struct rdt_resource = *r) =20 /* Detect list of bandwidth sources that can be tracked */ cpuid_count(0x80000020, 3, &eax, &ebx, &ecx, &edx); - hw_res->mbm_cfg_mask =3D ecx & MAX_EVT_CONFIG_BITS; + r->mbm_cfg_mask =3D ecx & MAX_EVT_CONFIG_BITS; } =20 r->mon_capable =3D true; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index e7d1d8b6983d..a388ef66ef4c 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -1731,7 +1731,6 @@ static void mbm_config_write_domain(struct rdt_resour= ce *r, =20 static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid) { - struct rdt_hw_resource *hw_res =3D resctrl_to_arch_res(r); char *dom_str =3D NULL, *id_str; unsigned long dom_id, val; struct rdt_mon_domain *d; @@ -1758,9 +1757,9 @@ static int mon_config_write(struct rdt_resource *r, c= har *tok, u32 evtid) } =20 /* Value from user cannot be more than the supported set of events */ - if ((val & hw_res->mbm_cfg_mask) !=3D val) { + if ((val & r->mbm_cfg_mask) !=3D val) { rdt_last_cmd_printf("Invalid event configuration: max valid mask is 0x%0= 2x\n", - hw_res->mbm_cfg_mask); + r->mbm_cfg_mask); return -EINVAL; } =20 diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 90b6563cf532..914df24ffe48 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -217,6 +217,8 @@ enum resctrl_schema_fmt { * @name: Name to use in "schemata" file. * @schema_fmt: Which format string and parser is used for this schema. * @evt_list: List of monitoring events + * @mbm_cfg_mask: Bandwidth sources that can be tracked when bandwidth + * monitoring events can be configured. * @cdp_capable: Is the CDP feature available on this resource */ struct rdt_resource { @@ -233,6 +235,7 @@ struct rdt_resource { char *name; enum resctrl_schema_fmt schema_fmt; struct list_head evt_list; + unsigned int mbm_cfg_mask; bool cdp_capable; }; =20 --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B9D6A263F42 for ; Tue, 11 Mar 2025 18:41:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718496; cv=none; b=X9vkrR1k+bhR+zL4/c66czunXO4d8QKHtyPrOK8eN1/rIeCt1y0yGTLxjJX/0eeWpihAxnGMjf2xFK+wKneayHFIdKmR/yz4HbQwosyppr6KjpElkIiV9FHmqWa13LlCB+fODAbYAelfrp1aApHoo0jFmfq3IRCVZpz0WGDYdkQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718496; c=relaxed/simple; bh=5fxu9KRrP17JgsoMLbOFGuAdmB2ILif9aNVpfILl/xo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rzDV1/3JJJMrA2AYJ6ujISSJVNfXXY0ijoTYyULseSqYAwexEGzdrDwoTVzlZfDHvvpnvyUzJJ7jhWeBnrJ99+QPpUF2fma6fJz9+GATyYqWfXQaONtyVxk/IGLxSZslieD9jI5I3I32LJyDYEEn9AGWtFmtPMZal5VKltEWljQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 58A8F1515; Tue, 11 Mar 2025 11:41:45 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D58813F694; Tue, 11 Mar 2025 11:41:30 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 23/30] x86/resctrl: Add resctrl_arch_ prefix to pseudo lock functions Date: Tue, 11 Mar 2025 18:37:08 +0000 Message-Id: <20250311183715.16445-24-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" resctrl's pseudo lock has some copy-to-cache and measurement functions that are micro-architecture specific. For example, pseudo_lock_fn() is not at all portable. Label these 'resctrl_arch_' so they stay under /arch/x86. To expose these functions to the filesystem code they need an entry in a header file, and can't be marked static. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v6: * Removed a space. Changes since v5: * Expanded commit message. --- arch/x86/include/asm/resctrl.h | 5 ++++ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 36 ++++++++++++----------- 2 files changed, 24 insertions(+), 17 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 6d4c7ea2c9e3..86407dbde583 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -203,6 +203,11 @@ static inline void *resctrl_arch_mon_ctx_alloc(struct = rdt_resource *r, int evtid static inline void resctrl_arch_mon_ctx_free(struct rdt_resource *r, int e= vtid, void *ctx) { }; =20 +u64 resctrl_arch_get_prefetch_disable_bits(void); +int resctrl_arch_pseudo_lock_fn(void *_rdtgrp); +int resctrl_arch_measure_cycles_lat_fn(void *_plr); +int resctrl_arch_measure_l2_residency(void *_plr); +int resctrl_arch_measure_l3_residency(void *_plr); void resctrl_cpu_detect(struct cpuinfo_x86 *c); =20 #else diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cp= u/resctrl/pseudo_lock.c index 42cc162f7fc9..1f42c1190d26 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -61,7 +61,8 @@ static const struct class pseudo_lock_class =3D { }; =20 /** - * get_prefetch_disable_bits - prefetch disable bits of supported platforms + * resctrl_arch_get_prefetch_disable_bits - prefetch disable bits of suppo= rted + * platforms * @void: It takes no parameters. * * Capture the list of platforms that have been validated to support @@ -75,13 +76,13 @@ static const struct class pseudo_lock_class =3D { * in the SDM. * * When adding a platform here also add support for its cache events to - * measure_cycles_perf_fn() + * resctrl_arch_measure_l*_residency() * * Return: * If platform is supported, the bits to disable hardware prefetchers, 0 * if platform is not supported. */ -static u64 get_prefetch_disable_bits(void) +u64 resctrl_arch_get_prefetch_disable_bits(void) { if (boot_cpu_data.x86_vendor !=3D X86_VENDOR_INTEL || boot_cpu_data.x86 !=3D 6) @@ -408,7 +409,7 @@ static void pseudo_lock_free(struct rdtgroup *rdtgrp) } =20 /** - * pseudo_lock_fn - Load kernel memory into cache + * resctrl_arch_pseudo_lock_fn - Load kernel memory into cache * @_rdtgrp: resource group to which pseudo-lock region belongs * * This is the core pseudo-locking flow. @@ -426,7 +427,7 @@ static void pseudo_lock_free(struct rdtgroup *rdtgrp) * * Return: 0. Waiter on waitqueue will be woken on completion. */ -static int pseudo_lock_fn(void *_rdtgrp) +int resctrl_arch_pseudo_lock_fn(void *_rdtgrp) { struct rdtgroup *rdtgrp =3D _rdtgrp; struct pseudo_lock_region *plr =3D rdtgrp->plr; @@ -712,7 +713,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp) * Not knowing the bits to disable prefetching implies that this * platform does not support Cache Pseudo-Locking. */ - prefetch_disable_bits =3D get_prefetch_disable_bits(); + prefetch_disable_bits =3D resctrl_arch_get_prefetch_disable_bits(); if (prefetch_disable_bits =3D=3D 0) { rdt_last_cmd_puts("Pseudo-locking not supported\n"); return -EINVAL; @@ -872,7 +873,8 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_ctr= l_domain *d) } =20 /** - * measure_cycles_lat_fn - Measure cycle latency to read pseudo-locked mem= ory + * resctrl_arch_measure_cycles_lat_fn - Measure cycle latency to read + * pseudo-locked memory * @_plr: pseudo-lock region to measure * * There is no deterministic way to test if a memory region is cached. One @@ -885,7 +887,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_ctr= l_domain *d) * * Return: 0. Waiter on waitqueue will be woken on completion. */ -static int measure_cycles_lat_fn(void *_plr) +int resctrl_arch_measure_cycles_lat_fn(void *_plr) { struct pseudo_lock_region *plr =3D _plr; u32 saved_low, saved_high; @@ -1069,7 +1071,7 @@ static int measure_residency_fn(struct perf_event_att= r *miss_attr, return 0; } =20 -static int measure_l2_residency(void *_plr) +int resctrl_arch_measure_l2_residency(void *_plr) { struct pseudo_lock_region *plr =3D _plr; struct residency_counts counts =3D {0}; @@ -1107,7 +1109,7 @@ static int measure_l2_residency(void *_plr) return 0; } =20 -static int measure_l3_residency(void *_plr) +int resctrl_arch_measure_l3_residency(void *_plr) { struct pseudo_lock_region *plr =3D _plr; struct residency_counts counts =3D {0}; @@ -1205,14 +1207,14 @@ static int pseudo_lock_measure_cycles(struct rdtgro= up *rdtgrp, int sel) plr->cpu =3D cpu; =20 if (sel =3D=3D 1) - thread =3D kthread_run_on_cpu(measure_cycles_lat_fn, plr, - cpu, "pseudo_lock_measure/%u"); + thread =3D kthread_run_on_cpu(resctrl_arch_measure_cycles_lat_fn, + plr, cpu, "pseudo_lock_measure/%u"); else if (sel =3D=3D 2) - thread =3D kthread_run_on_cpu(measure_l2_residency, plr, - cpu, "pseudo_lock_measure/%u"); + thread =3D kthread_run_on_cpu(resctrl_arch_measure_l2_residency, + plr, cpu, "pseudo_lock_measure/%u"); else if (sel =3D=3D 3) - thread =3D kthread_run_on_cpu(measure_l3_residency, plr, - cpu, "pseudo_lock_measure/%u"); + thread =3D kthread_run_on_cpu(resctrl_arch_measure_l3_residency, + plr, cpu, "pseudo_lock_measure/%u"); else goto out; =20 @@ -1307,7 +1309,7 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtg= rp) =20 plr->thread_done =3D 0; =20 - thread =3D kthread_run_on_cpu(pseudo_lock_fn, rdtgrp, + thread =3D kthread_run_on_cpu(resctrl_arch_pseudo_lock_fn, rdtgrp, plr->cpu, "pseudo_lock/%u"); if (IS_ERR(thread)) { ret =3D PTR_ERR(thread); --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 69DBC266565 for ; Tue, 11 Mar 2025 18:41:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718500; cv=none; b=DbkpX761iujhgtMSOz1ZZ4VS93Cnp+7f8wQaBdl+7jjwYXqSqNNEnPOz5FH/qscFzLgsL6l1OWb7lp3/Ejc+/y1pBeLQPfiNCo67A9QofrhgwPNDwVkGzRw55ITF2DC6mJJ+ZFMBYHRLQ3ikuXJtvlHoI+ePSb1IZNBtnRC3idg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718500; c=relaxed/simple; bh=mssoB0ImMUQKadmQ5O3JX8EwHj5EcFbn72hByUTsbQI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=s7ExqxCfPSUrwslIlN3K4+AfwLXnIQWTLxbzzEcn/6f2pOYtj76zGNvAFPPpZYgw6N5Zln4yGixWH7/VReAKYM1Gg7P+vjZZDgwLbsWURibgujx1ke+rAdyvDt9eO4rwjfPXOOL02DPPkKwvuZILPRNJSUDUCHV0Qp7qC6MhUW8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 12F881713; Tue, 11 Mar 2025 11:41:49 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8FF713F694; Tue, 11 Mar 2025 11:41:34 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 24/30] x86/resctrl: Allow an architecture to disable pseudo lock Date: Tue, 11 Mar 2025 18:37:09 +0000 Message-Id: <20250311183715.16445-25-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Pseudo-lock relies on knowledge of the micro-architecture to disable prefetchers etc. On arm64 these controls are typically secure only, meaning linux can't access them. Arm's cache-lockdown feature works in a very different way. Resctrl's pseudo-lock isn't going to be used on arm64 platforms. Add a Kconfig symbol that can be selected by the architecture. This enables or disables building of the pseudo_lock.c file, and replaces the functions with stubs. An additional IS_ENABLED() check is needed in rdtgroup_mode_write() so that attempting to enable pseudo-lock reports an "Unknown or unsupported mode" to user-space via the last_cmd_status file. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v4: * "last_cmd file" -> "last_cmd_status file" Changes since v2: * Clarified the commit message as to where the error string is printed. Changes since v1: * [Commit message only] Typo fix: s/psuedo/pseudo/g --- arch/x86/Kconfig | 7 ++++ arch/x86/kernel/cpu/resctrl/Makefile | 5 +-- arch/x86/kernel/cpu/resctrl/internal.h | 49 +++++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 +- 4 files changed, 53 insertions(+), 11 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index be2c311f5118..1a89574a58f7 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -505,6 +505,7 @@ config X86_CPU_RESCTRL depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD) select KERNFS select PROC_CPU_RESCTRL if PROC_FS + select RESCTRL_FS_PSEUDO_LOCK help Enable x86 CPU resource control support. =20 @@ -521,6 +522,12 @@ config X86_CPU_RESCTRL =20 Say N if unsure. =20 +config RESCTRL_FS_PSEUDO_LOCK + bool + help + Software mechanism to pin data in a cache portion using + micro-architecture specific knowledge. + config X86_FRED bool "Flexible Return and Event Delivery" depends on X86_64 diff --git a/arch/x86/kernel/cpu/resctrl/Makefile b/arch/x86/kernel/cpu/res= ctrl/Makefile index 4a06c37b9cf1..0c13b0befd8a 100644 --- a/arch/x86/kernel/cpu/resctrl/Makefile +++ b/arch/x86/kernel/cpu/resctrl/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_X86_CPU_RESCTRL) +=3D core.o rdtgroup.o monitor.o -obj-$(CONFIG_X86_CPU_RESCTRL) +=3D ctrlmondata.o pseudo_lock.o +obj-$(CONFIG_X86_CPU_RESCTRL) +=3D core.o rdtgroup.o monitor.o +obj-$(CONFIG_X86_CPU_RESCTRL) +=3D ctrlmondata.o +obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK) +=3D pseudo_lock.o CFLAGS_pseudo_lock.o =3D -I$(src) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 725f223ea07b..8d35bb423aad 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -512,14 +512,6 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource = *r, struct rdt_ctrl_domain unsigned long cbm); enum rdtgrp_mode rdtgroup_mode_by_closid(int closid); int rdtgroup_tasks_assigned(struct rdtgroup *r); -int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp); -int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp); -bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_ctrl_domain *d, unsign= ed long cbm); -bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_ctrl_domain *d); -int rdt_pseudo_lock_init(void); -void rdt_pseudo_lock_release(void); -int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp); -void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp); struct rdt_ctrl_domain *get_ctrl_domain_from_cpu(int cpu, struct rdt_resou= rce *r); struct rdt_mon_domain *get_mon_domain_from_cpu(int cpu, struct rdt_resourc= e *r); int closids_supported(void); @@ -551,4 +543,45 @@ void resctrl_file_fflags_init(const char *config, unsi= gned long fflags); void rdt_staged_configs_clear(void); bool closid_allocated(unsigned int closid); int resctrl_find_cleanest_closid(void); + +#ifdef CONFIG_RESCTRL_FS_PSEUDO_LOCK +int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp); +int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp); +bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_ctrl_domain *d, unsign= ed long cbm); +bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_ctrl_domain *d); +int rdt_pseudo_lock_init(void); +void rdt_pseudo_lock_release(void); +int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp); +void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp); +#else +static inline int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp) +{ + return -EOPNOTSUPP; +} + +static inline int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) +{ + return -EOPNOTSUPP; +} + +static inline bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_ctrl_dom= ain *d, unsigned long cbm) +{ + return false; +} + +static inline bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_ctrl_dom= ain *d) +{ + return false; +} + +static inline int rdt_pseudo_lock_init(void) { return 0; } +static inline void rdt_pseudo_lock_release(void) { } +static inline int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp) +{ + return -EOPNOTSUPP; +} + +static inline void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp) { } +#endif /* CONFIG_RESCTRL_FS_PSEUDO_LOCK */ + #endif /* _ASM_X86_RESCTRL_INTERNAL_H */ diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index a388ef66ef4c..e59271515a46 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -1453,7 +1453,8 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open= _file *of, goto out; } rdtgrp->mode =3D RDT_MODE_EXCLUSIVE; - } else if (!strcmp(buf, "pseudo-locksetup")) { + } else if (IS_ENABLED(CONFIG_RESCTRL_FS_PSEUDO_LOCK) && + !strcmp(buf, "pseudo-locksetup")) { ret =3D rdtgroup_locksetup_enter(rdtgrp); if (ret) goto out; --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 76572264639 for ; Tue, 11 Mar 2025 18:41:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718507; cv=none; b=cklJY860P+EQi3av3EVKz/2gumBwuXcNTNVwm8npAQW/tV5BE0hMp4FYvKXFF1cKuAW3r5I8CdtLEIa9+T7hGIlYMEtT6POgzlfih2pJFCox73YqvUHCbdL+NZDISi6SwokwJL47y461gbp67hqecOmwG1bW3AqrI+gFB5b5jHs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718507; c=relaxed/simple; bh=3+J6oDEu1X6g5Auuhbmfyh+8zqtaX8I7/EFan5gk9Ec=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nrSL8a0Fue1KRSMn+2MBQvfLok65T56oAnkjSwXlYFie2xQYF8qqmAwYI3pu1qozqNj1jezcRQcVkDFzVFJSbEAO6p6clMRps4Om2ALYM+NVb/uELVz3wSD2FOPfTl7l7iQ5+GT/Bi2gdsZfqbZAcYrqXjbQEP9JJgdhlwSW7js= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C31B11762; Tue, 11 Mar 2025 11:41:52 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4A4553F694; Tue, 11 Mar 2025 11:41:38 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 25/30] x86/resctrl: Make prefetch_disable_bits belong to the arch code Date: Tue, 11 Mar 2025 18:37:10 +0000 Message-Id: <20250311183715.16445-26-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" prefetch_disable_bits is set by rdtgroup_locksetup_enter() from a value provided by the architecture, but is largely read by other architecture helpers. Make resctrl_arch_get_prefetch_disable_bits() set prefetch_disable_bits so that it can be isolated to arch-code from where the other arch-code helpers can use its cached-value. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v5: * Tweaked the word 'export'. * Swapped the second paragraph for Reinette's version. --- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cp= u/resctrl/pseudo_lock.c index 1f42c1190d26..90044a01d000 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -84,6 +84,8 @@ static const struct class pseudo_lock_class =3D { */ u64 resctrl_arch_get_prefetch_disable_bits(void) { + prefetch_disable_bits =3D 0; + if (boot_cpu_data.x86_vendor !=3D X86_VENDOR_INTEL || boot_cpu_data.x86 !=3D 6) return 0; @@ -99,7 +101,8 @@ u64 resctrl_arch_get_prefetch_disable_bits(void) * 3 DCU IP Prefetcher Disable (R/W) * 63:4 Reserved */ - return 0xF; + prefetch_disable_bits =3D 0xF; + break; case INTEL_ATOM_GOLDMONT: case INTEL_ATOM_GOLDMONT_PLUS: /* @@ -110,10 +113,11 @@ u64 resctrl_arch_get_prefetch_disable_bits(void) * 2 DCU Hardware Prefetcher Disable (R/W) * 63:3 Reserved */ - return 0x5; + prefetch_disable_bits =3D 0x5; + break; } =20 - return 0; + return prefetch_disable_bits; } =20 /** @@ -713,8 +717,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp) * Not knowing the bits to disable prefetching implies that this * platform does not support Cache Pseudo-Locking. */ - prefetch_disable_bits =3D resctrl_arch_get_prefetch_disable_bits(); - if (prefetch_disable_bits =3D=3D 0) { + if (resctrl_arch_get_prefetch_disable_bits() =3D=3D 0) { rdt_last_cmd_puts("Pseudo-locking not supported\n"); return -EINVAL; } --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D324C263F23 for ; Tue, 11 Mar 2025 18:41:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718507; cv=none; b=j4ej9fqyhrEW9OEiMQGtXYEmv8pe82JsJsimwUXP5Ag9TvVMB98VWXQuySa1SEdoV5dav2mDqqiUIhFq5UBZjFVDbea9AmyZt+jl/JIkYjnrUdZaJBTLeItqCvNKMqUHJFon10NnKexUiPgWq0UmDOSKkKS/MeNDXV5rXbyOMxQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718507; c=relaxed/simple; bh=BdVniiJUDyr/th7k+CUfaRiLjsqgs/UClqpGOsrUX1M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kPTrZe+ArDqK/IkFimVkettly5zNsrFnJ7WZX9zopSuEtkGc4WOMlVdhLZlZ9ymQv+zg39XYo20TLz9VbOCrEk0Wzue+niOFF5x4DsVCIfKcuY5Tm7ku+SO9JrV6Tc6zfh2lm5Z+WideVTrTpqghIck9jZeKIl4I0gQ8L1SIb0s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7EC9C1515; Tue, 11 Mar 2025 11:41:56 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 068293F694; Tue, 11 Mar 2025 11:41:41 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 26/30] x86/resctrl: Make resctrl_arch_pseudo_lock_fn() take a plr Date: Tue, 11 Mar 2025 18:37:11 +0000 Message-Id: <20250311183715.16445-27-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" resctrl_arch_pseudo_lock_fn() has architecture specific behaviour, and takes a struct rdtgroup as an argument. After the filesystem code moves to /fs/, the definition of struct rdtgroup will not be available to the architecture code. The only reason resctrl_arch_pseudo_lock_fn() wants the rdtgroup is for the CLOSID. Embed that in the pseudo_lock_region as a closid, and move the definition of struct pseudo_lock_region to resctrl.h. Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Change since v1: * [Commit message only] Typo fix: s/hw_closid/closid/g --- arch/x86/include/asm/resctrl.h | 2 +- arch/x86/kernel/cpu/resctrl/internal.h | 37 --------------------- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 12 +++---- include/linux/resctrl.h | 39 +++++++++++++++++++++++ 4 files changed, 46 insertions(+), 44 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 86407dbde583..011bf67a1866 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -204,7 +204,7 @@ static inline void resctrl_arch_mon_ctx_free(struct rdt= _resource *r, int evtid, void *ctx) { }; =20 u64 resctrl_arch_get_prefetch_disable_bits(void); -int resctrl_arch_pseudo_lock_fn(void *_rdtgrp); +int resctrl_arch_pseudo_lock_fn(void *_plr); int resctrl_arch_measure_cycles_lat_fn(void *_plr); int resctrl_arch_measure_l2_residency(void *_plr); int resctrl_arch_measure_l3_residency(void *_plr); diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 8d35bb423aad..0d13006e920b 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -208,43 +208,6 @@ struct mongroup { u32 rmid; }; =20 -/** - * struct pseudo_lock_region - pseudo-lock region information - * @s: Resctrl schema for the resource to which this - * pseudo-locked region belongs - * @d: RDT domain to which this pseudo-locked region - * belongs - * @cbm: bitmask of the pseudo-locked region - * @lock_thread_wq: waitqueue used to wait on the pseudo-locking thread - * completion - * @thread_done: variable used by waitqueue to test if pseudo-locking - * thread completed - * @cpu: core associated with the cache on which the setup code - * will be run - * @line_size: size of the cache lines - * @size: size of pseudo-locked region in bytes - * @kmem: the kernel memory associated with pseudo-locked region - * @minor: minor number of character device associated with this - * region - * @debugfs_dir: pointer to this region's directory in the debugfs - * filesystem - * @pm_reqs: Power management QoS requests related to this region - */ -struct pseudo_lock_region { - struct resctrl_schema *s; - struct rdt_ctrl_domain *d; - u32 cbm; - wait_queue_head_t lock_thread_wq; - int thread_done; - int cpu; - unsigned int line_size; - unsigned int size; - void *kmem; - unsigned int minor; - struct dentry *debugfs_dir; - struct list_head pm_reqs; -}; - /** * struct rdtgroup - store rdtgroup's data in resctrl file system. * @kn: kernfs node diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cp= u/resctrl/pseudo_lock.c index 90044a01d000..01fa7890b43f 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -414,7 +414,7 @@ static void pseudo_lock_free(struct rdtgroup *rdtgrp) =20 /** * resctrl_arch_pseudo_lock_fn - Load kernel memory into cache - * @_rdtgrp: resource group to which pseudo-lock region belongs + * @_plr: the pseudo-lock region descriptor * * This is the core pseudo-locking flow. * @@ -431,10 +431,9 @@ static void pseudo_lock_free(struct rdtgroup *rdtgrp) * * Return: 0. Waiter on waitqueue will be woken on completion. */ -int resctrl_arch_pseudo_lock_fn(void *_rdtgrp) +int resctrl_arch_pseudo_lock_fn(void *_plr) { - struct rdtgroup *rdtgrp =3D _rdtgrp; - struct pseudo_lock_region *plr =3D rdtgrp->plr; + struct pseudo_lock_region *plr =3D _plr; u32 rmid_p, closid_p; unsigned long i; u64 saved_msr; @@ -494,7 +493,8 @@ int resctrl_arch_pseudo_lock_fn(void *_rdtgrp) * pseudo-locked followed by reading of kernel memory to load it * into the cache. */ - __wrmsr(MSR_IA32_PQR_ASSOC, rmid_p, rdtgrp->closid); + __wrmsr(MSR_IA32_PQR_ASSOC, rmid_p, plr->closid); + /* * Cache was flushed earlier. Now access kernel memory to read it * into cache region associated with just activated plr->closid. @@ -1312,7 +1312,7 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtg= rp) =20 plr->thread_done =3D 0; =20 - thread =3D kthread_run_on_cpu(resctrl_arch_pseudo_lock_fn, rdtgrp, + thread =3D kthread_run_on_cpu(resctrl_arch_pseudo_lock_fn, plr, plr->cpu, "pseudo_lock/%u"); if (IS_ERR(thread)) { ret =3D PTR_ERR(thread); diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 914df24ffe48..226cc5c0d765 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -58,6 +58,45 @@ enum resctrl_conf_type { =20 #define CDP_NUM_TYPES (CDP_DATA + 1) =20 +/* + * struct pseudo_lock_region - pseudo-lock region information + * @s: Resctrl schema for the resource to which this + * pseudo-locked region belongs + * @closid: The closid that this pseudo-locked region uses + * @d: RDT domain to which this pseudo-locked region + * belongs + * @cbm: bitmask of the pseudo-locked region + * @lock_thread_wq: waitqueue used to wait on the pseudo-locking thread + * completion + * @thread_done: variable used by waitqueue to test if pseudo-locking + * thread completed + * @cpu: core associated with the cache on which the setup code + * will be run + * @line_size: size of the cache lines + * @size: size of pseudo-locked region in bytes + * @kmem: the kernel memory associated with pseudo-locked region + * @minor: minor number of character device associated with this + * region + * @debugfs_dir: pointer to this region's directory in the debugfs + * filesystem + * @pm_reqs: Power management QoS requests related to this region + */ +struct pseudo_lock_region { + struct resctrl_schema *s; + u32 closid; + struct rdt_ctrl_domain *d; + u32 cbm; + wait_queue_head_t lock_thread_wq; + int thread_done; + int cpu; + unsigned int line_size; + unsigned int size; + void *kmem; + unsigned int minor; + struct dentry *debugfs_dir; + struct list_head pm_reqs; +}; + /** * struct resctrl_staged_config - parsed configuration to be applied * @new_ctrl: new ctrl value to be loaded --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A8C1E266588 for ; Tue, 11 Mar 2025 18:41:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718511; cv=none; b=SKMh/KSlTp46LDbZnR3hMr+G476MaG1jjC5u3Ex6dqpQF/xg6nMNuoaJnk9115opx6DgirhidzblAO1vsQXv7URhFCDrdt0sUcj9Anu97tqNSgrlEFdgHW45hBqja2n/G+npvfz/hCywpxu0SQF8SitZkhII0oTH5ivAWAkkPOc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718511; c=relaxed/simple; bh=hSk8lB0mxQLdff/yB0fkodC5DIeCq8cy48atAAOobwY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iZ9NTZ8jAUhusXsvDWwOlqAJCwVCdfPDX8iCJLAdB5Home+OCo1YrOfT5sm2BIMFq1RU9O8P6qCmZk0cEmshWDkHLHRkGSkgOn8T9Adu1d8+Ub+cZ3hc4Die2Em615IbKGEoEBv4c/F2mRuJLSkY2oHr7Elr4IHCIfKuQ2TmCSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1EF881713; Tue, 11 Mar 2025 11:42:00 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B59233F694; Tue, 11 Mar 2025 11:41:45 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger Subject: [PATCH v8:for-boris 27/30] x86/resctrl: Move RFTYPE flags to be managed by resctrl Date: Tue, 11 Mar 2025 18:37:12 +0000 Message-Id: <20250311183715.16445-28-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" resctrl_file_fflags_init() is called from the architecture specific code to make the 'thread_throttle_mode' file visible. The architecture specific code has already set the membw.throttle_mode in the rdt_resource. This forces the RFTYPE flags used by resctrl to be exposed to the architecture specific code. This doesn't need to be specific to the architecture, the throttle_mode can be used by resctrl to determine if the 'thread_throttle_mode' file should be visible. This allows the RFTYPE flags to be private to resctrl. Add thread_throttle_mode_init(), and use it to call resctrl_file_fflags_init() from resctrl_init(). This avoids publishing an extra function between the architecture and filesystem code. Signed-off-by: James Morse Tested-by: Peter Newman Tested-by: Shaopeng Tan Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v6: * Refer to resctrl_init() in the commit message. * Remove a space. Changes since v5: * Added checking for SMBA. * Added printing of undefined to rdt_thread_throttle_mode_show(). * Major juggling around commit 2937f9c361f7 ("x86/resctrl: Introduce resct= rl_file_fflags_init() to initialize fflags") * Dropped tags, split patch --- arch/x86/kernel/cpu/resctrl/core.c | 3 --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++++++++++++ 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index b9b74f53b8ee..e590dd347fa7 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -227,9 +227,6 @@ static __init bool __get_mem_config_intel(struct rdt_re= source *r) else r->membw.throttle_mode =3D THREAD_THROTTLE_MAX; =20 - resctrl_file_fflags_init("thread_throttle_mode", - RFTYPE_CTRL_INFO | RFTYPE_RES_MB); - r->alloc_capable =3D true; =20 return true; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index e59271515a46..58feba3feefd 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2064,6 +2064,16 @@ static struct rftype *rdtgroup_get_rftype_by_name(co= nst char *name) return NULL; } =20 +static void thread_throttle_mode_init(void) +{ + struct rdt_resource *r_mba; + + r_mba =3D resctrl_arch_get_resource(RDT_RESOURCE_MBA); + if (r_mba->membw.throttle_mode !=3D THREAD_THROTTLE_UNDEFINED) + resctrl_file_fflags_init("thread_throttle_mode", + RFTYPE_CTRL_INFO | RFTYPE_RES_MB); +} + void resctrl_file_fflags_init(const char *config, unsigned long fflags) { struct rftype *rft; @@ -4277,6 +4287,8 @@ int __init resctrl_init(void) =20 rdtgroup_setup_default(); =20 + thread_throttle_mode_init(); + ret =3D resctrl_mon_resource_init(); if (ret) return ret; --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 26379264A6B for ; Tue, 11 Mar 2025 18:41:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718514; cv=none; b=ZFzvDzpC6TqyCt0U2WMSRojcQpuf1wappiTgqbFvBmV9bZAQdhaEHMKu5qU+EaO2D54p5WmVPzeQqM9DnQET1F2Hs0DoqhhiQkmgybaqmov3W7MJngUKhT4mr9nKC+Tu3IJo4gRMh3Yy2QAWh2+4w3wlyvOV9mQWpZ6lhTm1hO0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718514; c=relaxed/simple; bh=YFPWpHhDg2XqafTvBY/CdA4ujctbyQwWVJLDDnG3HRw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qsP1Cbzgp1KfcFNE0VqbNcbkrEJN3htfnmQRuszHIRoXKEF7hIsZk9rRznCQGcT37RoFH9bQ5rYT/WeWmEqWjjPh5IMqlFwWSXyYAxrHkRBANrfxM8JnLB9FIBhIHiLWfCGtDeceFC78dG0Sk+wHcpOTcQWMG1asop0AJWQFU2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B53241515; Tue, 11 Mar 2025 11:42:03 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 566BA3F694; Tue, 11 Mar 2025 11:41:49 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger Subject: [PATCH v8:for-boris 28/30] x86/resctrl: Handle throttle_mode for SMBA resources Date: Tue, 11 Mar 2025 18:37:13 +0000 Message-Id: <20250311183715.16445-29-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the visibility of throttle_mode is being managed by resctrl, it should consider resources other than MBA that may have a throttle_mode. SMBA is one such resource. Extend thread_throttle_mode_init() to check SMBA for a throttle_mode. Adding support for multiple resources means it is possible for a platform with both MBA and SMBA, but an undefined throttle_mode on one of them to make the file visible. Add the 'undefined' case to rdt_thread_throttle_mode_show(). Signed-off-by: James Morse Tested-by: Peter Newman Tested-by: Shaopeng Tan Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Reinette Chatre Reviewed-by: Fenghua Yu Reviewed-by: Babu Moger --- Changes since v6: * Refer to thread_throttle_mode_init() in the commit message. Changes since v5: * This change split out of the previous patch. --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 33 +++++++++++++++++++++----- 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 58feba3feefd..5fc60c9ce28f 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -1188,10 +1188,19 @@ static int rdt_thread_throttle_mode_show(struct ker= nfs_open_file *of, struct resctrl_schema *s =3D of->kn->parent->priv; struct rdt_resource *r =3D s->res; =20 - if (r->membw.throttle_mode =3D=3D THREAD_THROTTLE_PER_THREAD) + switch (r->membw.throttle_mode) { + case THREAD_THROTTLE_PER_THREAD: seq_puts(seq, "per-thread\n"); - else + return 0; + case THREAD_THROTTLE_MAX: seq_puts(seq, "max\n"); + return 0; + case THREAD_THROTTLE_UNDEFINED: + seq_puts(seq, "undefined\n"); + return 0; + } + + WARN_ON_ONCE(1); =20 return 0; } @@ -2066,12 +2075,24 @@ static struct rftype *rdtgroup_get_rftype_by_name(c= onst char *name) =20 static void thread_throttle_mode_init(void) { - struct rdt_resource *r_mba; + enum membw_throttle_mode throttle_mode =3D THREAD_THROTTLE_UNDEFINED; + struct rdt_resource *r_mba, *r_smba; =20 r_mba =3D resctrl_arch_get_resource(RDT_RESOURCE_MBA); - if (r_mba->membw.throttle_mode !=3D THREAD_THROTTLE_UNDEFINED) - resctrl_file_fflags_init("thread_throttle_mode", - RFTYPE_CTRL_INFO | RFTYPE_RES_MB); + if (r_mba->alloc_capable && + r_mba->membw.throttle_mode !=3D THREAD_THROTTLE_UNDEFINED) + throttle_mode =3D r_mba->membw.throttle_mode; + + r_smba =3D resctrl_arch_get_resource(RDT_RESOURCE_SMBA); + if (r_smba->alloc_capable && + r_smba->membw.throttle_mode !=3D THREAD_THROTTLE_UNDEFINED) + throttle_mode =3D r_smba->membw.throttle_mode; + + if (throttle_mode =3D=3D THREAD_THROTTLE_UNDEFINED) + return; + + resctrl_file_fflags_init("thread_throttle_mode", + RFTYPE_CTRL_INFO | RFTYPE_RES_MB); } =20 void resctrl_file_fflags_init(const char *config, unsigned long fflags) --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E0D23264A67 for ; Tue, 11 Mar 2025 18:41:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718518; cv=none; b=uZ5ilf/DkYXuk3VIjDwkuXTMGKNPebUcpbULPC4Ktp80CI6zmA4GXDqufI+hwgJjNPWJQh5OYORa7G0MrcD488h4/wr2+NIoYesu6kBLGSd1AByNUn8ffXnCGHBXVW4q5ShH+tb9CiqtO3YXvq6a6ecj2uOFU3XeGQpd4HEI+nY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718518; c=relaxed/simple; bh=V3ADyWurwLqs9Gx2RToM+kdldoyovN0x21VsziBo4OE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NLz9sgR3qEPbX1uB+BvkqQEYuRaPawjBQ8akSPPIiJrTHbccVKFYqrOwrSAEsWAqsf2PXJfdRSuY0YYFrjqF3OvpYHX/Qay2qMVUeb+5eBS3Es98EOZT1sNjNgYpCWMfQxRLaTCbLXx7n2OyL4SjL8GmMFu1Xd9s+qGuwoCnkbk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8B7ED1713; Tue, 11 Mar 2025 11:42:07 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EAE0E3F694; Tue, 11 Mar 2025 11:41:52 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Dave Martin , Shaopeng Tan , Babu Moger , Tony Luck Subject: [PATCH v8:for-boris 29/30] x86/resctrl: Move get_config_index() to a header Date: Tue, 11 Mar 2025 18:37:14 +0000 Message-Id: <20250311183715.16445-30-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" get_config_index() is used by the architecture specific code to map a CLOSID+type pair to an index in the configuration arrays. MPAM needs to do this too to preserve the ABI to user-space, there is no reason to do it differently. Move the helper to a header file to allow all architectures that either use or emulate CDP to use the same pattern of CLOSID values. Moving this to a header file means it must be marked inline, which matches the existing compiler choice for this static function. Co-developed-by: Dave Martin Signed-off-by: Dave Martin Signed-off-by: James Morse Tested-by: Carl Worth # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Amit Singh Tomar # arm64 Tested-by: Babu Moger Reviewed-by: Shaopeng Tan Reviewed-by: Tony Luck Reviewed-by: Fenghua Yu Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v6: * Expanded the last paragraph of the commit message. Changes since v1: * Reindent resctrl_get_config_index() as per coding-style.rst rules. --- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 19 +++---------------- include/linux/resctrl.h | 15 +++++++++++++++ 2 files changed, 18 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cp= u/resctrl/ctrlmondata.c index 1ecc93282b7d..0a0ac5f6112e 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -287,25 +287,12 @@ static int parse_line(char *line, struct resctrl_sche= ma *s, return -EINVAL; } =20 -static u32 get_config_index(u32 closid, enum resctrl_conf_type type) -{ - switch (type) { - default: - case CDP_NONE: - return closid; - case CDP_CODE: - return closid * 2 + 1; - case CDP_DATA: - return closid * 2; - } -} - int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain= *d, u32 closid, enum resctrl_conf_type t, u32 cfg_val) { struct rdt_hw_ctrl_domain *hw_dom =3D resctrl_to_arch_ctrl_dom(d); struct rdt_hw_resource *hw_res =3D resctrl_to_arch_res(r); - u32 idx =3D get_config_index(closid, t); + u32 idx =3D resctrl_get_config_index(closid, t); struct msr_param msr_param; =20 if (!cpumask_test_cpu(smp_processor_id(), &d->hdr.cpu_mask)) @@ -342,7 +329,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r,= u32 closid) if (!cfg->have_new_ctrl) continue; =20 - idx =3D get_config_index(closid, t); + idx =3D resctrl_get_config_index(closid, t); if (cfg->new_ctrl =3D=3D hw_dom->ctrl_val[idx]) continue; hw_dom->ctrl_val[idx] =3D cfg->new_ctrl; @@ -462,7 +449,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, str= uct rdt_ctrl_domain *d, u32 closid, enum resctrl_conf_type type) { struct rdt_hw_ctrl_domain *hw_dom =3D resctrl_to_arch_ctrl_dom(d); - u32 idx =3D get_config_index(closid, type); + u32 idx =3D resctrl_get_config_index(closid, type); =20 return hw_dom->ctrl_val[idx]; } diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 226cc5c0d765..880351ca3dfc 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -384,6 +384,21 @@ void resctrl_arch_mon_event_config_write(void *config_= info); */ void resctrl_arch_mon_event_config_read(void *config_info); =20 +/* For use by arch code to remap resctrl's smaller CDP CLOSID range */ +static inline u32 resctrl_get_config_index(u32 closid, + enum resctrl_conf_type type) +{ + switch (type) { + default: + case CDP_NONE: + return closid; + case CDP_CODE: + return closid * 2 + 1; + case CDP_DATA: + return closid * 2; + } +} + /* * Update the ctrl_val and apply this config right now. * Must be called on one of the domain's CPUs. --=20 2.39.5 From nobody Sun Feb 8 14:24:05 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CBE20264A67 for ; Tue, 11 Mar 2025 18:42:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718523; cv=none; b=Zwb4MmJe6IC/LI8IB/PAJpL63mrEuYuO0de9AllcNDsyVLYKCwRpJrjDTeYOVVEGrvEHHCOn0ibfqBjOeBVipgclQFC3aL7d0m8BWD+pdTQSikNBR1w10Sd+H63EZ7ddbsL6/7iGTAp3dSXENK0twTF+7DRjckX1gbY+JsZgfB4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741718523; c=relaxed/simple; bh=0+2x5TG9cq0J0FqZvGwwTpf41DpnTNH0saG60xn9FdQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gNYzV5SNq6P2suvTUOsUT/GQ5AaNlL4BidqP1/3KuAFurOsh80oqoY8A8M9SdTGc0lMeyCZL/lxE1pQTPOgA+Ap0GK1HAbUGVZJhr+6MrR8rB97N9LtY60VNGItvV35X8M7UVuX7Rl/L3L2qpEMTg23Z2O2Bws6VzTrOQqTYqhs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 36DB31762; Tue, 11 Mar 2025 11:42:11 -0700 (PDT) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C55BB3F694; Tue, 11 Mar 2025 11:41:56 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Rex Nie , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, Shaopeng Tan , Babu Moger Subject: [PATCH v8:for-boris 30/30] x86/resctrl: Move get_{mon,ctrl}_domain_from_cpu() to live with their callers Date: Tue, 11 Mar 2025 18:37:15 +0000 Message-Id: <20250311183715.16445-31-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20250311183715.16445-1-james.morse@arm.com> References: <20250311183715.16445-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Each of get_{mon,ctrl}_domain_from_cpu() only has one caller. Once the filesystem code is moved to /fs/, there is no equivalent to core.c. Move these functions to each live next to their caller. This allows them to be made static and the header file entries to be removed. Signed-off-by: James Morse Tested-by: Peter Newman Tested-by: Shaopeng Tan Tested-by: Amit Singh Tomar # arm64 Tested-by: Shanker Donthineni # arm64 Tested-by: Babu Moger Reviewed-by: Fenghua Yu Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v7: * Typo: teh->the Changes since v6: * This patch replaces one that moved these to a header, any arch code will need its own way of finding the domain to online/offline from a cpu number. --- arch/x86/kernel/cpu/resctrl/core.c | 30 -------------------------- arch/x86/kernel/cpu/resctrl/internal.h | 2 -- arch/x86/kernel/cpu/resctrl/monitor.c | 16 ++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 ++++++++++++++ 4 files changed, 32 insertions(+), 32 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index e590dd347fa7..cf29681d01e0 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -347,36 +347,6 @@ static void cat_wrmsr(struct msr_param *m) wrmsrl(hw_res->msr_base + i, hw_dom->ctrl_val[i]); } =20 -struct rdt_ctrl_domain *get_ctrl_domain_from_cpu(int cpu, struct rdt_resou= rce *r) -{ - struct rdt_ctrl_domain *d; - - lockdep_assert_cpus_held(); - - list_for_each_entry(d, &r->ctrl_domains, hdr.list) { - /* Find the domain that contains this CPU */ - if (cpumask_test_cpu(cpu, &d->hdr.cpu_mask)) - return d; - } - - return NULL; -} - -struct rdt_mon_domain *get_mon_domain_from_cpu(int cpu, struct rdt_resourc= e *r) -{ - struct rdt_mon_domain *d; - - lockdep_assert_cpus_held(); - - list_for_each_entry(d, &r->mon_domains, hdr.list) { - /* Find the domain that contains this CPU */ - if (cpumask_test_cpu(cpu, &d->hdr.cpu_mask)) - return d; - } - - return NULL; -} - u32 resctrl_arch_get_num_closid(struct rdt_resource *r) { return resctrl_to_arch_res(r)->num_closid; diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index 0d13006e920b..c44c5b496355 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -475,8 +475,6 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *= r, struct rdt_ctrl_domain unsigned long cbm); enum rdtgrp_mode rdtgroup_mode_by_closid(int closid); int rdtgroup_tasks_assigned(struct rdtgroup *r); -struct rdt_ctrl_domain *get_ctrl_domain_from_cpu(int cpu, struct rdt_resou= rce *r); -struct rdt_mon_domain *get_mon_domain_from_cpu(int cpu, struct rdt_resourc= e *r); int closids_supported(void); void closid_free(int closid); int alloc_rmid(u32 closid); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index 83f90128d768..a93ed7d2a160 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -718,6 +718,22 @@ void mon_event_count(void *info) rr->err =3D 0; } =20 +static struct rdt_ctrl_domain *get_ctrl_domain_from_cpu(int cpu, + struct rdt_resource *r) +{ + struct rdt_ctrl_domain *d; + + lockdep_assert_cpus_held(); + + list_for_each_entry(d, &r->ctrl_domains, hdr.list) { + /* Find the domain that contains this CPU */ + if (cpumask_test_cpu(cpu, &d->hdr.cpu_mask)) + return d; + } + + return NULL; +} + /* * Feedback loop for MBA software controller (mba_sc) * diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index 5fc60c9ce28f..c6274d40b217 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -4257,6 +4257,22 @@ static void clear_childcpus(struct rdtgroup *r, unsi= gned int cpu) } } =20 +static struct rdt_mon_domain *get_mon_domain_from_cpu(int cpu, + struct rdt_resource *r) +{ + struct rdt_mon_domain *d; + + lockdep_assert_cpus_held(); + + list_for_each_entry(d, &r->mon_domains, hdr.list) { + /* Find the domain that contains this CPU */ + if (cpumask_test_cpu(cpu, &d->hdr.cpu_mask)) + return d; + } + + return NULL; +} + void resctrl_offline_cpu(unsigned int cpu) { struct rdt_resource *l3 =3D resctrl_arch_get_resource(RDT_RESOURCE_L3); --=20 2.39.5