From nobody Mon Feb 9 08:54:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD62AC77B7E for ; Thu, 25 May 2023 18:02:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240819AbjEYSCu (ORCPT ); Thu, 25 May 2023 14:02:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60588 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240536AbjEYSCn (ORCPT ); Thu, 25 May 2023 14:02:43 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 644971B3 for ; Thu, 25 May 2023 11:02:35 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 36F1415BF; Thu, 25 May 2023 11:03:20 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6AB793F6C4; Thu, 25 May 2023 11:02:32 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com Subject: [PATCH v4 02/24] x86/resctrl: Access per-rmid structures by index Date: Thu, 25 May 2023 18:01:47 +0000 Message-Id: <20230525180209.19497-3-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230525180209.19497-1-james.morse@arm.com> References: <20230525180209.19497-1-james.morse@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Because of the differences between Intel RDT/AMD QoS and Arm's MPAM monitors, RMID values on arm64 are not unique unless the CLOSID is also included. Bitmaps like rmid_busy_llc need to be sized by the number of unique entries for this resource. Add helpers to encode/decode the CLOSID and RMID to an index. The domain's rmid_busy_llc and rmid_ptrs[] are then sized by index, as are the domain mbm_local and mbm_total arrays. On x86, the index is always just the RMID, so all these structures remain the same size. The index gives resctrl a unique value it can use to store monitor values, and allows MPAM to decode the CLOSID when reading the hardware counters. Tested-by: Shaopeng Tan Signed-off-by: James Morse --- Changes since v1: * Added X86_BAD_CLOSID macro to make it clear what this value means * Added second WARN_ON() for closid checking, and made both _ONCE() Changes since v2: * Added RESCTRL_RESERVED_CLOSID * Removed a newline * Repharsed some comments * Renamed a variable 'ignore'd * Moved X86_RESCTRL_BAD_CLOSID to a previous patch Changes since v3: * Changed a variable name * Fixed various typos --- arch/x86/include/asm/resctrl.h | 17 ++++++ arch/x86/kernel/cpu/resctrl/core.c | 2 +- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 84 +++++++++++++++++--------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 7 ++- include/linux/resctrl.h | 3 + 6 files changed, 83 insertions(+), 31 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index e906070285fb..dd9b638d43c8 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -101,6 +101,23 @@ static inline void resctrl_sched_in(struct task_struct= *tsk) __resctrl_sched_in(tsk); } =20 +static inline u32 resctrl_arch_system_num_rmid_idx(void) +{ + /* RMID are independent numbers for x86. num_rmid_idx =3D=3D num_rmid */ + return boot_cpu_data.x86_cache_max_rmid + 1; +} + +static inline void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 = *rmid) +{ + *rmid =3D idx; + *closid =3D X86_RESCTRL_BAD_CLOSID; +} + +static inline u32 resctrl_arch_rmid_idx_encode(u32 ignored, u32 rmid) +{ + return rmid; +} + void resctrl_cpu_detect(struct cpuinfo_x86 *c); =20 #else diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resct= rl/core.c index 030d3b409768..4bea032d072e 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -600,7 +600,7 @@ static void clear_closid_rmid(int cpu) state->default_rmid =3D 0; state->cur_closid =3D 0; state->cur_rmid =3D 0; - wrmsr(MSR_IA32_PQR_ASSOC, 0, 0); + wrmsr(MSR_IA32_PQR_ASSOC, 0, RESCTRL_RESERVED_CLOSID); } =20 static int resctrl_online_cpu(unsigned int cpu) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/r= esctrl/internal.h index f2da908bb079..d571da4848a4 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -7,6 +7,7 @@ #include #include #include +#include =20 #define L3_QOS_CDP_ENABLE 0x01ULL =20 diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/re= sctrl/monitor.c index 86574adedd64..bcc25f5339c0 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -142,12 +142,29 @@ static inline u64 get_corrected_mbm_count(u32 rmid, u= nsigned long val) return val; } =20 -static inline struct rmid_entry *__rmid_entry(u32 closid, u32 rmid) +/* + * x86 and arm64 differ in their handling of monitoring. + * x86's RMID are an independent number, there is only one source of traff= ic + * with an RMID value of '1'. + * arm64's PMG extend the PARTID/CLOSID space, there are multiple sources = of + * traffic with a PMG value of '1', one for each CLOSID, meaning the RMID + * value is no longer unique. + * To account for this, resctrl uses an index. On x86 this is just the RMI= D, + * on arm64 it encodes the CLOSID and RMID. This gives a unique number. + * + * The domain's rmid_busy_llc and rmid_ptrs are sized by index. The arch c= ode + * must accept an attempt to read every index. + */ +static inline struct rmid_entry *__rmid_entry(u32 idx) { struct rmid_entry *entry; + u32 closid, rmid; =20 - entry =3D &rmid_ptrs[rmid]; - WARN_ON(entry->rmid !=3D rmid); + entry =3D &rmid_ptrs[idx]; + resctrl_arch_rmid_idx_decode(idx, &closid, &rmid); + + WARN_ON_ONCE(entry->closid !=3D closid); + WARN_ON_ONCE(entry->rmid !=3D rmid); =20 return entry; } @@ -277,8 +294,9 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, stru= ct rdt_domain *d, void __check_limbo(struct rdt_domain *d, bool force_free) { struct rdt_resource *r =3D &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + u32 idx_limit =3D resctrl_arch_system_num_rmid_idx(); struct rmid_entry *entry; - u32 crmid =3D 1, nrmid; + u32 idx, cur_idx =3D 1; bool rmid_dirty; u64 val =3D 0; =20 @@ -289,12 +307,11 @@ void __check_limbo(struct rdt_domain *d, bool force_f= ree) * RMID and move it to the free list when the counter reaches 0. */ for (;;) { - nrmid =3D find_next_bit(d->rmid_busy_llc, r->num_rmid, crmid); - if (nrmid >=3D r->num_rmid) + idx =3D find_next_bit(d->rmid_busy_llc, idx_limit, cur_idx); + if (idx >=3D idx_limit) break; =20 - entry =3D __rmid_entry(X86_RESCTRL_BAD_CLOSID, nrmid);// temporary - + entry =3D __rmid_entry(idx); if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid, QOS_L3_OCCUP_EVENT_ID, &val)) { rmid_dirty =3D true; @@ -303,19 +320,21 @@ void __check_limbo(struct rdt_domain *d, bool force_f= ree) } =20 if (force_free || !rmid_dirty) { - clear_bit(entry->rmid, d->rmid_busy_llc); + clear_bit(idx, d->rmid_busy_llc); if (!--entry->busy) { rmid_limbo_count--; list_add_tail(&entry->list, &rmid_free_lru); } } - crmid =3D nrmid + 1; + cur_idx =3D idx + 1; } } =20 bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d) { - return find_first_bit(d->rmid_busy_llc, r->num_rmid) !=3D r->num_rmid; + u32 idx_limit =3D resctrl_arch_system_num_rmid_idx(); + + return find_first_bit(d->rmid_busy_llc, idx_limit) !=3D idx_limit; } =20 /* @@ -345,6 +364,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) struct rdt_domain *d; int cpu, err; u64 val =3D 0; + u32 idx; + + idx =3D resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid); =20 entry->busy =3D 0; cpu =3D get_cpu(); @@ -364,7 +386,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) */ if (!has_busy_rmid(r, d)) cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL); - set_bit(entry->rmid, d->rmid_busy_llc); + set_bit(idx, d->rmid_busy_llc); entry->busy++; } put_cpu(); @@ -377,14 +399,16 @@ static void add_rmid_to_limbo(struct rmid_entry *entr= y) =20 void free_rmid(u32 closid, u32 rmid) { + u32 idx =3D resctrl_arch_rmid_idx_encode(closid, rmid); struct rmid_entry *entry; =20 - if (!rmid) - return; - lockdep_assert_held(&rdtgroup_mutex); =20 - entry =3D __rmid_entry(closid, rmid); + /* do not allow the default rmid to be free'd */ + if (!idx) + return; + + entry =3D __rmid_entry(idx); =20 if (is_llc_occupancy_enabled()) add_rmid_to_limbo(entry); @@ -395,11 +419,13 @@ void free_rmid(u32 closid, u32 rmid) static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 closid, u32 rmid, enum resctrl_event_id evtid) { + u32 idx =3D resctrl_arch_rmid_idx_encode(closid, rmid); + switch (evtid) { case QOS_L3_MBM_TOTAL_EVENT_ID: - return &d->mbm_total[rmid]; + return &d->mbm_total[idx]; case QOS_L3_MBM_LOCAL_EVENT_ID: - return &d->mbm_local[rmid]; + return &d->mbm_local[idx]; default: return NULL; } @@ -441,7 +467,8 @@ static int __mon_event_count(u32 closid, u32 rmid, stru= ct rmid_read *rr) */ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) { - struct mbm_state *m =3D &rr->d->mbm_local[rmid]; + u32 idx =3D resctrl_arch_rmid_idx_encode(closid, rmid); + struct mbm_state *m =3D &rr->d->mbm_local[idx]; u64 cur_bw, bytes, cur_bytes; =20 cur_bytes =3D rr->val; @@ -531,7 +558,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct= rdt_domain *dom_mbm) { u32 closid, rmid, cur_msr_val, new_msr_val; struct mbm_state *pmbm_data, *cmbm_data; - u32 cur_bw, delta_bw, user_bw; + u32 cur_bw, delta_bw, user_bw, idx; struct rdt_resource *r_mba; struct rdt_domain *dom_mba; struct list_head *head; @@ -544,7 +571,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct= rdt_domain *dom_mbm) =20 closid =3D rgrp->closid; rmid =3D rgrp->mon.rmid; - pmbm_data =3D &dom_mbm->mbm_local[rmid]; + idx =3D resctrl_arch_rmid_idx_encode(closid, rmid); + pmbm_data =3D &dom_mbm->mbm_local[idx]; =20 dom_mba =3D get_domain_from_cpu(smp_processor_id(), r_mba); if (!dom_mba) { @@ -727,19 +755,20 @@ void mbm_setup_overflow_handler(struct rdt_domain *do= m, unsigned long delay_ms) =20 static int dom_data_init(struct rdt_resource *r) { + u32 idx_limit =3D resctrl_arch_system_num_rmid_idx(); struct rmid_entry *entry =3D NULL; - int i, nr_rmids; + u32 idx; + int i; =20 - nr_rmids =3D r->num_rmid; - rmid_ptrs =3D kcalloc(nr_rmids, sizeof(struct rmid_entry), GFP_KERNEL); + rmid_ptrs =3D kcalloc(idx_limit, sizeof(struct rmid_entry), GFP_KERNEL); if (!rmid_ptrs) return -ENOMEM; =20 - for (i =3D 0; i < nr_rmids; i++) { + for (i =3D 0; i < idx_limit; i++) { entry =3D &rmid_ptrs[i]; INIT_LIST_HEAD(&entry->list); =20 - entry->rmid =3D i; + resctrl_arch_rmid_idx_decode(i, &entry->closid, &entry->rmid); list_add_tail(&entry->list, &rmid_free_lru); } =20 @@ -748,7 +777,8 @@ static int dom_data_init(struct rdt_resource *r) * default_rdtgroup control group, which will be setup later. See * rdtgroup_setup_root(). */ - entry =3D __rmid_entry(0, 0); + idx =3D resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID, 0); + entry =3D __rmid_entry(idx); list_del(&entry->list); =20 return 0; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/r= esctrl/rdtgroup.c index ff9ccfcd18bd..023eae69f29e 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -3604,16 +3604,17 @@ void resctrl_offline_domain(struct rdt_resource *r,= struct rdt_domain *d) =20 static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domai= n *d) { + u32 idx_limit =3D resctrl_arch_system_num_rmid_idx(); size_t tsize; =20 if (is_llc_occupancy_enabled()) { - d->rmid_busy_llc =3D bitmap_zalloc(r->num_rmid, GFP_KERNEL); + d->rmid_busy_llc =3D bitmap_zalloc(idx_limit, GFP_KERNEL); if (!d->rmid_busy_llc) return -ENOMEM; } if (is_mbm_total_enabled()) { tsize =3D sizeof(*d->mbm_total); - d->mbm_total =3D kcalloc(r->num_rmid, tsize, GFP_KERNEL); + d->mbm_total =3D kcalloc(idx_limit, tsize, GFP_KERNEL); if (!d->mbm_total) { bitmap_free(d->rmid_busy_llc); return -ENOMEM; @@ -3621,7 +3622,7 @@ static int domain_setup_mon_state(struct rdt_resource= *r, struct rdt_domain *d) } if (is_mbm_local_enabled()) { tsize =3D sizeof(*d->mbm_local); - d->mbm_local =3D kcalloc(r->num_rmid, tsize, GFP_KERNEL); + d->mbm_local =3D kcalloc(idx_limit, tsize, GFP_KERNEL); if (!d->mbm_local) { bitmap_free(d->rmid_busy_llc); kfree(d->mbm_total); diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 7d80bae05f59..ff7452f644e4 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -6,6 +6,9 @@ #include #include =20 +/* CLOSID value used by the default control group */ +#define RESCTRL_RESERVED_CLOSID 0 + #ifdef CONFIG_PROC_CPU_RESCTRL =20 int proc_resctrl_show(struct seq_file *m, --=20 2.39.2