From nobody Tue Dec 16 14:57:43 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 762862F7AB1 for ; Fri, 5 Dec 2025 22:00:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764972045; cv=none; b=ZhIFYAMrZSq9AC1ZVQVywgjK6ITFXXmVZVI61AJUi5vQeZLOzVB+fwF7q/ywgk1BcDi8C3cIeuqQHaEkLsmIiHvGGo9GlTPAVbC9dUVwj0uLe0rwqUdEYOHS1EghcIu4oMUUAsUjNhT2J2LnmIm3C+3JQdUOBLNdDl90ctrXiJ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764972045; c=relaxed/simple; bh=L6Wo+78BzlPsdlQkXQR95Tj2GKrQ4Et9yr4EnE+nGCA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rhw1muys3G7AAGC6A9BMfwhzLa+GGHRaeD9lz4q6tWVWavdPo689dlWG3CNHcJWCgb4zr19iVKZ8vtCb+ixnqa1CwmnTUI/dp12XxN/fIHuRYVtPgrJ0lSZ+6+iUh8PAkTt+S85AcZkSy0bv0CmvFPesqPx1bdnolmkger7ofCA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 72D181AED; Fri, 5 Dec 2025 14:00:34 -0800 (PST) Received: from merodach.members.linode.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 102523F740; Fri, 5 Dec 2025 14:00:37 -0800 (PST) From: James Morse To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: James Morse , D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Dave Martin , Koba Ko , Shanker Donthineni , fenghuay@nvidia.com, baisheng.gao@unisoc.com, Jonathan Cameron , Gavin Shan , Ben Horgan , rohit.mathew@arm.com, reinette.chatre@intel.com, Punit Agrawal Subject: [RFC PATCH 18/38] arm_mpam: resctrl: Add support for csu counters Date: Fri, 5 Dec 2025 21:58:41 +0000 Message-Id: <20251205215901.17772-19-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20251205215901.17772-1-james.morse@arm.com> References: <20251205215901.17772-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" resctrl exposes a counter via a file named llc_occupancy. This isn't really a counter as its value goes up and down, this is a snapshot of the cache storage usage monitor. Add some picking code to find a cache as close as possible to the L3 that supports the CSU monitor. If there is an L3, but it doesn't have any controls, force the L3 resource to exist. The existing topology_matches_l3() and mpam_resctrl_domain_hdr_init() code will ensure this looks like the L3, even if the class belongs to a later cache. Signed-off-by: James Morse --- drivers/resctrl/mpam_internal.h | 6 ++ drivers/resctrl/mpam_resctrl.c | 148 ++++++++++++++++++++++++++++++++ 2 files changed, 154 insertions(+) diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_interna= l.h index 8684bd35d4ab..f9d2a1004c32 100644 --- a/drivers/resctrl/mpam_internal.h +++ b/drivers/resctrl/mpam_internal.h @@ -348,6 +348,12 @@ struct mpam_resctrl_res { struct rdt_resource resctrl_res; }; =20 +struct mpam_resctrl_mon { + struct mpam_class *class; + + /* per-class data that resctrl needs will live here */ +}; + static inline int mpam_alloc_csu_mon(struct mpam_class *class) { struct mpam_props *cprops =3D &class->props; diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c index fe830524639e..fc1f054f187e 100644 --- a/drivers/resctrl/mpam_resctrl.c +++ b/drivers/resctrl/mpam_resctrl.c @@ -31,6 +31,16 @@ static struct mpam_resctrl_res mpam_resctrl_controls[RDT= _NUM_RESOURCES]; /* The lock for modifying resctrl's domain lists from cpuhp callbacks. */ static DEFINE_MUTEX(domain_list_lock); =20 +/* + * The classes we've picked to map to resctrl events. + * Resctrl believes all the worlds a Xeon, and these are all on the L3. Th= is + * array lets us find the actual class backing the event counters. e.g. + * the only memory bandwidth counters may be on the memory controller, but= to + * make use of them, we pretend they are on L3. + * Class pointer may be NULL. + */ +static struct mpam_resctrl_mon mpam_resctrl_counters[QOS_NUM_EVENTS]; + static bool exposed_alloc_capable; static bool exposed_mon_capable; =20 @@ -258,6 +268,28 @@ static bool class_has_usable_mba(struct mpam_props *cp= rops) return mba_class_use_mbw_max(cprops); } =20 +static bool cache_has_usable_csu(struct mpam_class *class) +{ + struct mpam_props *cprops; + + if (!class) + return false; + + cprops =3D &class->props; + + if (!mpam_has_feature(mpam_feat_msmon_csu, cprops)) + return false; + + /* + * CSU counters settle on the value, so we can get away with + * having only one. + */ + if (!cprops->num_csu_mon) + return false; + + return (mpam_partid_max > 1) || (mpam_pmg_max !=3D 0); +} + /* * Calculate the worst-case percentage change from each implemented step * in the control. @@ -499,6 +531,64 @@ static void mpam_resctrl_pick_mba(void) } } =20 +static void counter_update_class(enum resctrl_event_id evt_id, + struct mpam_class *class) +{ + struct mpam_class *existing_class =3D mpam_resctrl_counters[evt_id].class; + + if (existing_class) { + if (class->level =3D=3D 3) { + pr_debug("Existing class is L3 - L3 wins\n"); + return; + } else if (existing_class->level < class->level) { + pr_debug("Existing class is closer to L3, %u versus %u - closer is bett= er\n", + existing_class->level, class->level); + return; + } + } + + mpam_resctrl_counters[evt_id].class =3D class; + exposed_mon_capable =3D true; +} + +static void mpam_resctrl_pick_counters(void) +{ + struct mpam_class *class; + bool has_csu; + + lockdep_assert_cpus_held(); + + guard(srcu)(&mpam_srcu); + list_for_each_entry_srcu(class, &mpam_classes, classes_list, + srcu_read_lock_held(&mpam_srcu)) { + if (class->level < 3) { + pr_debug("class %u is before L3", class->level); + continue; + } + + if (!cpumask_equal(&class->affinity, cpu_possible_mask)) { + pr_debug("class %u does not cover all CPUs", + class->level); + continue; + } + + has_csu =3D cache_has_usable_csu(class); + if (has_csu && topology_matches_l3(class)) { + pr_debug("class %u has usable CSU, and matches L3 topology", + class->level); + + /* CSU counters only make sense on a cache. */ + switch (class->type) { + case MPAM_CLASS_CACHE: + counter_update_class(QOS_L3_OCCUP_EVENT_ID, class); + return; + default: + return; + } + } + } +} + static int mpam_resctrl_control_init(struct mpam_resctrl_res *res, enum resctrl_res_level type) { @@ -578,6 +668,50 @@ static int mpam_resctrl_pick_domain_id(int cpu, struct= mpam_component *comp) return comp->comp_id; } =20 +static void mpam_resctrl_monitor_init(struct mpam_resctrl_mon *mon, + enum resctrl_event_id type) +{ + struct mpam_resctrl_res *res =3D &mpam_resctrl_controls[RDT_RESOURCE_L3]; + struct rdt_resource *l3 =3D &res->resctrl_res; + + lockdep_assert_cpus_held(); + + /* There also needs to be an L3 cache present */ + if (get_cpu_cacheinfo_id(smp_processor_id(), 3) =3D=3D -1) + return; + + /* + * If there are no MPAM resources on L3, force it into existence. + * topology_matches_l3() already ensures this looks like the L3. + * The domain-ids will be fixed up by mpam_resctrl_domain_hdr_init(). + */ + if (!res->class) { + pr_warn_once("Faking L3 MSC to enable counters.\n"); + res->class =3D mpam_resctrl_counters[type].class; + } + + /* Called multiple times!, once per event type */ + if (exposed_mon_capable) { + l3->mon_capable =3D true; + + /* Setting name is necessary on monitor only platforms */ + l3->name =3D "L3"; + l3->mon_scope =3D RESCTRL_L3_CACHE; + + resctrl_enable_mon_event(type); + + /* + * Unfortunately, num_rmid doesn't mean anything for + * mpam, and its exposed to user-space! + * num-rmid is supposed to mean the number of groups + * that can be created, both control or monitor groups. + * For mpam, each control group has its own pmg/rmid + * space. + */ + l3->mon.num_rmid =3D 1; + } +} + u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain= *d, u32 closid, enum resctrl_conf_type type) { @@ -939,8 +1073,10 @@ void mpam_resctrl_offline_cpu(unsigned int cpu) int mpam_resctrl_setup(void) { int err =3D 0; + enum resctrl_event_id j; enum resctrl_res_level i; struct mpam_resctrl_res *res; + struct mpam_resctrl_mon *mon; =20 cpus_read_lock(); for (i =3D 0; i < RDT_NUM_RESOURCES; i++) { @@ -966,6 +1102,18 @@ int mpam_resctrl_setup(void) break; } } + + /* Find some classes to use for monitors */ + mpam_resctrl_pick_counters(); + + for (j =3D 0; j < QOS_NUM_EVENTS; j++) { + mon =3D &mpam_resctrl_counters[j]; + if (!mon->class) + continue; // dummy resource + + mpam_resctrl_monitor_init(mon, j); + } + cpus_read_unlock(); =20 if (err || (!exposed_alloc_capable && !exposed_mon_capable)) { --=20 2.39.5