From nobody Mon Apr 6 12:17:08 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AD1F63EDAA3 for ; Thu, 19 Mar 2026 16:56:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773939366; cv=none; b=TNBS4lTITdfGtbibbbHII8x2e8277t1+U0uCCvTZDY2n2168M+PEzvWk/sttQB9hfzIZ8QRDw7Xzwlq9ays/Da8CKqb8AC9mbcs0xDKz/ZrnXSXtdf8tr+jXmhhE053SyfUZ+EwbWiMqVsUhOTnvr6q6RbYGhjxbTTO2JefjqyE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773939366; c=relaxed/simple; bh=+uvbBk/jZ1wwPlmdpeHSy39DQSGvIZfUL2hdYQjFGQo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OW6y0ugffIsEOtPmBl9PRma4aXnE93Gn6bWcZDnKMq9mo4C8rd32DP5g2NAqN/0CqGm3AAalj5HVjIWeMN80CrcZeMOiFxTnk9W6qB0+DtpTMzC5NiVPyWB6dYn3tEE+hrnpXyuG0xbA0h5AqIjGb+wZtIkAPI8hzEUfRxYeWTc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 59B4420E3; Thu, 19 Mar 2026 09:55:53 -0700 (PDT) Received: from e134344.cambridge.arm.com (e134344.arm.com [10.1.196.46]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 794683F7BD; Thu, 19 Mar 2026 09:55:56 -0700 (PDT) From: Ben Horgan To: ben.horgan@arm.com Cc: amitsinght@marvell.com, baisheng.gao@unisoc.com, baolin.wang@linux.alibaba.com, carl@os.amperecomputing.com, dave.martin@arm.com, david@kernel.org, dfustini@baylibre.com, fenghuay@nvidia.com, gshan@redhat.com, james.morse@arm.com, jonathan.cameron@huawei.com, kobak@nvidia.com, lcherian@marvell.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, peternewman@google.com, punit.agrawal@oss.qualcomm.com, quic_jiles@quicinc.com, reinette.chatre@intel.com, rohit.mathew@arm.com, scott@os.amperecomputing.com, sdonthineni@nvidia.com, tan.shaopeng@fujitsu.com, xhao@linux.alibaba.com, zengheng4@huawei.com Subject: [RFC PATCH v2 2/5] arm_mpam: resctrl: Pre-allocate assignable monitors Date: Thu, 19 Mar 2026 16:55:37 +0000 Message-ID: <20260319165540.381410-3-ben.horgan@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260319165540.381410-1-ben.horgan@arm.com> References: <20260319165540.381410-1-ben.horgan@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" MPAM is able to emulate ABMC, i.e. mbm_event mode, by making memory bandwidth monitors assignable. Rather than supporting the 'default' mbm_assign_mode always use 'mbm_event'mode even if there are sufficient memory bandwidth monitors. The per monitor event configuration is only provided by resctrl when in 'mbm_event' mode and so only allowing 'mbm_event' mode will make it easier to support per-monitor event configuration for MPAM. For the moment, the only event supported is mbm_total_event with no bandwidth type configuration. The 'mbm_assign_mode' file will still show 'default' when there is no support for memory bandwidth monitoring. The monitors need to be allocated from the driver, and mapped to whichever control/monitor group resctrl wants to use them with. Add a second array to hold the monitor values indexed by resctrl's cntr_id. When CDP is in use, two monitors are needed so the available number of counters halves. Platforms with one monitor will have zero monitors when CDP is in use. Co-developed-by: James Morse Signed-off-by: James Morse Signed-off-by: Ben Horgan --- Changes since rfc v1: abmc enabled even if enough counters Helpers from dropped free running commits carry on with zero counters if using cdp set config bits use kmalloc_objs drop tags for rework Configure mbm_cntr_configurable, mbm_cntr_assign_fixed --- drivers/resctrl/mpam_internal.h | 6 +- drivers/resctrl/mpam_resctrl.c | 135 +++++++++++++++++++++++++++++++- 2 files changed, 137 insertions(+), 4 deletions(-) diff --git a/drivers/resctrl/mpam_internal.h b/drivers/resctrl/mpam_interna= l.h index dbb99d9b0795..02807531bd1b 100644 --- a/drivers/resctrl/mpam_internal.h +++ b/drivers/resctrl/mpam_internal.h @@ -415,7 +415,11 @@ struct mpam_resctrl_res { struct mpam_resctrl_mon { struct mpam_class *class; =20 - /* per-class data that resctrl needs will live here */ + /* Array of allocated MBWU monitors, indexed by (closid, rmid). */ + int *mbwu_idx_to_mon; + + /* Array of assigned MBWU monitors, indexed by idx argument. */ + int *assigned_counters; }; =20 static inline int mpam_alloc_csu_mon(struct mpam_class *class) diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c index c17577e52f58..74b6ca59ce4a 100644 --- a/drivers/resctrl/mpam_resctrl.c +++ b/drivers/resctrl/mpam_resctrl.c @@ -75,6 +75,8 @@ static DECLARE_WAIT_QUEUE_HEAD(wait_cacheinfo_ready); */ static bool resctrl_enabled; =20 +static unsigned int l3_num_allocated_mbwu =3D ~0; + bool resctrl_arch_alloc_capable(void) { struct mpam_resctrl_res *res; @@ -140,7 +142,7 @@ int resctrl_arch_cntr_read(struct rdt_resource *r, stru= ct rdt_l3_mon_domain *d, =20 bool resctrl_arch_mbm_cntr_assign_enabled(struct rdt_resource *r) { - return false; + return (r =3D=3D &mpam_resctrl_controls[RDT_RESOURCE_L3].resctrl_res); } =20 int resctrl_arch_mbm_cntr_assign_set(struct rdt_resource *r, bool enable) @@ -185,6 +187,22 @@ static void resctrl_reset_task_closids(void) read_unlock(&tasklist_lock); } =20 +static void mpam_resctrl_monitor_sync_abmc_vals(struct rdt_resource *l3) +{ + l3->mon.num_mbm_cntrs =3D l3_num_allocated_mbwu; + if (cdp_enabled) + l3->mon.num_mbm_cntrs /=3D 2; + + /* + * Continue as normal even if there are zero counters to avoid giving + * resctrl mixed messages. + */ + l3->mon.mbm_cntr_assignable =3D true; + l3->mon.mbm_assign_on_mkdir =3D true; + l3->mon.mbm_cntr_configurable =3D false; + l3->mon.mbm_cntr_assign_fixed =3D true; +} + int resctrl_arch_set_cdp_enabled(enum resctrl_res_level rid, bool enable) { u32 partid_i =3D RESCTRL_RESERVED_CLOSID, partid_d =3D RESCTRL_RESERVED_C= LOSID; @@ -236,6 +254,7 @@ int resctrl_arch_set_cdp_enabled(enum resctrl_res_level= rid, bool enable) WRITE_ONCE(arm64_mpam_global_default, mpam_get_regval(current)); =20 resctrl_reset_task_closids(); + mpam_resctrl_monitor_sync_abmc_vals(l3); =20 for_each_possible_cpu(cpu) mpam_set_cpu_defaults(cpu, partid_d, partid_i, 0, 0); @@ -605,6 +624,9 @@ static bool class_has_usable_mbwu(struct mpam_class *cl= ass) if (!mpam_has_feature(mpam_feat_msmon_mbwu, cprops)) return false; =20 + if (!cprops->num_mbwu_mon) + return false; + return true; } =20 @@ -933,6 +955,52 @@ static void mpam_resctrl_pick_mba(void) } } =20 +static void __free_mbwu_mon(struct mpam_class *class, int *array, + u16 num_mbwu_mon) +{ + for (int i =3D 0; i < num_mbwu_mon; i++) { + if (array[i] < 0) + continue; + + mpam_free_mbwu_mon(class, array[i]); + array[i] =3D ~0; + } +} + +static int __alloc_mbwu_mon(struct mpam_class *class, int *array, + u16 num_mbwu_mon) +{ + for (int i =3D 0; i < num_mbwu_mon; i++) { + int mbwu_mon =3D mpam_alloc_mbwu_mon(class); + + if (mbwu_mon < 0) { + __free_mbwu_mon(class, array, num_mbwu_mon); + return mbwu_mon; + } + array[i] =3D mbwu_mon; + } + + l3_num_allocated_mbwu =3D min(l3_num_allocated_mbwu, num_mbwu_mon); + + return 0; +} + +static int *__alloc_mbwu_array(struct mpam_class *class, u16 num_mbwu_mon) +{ + int err; + int *array __free(kfree) =3D kmalloc_objs(*array, num_mbwu_mon); + + if (!array) + return ERR_PTR(-ENOMEM); + + memset(array, -1, num_mbwu_mon * sizeof(*array)); + + err =3D __alloc_mbwu_mon(class, array, num_mbwu_mon); + if (err) + return ERR_PTR(err); + return_ptr(array); +} + static void counter_update_class(enum resctrl_event_id evt_id, struct mpam_class *class) { @@ -1087,9 +1155,46 @@ static int mpam_resctrl_pick_domain_id(int cpu, stru= ct mpam_component *comp) return comp->comp_id; } =20 +/* + * This must run after all event counters have been picked so that any free + * running counters have already been allocated. + */ +static int mpam_resctrl_monitor_init_abmc(struct mpam_resctrl_mon *mon) +{ + struct mpam_resctrl_res *res =3D &mpam_resctrl_controls[RDT_RESOURCE_L3]; + struct rdt_resource *l3 =3D &res->resctrl_res; + struct mpam_class *class =3D mon->class; + u16 num_mbwu_mon; + size_t num_rmid =3D resctrl_arch_system_num_rmid_idx(); + + if (mon->mbwu_idx_to_mon) { + pr_debug("monitors free running\n"); + return 0; + } + + int *rmid_array __free(kfree) =3D kmalloc_objs(*rmid_array, num_rmid); + + if (!rmid_array) { + pr_debug("Failed to allocate RMID array\n"); + return -ENOMEM; + } + memset(rmid_array, -1, num_rmid * sizeof(*rmid_array)); + + num_mbwu_mon =3D class->props.num_mbwu_mon; + mon->assigned_counters =3D __alloc_mbwu_array(mon->class, num_mbwu_mon); + if (IS_ERR(mon->assigned_counters)) + return PTR_ERR(mon->assigned_counters); + mon->mbwu_idx_to_mon =3D no_free_ptr(rmid_array); + + mpam_resctrl_monitor_sync_abmc_vals(l3); + + return 0; +} + static int mpam_resctrl_monitor_init(struct mpam_resctrl_mon *mon, enum resctrl_event_id type) { + int err =3D 0; struct mpam_resctrl_res *res =3D &mpam_resctrl_controls[RDT_RESOURCE_L3]; struct rdt_resource *l3 =3D &res->resctrl_res; =20 @@ -1131,8 +1236,19 @@ static int mpam_resctrl_monitor_init(struct mpam_res= ctrl_mon *mon, */ l3->mon.num_rmid =3D resctrl_arch_system_num_rmid_idx(); =20 - if (resctrl_enable_mon_event(type, false, 0, NULL)) - l3->mon_capable =3D true; + if (type =3D=3D QOS_L3_MBM_TOTAL_EVENT_ID) { + err =3D mpam_resctrl_monitor_init_abmc(mon); + if (err) + return err; + + static_assert(MAX_EVT_CONFIG_BITS =3D=3D 0x7f); + l3->mon.mbm_cfg_mask =3D MAX_EVT_CONFIG_BITS; + } + + if (!resctrl_enable_mon_event(type, false, 0, NULL)) + return -EINVAL; + + l3->mon_capable =3D true; =20 return 0; } @@ -1695,6 +1811,17 @@ void mpam_resctrl_exit(void) resctrl_exit(); } =20 +static void mpam_resctrl_teardown_mon(struct mpam_resctrl_mon *mon, struct= mpam_class *class) +{ + u32 num_mbwu_mon =3D resctrl_arch_system_num_rmid_idx(); + + if (!mon->mbwu_idx_to_mon) + return; + + __free_mbwu_mon(class, mon->mbwu_idx_to_mon, num_mbwu_mon); + mon->mbwu_idx_to_mon =3D NULL; +} + /* * The driver is detaching an MSC from this class, if resctrl was using it, * pull on resctrl_exit(). @@ -1717,6 +1844,8 @@ void mpam_resctrl_teardown_class(struct mpam_class *c= lass) for_each_mpam_resctrl_mon(mon, eventid) { if (mon->class =3D=3D class) { mon->class =3D NULL; + + mpam_resctrl_teardown_mon(mon, class); break; } } --=20 2.43.0