From nobody Wed Dec 31 00:37:09 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6F2BC4167B for ; Mon, 13 Nov 2023 11:28:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230454AbjKML2C (ORCPT ); Mon, 13 Nov 2023 06:28:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230056AbjKML1s (ORCPT ); Mon, 13 Nov 2023 06:27:48 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0A3BE49CB; Mon, 13 Nov 2023 03:26:06 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AC2711007; Mon, 13 Nov 2023 03:26:41 -0800 (PST) Received: from e127643.arm.com (unknown [10.57.71.191]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D16533F6C4; Mon, 13 Nov 2023 03:25:53 -0800 (PST) From: James Clark To: linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, suzuki.poulose@arm.com, will@kernel.org, mark.rutland@arm.com Cc: James Clark , Catalin Marinas , Jonathan Corbet , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 1/3] arm64: perf: Include threshold control fields in PMEVTYPER mask Date: Mon, 13 Nov 2023 11:25:04 +0000 Message-Id: <20231113112507.917107-2-james.clark@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231113112507.917107-1-james.clark@arm.com> References: <20231113112507.917107-1-james.clark@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" FEAT_PMUv3_TH (Armv8.8) adds two new fields to PMEVTYPER, so include them in the mask. These aren't writable on 32 bit kernels as they are in the high part of the register, so only include them for arm64. It would be difficult to do this statically in the asm header files for each platform without resulting in circular includes or #ifdefs inline in the code. For that reason the ARMV8_PMU_EVTYPE_MASK definition has been removed and the mask is constructed programmatically. Signed-off-by: James Clark Reviewed-by: Anshuman Khandual Reviewed-by: Suzuki K Poulose --- drivers/perf/arm_pmuv3.c | 9 ++++++++- include/linux/perf/arm_pmuv3.h | 3 ++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c index 6ca7be05229c..1d40d794f5e4 100644 --- a/drivers/perf/arm_pmuv3.c +++ b/drivers/perf/arm_pmuv3.c @@ -555,8 +555,15 @@ static void armv8pmu_write_counter(struct perf_event *= event, u64 value) static inline void armv8pmu_write_evtype(int idx, u32 val) { u32 counter =3D ARMV8_IDX_TO_COUNTER(idx); + unsigned long mask =3D ARMV8_PMU_EVTYPE_EVENT | + ARMV8_PMU_INCLUDE_EL2 | + ARMV8_PMU_EXCLUDE_EL0 | + ARMV8_PMU_EXCLUDE_EL1; =20 - val &=3D ARMV8_PMU_EVTYPE_MASK; + if (IS_ENABLED(CONFIG_ARM64)) + mask |=3D ARMV8_PMU_EVTYPE_TC | ARMV8_PMU_EVTYPE_TH; + + val &=3D mask; write_pmevtypern(counter, val); } =20 diff --git a/include/linux/perf/arm_pmuv3.h b/include/linux/perf/arm_pmuv3.h index 9c226adf938a..ddd1fec86739 100644 --- a/include/linux/perf/arm_pmuv3.h +++ b/include/linux/perf/arm_pmuv3.h @@ -228,7 +228,8 @@ /* * PMXEVTYPER: Event selection reg */ -#define ARMV8_PMU_EVTYPE_MASK 0xc800ffff /* Mask for writable bits */ +#define ARMV8_PMU_EVTYPE_TH GENMASK(43, 32) +#define ARMV8_PMU_EVTYPE_TC GENMASK(63, 61) #define ARMV8_PMU_EVTYPE_EVENT 0xffff /* Mask for EVENT bits */ =20 /* --=20 2.34.1 From nobody Wed Dec 31 00:37:09 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E78CC4332F for ; Mon, 13 Nov 2023 11:27:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231404AbjKML1X (ORCPT ); Mon, 13 Nov 2023 06:27:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231231AbjKML1H (ORCPT ); Mon, 13 Nov 2023 06:27:07 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id EC4BB19B1; Mon, 13 Nov 2023 03:26:07 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BD6BB14BF; Mon, 13 Nov 2023 03:26:44 -0800 (PST) Received: from e127643.arm.com (unknown [10.57.71.191]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 39BF13F6C4; Mon, 13 Nov 2023 03:25:57 -0800 (PST) From: James Clark To: linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, suzuki.poulose@arm.com, will@kernel.org, mark.rutland@arm.com Cc: James Clark , Catalin Marinas , Jonathan Corbet , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 2/3] arm64: perf: Add support for event counting threshold Date: Mon, 13 Nov 2023 11:25:05 +0000 Message-Id: <20231113112507.917107-3-james.clark@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231113112507.917107-1-james.clark@arm.com> References: <20231113112507.917107-1-james.clark@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" FEAT_PMUv3_TH (Armv8.8) permits a PMU counter to increment only on events whose count meets a specified threshold condition. For example if PMEVTYPERn.TC (Threshold Control) is set to 0b101 (Greater than or equal, count), and the threshold is set to 2, then the PMU counter will now only increment by 1 when an event would have previously incremented the PMU counter by 2 or more on a single processor cycle. Three new Perf event config fields, 'threshold', 'threshold_compare' and 'threshold_count' have been added to control the feature. threshold_compare maps to the upper two bits of PMEVTYPERn.TC and threshold_count maps to the first bit of TC. These separate attributes have been picked rather than enumerating all the possible combinations of the TC field as in the Arm ARM. The attributes would be used on a Perf command line like this: $ perf stat -e stall_slot/threshold=3D2,threshold_compare=3D2/ A new capability for reading out the maximum supported threshold value has also been added: $ cat /sys/bus/event_source/devices/armv8_pmuv3/caps/threshold_max 0x000000ff If a threshold higher than threshold_max is provided, then no error is generated but the threshold is clamped to the max value. If FEAT_PMUv3_TH isn't implemented or a 32 bit kernel is running, then threshold_max reads zero, and neither the 'threshold' nor 'threshold_control' parameters will be used. The threshold is per PMU counter, and there are potentially different threshold_max values per PMU type on heterogeneous systems. Bits higher than 32 now need to be written into PMEVTYPER, so armv8pmu_write_evtype() has to be updated to take an unsigned long value rather than u32 which gives the correct behavior on both aarch32 and 64. Signed-off-by: James Clark Reviewed-by: Suzuki K Poulose --- drivers/perf/arm_pmuv3.c | 79 +++++++++++++++++++++++++++++++++- include/linux/perf/arm_pmuv3.h | 1 + 2 files changed, 79 insertions(+), 1 deletion(-) diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c index 1d40d794f5e4..694d914ffc08 100644 --- a/drivers/perf/arm_pmuv3.c +++ b/drivers/perf/arm_pmuv3.c @@ -15,6 +15,7 @@ #include =20 #include +#include #include #include #include @@ -294,9 +295,18 @@ static const struct attribute_group armv8_pmuv3_events= _attr_group =3D { .is_visible =3D armv8pmu_event_attr_is_visible, }; =20 +#define TH_LO 2 +#define TH_HI 13 +#define TH_CNT 14 +#define TH_CMP_LO 15 +#define TH_CMP_HI 16 + PMU_FORMAT_ATTR(event, "config:0-15"); PMU_FORMAT_ATTR(long, "config1:0"); PMU_FORMAT_ATTR(rdpmc, "config1:1"); +PMU_FORMAT_ATTR(threshold, "config1:" __stringify(TH_LO) "-" __stringify(T= H_HI)); +PMU_FORMAT_ATTR(threshold_compare, "config1:" __stringify(TH_CMP_LO) "-" _= _stringify(TH_CMP_HI)); +PMU_FORMAT_ATTR(threshold_count, "config1:" __stringify(TH_CNT)); =20 static int sysctl_perf_user_access __read_mostly; =20 @@ -310,10 +320,32 @@ static inline bool armv8pmu_event_want_user_access(st= ruct perf_event *event) return event->attr.config1 & 0x2; } =20 +static inline u32 armv8pmu_event_threshold(struct perf_event_attr *attr) +{ + return FIELD_GET(GENMASK(TH_HI, TH_LO), attr->config1); +} + +static inline u8 armv8pmu_event_threshold_control(struct perf_event_attr *= attr) +{ + u8 th_compare =3D FIELD_GET(GENMASK(TH_CMP_HI, TH_CMP_LO), attr->config1); + u8 th_count =3D FIELD_GET(BIT(TH_CNT), attr->config1); + + /* + * The count bit is always the bottom bit of the full control field, and + * the comparison is the upper two bits, but it's not explicitly + * labelled in the Arm ARM. For the Perf interface we split it into two + * fields, so reconstruct it here. + */ + return (th_compare << 1) | th_count; +} + static struct attribute *armv8_pmuv3_format_attrs[] =3D { &format_attr_event.attr, &format_attr_long.attr, &format_attr_rdpmc.attr, + &format_attr_threshold.attr, + &format_attr_threshold_compare.attr, + &format_attr_threshold_count.attr, NULL, }; =20 @@ -365,10 +397,38 @@ static ssize_t bus_width_show(struct device *dev, str= uct device_attribute *attr, =20 static DEVICE_ATTR_RO(bus_width); =20 +static u32 threshold_max(struct arm_pmu *cpu_pmu) +{ + /* + * PMMIR.WIDTH is readable and non-zero on aarch32, but it would be + * impossible to write the threshold in the upper 32 bits of PMEVTYPER. + */ + if (IS_ENABLED(CONFIG_ARM)) + return 0; + + /* + * The largest value that can be written to PMEVTYPER_EL0.TH is + * (2 ^ PMMIR.THWIDTH) - 1. + */ + return (1 << FIELD_GET(ARMV8_PMU_THWIDTH, cpu_pmu->reg_pmmir)) - 1; +} + +static ssize_t threshold_max_show(struct device *dev, + struct device_attribute *attr, char *page) +{ + struct pmu *pmu =3D dev_get_drvdata(dev); + struct arm_pmu *cpu_pmu =3D container_of(pmu, struct arm_pmu, pmu); + + return sysfs_emit(page, "0x%08x\n", threshold_max(cpu_pmu)); +} + +static DEVICE_ATTR_RO(threshold_max); + static struct attribute *armv8_pmuv3_caps_attrs[] =3D { &dev_attr_slots.attr, &dev_attr_bus_slots.attr, &dev_attr_bus_width.attr, + &dev_attr_threshold_max.attr, NULL, }; =20 @@ -552,7 +612,7 @@ static void armv8pmu_write_counter(struct perf_event *e= vent, u64 value) armv8pmu_write_hw_counter(event, value); } =20 -static inline void armv8pmu_write_evtype(int idx, u32 val) +static inline void armv8pmu_write_evtype(int idx, unsigned long val) { u32 counter =3D ARMV8_IDX_TO_COUNTER(idx); unsigned long mask =3D ARMV8_PMU_EVTYPE_EVENT | @@ -921,6 +981,10 @@ static int armv8pmu_set_event_filter(struct hw_perf_ev= ent *event, struct perf_event_attr *attr) { unsigned long config_base =3D 0; + struct perf_event *perf_event =3D container_of(attr, struct perf_event, + attr); + struct arm_pmu *cpu_pmu =3D to_arm_pmu(perf_event->pmu); + u32 th, th_max; =20 if (attr->exclude_idle) return -EPERM; @@ -952,6 +1016,19 @@ static int armv8pmu_set_event_filter(struct hw_perf_e= vent *event, if (attr->exclude_user) config_base |=3D ARMV8_PMU_EXCLUDE_EL0; =20 + /* + * Insert event counting threshold (FEAT_PMUv3_TH) values. If + * FEAT_PMUv3_TH isn't implemented, then THWIDTH (threshold_max) will be + * 0 and no values will be written. + */ + th_max =3D threshold_max(cpu_pmu); + if (IS_ENABLED(CONFIG_ARM64) && th_max) { + th =3D min(armv8pmu_event_threshold(attr), th_max); + config_base |=3D FIELD_PREP(ARMV8_PMU_EVTYPE_TH, th); + config_base |=3D FIELD_PREP(ARMV8_PMU_EVTYPE_TC, + armv8pmu_event_threshold_control(attr)); + } + /* * Install the filter into config_base as this is used to * construct the event type. diff --git a/include/linux/perf/arm_pmuv3.h b/include/linux/perf/arm_pmuv3.h index ddd1fec86739..ccbc0f9a74d8 100644 --- a/include/linux/perf/arm_pmuv3.h +++ b/include/linux/perf/arm_pmuv3.h @@ -258,6 +258,7 @@ #define ARMV8_PMU_BUS_SLOTS_MASK 0xff #define ARMV8_PMU_BUS_WIDTH_SHIFT 16 #define ARMV8_PMU_BUS_WIDTH_MASK 0xf +#define ARMV8_PMU_THWIDTH GENMASK(23, 20) =20 /* * This code is really good --=20 2.34.1 From nobody Wed Dec 31 00:37:09 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08954C4167B for ; Mon, 13 Nov 2023 11:27:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231992AbjKML10 (ORCPT ); Mon, 13 Nov 2023 06:27:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231232AbjKML1I (ORCPT ); Mon, 13 Nov 2023 06:27:08 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0157044B8; Mon, 13 Nov 2023 03:26:07 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A8AAD150C; Mon, 13 Nov 2023 03:26:47 -0800 (PST) Received: from e127643.arm.com (unknown [10.57.71.191]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 191A73F6C4; Mon, 13 Nov 2023 03:25:59 -0800 (PST) From: James Clark To: linux-arm-kernel@lists.infradead.org, linux-perf-users@vger.kernel.org, suzuki.poulose@arm.com, will@kernel.org, mark.rutland@arm.com Cc: James Clark , Catalin Marinas , Jonathan Corbet , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 3/3] Documentation: arm64: Document the PMU event counting threshold feature Date: Mon, 13 Nov 2023 11:25:06 +0000 Message-Id: <20231113112507.917107-4-james.clark@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231113112507.917107-1-james.clark@arm.com> References: <20231113112507.917107-1-james.clark@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add documentation for the new Perf event open parameters and the threshold_max capability file. Signed-off-by: James Clark --- Documentation/arch/arm64/perf.rst | 56 +++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/Documentation/arch/arm64/perf.rst b/Documentation/arch/arm64/p= erf.rst index 1f87b57c2332..36b8111a710d 100644 --- a/Documentation/arch/arm64/perf.rst +++ b/Documentation/arch/arm64/perf.rst @@ -164,3 +164,59 @@ and should be used to mask the upper bits as needed. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree= /tools/perf/arch/arm64/tests/user-events.c .. _tools/lib/perf/tests/test-evsel.c: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree= /tools/lib/perf/tests/test-evsel.c + +Event Counting Threshold +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Overview +-------- + +FEAT_PMUv3_TH (Armv8.8) permits a PMU counter to increment only on +events whose count meets a specified threshold condition. For example if +threshold_compare is set to 2 ('Greater than or equal'), and the +threshold is set to 2, then the PMU counter will now only increment by +when an event would have previously incremented the PMU counter by 2 or +more on a single processor cycle. + +To increment by 1 after passing the threshold condition instead of the +number of events on that cycle, add the 'threshold_count' option to the +commandline. + +How-to +------ + +The threshold, threshold_compare and threshold_count values can be +provided per event: + +.. code-block:: sh + + perf stat -e stall_slot/threshold=3D2,threshold_compare=3D2/ \ + -e dtlb_walk/threshold=3D10,threshold_compare=3D3,threshold_co= unt/ + +And the following comparison values are supported: + +.. code-block:: + + 0: Not-equal + 1: Equals + 2: Greater-than-or-equal + 3: Less-than + +The maximum supported threshold value can be read from the caps of each +PMU, for example: + +.. code-block:: sh + + cat /sys/bus/event_source/devices/armv8_pmuv3/caps/threshold_max + + 0x000000ff + +If a value higher than this is given, then it will be silently clamped +to the maximum. The highest possible maximum is 4095, as the config +field for threshold is limited to 12 bits, and the Perf tool will refuse +to parse higher values. + +If the PMU doesn't support FEAT_PMUv3_TH, then threshold_max will read +0, and both threshold and threshold_compare will be silently ignored. +threshold_max will also read as 0 on aarch32 guests, even if the host +is running on hardware with the feature. --=20 2.34.1