From nobody Fri Oct 3 15:34:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1EA625A2B5 for ; Fri, 29 Aug 2025 03:07:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756436873; cv=none; b=on80aVS5NahEjFvtEcxycM5aPj8fN/v9g//L/y3IzmX89SJ2cSIKqtbAYKzy7rlvvC3p3g73zT/UxlmVNuFWwS9YW6SwZjN0Y/Rpir5vVKs1V7BEciIeOuZdTonJt4bZJlCNcSF7NMhCmTDGA+ne52u8Vvo5C2rJPklVLHch2jc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756436873; c=relaxed/simple; bh=FJ2GIBG1BsAUWQpMLjlonQ8xF0TkI0HYkC/wcGmiYkc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Content-Type; b=a1Xjv2XdM8vbaiRASwaK08RmCjUEa6KHwFhMNCNFZHZJP66kAALdRrx+nII8VdlJUz0YodSIhIMgM4lILW1DZ7JU5hoZSoSxdIUiMMiBUZxBhvQLuWq73wDJ0vweJIIUO8gbXIiPbKmVojM/5Jg66l8TtQYGuec7g60Lx3Pchw0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GOLo0W5K; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GOLo0W5K" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-32752f91beaso2149477a91.2 for ; Thu, 28 Aug 2025 20:07:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1756436871; x=1757041671; darn=vger.kernel.org; h=to:from:subject:message-id:references:mime-version:in-reply-to:date :from:to:cc:subject:date:message-id:reply-to; bh=VdHUScYT7W2VGRMgw68PcJ7pKiSlfY+0dw+HmuAdspg=; b=GOLo0W5Kd1crfouTzg6htZZVdqQjta/Q1vHBDAX4m2aY6+kYd22EuKQ6UaQMg7U4NA yr0xzfeK4MO2mfgtXfZWYBSNW4zuYYhSate1UWm5dbro8guIWNYLJeuD98RPuxcT0hSM m/2pW6w1+AgUR43iqMq7Po8f2BKQbr4dDc15F39wXabtd/jHNrIeBEvkoOZpLcY4mTWh YSWC2Lsd8jgnzj5bpsL3A5sA/ySFi4+dJZnGXcY2e+WzlPuFHBTXAp9kr1lxJ+CGw/zW pe3nCqNT/Sley0RYvOiAkVMf+2Ec9zyc/W46NiZlGLiYQ2VqEYVBv0zKyTq6oZvfTSzb E2+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756436871; x=1757041671; h=to:from:subject:message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VdHUScYT7W2VGRMgw68PcJ7pKiSlfY+0dw+HmuAdspg=; b=j3WWXTA1YWVX7KYDR4Svr7DEyDAfDClPBpI6Ng7j3kBrcclHvxsmHxdHghdrWkbJHC fa6WMefcuU0ffjRo7YrYX9YdSJ5T7LNjo6nZlfx6I4u/3z5uV947rAhWJfKNpI1XjcMJ 8g7vqhUJHS5md/jnad22mQ/NAG2ky/j7jfe7ltFyqySGlLqRGRacrCAxwPlXBqN3oFmQ VmoL1NDza9S6gu7J/4THSY4PnZAyd9SXDW2pYkzVHfFl5V02NZY6/1Je7EHqFcL1BkI1 J1IHDtotpUj1pZvrVxBDgWwc9YSFhXzYO04N1eAKPxGTTq4M2fuRT0/PiQ0VKTcpnUzJ ltjQ== X-Forwarded-Encrypted: i=1; AJvYcCWljnDbR/jbBX2pkcwQLFtk7rf74y09gtbfihjHXDB0q5QM8gdNbaSvqMJag9BLCgLB8qUPNnFvC77OiO8=@vger.kernel.org X-Gm-Message-State: AOJu0Yy2UxvRyb74m5g/KqKN3hLSYDiaQbclWq/JG6WgA+MDVC9uX6H+ 11JOUEeXISkYHnX5j4QEFA/T/59WSgZBlV3Dypbh8dtA5QTTYuk9F6o17ogleY5uQr7yJZhEXph uQI6MF4/2YA== X-Google-Smtp-Source: AGHT+IG314ujKFfTaDY0VW2hUce2/LUImMQR+ihljf4A+twxxq05CUzMuW4zeIgA+if1zd/ANQmJ0eI6fV4N X-Received: from plem10.prod.google.com ([2002:a17:902:e40a:b0:248:d08b:a337]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e80c:b0:246:b3d4:5c82 with SMTP id d9443c01a7336-246b3d4620amr283293095ad.16.1756436871187; Thu, 28 Aug 2025 20:07:51 -0700 (PDT) Date: Thu, 28 Aug 2025 20:07:25 -0700 In-Reply-To: <20250829030727.4159703-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250829030727.4159703-1-irogers@google.com> X-Mailer: git-send-email 2.51.0.318.gd7df087d1a-goog Message-ID: <20250829030727.4159703-12-irogers@google.com> Subject: [PATCH v5 11/12] perf jevents: Build support for generating metrics from python From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , James Clark , Xu Yang , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, John Garry , Jing Zhang , Sandipan Das , Benjamin Gray Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Generate extra-metrics.json and extra-metricgroups.json from python architecture specific scripts. The metrics themselves will be added in later patches. If a build takes place in tools/perf/ then extra-metrics.json and extra-metricgroups.json are generated in that directory and so added to .gitignore. If there is an OUTPUT directory then the tools/perf/pmu-events/arch files are copied to it so the generated extra-metrics.json and extra-metricgroups.json can be added/generated there. Signed-off-by: Ian Rogers --- tools/perf/.gitignore | 5 +++ tools/perf/Makefile.perf | 2 + tools/perf/pmu-events/Build | 51 +++++++++++++++++++++++++- tools/perf/pmu-events/amd_metrics.py | 38 +++++++++++++++++++ tools/perf/pmu-events/arm64_metrics.py | 39 ++++++++++++++++++++ tools/perf/pmu-events/intel_metrics.py | 38 +++++++++++++++++++ 6 files changed, 172 insertions(+), 1 deletion(-) create mode 100755 tools/perf/pmu-events/amd_metrics.py create mode 100755 tools/perf/pmu-events/arm64_metrics.py create mode 100755 tools/perf/pmu-events/intel_metrics.py diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore index b64302a76144..133e343bf44e 100644 --- a/tools/perf/.gitignore +++ b/tools/perf/.gitignore @@ -42,6 +42,11 @@ pmu-events/metric_test.log pmu-events/empty-pmu-events.log pmu-events/test-empty-pmu-events.c *.shellcheck_log +pmu-events/arch/**/extra-metrics.json +pmu-events/arch/**/extra-metricgroups.json +tests/shell/*.shellcheck_log +tests/shell/coresight/*.shellcheck_log +tests/shell/lib/*.shellcheck_log feature/ libapi/ libbpf/ diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf index cc1635335586..96ad9ea60f68 100644 --- a/tools/perf/Makefile.perf +++ b/tools/perf/Makefile.perf @@ -1279,6 +1279,8 @@ ifeq ($(OUTPUT),) pmu-events/metric_test.log \ pmu-events/test-empty-pmu-events.c \ pmu-events/empty-pmu-events.log + $(Q)find pmu-events/arch -name 'extra-metrics.json' -delete -o \ + -name 'extra-metricgroups.json' -delete else # When an OUTPUT directory is present, clean up the copied pmu-events= /arch directory. $(call QUIET_CLEAN, pmu-events) $(RM) -r $(OUTPUT)pmu-events/arch \ $(OUTPUT)pmu-events/pmu-events.c \ diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build index 4ebf37c14978..162d5f74cbe8 100644 --- a/tools/perf/pmu-events/Build +++ b/tools/perf/pmu-events/Build @@ -30,6 +30,10 @@ $(PMU_EVENTS_C): $(EMPTY_PMU_EVENTS_C) $(call rule_mkdir) $(Q)$(call echo-cmd,gen)cp $< $@ else +# Extract the model from a extra-metrics.json or extra-metricgroups.json p= ath +model_name =3D $(shell echo $(1)|sed -e 's@.\+/\(.*\)/extra-metric.*\.json= @\1@') +vendor_name =3D $(shell echo $(1)|sed -e 's@.\+/\(.*\)/[^/]*/extra-metric.= *\.json@\1@') + # Copy checked-in json for generation. $(OUTPUT)pmu-events/arch/%: pmu-events/arch/% $(call rule_mkdir) @@ -39,7 +43,52 @@ $(LEGACY_CACHE_JSON): $(LEGACY_CACHE_PY) $(call rule_mkdir) $(Q)$(call echo-cmd,gen)$(PYTHON) $(LEGACY_CACHE_PY) > $@ =20 -GEN_JSON =3D $(patsubst %,$(OUTPUT)%,$(JSON)) $(LEGACY_CACHE_JSON) +GEN_METRIC_DEPS :=3D pmu-events/metric.py + +# Generate AMD Json +ZENS =3D $(shell ls -d pmu-events/arch/x86/amdzen*) +ZEN_METRICS =3D $(foreach x,$(ZENS),$(OUTPUT)$(x)/extra-metrics.json) +ZEN_METRICGROUPS =3D $(foreach x,$(ZENS),$(OUTPUT)$(x)/extra-metricgroups.= json) + +$(ZEN_METRICS): pmu-events/amd_metrics.py $(GEN_METRIC_DEPS) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(PYTHON) $< $(call model_name,$@) arch > $@ + +$(ZEN_METRICGROUPS): pmu-events/amd_metrics.py $(GEN_METRIC_DEPS) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(PYTHON) $< -metricgroups $(call model_name,$@) = arch > $@ + +# Generate ARM Json +ARMS =3D $(shell ls -d pmu-events/arch/arm64/arm/*) +ARM_METRICS =3D $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metrics.json) +ARM_METRICGROUPS =3D $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metricgroups.= json) + +$(ARM_METRICS): pmu-events/arm64_metrics.py $(GEN_METRIC_DEPS) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(PYTHON) $< $(call vendor_name,$@) $(call model_= name,$@) arch > $@ + +$(ARM_METRICGROUPS): pmu-events/arm64_metrics.py $(GEN_METRIC_DEPS) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(PYTHON) $< -metricgroups $(call vendor_name,$@)= $(call model_name,$@) arch > $@ + +# Generate Intel Json +INTELS =3D $(shell ls -d pmu-events/arch/x86/*|grep -v amdzen|grep -v mapf= ile.csv) +INTEL_METRICS =3D $(foreach x,$(INTELS),$(OUTPUT)$(x)/extra-metrics.json) +INTEL_METRICGROUPS =3D $(foreach x,$(INTELS),$(OUTPUT)$(x)/extra-metricgro= ups.json) + +$(INTEL_METRICS): pmu-events/intel_metrics.py $(GEN_METRIC_DEPS) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(PYTHON) $< $(call model_name,$@) arch > $@ + +$(INTEL_METRICGROUPS): pmu-events/intel_metrics.py $(GEN_METRIC_DEPS) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(PYTHON) $< -metricgroups $(call model_name,$@) = arch > $@ + +GEN_JSON =3D $(patsubst %,$(OUTPUT)%,$(JSON)) \ + $(LEGACY_CACHE_JSON) \ + $(ZEN_METRICS) $(ZEN_METRICGROUPS) \ + $(ARM_METRICS) $(ARM_METRICGROUPS) \ + $(INTEL_METRICS) $(INTEL_METRICGROUPS) =20 $(METRIC_TEST_LOG): $(METRIC_TEST_PY) $(METRIC_PY) $(call rule_mkdir) diff --git a/tools/perf/pmu-events/amd_metrics.py b/tools/perf/pmu-events/a= md_metrics.py new file mode 100755 index 000000000000..7ab2ee4fdb17 --- /dev/null +++ b/tools/perf/pmu-events/amd_metrics.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, M= etricGroup) +import argparse +import json +import os + +# Global command line arguments. +_args =3D None + +def main() -> None: + global _args + + def dir_path(path: str) -> str: + """Validate path is a directory for argparse.""" + if os.path.isdir(path): + return path + raise argparse.ArgumentTypeError(f'\'{path}\' is not a valid directory= ') + + parser =3D argparse.ArgumentParser(description=3D"AMD perf json generato= r") + parser.add_argument("-metricgroups", help=3D"Generate metricgroups data"= , action=3D'store_true') + parser.add_argument("model", help=3D"e.g. amdzen[123]") + parser.add_argument( + 'events_path', + type=3Ddir_path, + help=3D'Root of tree containing architecture directories containing = json files' + ) + _args =3D parser.parse_args() + + all_metrics =3D MetricGroup("",[]) + + if _args.metricgroups: + print(JsonEncodeMetricGroupDescriptions(all_metrics)) + else: + print(JsonEncodeMetric(all_metrics)) + +if __name__ =3D=3D '__main__': + main() diff --git a/tools/perf/pmu-events/arm64_metrics.py b/tools/perf/pmu-events= /arm64_metrics.py new file mode 100755 index 000000000000..a9f0e6bc751b --- /dev/null +++ b/tools/perf/pmu-events/arm64_metrics.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, M= etricGroup) +import argparse +import json +import os + +# Global command line arguments. +_args =3D None + +def main() -> None: + global _args + + def dir_path(path: str) -> str: + """Validate path is a directory for argparse.""" + if os.path.isdir(path): + return path + raise argparse.ArgumentTypeError(f'\'{path}\' is not a valid directory= ') + + parser =3D argparse.ArgumentParser(description=3D"ARM perf json generato= r") + parser.add_argument("-metricgroups", help=3D"Generate metricgroups data"= , action=3D'store_true') + parser.add_argument("vendor", help=3D"e.g. arm") + parser.add_argument("model", help=3D"e.g. neoverse-n1") + parser.add_argument( + 'events_path', + type=3Ddir_path, + help=3D'Root of tree containing architecture directories containing = json files' + ) + _args =3D parser.parse_args() + + all_metrics =3D MetricGroup("",[]) + + if _args.metricgroups: + print(JsonEncodeMetricGroupDescriptions(all_metrics)) + else: + print(JsonEncodeMetric(all_metrics)) + +if __name__ =3D=3D '__main__': + main() diff --git a/tools/perf/pmu-events/intel_metrics.py b/tools/perf/pmu-events= /intel_metrics.py new file mode 100755 index 000000000000..f004c27640d2 --- /dev/null +++ b/tools/perf/pmu-events/intel_metrics.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, M= etricGroup) +import argparse +import json +import os + +# Global command line arguments. +_args =3D None + +def main() -> None: + global _args + + def dir_path(path: str) -> str: + """Validate path is a directory for argparse.""" + if os.path.isdir(path): + return path + raise argparse.ArgumentTypeError(f'\'{path}\' is not a valid directory= ') + + parser =3D argparse.ArgumentParser(description=3D"Intel perf json genera= tor") + parser.add_argument("-metricgroups", help=3D"Generate metricgroups data"= , action=3D'store_true') + parser.add_argument("model", help=3D"e.g. skylakex") + parser.add_argument( + 'events_path', + type=3Ddir_path, + help=3D'Root of tree containing architecture directories containing = json files' + ) + _args =3D parser.parse_args() + + all_metrics =3D MetricGroup("",[]) + + if _args.metricgroups: + print(JsonEncodeMetricGroupDescriptions(all_metrics)) + else: + print(JsonEncodeMetric(all_metrics)) + +if __name__ =3D=3D '__main__': + main() --=20 2.51.0.318.gd7df087d1a-goog