From nobody Fri Nov 29 00:42:10 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E513816190C for ; Thu, 26 Sep 2024 17:36:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727372198; cv=none; b=ZoUNjIIvJewzraaBnpVny93Xd7sBeD4bQE3abPjDL1ZPUGEeaAGKNXG1fofBBa9pm+pOHnhPL8frUAANNAGfBL43Hl9GCsSb2+5hV9gh903nLYMI1nzu+JkEe08i3Krw4MjcaWQZ8E08kVWKUNMBV3WzpGGJQIZE2Kx0a5KP5e4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727372198; c=relaxed/simple; bh=ds/6QN0N+eX7ozRww2ZlsCUbFrTYWR2Pkujfts4WTag=; h=Date:In-Reply-To:Message-Id:Mime-Version:References:Subject:From: To:Content-Type; b=mx/yGGcbzuEeG19agbb13hl2ASRvK3Sp3F4dxp3ywc5oayWY1zBzqsi+c4VEG3UrjL3qZXo+CK4dpUCUNEHkUeUez1hkkoUzXVHGPlYHnvhLMUKFfQlV2hJ5gbTU+iZHeTSTXjPQTxTLrCx5dJV80lI5OE2ofXcI4qqg2TbxkWY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JsIX3bqu; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JsIX3bqu" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6d4bd76f5a8so24383567b3.0 for ; Thu, 26 Sep 2024 10:36:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727372196; x=1727976996; darn=vger.kernel.org; h=to:from:subject:references:mime-version:message-id:in-reply-to:date :from:to:cc:subject:date:message-id:reply-to; bh=h1yvFCEwKQ3wd3i4rPKz93fgEiuuae9SHgL1DUf9d6U=; b=JsIX3bqu+Kpvoj/gIONL8vWDX2paS8wZorIokPrYHznG+qPamXcj1Yd9R5XclrPkK8 8d0gkVrqEq4b3CTGnOwyE1btwYwwyVLzMkx79oc/AWtBlzzXyzzBsNC5o2XMsUouVmt/ WbhwiOfq7o7QPQSse7STVuSQQ3gp48WSP77bLriLOXS0KqWHEU2TCMShSwK9jiToVrKa xlOBIOOF5pRWxX7gOVa/VhiymUGMuX61R1Ed9mfQ6VxUMbzeuCQ4M4Qx7lEGYFJnOeHQ 9GXIxQ0NjEf/IEs/uTya0VN6rr+o+aH/nCJ0n6xNO6igIAu/nBY6Iw1+pIFH8Oy0i/Mp +kAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727372196; x=1727976996; h=to:from:subject:references:mime-version:message-id:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=h1yvFCEwKQ3wd3i4rPKz93fgEiuuae9SHgL1DUf9d6U=; b=BWzFu7ZNVLV83RObbPqhj7AmYXHyGwmYBjpN4UjoFnRDyboqNkv4fbu+TFFusfR8V+ arWCPnG0AWCsf9mjen+WbzoydaZV2POquYdjvmckNF/1nC5ut505ohVVSpaNxO2MqATz MkBZVZzLD8yLQTeHeUxJvxuDE1wdKNnExJbeSTngraHMgZtG5Jjsq0eLfH71xgXk95iH HaXz3K+8EPagclqOw/syMNGVBfH+Fje3gXZQqNAYG+Xf9tdwjujkrOsX0fK3d0hz3EJb T17o0HsvVhkBlqOFTxM08c2GNVukWXX1sHYtbI2yOMJ6ic5ZQxpFWJi76g+dejqZnr3U 3soA== X-Forwarded-Encrypted: i=1; AJvYcCXRv4GgOYrDWW5JWBKnKXU6j45IbSRYCsOcYQ2C2CBEXMeH7SigEUt7vEqdgM3i3nXd75WK5qwViD31wbs=@vger.kernel.org X-Gm-Message-State: AOJu0YzCsN+p/HYabzBR+X0oSKwPsHTPFBVXUmhxRNsrbC5TXywxBslU HlLfD0+Mh/9yIuwTO3OJffmTTnF2n/nB93Qzy/e9IPAzz4lrIdmagkxEWzpl20/nD4cFDI+E6B6 uAXQ9Xw== X-Google-Smtp-Source: AGHT+IFK0k8Rkerkrh3QirrpvkNRJSdw/gsRuMqQtRFrcu08gzmyE1D8gAexa1TODfbWh22et09fOWTWlqkv X-Received: from irogers.svl.corp.google.com ([2620:15c:2a3:200:8872:3585:18ed:a056]) (user=irogers job=sendgmr) by 2002:a05:690c:2011:b0:6e2:3e17:1838 with SMTP id 00721157ae682-6e247518b3emr9357b3.1.1727372195414; Thu, 26 Sep 2024 10:36:35 -0700 (PDT) Date: Thu, 26 Sep 2024 10:35:54 -0700 In-Reply-To: <20240926173554.404411-1-irogers@google.com> Message-Id: <20240926173554.404411-13-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240926173554.404411-1-irogers@google.com> X-Mailer: git-send-email 2.46.1.824.gd892dcdcdd-goog Subject: [PATCH v4 12/12] perf jevents: Add load event json to verify and allow fallbacks From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , John Garry , Jing Zhang , Sandipan Das , Benjamin Gray , Xu Yang , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a LoadEvents function that loads all event json files in a directory. In the Event constructor ensure all events are defined in the event json except for legacy events like "cycles". If the initial event isn't found then legacy_event1 is used, and if that isn't found legacy_event2 is used. This allows a single Event to have multiple event names as models will often rename the same event over time. If the event doesn't exist an exception is raised. So that references to metrics can be added, add the MetricRef class. This doesn't validate as an event name and so provides an escape hatch for metrics to refer to each other. Signed-off-by: Ian Rogers --- tools/perf/pmu-events/Build | 12 ++-- tools/perf/pmu-events/amd_metrics.py | 6 +- tools/perf/pmu-events/arm64_metrics.py | 6 +- tools/perf/pmu-events/intel_metrics.py | 6 +- tools/perf/pmu-events/metric.py | 77 +++++++++++++++++++++++++- 5 files changed, 95 insertions(+), 12 deletions(-) diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build index d89275c4f485..f3bc6c093360 100644 --- a/tools/perf/pmu-events/Build +++ b/tools/perf/pmu-events/Build @@ -46,11 +46,11 @@ ZEN_METRICGROUPS =3D $(foreach x,$(ZENS),$(OUTPUT)$(x)/= extra-metricgroups.json) =20 $(ZEN_METRICS): pmu-events/amd_metrics.py $(GEN_METRIC_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $< $(call model_name,$@) arch > $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $< $(call model_name,$@) pmu-events/arc= h > $@ =20 $(ZEN_METRICGROUPS): pmu-events/amd_metrics.py $(GEN_METRIC_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $< -metricgroups $(call model_name,$@) = arch > $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $< -metricgroups $(call model_name,$@) = pmu-events/arch > $@ =20 # Generate ARM Json ARMS =3D $(shell ls -d pmu-events/arch/arm64/arm/*) @@ -59,11 +59,11 @@ ARM_METRICGROUPS =3D $(foreach x,$(ARMS),$(OUTPUT)$(x)/= extra-metricgroups.json) =20 $(ARM_METRICS): pmu-events/arm64_metrics.py $(GEN_METRIC_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $< $(call vendor_name,$@) $(call model_= name,$@) arch > $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $< $(call vendor_name,$@) $(call model_= name,$@) pmu-events/arch > $@ =20 $(ARM_METRICGROUPS): pmu-events/arm64_metrics.py $(GEN_METRIC_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $< -metricgroups $(call vendor_name,$@)= $(call model_name,$@) arch > $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $< -metricgroups $(call vendor_name,$@)= $(call model_name,$@) pmu-events/arch > $@ =20 # Generate Intel Json INTELS =3D $(shell ls -d pmu-events/arch/x86/*|grep -v amdzen|grep -v mapf= ile.csv) @@ -72,11 +72,11 @@ INTEL_METRICGROUPS =3D $(foreach x,$(INTELS),$(OUTPUT)$= (x)/extra-metricgroups.json =20 $(INTEL_METRICS): pmu-events/intel_metrics.py $(GEN_METRIC_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $< $(call model_name,$@) arch > $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $< $(call model_name,$@) pmu-events/arc= h > $@ =20 $(INTEL_METRICGROUPS): pmu-events/intel_metrics.py $(GEN_METRIC_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $< -metricgroups $(call model_name,$@) = arch > $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $< -metricgroups $(call model_name,$@) = pmu-events/arch > $@ =20 GEN_JSON =3D $(patsubst %,$(OUTPUT)%,$(JSON)) \ $(ZEN_METRICS) $(ZEN_METRICGROUPS) \ diff --git a/tools/perf/pmu-events/amd_metrics.py b/tools/perf/pmu-events/a= md_metrics.py index 7ab2ee4fdb17..4f728e7aae4a 100755 --- a/tools/perf/pmu-events/amd_metrics.py +++ b/tools/perf/pmu-events/amd_metrics.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) -from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, M= etricGroup) +from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, L= oadEvents, + MetricGroup) import argparse import json import os @@ -27,6 +28,9 @@ def main() -> None: ) _args =3D parser.parse_args() =20 + directory =3D f"{_args.events_path}/x86/{_args.model}/" + LoadEvents(directory) + all_metrics =3D MetricGroup("",[]) =20 if _args.metricgroups: diff --git a/tools/perf/pmu-events/arm64_metrics.py b/tools/perf/pmu-events= /arm64_metrics.py index a9f0e6bc751b..c9aa2d827a82 100755 --- a/tools/perf/pmu-events/arm64_metrics.py +++ b/tools/perf/pmu-events/arm64_metrics.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) -from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, M= etricGroup) +from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, L= oadEvents, + MetricGroup) import argparse import json import os @@ -30,6 +31,9 @@ def main() -> None: =20 all_metrics =3D MetricGroup("",[]) =20 + directory =3D f"{_args.events_path}/arm64/{_args.vendor}/{_args.model}/" + LoadEvents(directory) + if _args.metricgroups: print(JsonEncodeMetricGroupDescriptions(all_metrics)) else: diff --git a/tools/perf/pmu-events/intel_metrics.py b/tools/perf/pmu-events= /intel_metrics.py index f004c27640d2..04a19d05c6c1 100755 --- a/tools/perf/pmu-events/intel_metrics.py +++ b/tools/perf/pmu-events/intel_metrics.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) -from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, M= etricGroup) +from metric import (JsonEncodeMetric, JsonEncodeMetricGroupDescriptions, L= oadEvents, + MetricGroup) import argparse import json import os @@ -27,6 +28,9 @@ def main() -> None: ) _args =3D parser.parse_args() =20 + directory =3D f"{_args.events_path}/x86/{_args.model}/" + LoadEvents(directory) + all_metrics =3D MetricGroup("",[]) =20 if _args.metricgroups: diff --git a/tools/perf/pmu-events/metric.py b/tools/perf/pmu-events/metric= .py index dd8fd06940e6..03312cd6d491 100644 --- a/tools/perf/pmu-events/metric.py +++ b/tools/perf/pmu-events/metric.py @@ -3,10 +3,50 @@ import ast import decimal import json +import os import re from enum import Enum from typing import Dict, List, Optional, Set, Tuple, Union =20 +all_events =3D set() + +def LoadEvents(directory: str) -> None: + """Populate a global set of all known events for the purpose of validati= ng Event names""" + global all_events + all_events =3D { + "context\-switches", + "cycles", + "duration_time", + "instructions", + "l2_itlb_misses", + } + for file in os.listdir(os.fsencode(directory)): + filename =3D os.fsdecode(file) + if filename.endswith(".json"): + for x in json.load(open(f"{directory}/{filename}")): + if "EventName" in x: + all_events.add(x["EventName"]) + elif "ArchStdEvent" in x: + all_events.add(x["ArchStdEvent"]) + + +def CheckEvent(name: str) -> bool: + """Check the event name exists in the set of all loaded events""" + global all_events + if len(all_events) =3D=3D 0: + # No events loaded so assume any event is good. + return True + + if ':' in name: + # Remove trailing modifier. + name =3D name[:name.find(':')] + elif '/' in name: + # Name could begin with a PMU or an event, for now assume it is good. + return True + + return name in all_events + + class MetricConstraint(Enum): GROUPED_EVENTS =3D 0 NO_GROUP_EVENTS =3D 1 @@ -317,9 +357,18 @@ def _FixEscapes(s: str) -> str: class Event(Expression): """An event in an expression.""" =20 - def __init__(self, name: str, legacy_name: str =3D ''): - self.name =3D _FixEscapes(name) - self.legacy_name =3D _FixEscapes(legacy_name) + def __init__(self, *args: str): + error =3D "" + for name in args: + if CheckEvent(name): + self.name =3D _FixEscapes(name) + return + if error: + error +=3D " or " + name + else: + error =3D name + global all_events + raise Exception(f"No event {error} in:\n{all_events}") =20 def ToPerfJson(self): result =3D re.sub('/', '@', self.name) @@ -338,6 +387,28 @@ class Event(Expression): return self =20 =20 +class MetricRef(Expression): + """A metric reference in an expression.""" + + def __init__(self, name: str): + self.name =3D _FixEscapes(name) + + def ToPerfJson(self): + return self.name + + def ToPython(self): + return f'MetricRef(r"{self.name}")' + + def Simplify(self) -> Expression: + return self + + def Equals(self, other: Expression) -> bool: + return isinstance(other, MetricRef) and self.name =3D=3D other.name + + def Substitute(self, name: str, expression: Expression) -> Expression: + return self + + class Constant(Expression): """A constant within the expression tree.""" =20 --=20 2.46.1.824.gd892dcdcdd-goog