From nobody Wed Oct 8 19:57:27 2025 Received: from mail-oa1-f73.google.com (mail-oa1-f73.google.com [209.85.160.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52ADF26AA91 for ; Tue, 24 Jun 2025 23:19:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750807147; cv=none; b=rLwSw+yaxypCenrJkllJbB751u0Bwe4ZVw0KFKClD7IgEupf2OwgBczjQxNse/njoGDWzkP4sYilmWsbdBsPNJesUxV/qcUztLqt9V+idHTFJGSdeyqO1yYdthc7e+yDkly3CSM03qMXhZ6sIunQ5qkcJfqU7xTTBAyJmTMD0ZM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750807147; c=relaxed/simple; bh=UP7c8EZMJCmdtSRgS/00+UGz7KyMyLEWykYE2tYt5nQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VxsHCAWqNeUALGkvnkB3Td77uKSTmSU3fV3fFTpGi6BvZ5KEcFI1Ezv6yaS0ZMGb0ACajSolwye1oulwGOIIoFoMNPwHnvHhPcGnTFDLSsgYGDe05+lpRTcurPyCFI67DNcnFNnqlorW0HkBSaHUFge1kKFIUAnE0lKti6BpAdQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=s35A0cot; arc=none smtp.client-ip=209.85.160.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="s35A0cot" Received: by mail-oa1-f73.google.com with SMTP id 586e51a60fabf-2e933923303so3944863fac.1 for ; Tue, 24 Jun 2025 16:19:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1750807144; x=1751411944; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=c3nTtbe0Tv/hCcFjA75rcndb1ZtC/Z05byf6Qgte1y0=; b=s35A0cotC0RZvQYmz67TRf3d8l/OuEH9XcTfCAAAEiN+SA+R5WlYmu8RmDQyJ2SWnX JmUpl40WHHbZ8JiuiYKVvuSSa00X565vnqN0Vxpqa9yrLHA7nQg52Gs6s3wV1gMCR8nJ jhAHtd1M2XaYVw+B04uoeaUz++HtFSkFBauWSESAg7pNy9GIQbyduXS26vMHRCGceLcB YlsCc6yJgzJoCqyg+CQrcmWxxVkufRWJuRMa9cWCkw9xIbon0s/DTT3RlYeMt2/pm/72 MyaoeoacT3kR+Pno5/scNW3oQWTrgL5pUjLzyuprrGKVPvoNGP6ZwA/VxYcIQeKA25dE gc3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750807144; x=1751411944; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=c3nTtbe0Tv/hCcFjA75rcndb1ZtC/Z05byf6Qgte1y0=; b=K8GQ1OmqUeWv3OPfpUsoYYSgym1tRLk02wrDYJxSkL8BtXiZDSKDIp89DyMX3uliZw JeyG3P/KSZA7xUkT65wQytcoLOCOZmL5lW9HrSrGlDuvtthzhP1y68MEEFHqU+rFMO3Z nuQwkyEfrjKHm6sl06TC915aFq3FUFyv3aEaOA1wDTBqb371urLaDfh9iiDihYKdwdiN xtGurUTYmJTSVrZkHTJkFz4HXv2awUqFB7SIXkoNZBgGTaAGeYgmgXkaX20F6x7Qtdvg DwxFCguEfonAOvIq8d/Fv4rp+1GXeabvXphiapSfl3gBkxxLP06TVJsdq/2Osfj88uzU N63A== X-Forwarded-Encrypted: i=1; AJvYcCXzmT8wmvMGIhLW3iLmedQ8P6CyTrtT5KyS3zndRsVCpdV63dIP/ze+Ov48MR6AHTqg7VyiuteVfXT81KU=@vger.kernel.org X-Gm-Message-State: AOJu0YwfPgt37qsFq9MGJnixV9FltQzSqHFZgDrA8dfdALL4a9vFMjys C/PmBQfXJq1BTZi9eBv+icGUWdAG2aYR88kZKm2GmUGPTbZZopZYAZCN6dxixoIrcXos/FXZisq fQPogzegqOA== X-Google-Smtp-Source: AGHT+IFXWZZ1CRCWcrDta6qiLM+gayDxk+xFJqGOzfqYGuNTUXFKwCrv6FyBo0yt7uR6ntZsx4wgNnWjQYGt X-Received: from oabtd5.prod.google.com ([2002:a05:6871:8785:b0:2d5:2492:cc14]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6871:6101:b0:2bc:7811:5bb8 with SMTP id 586e51a60fabf-2efb2367241mr929848fac.18.1750807144437; Tue, 24 Jun 2025 16:19:04 -0700 (PDT) Date: Tue, 24 Jun 2025 16:18:35 -0700 In-Reply-To: <20250624231837.179536-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250624231837.179536-1-irogers@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250624231837.179536-2-irogers@google.com> Subject: [PATCH v4 1/3] perf parse-events: Avoid scanning PMUs that can't contain events From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Adrian Hunter , Kan Liang , "Masami Hiramatsu (Google)" , James Clark , Weilin Wang , Dominique Martinet , Thomas Richter , Junhao He , Jean-Philippe Romain , matthew.olson@intel.com, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, dri-devel@lists.freedesktop.org Cc: Ian Rogers Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add perf_pmus__scan_for_event that only reads sysfs for pmus that could contain a given event. Signed-off-by: Ian Rogers --- tools/perf/util/parse-events.c | 33 ++++++++------- tools/perf/util/pmus.c | 77 ++++++++++++++++++++++++++++++++++ tools/perf/util/pmus.h | 2 + 3 files changed, 97 insertions(+), 15 deletions(-) diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c index d1965a7b97ed..4cd64ffa4fcd 100644 --- a/tools/perf/util/parse-events.c +++ b/tools/perf/util/parse-events.c @@ -490,7 +490,7 @@ int parse_events_add_cache(struct list_head *list, int = *idx, const char *name, int ret =3D 0; struct evsel *first_wildcard_match =3D NULL; =20 - while ((pmu =3D perf_pmus__scan(pmu)) !=3D NULL) { + while ((pmu =3D perf_pmus__scan_for_event(pmu, name)) !=3D NULL) { LIST_HEAD(config_terms); struct perf_event_attr attr; =20 @@ -1681,7 +1681,8 @@ int parse_events_multi_pmu_add(struct parse_events_st= ate *parse_state, =20 INIT_LIST_HEAD(list); =20 - while ((pmu =3D perf_pmus__scan(pmu)) !=3D NULL) { + while ((pmu =3D perf_pmus__scan_for_event(pmu, event_name)) !=3D NULL) { + if (parse_events__filter_pmu(parse_state, pmu)) continue; =20 @@ -1760,19 +1761,21 @@ int parse_events_multi_pmu_add_or_add_pmu(struct pa= rse_events_state *parse_state =20 pmu =3D NULL; /* Failed to add, try wildcard expansion of event_or_pmu as a PMU name. */ - while ((pmu =3D perf_pmus__scan(pmu)) !=3D NULL) { - if (!parse_events__filter_pmu(parse_state, pmu) && - perf_pmu__wildcard_match(pmu, event_or_pmu)) { - if (!parse_events_add_pmu(parse_state, *listp, pmu, - const_parsed_terms, - first_wildcard_match, - /*alternate_hw_config=3D*/PERF_COUNT_HW_MAX)) { - ok++; - parse_state->wild_card_pmus =3D true; - } - if (first_wildcard_match =3D=3D NULL) - first_wildcard_match =3D - container_of((*listp)->prev, struct evsel, core.node); + while ((pmu =3D perf_pmus__scan_matching_wildcard(pmu, event_or_pmu)) != =3D NULL) { + + if (parse_events__filter_pmu(parse_state, pmu)) + continue; + + if (!parse_events_add_pmu(parse_state, *listp, pmu, + const_parsed_terms, + first_wildcard_match, + /*alternate_hw_config=3D*/PERF_COUNT_HW_MAX)) { + ok++; + parse_state->wild_card_pmus =3D true; + } + if (first_wildcard_match =3D=3D NULL) { + first_wildcard_match =3D + container_of((*listp)->prev, struct evsel, core.node); } } if (ok) diff --git a/tools/perf/util/pmus.c b/tools/perf/util/pmus.c index 3bbd26fec78a..e0094f56b8e7 100644 --- a/tools/perf/util/pmus.c +++ b/tools/perf/util/pmus.c @@ -19,6 +19,7 @@ #include "tool_pmu.h" #include "print-events.h" #include "strbuf.h" +#include "string2.h" =20 /* * core_pmus: A PMU belongs to core_pmus if it's name is "cpu" or it's sy= sfs @@ -350,6 +351,82 @@ struct perf_pmu *perf_pmus__scan_core(struct perf_pmu = *pmu) return NULL; } =20 +struct perf_pmu *perf_pmus__scan_for_event(struct perf_pmu *pmu, const cha= r *event) +{ + bool use_core_pmus =3D !pmu || pmu->is_core; + + if (!pmu) { + /* Hwmon filename values that aren't used. */ + enum hwmon_type type; + int number; + /* + * Core PMUs, other sysfs PMUs and tool PMU can take all event + * types or aren't wother optimizing for. + */ + unsigned int to_read_pmus =3D PERF_TOOL_PMU_TYPE_PE_CORE_MASK | + PERF_TOOL_PMU_TYPE_PE_OTHER_MASK | + PERF_TOOL_PMU_TYPE_TOOL_MASK; + + /* Could the event be a hwmon event? */ + if (parse_hwmon_filename(event, &type, &number, /*item=3D*/NULL, /*alarm= =3D*/NULL)) + to_read_pmus |=3D PERF_TOOL_PMU_TYPE_HWMON_MASK; + + pmu_read_sysfs(to_read_pmus); + pmu =3D list_prepare_entry(pmu, &core_pmus, list); + } + if (use_core_pmus) { + list_for_each_entry_continue(pmu, &core_pmus, list) + return pmu; + + pmu =3D NULL; + pmu =3D list_prepare_entry(pmu, &other_pmus, list); + } + list_for_each_entry_continue(pmu, &other_pmus, list) + return pmu; + return NULL; +} + +struct perf_pmu *perf_pmus__scan_matching_wildcard(struct perf_pmu *pmu, c= onst char *wildcard) +{ + bool use_core_pmus =3D !pmu || pmu->is_core; + + if (!pmu) { + /* + * Core PMUs, other sysfs PMUs and tool PMU can have any name or + * aren't wother optimizing for. + */ + unsigned int to_read_pmus =3D PERF_TOOL_PMU_TYPE_PE_CORE_MASK | + PERF_TOOL_PMU_TYPE_PE_OTHER_MASK | + PERF_TOOL_PMU_TYPE_TOOL_MASK; + + /* + * Hwmon PMUs have an alias from a sysfs name like hwmon0, + * hwmon1, etc. or have a name of hwmon_. They therefore + * can only have a wildcard match if the wildcard begins with + * "hwmon". + */ + if (strisglob(wildcard) || + (strlen(wildcard) >=3D 5 && strncmp("hwmon", wildcard, 5) =3D=3D 0)) + to_read_pmus |=3D PERF_TOOL_PMU_TYPE_HWMON_MASK; + + pmu_read_sysfs(to_read_pmus); + pmu =3D list_prepare_entry(pmu, &core_pmus, list); + } + if (use_core_pmus) { + list_for_each_entry_continue(pmu, &core_pmus, list) { + if (perf_pmu__wildcard_match(pmu, wildcard)) + return pmu; + } + pmu =3D NULL; + pmu =3D list_prepare_entry(pmu, &other_pmus, list); + } + list_for_each_entry_continue(pmu, &other_pmus, list) { + if (perf_pmu__wildcard_match(pmu, wildcard)) + return pmu; + } + return NULL; +} + static struct perf_pmu *perf_pmus__scan_skip_duplicates(struct perf_pmu *p= mu) { bool use_core_pmus =3D !pmu || pmu->is_core; diff --git a/tools/perf/util/pmus.h b/tools/perf/util/pmus.h index 8def20e615ad..2794d8c3a466 100644 --- a/tools/perf/util/pmus.h +++ b/tools/perf/util/pmus.h @@ -19,6 +19,8 @@ struct perf_pmu *perf_pmus__find_by_type(unsigned int typ= e); =20 struct perf_pmu *perf_pmus__scan(struct perf_pmu *pmu); struct perf_pmu *perf_pmus__scan_core(struct perf_pmu *pmu); +struct perf_pmu *perf_pmus__scan_for_event(struct perf_pmu *pmu, const cha= r *event); +struct perf_pmu *perf_pmus__scan_matching_wildcard(struct perf_pmu *pmu, c= onst char *wildcard); =20 const struct perf_pmu *perf_pmus__pmu_for_pmu_filter(const char *str); =20 --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Oct 8 19:57:27 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24AF926B0BE for ; Tue, 24 Jun 2025 23:19:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750807150; cv=none; b=aJTLdTyVr/jsmEzDqIY3TmIpAT9VaEpkNrCSH/eFJtz2oCiSr0Itmv6KNWxtG1k/zn/SzKlm5+ZX2WeaPdZjH4p9FrcjkPuG/jV84ET8COu7jBARwlaLfvlzbW5OW+MSA4GcUG9ToUd9I83oy+sLrfh4AIJRyFSERGvOvZgxQ0M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750807150; c=relaxed/simple; bh=NwoBokPzEAii9dT62vWaxzEq5TfQy6cDgkOZ9Uo5g3E=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GkRLSZxtMbkjK8OsF6js2lyZNkX7ZUiOzsgCxxPF9P5E2ppkrDj3LRPAh9yhalk6u6/r43Ypru/Cqf8/QcNNV0rUYfo8cR4wBr9NH18nBeLPv6jB2+aRjw1x+8ahcxazEfqH34ifSf635LE4tQU+Mt6Z8kuanf2N3nlNZ4tAMlo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rdYKrB8W; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rdYKrB8W" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-23689228a7fso14960755ad.1 for ; Tue, 24 Jun 2025 16:19:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1750807146; x=1751411946; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=3nwFWZ0/+3dK9npxKNdbEfqlbp9HoGW45vCO3uch9Ac=; b=rdYKrB8WVpQ7NJumjEe08Iqo8aMh3StXXk4U2mtSLHkbDKVr+yU/rw/lgJamUGuBZ6 1XB4BWhmOPfTeLwESQRcxAfaSgNyTA/1NhExQmd5qPSXwiwEWsI2d8vhGXoDCxMKNZkt ynGqdF5qhSTXneXbSeILRd5EnwZVjRJoSkeZdPoc7Y+CafQ0yzjmoN12KGLum4gig8uh OeMzKtYri0jN08I0YqwH8yp/lo4EKb1mH1jGfmS7aSXSh0Y0XrbQc55BK/48uN58YR6p TN+9Y8ukJyaFgHVRNaOtILKpuupqIViHmUT8ZEUdyHi6Bq3IWGTs3Sz59nepolOhf+ki MJZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750807146; x=1751411946; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=3nwFWZ0/+3dK9npxKNdbEfqlbp9HoGW45vCO3uch9Ac=; b=R0aEFzaOHVCj0elJJxtEmRhzIsR6+ywOi6jRtO2ACq63aD2BfGdShaLkI/gF6g06fJ Ohg80wSFHjA2JNkaPn0nwbHjb2CD4qt9TPWdzeF6QhqdFpIFpl2OhVTVrCvBKxd8gRY2 D+rPKFRuq1GIfl2qfIsm3k4YEr0aVPa2r/GNJ5emjPX0lMAbX7Kal1ZB6ssPfIY30n9U rL5WevsTHxeqBjKCgk82iG+muANXUMHEMUQED05a1cMs7DfBVAInI8sMOnA3xUjhEqP1 UvoJU27Zp/WlKaHZ2PdYMT78pMV/IV7H8+z3wNYRr6x28F8x47ZmCNMCX7zJZmJ704Pe Y2/w== X-Forwarded-Encrypted: i=1; AJvYcCUaMnw2giDpoJrxdGLooTIZwn7AZEv2NHN9DjYIkLWytw0cQ4MtQ8TypXlCOfsv+yOWHzlLCJ+NaWpnLeY=@vger.kernel.org X-Gm-Message-State: AOJu0YzCQCIpgzKrdMNy1ba6NTkU0s6ZuIXjLjQozHh9Yi4M5BdCliBt AEqYySTcKS++xjPsTj40I4LSFN67vY9qaT+S47tvMk+ZfHjj6H2hjqxMUy33f/7KAZ49rsAIfYu z0vvZ+xt1Sw== X-Google-Smtp-Source: AGHT+IHWOwvLhYykzD+ZJA1i2ggSdf+07569Kz5g+DXPmhbrE5iJHUl3UItVNQ3G63046gvJDJHwRZWQYLcR X-Received: from plbkg8.prod.google.com ([2002:a17:903:608:b0:223:5693:a4e9]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:f684:b0:234:c5c1:9b84 with SMTP id d9443c01a7336-23824050c20mr15632015ad.37.1750807146517; Tue, 24 Jun 2025 16:19:06 -0700 (PDT) Date: Tue, 24 Jun 2025 16:18:36 -0700 In-Reply-To: <20250624231837.179536-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250624231837.179536-1-irogers@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250624231837.179536-3-irogers@google.com> Subject: [PATCH v4 2/3] perf drm_pmu: Add a tool like PMU to expose DRM information From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Adrian Hunter , Kan Liang , "Masami Hiramatsu (Google)" , James Clark , Weilin Wang , Dominique Martinet , Thomas Richter , Junhao He , Jean-Philippe Romain , matthew.olson@intel.com, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, dri-devel@lists.freedesktop.org Cc: Ian Rogers Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" DRM clients expose information through usage stats as documented in Documentation/gpu/drm-usage-stats.rst (available online at https://docs.kernel.org/gpu/drm-usage-stats.html). Add a tool like PMU, similar to the hwmon PMU, that exposes DRM information. For example on a tigerlake laptop: ``` $ perf list drm List of pre-defined events (to be used in -e or -M): drm: drm-active-stolen-system0 [Total memory active in one or more engines. Unit: drm_i915] drm-active-system0 [Total memory active in one or more engines. Unit: drm_i915] drm-engine-capacity-video [Engine capacity. Unit: drm_i915] drm-engine-copy [Utilization in ns. Unit: drm_i915] drm-engine-render [Utilization in ns. Unit: drm_i915] drm-engine-video [Utilization in ns. Unit: drm_i915] drm-engine-video-enhance [Utilization in ns. Unit: drm_i915] drm-purgeable-stolen-system0 [Size of resident and purgeable memory bufers. Unit: drm_i915] drm-purgeable-system0 [Size of resident and purgeable memory bufers. Unit: drm_i915] drm-resident-stolen-system0 [Size of resident memory bufers. Unit: drm_i915] drm-resident-system0 [Size of resident memory bufers. Unit: drm_i915] drm-shared-stolen-system0 [Size of shared memory bufers. Unit: drm_i915] drm-shared-system0 [Size of shared memory bufers. Unit: drm_i915] drm-total-stolen-system0 [Size of shared and private memory. Unit: drm_i915] drm-total-system0 [Size of shared and private memory. Unit: drm_i915] ``` System wide data can be gathered: ``` $ perf stat -x, -I 1000 -e drm-active-stolen-system0,drm-active-system0,drm= -engine-capacity-video,drm-engine-copy,drm-engine-render,drm-engine-video,d= rm-engine-video-enhance,drm-purgeable-stolen-system0,drm-purgeable-system0,= drm-resident-stolen-system0,drm-resident-system0,drm-shared-stolen-system0,= drm-shared-system0,drm-total-stolen-system0,drm-total-system0 1.000904910,0,bytes,drm-active-stolen-system0,1,100.00,, 1.000904910,0,bytes,drm-active-system0,1,100.00,, 1.000904910,36,capacity,drm-engine-capacity-video,1,100.00,, 1.000904910,0,ns,drm-engine-copy,1,100.00,, 1.000904910,1472970566175,ns,drm-engine-render,1,100.00,, 1.000904910,0,ns,drm-engine-video,1,100.00,, 1.000904910,0,ns,drm-engine-video-enhance,1,100.00,, 1.000904910,0,bytes,drm-purgeable-stolen-system0,1,100.00,, 1.000904910,38199296,bytes,drm-purgeable-system0,1,100.00,, 1.000904910,0,bytes,drm-resident-stolen-system0,1,100.00,, 1.000904910,4643196928,bytes,drm-resident-system0,1,100.00,, 1.000904910,0,bytes,drm-shared-stolen-system0,1,100.00,, 1.000904910,1886871552,bytes,drm-shared-system0,1,100.00,, 1.000904910,0,bytes,drm-total-stolen-system0,1,100.00,, 1.000904910,4643196928,bytes,drm-total-system0,1,100.00,, 2.264426839,0,bytes,drm-active-stolen-system0,1,100.00,, ``` Or for a particular process: ``` $ perf stat -x, -I 1000 -e drm-active-stolen-system0,drm-active-system0,drm= -engine-capacity-video,drm-engine-copy,drm-engine-render,drm-engine-video,d= rm-engine-video-enhance,drm-purgeable-stolen-system0,drm-purgeable-system0,= drm-resident-stolen-system0,drm-resident-system0,drm-shared-stolen-system0,= drm-shared-system0,drm-total-stolen-system0,drm-total-system0 -p 200027 1.001040274,0,bytes,drm-active-stolen-system0,6,100.00,, 1.001040274,0,bytes,drm-active-system0,6,100.00,, 1.001040274,12,capacity,drm-engine-capacity-video,6,100.00,, 1.001040274,0,ns,drm-engine-copy,6,100.00,, 1.001040274,1542300,ns,drm-engine-render,6,100.00,, 1.001040274,0,ns,drm-engine-video,6,100.00,, 1.001040274,0,ns,drm-engine-video-enhance,6,100.00,, 1.001040274,0,bytes,drm-purgeable-stolen-system0,6,100.00,, 1.001040274,13516800,bytes,drm-purgeable-system0,6,100.00,, 1.001040274,0,bytes,drm-resident-stolen-system0,6,100.00,, 1.001040274,27746304,bytes,drm-resident-system0,6,100.00,, 1.001040274,0,bytes,drm-shared-stolen-system0,6,100.00,, 1.001040274,0,bytes,drm-shared-system0,6,100.00,, 1.001040274,0,bytes,drm-total-stolen-system0,6,100.00,, 1.001040274,27746304,bytes,drm-total-system0,6,100.00,, 2.016629075,0,bytes,drm-active-stolen-system0,6,100.00,, ``` As with the hwmon PMU, high numbered PMU types are used to encode multiple possible "DRM" PMUs. The appropriate fdinfo is found by scanning /proc and filtering which fdinfos to read with stat. To avoid some unneeding scanning, events not starting with "drm-" are ignored. The patch builds on commit 57e13264dcea ("perf pmus: Restructure pmu_read_sysfs to scan fewer PMUs") and later so that only if full wild carding is being done, the PMU starts with "drm_" or the event starts with "drm-" will /proc be scanned. That is there should be little to no cost in this PMU unless DRM events are requested. Signed-off-by: Ian Rogers --- tools/perf/util/Build | 1 + tools/perf/util/drm_pmu.c | 686 ++++++++++++++++++++++++++++++++++++++ tools/perf/util/drm_pmu.h | 39 +++ tools/perf/util/evsel.c | 9 + tools/perf/util/pmu.c | 15 + tools/perf/util/pmu.h | 4 +- tools/perf/util/pmus.c | 30 +- 7 files changed, 779 insertions(+), 5 deletions(-) create mode 100644 tools/perf/util/drm_pmu.c create mode 100644 tools/perf/util/drm_pmu.h diff --git a/tools/perf/util/Build b/tools/perf/util/Build index 7910d908c814..8a23eb767fb2 100644 --- a/tools/perf/util/Build +++ b/tools/perf/util/Build @@ -84,6 +84,7 @@ perf-util-y +=3D pmu.o perf-util-y +=3D pmus.o perf-util-y +=3D pmu-flex.o perf-util-y +=3D pmu-bison.o +perf-util-y +=3D drm_pmu.o perf-util-y +=3D hwmon_pmu.o perf-util-y +=3D tool_pmu.o perf-util-y +=3D svghelper.o diff --git a/tools/perf/util/drm_pmu.c b/tools/perf/util/drm_pmu.c new file mode 100644 index 000000000000..17385a10005b --- /dev/null +++ b/tools/perf/util/drm_pmu.c @@ -0,0 +1,686 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#include "drm_pmu.h" +#include "counts.h" +#include "cpumap.h" +#include "debug.h" +#include "evsel.h" +#include "pmu.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +enum drm_pmu_unit { + DRM_PMU_UNIT_BYTES, + DRM_PMU_UNIT_CAPACITY, + DRM_PMU_UNIT_CYCLES, + DRM_PMU_UNIT_HZ, + DRM_PMU_UNIT_NS, + + DRM_PMU_UNIT_MAX, +}; + +struct drm_pmu_event { + const char *name; + const char *desc; + enum drm_pmu_unit unit; +}; + +struct drm_pmu { + struct perf_pmu pmu; + struct drm_pmu_event *events; + int num_events; +}; + +static const char * const drm_pmu_unit_strs[DRM_PMU_UNIT_MAX] =3D { + "bytes", + "capacity", + "cycles", + "hz", + "ns", +}; + +static const char * const drm_pmu_scale_unit_strs[DRM_PMU_UNIT_MAX] =3D { + "1bytes", + "1capacity", + "1cycles", + "1hz", + "1ns", +}; + +bool perf_pmu__is_drm(const struct perf_pmu *pmu) +{ + return pmu && pmu->type >=3D PERF_PMU_TYPE_DRM_START && + pmu->type <=3D PERF_PMU_TYPE_DRM_END; +} + +bool evsel__is_drm(const struct evsel *evsel) +{ + return perf_pmu__is_drm(evsel->pmu); +} + +static struct drm_pmu *add_drm_pmu(struct list_head *pmus, char *line, siz= e_t line_len) +{ + struct drm_pmu *drm; + struct perf_pmu *pmu; + const char *name; + __u32 max_drm_pmu_type =3D 0, type; + int i =3D 12; + + if (line[line_len - 1] =3D=3D '\n') + line[line_len - 1] =3D '\0'; + while (isspace(line[i])) + i++; + + line[--i] =3D '_'; + line[--i] =3D 'm'; + line[--i] =3D 'r'; + line[--i] =3D 'd'; + name =3D &line[i]; + + list_for_each_entry(pmu, pmus, list) { + if (!perf_pmu__is_drm(pmu)) + continue; + if (pmu->type > max_drm_pmu_type) + max_drm_pmu_type =3D pmu->type; + if (!strcmp(pmu->name, name)) { + /* PMU already exists. */ + return NULL; + } + } + + if (max_drm_pmu_type !=3D 0) + type =3D max_drm_pmu_type + 1; + else + type =3D PERF_PMU_TYPE_DRM_START; + + if (type > PERF_PMU_TYPE_DRM_END) { + zfree(&drm); + pr_err("Unable to encode DRM PMU type for %s\n", name); + return NULL; + } + + drm =3D zalloc(sizeof(*drm)); + if (!drm) + return NULL; + + if (perf_pmu__init(&drm->pmu, type, name) !=3D 0) { + perf_pmu__delete(&drm->pmu); + return NULL; + } + + drm->pmu.cpus =3D perf_cpu_map__new("0"); + if (!drm->pmu.cpus) { + perf_pmu__delete(&drm->pmu); + return NULL; + } + return drm; +} + + +static bool starts_with(const char *str, const char *prefix) +{ + return !strncmp(prefix, str, strlen(prefix)); +} + +static int add_event(struct drm_pmu_event **events, int *num_events, + const char *line, enum drm_pmu_unit unit, const char *desc) +{ + const char *colon =3D strchr(line, ':'); + struct drm_pmu_event *tmp; + + if (!colon) + return -EINVAL; + + tmp =3D reallocarray(*events, *num_events + 1, sizeof(struct drm_pmu_even= t)); + if (!tmp) + return -ENOMEM; + tmp[*num_events].unit =3D unit; + tmp[*num_events].desc =3D desc; + tmp[*num_events].name =3D strndup(line, colon - line); + if (!tmp[*num_events].name) + return -ENOMEM; + (*num_events)++; + *events =3D tmp; + return 0; +} + +static int read_drm_pmus_cb(void *args, int fdinfo_dir_fd, const char *fd_= name) +{ + struct list_head *pmus =3D args; + char buf[640]; + struct io io; + char *line =3D NULL; + size_t line_len; + struct drm_pmu *drm =3D NULL; + struct drm_pmu_event *events =3D NULL; + int num_events =3D 0; + + io__init(&io, openat(fdinfo_dir_fd, fd_name, O_RDONLY), buf, sizeof(buf)); + if (io.fd =3D=3D -1) { + /* Failed to open file, ignore. */ + return 0; + } + + while (io__getline(&io, &line, &line_len) > 0) { + if (starts_with(line, "drm-driver:")) { + drm =3D add_drm_pmu(pmus, line, line_len); + if (!drm) + break; + continue; + } + /* + * Note the string matching below is alphabetical, with more + * specific matches appearing before less specific. + */ + if (starts_with(line, "drm-active-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_BYTES, + "Total memory active in one or more engines"); + continue; + } + if (starts_with(line, "drm-cycles-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_CYCLES, + "Busy cycles"); + continue; + } + if (starts_with(line, "drm-engine-capacity-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_CAPACITY, + "Engine capacity"); + continue; + } + if (starts_with(line, "drm-engine-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_NS, + "Utilization in ns"); + continue; + } + if (starts_with(line, "drm-maxfreq-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_HZ, + "Maximum frequency"); + continue; + } + if (starts_with(line, "drm-purgeable-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_BYTES, + "Size of resident and purgeable memory bufers"); + continue; + } + if (starts_with(line, "drm-resident-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_BYTES, + "Size of resident memory bufers"); + continue; + } + if (starts_with(line, "drm-shared-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_BYTES, + "Size of shared memory bufers"); + continue; + } + if (starts_with(line, "drm-total-cycles-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_BYTES, + "Total busy cycles"); + continue; + } + if (starts_with(line, "drm-total-")) { + add_event(&events, &num_events, line, DRM_PMU_UNIT_BYTES, + "Size of shared and private memory"); + continue; + } + if (verbose > 1 && starts_with(line, "drm-") && + !starts_with(line, "drm-client-id:") && + !starts_with(line, "drm-pdev:")) + pr_debug("Unhandled DRM PMU fdinfo line match '%s'\n", line); + } + if (drm) { + drm->events =3D events; + drm->num_events =3D num_events; + list_add_tail(&drm->pmu.list, pmus); + } + free(line); + if (io.fd !=3D -1) + close(io.fd); + return 0; +} + +void drm_pmu__exit(struct perf_pmu *pmu) +{ + struct drm_pmu *drm =3D container_of(pmu, struct drm_pmu, pmu); + + free(drm->events); +} + +bool drm_pmu__have_event(const struct perf_pmu *pmu, const char *name) +{ + struct drm_pmu *drm =3D container_of(pmu, struct drm_pmu, pmu); + + if (!starts_with(name, "drm-")) + return false; + + for (int i =3D 0; i < drm->num_events; i++) { + if (!strcasecmp(drm->events[i].name, name)) + return true; + } + return false; +} + +int drm_pmu__for_each_event(const struct perf_pmu *pmu, void *state, pmu_e= vent_callback cb) +{ + struct drm_pmu *drm =3D container_of(pmu, struct drm_pmu, pmu); + + for (int i =3D 0; i < drm->num_events; i++) { + char encoding_buf[128]; + struct pmu_event_info info =3D { + .pmu =3D pmu, + .name =3D drm->events[i].name, + .alias =3D NULL, + .scale_unit =3D drm_pmu_scale_unit_strs[drm->events[i].unit], + .desc =3D drm->events[i].desc, + .long_desc =3D NULL, + .encoding_desc =3D encoding_buf, + .topic =3D "drm", + .pmu_name =3D pmu->name, + .event_type_desc =3D "DRM event", + }; + int ret; + + snprintf(encoding_buf, sizeof(encoding_buf), "%s/config=3D0x%x/", pmu->n= ame, i); + + ret =3D cb(state, &info); + if (ret) + return ret; + } + return 0; +} + +size_t drm_pmu__num_events(const struct perf_pmu *pmu) +{ + const struct drm_pmu *drm =3D container_of(pmu, struct drm_pmu, pmu); + + return drm->num_events; +} + +static int drm_pmu__index_for_event(const struct drm_pmu *drm, const char = *name) +{ + for (int i =3D 0; i < drm->num_events; i++) { + if (!strcmp(drm->events[i].name, name)) + return i; + } + return -1; +} + +static int drm_pmu__config_term(const struct drm_pmu *drm, + struct perf_event_attr *attr, + struct parse_events_term *term, + struct parse_events_error *err) +{ + if (term->type_term =3D=3D PARSE_EVENTS__TERM_TYPE_USER) { + int i =3D drm_pmu__index_for_event(drm, term->config); + + if (i >=3D 0) { + attr->config =3D i; + return 0; + } + } + if (err) { + char *err_str; + + parse_events_error__handle(err, term->err_val, + asprintf(&err_str, + "unexpected drm event term (%s) %s", + parse_events__term_type_str(term->type_term), + term->config) < 0 + ? strdup("unexpected drm event term") + : err_str, + NULL); + } + return -EINVAL; +} + +int drm_pmu__config_terms(const struct perf_pmu *pmu, + struct perf_event_attr *attr, + struct parse_events_terms *terms, + struct parse_events_error *err) +{ + struct drm_pmu *drm =3D container_of(pmu, struct drm_pmu, pmu); + struct parse_events_term *term; + + list_for_each_entry(term, &terms->terms, list) { + if (drm_pmu__config_term(drm, attr, term, err)) + return -EINVAL; + } + + return 0; +} + +int drm_pmu__check_alias(const struct perf_pmu *pmu, struct parse_events_t= erms *terms, + struct perf_pmu_info *info, struct parse_events_error *err) +{ + struct drm_pmu *drm =3D container_of(pmu, struct drm_pmu, pmu); + struct parse_events_term *term =3D + list_first_entry(&terms->terms, struct parse_events_term, list); + + if (term->type_term =3D=3D PARSE_EVENTS__TERM_TYPE_USER) { + int i =3D drm_pmu__index_for_event(drm, term->config); + + if (i >=3D 0) { + info->unit =3D drm_pmu_unit_strs[drm->events[i].unit]; + info->scale =3D 1; + return 0; + } + } + if (err) { + char *err_str; + + parse_events_error__handle(err, term->err_val, + asprintf(&err_str, + "unexpected drm event term (%s) %s", + parse_events__term_type_str(term->type_term), + term->config) < 0 + ? strdup("unexpected drm event term") + : err_str, + NULL); + } + return -EINVAL; +} + +struct minor_info { + unsigned int *minors; + int minors_num, minors_len; +}; + +static int for_each_drm_fdinfo_in_dir(int (*cb)(void *args, int fdinfo_dir= _fd, const char *fd_name), + void *args, int proc_dir, const char *pid_name, + struct minor_info *minors) +{ + char buf[256]; + DIR *fd_dir; + struct dirent *fd_entry; + int fd_dir_fd, fdinfo_dir_fd =3D -1; + + + scnprintf(buf, sizeof(buf), "%s/fd", pid_name); + fd_dir_fd =3D openat(proc_dir, buf, O_DIRECTORY); + if (fd_dir_fd =3D=3D -1) + return 0; /* Presumably lost race to open. */ + fd_dir =3D fdopendir(fd_dir_fd); + if (!fd_dir) { + close(fd_dir_fd); + return -ENOMEM; + } + while ((fd_entry =3D readdir(fd_dir)) !=3D NULL) { + struct stat stat; + unsigned int minor; + bool is_dup =3D false; + int ret; + + if (fd_entry->d_type !=3D DT_LNK) + continue; + + if (fstatat(fd_dir_fd, fd_entry->d_name, &stat, 0) !=3D 0) + continue; + + if ((stat.st_mode & S_IFMT) !=3D S_IFCHR || major(stat.st_rdev) !=3D 226) + continue; + + minor =3D minor(stat.st_rdev); + for (int i =3D 0; i < minors->minors_num; i++) { + if (minor(stat.st_rdev) =3D=3D minors->minors[i]) { + is_dup =3D true; + break; + } + } + if (is_dup) + continue; + + if (minors->minors_num =3D=3D minors->minors_len) { + unsigned int *tmp =3D reallocarray(minors->minors, minors->minors_len += 4, + sizeof(unsigned int)); + + if (tmp) { + minors->minors =3D tmp; + minors->minors_len +=3D 4; + } + } + minors->minors[minors->minors_num++] =3D minor; + if (fdinfo_dir_fd =3D=3D -1) { + /* Open fdinfo dir if we have a DRM fd. */ + scnprintf(buf, sizeof(buf), "%s/fdinfo", pid_name); + fdinfo_dir_fd =3D openat(proc_dir, buf, O_DIRECTORY); + if (fdinfo_dir_fd =3D=3D -1) + continue; + } + ret =3D cb(args, fdinfo_dir_fd, fd_entry->d_name); + if (ret) + return ret; + } + if (fdinfo_dir_fd !=3D -1) + close(fdinfo_dir_fd); + closedir(fd_dir); + return 0; +} + +static int for_each_drm_fdinfo(bool skip_all_duplicates, + int (*cb)(void *args, int fdinfo_dir_fd, const char *fd_name), + void *args) +{ + DIR *proc_dir; + struct dirent *proc_entry; + int ret; + /* + * minors maintains an array of DRM minor device numbers seen for a pid, + * or for all pids if skip_all_duplicates is true, so that duplicates + * are ignored. + */ + struct minor_info minors =3D { + .minors =3D NULL, + .minors_num =3D 0, + .minors_len =3D 0, + }; + + proc_dir =3D opendir(procfs__mountpoint()); + if (!proc_dir) + return 0; + + /* Walk through the /proc directory. */ + while ((proc_entry =3D readdir(proc_dir)) !=3D NULL) { + if (proc_entry->d_type !=3D DT_DIR || + !isdigit(proc_entry->d_name[0])) + continue; + if (!skip_all_duplicates) { + /* Reset the seen minor numbers for each pid. */ + minors.minors_num =3D 0; + } + ret =3D for_each_drm_fdinfo_in_dir(cb, args, + dirfd(proc_dir), proc_entry->d_name, + &minors); + if (ret) + break; + } + free(minors.minors); + closedir(proc_dir); + return ret; +} + +int perf_pmus__read_drm_pmus(struct list_head *pmus) +{ + return for_each_drm_fdinfo(/*skip_all_duplicates=3D*/true, read_drm_pmus_= cb, pmus); +} + +int evsel__drm_pmu_open(struct evsel *evsel, + struct perf_thread_map *threads, + int start_cpu_map_idx, int end_cpu_map_idx) +{ + (void)evsel; + (void)threads; + (void)start_cpu_map_idx; + (void)end_cpu_map_idx; + return 0; +} + +static uint64_t read_count_and_apply_unit(const char *count_and_unit, enum= drm_pmu_unit unit) +{ + char *unit_ptr =3D NULL; + uint64_t count =3D strtoul(count_and_unit, &unit_ptr, 10); + + if (!unit_ptr) + return 0; + + while (isblank(*unit_ptr)) + unit_ptr++; + + switch (unit) { + case DRM_PMU_UNIT_BYTES: + if (*unit_ptr =3D=3D '\0') + assert(count =3D=3D 0); /* Generally undocumented, happens for 0. */ + else if (!strcmp(unit_ptr, "KiB")) + count *=3D 1024; + else if (!strcmp(unit_ptr, "MiB")) + count *=3D 1024 * 1024; + else + pr_err("Unexpected bytes unit '%s'\n", unit_ptr); + break; + case DRM_PMU_UNIT_CAPACITY: + /* No units expected. */ + break; + case DRM_PMU_UNIT_CYCLES: + /* No units expected. */ + break; + case DRM_PMU_UNIT_HZ: + if (!strcmp(unit_ptr, "Hz")) + count *=3D 1; + else if (!strcmp(unit_ptr, "KHz")) + count *=3D 1000; + else if (!strcmp(unit_ptr, "MHz")) + count *=3D 1000000; + else + pr_err("Unexpected hz unit '%s'\n", unit_ptr); + break; + case DRM_PMU_UNIT_NS: + /* Only unit ns expected. */ + break; + case DRM_PMU_UNIT_MAX: + default: + break; + } + return count; +} + +static uint64_t read_drm_event(int fdinfo_dir_fd, const char *fd_name, + const char *match, enum drm_pmu_unit unit) +{ + char buf[640]; + struct io io; + char *line =3D NULL; + size_t line_len; + uint64_t count =3D 0; + + io__init(&io, openat(fdinfo_dir_fd, fd_name, O_RDONLY), buf, sizeof(buf)); + if (io.fd =3D=3D -1) { + /* Failed to open file, ignore. */ + return 0; + } + while (io__getline(&io, &line, &line_len) > 0) { + size_t i =3D strlen(match); + + if (strncmp(line, match, i)) + continue; + if (line[i] !=3D ':') + continue; + while (isblank(line[++i])) + ; + if (line[line_len - 1] =3D=3D '\n') + line[line_len - 1] =3D '\0'; + count =3D read_count_and_apply_unit(&line[i], unit); + break; + } + free(line); + close(io.fd); + return count; +} + +struct read_drm_event_cb_args { + const char *match; + uint64_t count; + enum drm_pmu_unit unit; +}; + +static int read_drm_event_cb(void *vargs, int fdinfo_dir_fd, const char *f= d_name) +{ + struct read_drm_event_cb_args *args =3D vargs; + + args->count +=3D read_drm_event(fdinfo_dir_fd, fd_name, args->match, args= ->unit); + return 0; +} + +static uint64_t drm_pmu__read_system_wide(struct drm_pmu *drm, struct evse= l *evsel) +{ + struct read_drm_event_cb_args args =3D { + .count =3D 0, + .match =3D drm->events[evsel->core.attr.config].name, + .unit =3D drm->events[evsel->core.attr.config].unit, + }; + + for_each_drm_fdinfo(/*skip_all_duplicates=3D*/false, read_drm_event_cb, &= args); + return args.count; +} + +static uint64_t drm_pmu__read_for_pid(struct drm_pmu *drm, struct evsel *e= vsel, int pid) +{ + struct read_drm_event_cb_args args =3D { + .count =3D 0, + .match =3D drm->events[evsel->core.attr.config].name, + .unit =3D drm->events[evsel->core.attr.config].unit, + }; + struct minor_info minors =3D { + .minors =3D NULL, + .minors_num =3D 0, + .minors_len =3D 0, + }; + int proc_dir =3D open(procfs__mountpoint(), O_DIRECTORY); + char pid_name[12]; + int ret; + + if (proc_dir < 0) + return 0; + + snprintf(pid_name, sizeof(pid_name), "%d", pid); + ret =3D for_each_drm_fdinfo_in_dir(read_drm_event_cb, &args, proc_dir, pi= d_name, &minors); + free(minors.minors); + close(proc_dir); + return ret =3D=3D 0 ? args.count : 0; +} + +int evsel__drm_pmu_read(struct evsel *evsel, int cpu_map_idx, int thread) +{ + struct drm_pmu *drm =3D container_of(evsel->pmu, struct drm_pmu, pmu); + struct perf_counts_values *count, *old_count =3D NULL; + int pid =3D perf_thread_map__pid(evsel->core.threads, thread); + uint64_t counter; + + if (pid !=3D -1) + counter =3D drm_pmu__read_for_pid(drm, evsel, pid); + else + counter =3D drm_pmu__read_system_wide(drm, evsel); + + if (evsel->prev_raw_counts) + old_count =3D perf_counts(evsel->prev_raw_counts, cpu_map_idx, thread); + + count =3D perf_counts(evsel->counts, cpu_map_idx, thread); + if (old_count) { + count->val =3D old_count->val + counter; + count->run =3D old_count->run + 1; + count->ena =3D old_count->ena + 1; + } else { + count->val =3D counter; + count->run++; + count->ena++; + } + return 0; +} diff --git a/tools/perf/util/drm_pmu.h b/tools/perf/util/drm_pmu.h new file mode 100644 index 000000000000..e7f366fca8a4 --- /dev/null +++ b/tools/perf/util/drm_pmu.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __DRM_PMU_H +#define __DRM_PMU_H +/* + * Linux DRM clients expose information through usage stats as documented = in + * Documentation/gpu/drm-usage-stats.rst (available online at + * https://docs.kernel.org/gpu/drm-usage-stats.html). This is a tool like = PMU + * that exposes DRM information. + */ + +#include "pmu.h" +#include + +struct list_head; +struct perf_thread_map; + +void drm_pmu__exit(struct perf_pmu *pmu); +bool drm_pmu__have_event(const struct perf_pmu *pmu, const char *name); +int drm_pmu__for_each_event(const struct perf_pmu *pmu, void *state, pmu_e= vent_callback cb); +size_t drm_pmu__num_events(const struct perf_pmu *pmu); +int drm_pmu__config_terms(const struct perf_pmu *pmu, + struct perf_event_attr *attr, + struct parse_events_terms *terms, + struct parse_events_error *err); +int drm_pmu__check_alias(const struct perf_pmu *pmu, struct parse_events_t= erms *terms, + struct perf_pmu_info *info, struct parse_events_error *err); + + +bool perf_pmu__is_drm(const struct perf_pmu *pmu); +bool evsel__is_drm(const struct evsel *evsel); + +int perf_pmus__read_drm_pmus(struct list_head *pmus); + +int evsel__drm_pmu_open(struct evsel *evsel, + struct perf_thread_map *threads, + int start_cpu_map_idx, int end_cpu_map_idx); +int evsel__drm_pmu_read(struct evsel *evsel, int cpu_map_idx, int thread); + +#endif /* __DRM_PMU_H */ diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index d55482f094bf..9c50c3960487 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -56,6 +56,7 @@ #include "off_cpu.h" #include "pmu.h" #include "pmus.h" +#include "drm_pmu.h" #include "hwmon_pmu.h" #include "tool_pmu.h" #include "rlimit.h" @@ -1889,6 +1890,9 @@ int evsel__read_counter(struct evsel *evsel, int cpu_= map_idx, int thread) if (evsel__is_hwmon(evsel)) return evsel__hwmon_pmu_read(evsel, cpu_map_idx, thread); =20 + if (evsel__is_drm(evsel)) + return evsel__drm_pmu_read(evsel, cpu_map_idx, thread); + if (evsel__is_retire_lat(evsel)) return evsel__tpebs_read(evsel, cpu_map_idx, thread); =20 @@ -2610,6 +2614,11 @@ static int evsel__open_cpu(struct evsel *evsel, stru= ct perf_cpu_map *cpus, start_cpu_map_idx, end_cpu_map_idx); } + if (evsel__is_drm(evsel)) { + return evsel__drm_pmu_open(evsel, threads, + start_cpu_map_idx, + end_cpu_map_idx); + } =20 for (idx =3D start_cpu_map_idx; idx < end_cpu_map_idx; idx++) { cpu =3D perf_cpu_map__cpu(cpus, idx); diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c index 609828513f6c..f795883c233f 100644 --- a/tools/perf/util/pmu.c +++ b/tools/perf/util/pmu.c @@ -20,6 +20,7 @@ #include "debug.h" #include "evsel.h" #include "pmu.h" +#include "drm_pmu.h" #include "hwmon_pmu.h" #include "pmus.h" #include "tool_pmu.h" @@ -1627,6 +1628,8 @@ int perf_pmu__config_terms(const struct perf_pmu *pmu, =20 if (perf_pmu__is_hwmon(pmu)) return hwmon_pmu__config_terms(pmu, attr, terms, err); + if (perf_pmu__is_drm(pmu)) + return drm_pmu__config_terms(pmu, attr, terms, err); =20 list_for_each_entry(term, &terms->terms, list) { if (pmu_config_term(pmu, attr, term, terms, zero, apply_hardcoded, err)) @@ -1767,6 +1770,10 @@ int perf_pmu__check_alias(struct perf_pmu *pmu, stru= ct parse_events_terms *head_ ret =3D hwmon_pmu__check_alias(head_terms, info, err); goto out; } + if (perf_pmu__is_drm(pmu)) { + ret =3D drm_pmu__check_alias(pmu, head_terms, info, err); + goto out; + } =20 /* Fake PMU doesn't rewrite terms. */ if (perf_pmu__is_fake(pmu)) @@ -1949,6 +1956,8 @@ bool perf_pmu__have_event(struct perf_pmu *pmu, const= char *name) return false; if (perf_pmu__is_hwmon(pmu)) return hwmon_pmu__have_event(pmu, name); + if (perf_pmu__is_drm(pmu)) + return drm_pmu__have_event(pmu, name); if (perf_pmu__find_alias(pmu, name, /*load=3D*/ true) !=3D NULL) return true; if (pmu->cpu_aliases_added || !pmu->events_table) @@ -1962,6 +1971,8 @@ size_t perf_pmu__num_events(struct perf_pmu *pmu) =20 if (perf_pmu__is_hwmon(pmu)) return hwmon_pmu__num_events(pmu); + if (perf_pmu__is_drm(pmu)) + return drm_pmu__num_events(pmu); =20 pmu_aliases_parse(pmu); nr =3D pmu->sysfs_aliases + pmu->sys_json_aliases; @@ -2030,6 +2041,8 @@ int perf_pmu__for_each_event(struct perf_pmu *pmu, bo= ol skip_duplicate_pmus, =20 if (perf_pmu__is_hwmon(pmu)) return hwmon_pmu__for_each_event(pmu, state, cb); + if (perf_pmu__is_drm(pmu)) + return drm_pmu__for_each_event(pmu, state, cb); =20 strbuf_init(&sb, /*hint=3D*/ 0); pmu_aliases_parse(pmu); @@ -2511,6 +2524,8 @@ void perf_pmu__delete(struct perf_pmu *pmu) =20 if (perf_pmu__is_hwmon(pmu)) hwmon_pmu__exit(pmu); + else if (perf_pmu__is_drm(pmu)) + drm_pmu__exit(pmu); =20 perf_pmu__del_formats(&pmu->format); perf_pmu__del_aliases(pmu); diff --git a/tools/perf/util/pmu.h b/tools/perf/util/pmu.h index 71b8636fd07d..a4a08192154c 100644 --- a/tools/perf/util/pmu.h +++ b/tools/perf/util/pmu.h @@ -39,7 +39,9 @@ struct perf_pmu_caps { =20 enum { PERF_PMU_TYPE_PE_START =3D 0, - PERF_PMU_TYPE_PE_END =3D 0xFFFEFFFF, + PERF_PMU_TYPE_PE_END =3D 0xFFFDFFFF, + PERF_PMU_TYPE_DRM_START =3D 0xFFFE0000, + PERF_PMU_TYPE_DRM_END =3D 0xFFFEFFFF, PERF_PMU_TYPE_HWMON_START =3D 0xFFFF0000, PERF_PMU_TYPE_HWMON_END =3D 0xFFFFFFFD, PERF_PMU_TYPE_TOOL =3D 0xFFFFFFFE, diff --git a/tools/perf/util/pmus.c b/tools/perf/util/pmus.c index e0094f56b8e7..81c2ed689db2 100644 --- a/tools/perf/util/pmus.c +++ b/tools/perf/util/pmus.c @@ -12,6 +12,7 @@ #include #include "cpumap.h" #include "debug.h" +#include "drm_pmu.h" #include "evsel.h" #include "pmus.h" #include "pmu.h" @@ -43,16 +44,19 @@ enum perf_tool_pmu_type { PERF_TOOL_PMU_TYPE_PE_OTHER, PERF_TOOL_PMU_TYPE_TOOL, PERF_TOOL_PMU_TYPE_HWMON, + PERF_TOOL_PMU_TYPE_DRM, =20 #define PERF_TOOL_PMU_TYPE_PE_CORE_MASK (1 << PERF_TOOL_PMU_TYPE_PE_CORE) #define PERF_TOOL_PMU_TYPE_PE_OTHER_MASK (1 << PERF_TOOL_PMU_TYPE_PE_OTHER) #define PERF_TOOL_PMU_TYPE_TOOL_MASK (1 << PERF_TOOL_PMU_TYPE_TOOL) #define PERF_TOOL_PMU_TYPE_HWMON_MASK (1 << PERF_TOOL_PMU_TYPE_HWMON) +#define PERF_TOOL_PMU_TYPE_DRM_MASK (1 << PERF_TOOL_PMU_TYPE_DRM) =20 #define PERF_TOOL_PMU_TYPE_ALL_MASK (PERF_TOOL_PMU_TYPE_PE_CORE_MASK | \ PERF_TOOL_PMU_TYPE_PE_OTHER_MASK | \ PERF_TOOL_PMU_TYPE_TOOL_MASK | \ - PERF_TOOL_PMU_TYPE_HWMON_MASK) + PERF_TOOL_PMU_TYPE_HWMON_MASK | \ + PERF_TOOL_PMU_TYPE_DRM_MASK) }; static unsigned int read_pmu_types; =20 @@ -173,6 +177,8 @@ struct perf_pmu *perf_pmus__find(const char *name) /* Looking up an individual perf event PMU failed, check if a tool PMU sh= ould be read. */ if (!strncmp(name, "hwmon_", 6)) to_read_pmus |=3D PERF_TOOL_PMU_TYPE_HWMON_MASK; + else if (!strncmp(name, "drm_", 4)) + to_read_pmus |=3D PERF_TOOL_PMU_TYPE_DRM_MASK; else if (!strcmp(name, "tool")) to_read_pmus |=3D PERF_TOOL_PMU_TYPE_TOOL_MASK; =20 @@ -273,6 +279,10 @@ static void pmu_read_sysfs(unsigned int to_read_types) (read_pmu_types & PERF_TOOL_PMU_TYPE_HWMON_MASK) =3D=3D 0) perf_pmus__read_hwmon_pmus(&other_pmus); =20 + if ((to_read_types & PERF_TOOL_PMU_TYPE_DRM_MASK) !=3D 0 && + (read_pmu_types & PERF_TOOL_PMU_TYPE_DRM_MASK) =3D=3D 0) + perf_pmus__read_drm_pmus(&other_pmus); + list_sort(NULL, &other_pmus, pmus_cmp); =20 read_pmu_types |=3D to_read_types; @@ -305,6 +315,8 @@ struct perf_pmu *perf_pmus__find_by_type(unsigned int t= ype) if (type >=3D PERF_PMU_TYPE_PE_START && type <=3D PERF_PMU_TYPE_PE_END) { to_read_pmus =3D PERF_TOOL_PMU_TYPE_PE_CORE_MASK | PERF_TOOL_PMU_TYPE_PE_OTHER_MASK; + } else if (type >=3D PERF_PMU_TYPE_DRM_START && type <=3D PERF_PMU_TYPE_D= RM_END) { + to_read_pmus =3D PERF_TOOL_PMU_TYPE_DRM_MASK; } else if (type >=3D PERF_PMU_TYPE_HWMON_START && type <=3D PERF_PMU_TYPE= _HWMON_END) { to_read_pmus =3D PERF_TOOL_PMU_TYPE_HWMON_MASK; } else { @@ -371,6 +383,10 @@ struct perf_pmu *perf_pmus__scan_for_event(struct perf= _pmu *pmu, const char *eve if (parse_hwmon_filename(event, &type, &number, /*item=3D*/NULL, /*alarm= =3D*/NULL)) to_read_pmus |=3D PERF_TOOL_PMU_TYPE_HWMON_MASK; =20 + /* Could the event be a DRM event? */ + if (strlen(event) > 4 && strncmp("drm-", event, 4) =3D=3D 0) + to_read_pmus |=3D PERF_TOOL_PMU_TYPE_DRM_MASK; + pmu_read_sysfs(to_read_pmus); pmu =3D list_prepare_entry(pmu, &core_pmus, list); } @@ -403,11 +419,17 @@ struct perf_pmu *perf_pmus__scan_matching_wildcard(st= ruct perf_pmu *pmu, const c * Hwmon PMUs have an alias from a sysfs name like hwmon0, * hwmon1, etc. or have a name of hwmon_. They therefore * can only have a wildcard match if the wildcard begins with - * "hwmon". + * "hwmon". Similarly drm PMUs must start "drm_", avoid reading + * such events unless the PMU could match. */ - if (strisglob(wildcard) || - (strlen(wildcard) >=3D 5 && strncmp("hwmon", wildcard, 5) =3D=3D 0)) + if (strisglob(wildcard)) { + to_read_pmus |=3D PERF_TOOL_PMU_TYPE_HWMON_MASK | + PERF_TOOL_PMU_TYPE_DRM_MASK; + } else if (strlen(wildcard) >=3D 4 && strncmp("drm_", wildcard, 4) =3D= =3D 0) { + to_read_pmus |=3D PERF_TOOL_PMU_TYPE_DRM_MASK; + } else if (strlen(wildcard) >=3D 5 && strncmp("hwmon", wildcard, 5) =3D= =3D 0) { to_read_pmus |=3D PERF_TOOL_PMU_TYPE_HWMON_MASK; + } =20 pmu_read_sysfs(to_read_pmus); pmu =3D list_prepare_entry(pmu, &core_pmus, list); --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Oct 8 19:57:27 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CB6226B744 for ; Tue, 24 Jun 2025 23:19:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750807151; cv=none; b=ZRO/V6rdGqIXLEmuqIEpvnyr3lUO4ZLC3G7/2hjjkEESVWcEvc8V4+773oYXozKtNnCIhfWp3Ueb6BzTlj71PHqtSnmDVsZVK+859VvnPp1bZU8+CMoTQhTCL9wwmlMcKKA6OShwfax1zeJYD+H+X6H9R4gCfdN0aSUMH4/oki0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750807151; c=relaxed/simple; bh=Ov7KYM5urbb+duQlFNaUQMctYTgtbQGKJ1mEoTEkiAk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bVe+tVBtkeQx2tQ5GQpDO/z8Xg8Lq28+fpykPRCRHEdR4QTKy0P26aukUnzl8LSpBGa5SHzfXIjb2KBvh1wtBX8sHXZgnHT9uqmqfGDrd+lsIxUqevhga9PyI2Zt4kVhmzDpXxpOkT64B/l1CU44YTJZFSIvknxishroWOgtaAQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=StgIEZNo; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="StgIEZNo" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-74890390d17so716027b3a.2 for ; Tue, 24 Jun 2025 16:19:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1750807148; x=1751411948; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=B1kPy2JCJb4zSJWeggG32MlvxkW2zq73XjOq2Ztu1jk=; b=StgIEZNotgRr9KbKx1Hkb8wAaV2I80CrDLcL8H+MAojsJn6LXNJ1iKHEyYiGeypeij HTKFJsQ1FA/4jqcKChjtGrxjQY37MngWgANyuvZLwAWHLGi8vwy7e/Y7nPaTdjh40FBi TzS9L8m51/MRflJ2WQk0QmJY6eK4nYyOTcKh2p9GsFGR28QMyOcJ7hBGk6XjxO5yr2CT wyYk/KLgfXN4W6mHBzEN/yQ9eEG+m2yVFqNtmTUJmJsvk4xrSu2UHavbQ6ORSLtNxG9I qXu2SDBDJUGnlU623uJbgzCgS/dq4XXtkSYrbGCnL90yJhSbNdlD6lITncc2B09YBVxl P+TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750807148; x=1751411948; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=B1kPy2JCJb4zSJWeggG32MlvxkW2zq73XjOq2Ztu1jk=; b=GBRvKREuYPpQiMiSi/0QGEMsaykCDcKaSuQLW+7yuzCrWqi+q+SFJEIujhbrDc4lrk qbFMCLRuCYquYu6SqQPHaq35jsO6Cw8sRSlXlfsX2H3ITViW1pgNqwzfG2hJHX/JKPo5 424KgXQ061b8EWRQafVmYReOtyX7paWiZJwZ9nUlKSlpD+ep+kgKB/X9egagdFq2BZMl q63xQ+r2n4nq0ABK66/dtBwfd6psY2Q/7yi/GeKikcT2t9egdHPhLn7SRhTLViUmF7tB Y1X17TgapTUvafQu4cVJugvaJ9R8QwlU0VAmyEOOUffqsf0KI2SvyNZQLclVYBm8ZtoN QvOw== X-Forwarded-Encrypted: i=1; AJvYcCXiVmvaSUmZ3+Ax+1QJFOvRFmG/RHF8fdUI9hqgIY52syq5+ZO8oMBcqRN46st5/bGNIbtD/TS5rn5QRa0=@vger.kernel.org X-Gm-Message-State: AOJu0YwgX3OLR1gzysh963LxuqAeaIbz1lAgVH3VQeax88XI77tt1a9T 7LCch9coNie+skVpqk4MYOjHjSKid9BFmUYyeLl0i1H33k//pAvUL5u1VCwq7GtgZpri8qqBDfL EOs54Z9Hntg== X-Google-Smtp-Source: AGHT+IFM/3RFuKFIfuGzUEOUqlkqwyLCKg/Cqd+4H1d7Z9aSC+KCwBKw4SxuPZ/gbvXQWKoQuXWIxobHNqhB X-Received: from pfbfa15.prod.google.com ([2002:a05:6a00:2d0f:b0:749:2cc7:bd89]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1826:b0:748:fcfa:8bd5 with SMTP id d2e1a72fcca58-74ad4462622mr1658511b3a.3.1750807148521; Tue, 24 Jun 2025 16:19:08 -0700 (PDT) Date: Tue, 24 Jun 2025 16:18:37 -0700 In-Reply-To: <20250624231837.179536-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250624231837.179536-1-irogers@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250624231837.179536-4-irogers@google.com> Subject: [PATCH v4 3/3] perf tests: Add a DRM PMU test From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Adrian Hunter , Kan Liang , "Masami Hiramatsu (Google)" , James Clark , Weilin Wang , Dominique Martinet , Thomas Richter , Junhao He , Jean-Philippe Romain , matthew.olson@intel.com, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, dri-devel@lists.freedesktop.org Cc: Ian Rogers Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The test opens any DRM devices so that the shell has fdinfo files containing the DRM data. The test then uses perf stat to make sure the events can be read. Signed-off-by: Ian Rogers --- tools/perf/tests/shell/drm_pmu.sh | 78 +++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100755 tools/perf/tests/shell/drm_pmu.sh diff --git a/tools/perf/tests/shell/drm_pmu.sh b/tools/perf/tests/shell/drm= _pmu.sh new file mode 100755 index 000000000000..e629fe0e8463 --- /dev/null +++ b/tools/perf/tests/shell/drm_pmu.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# DRM PMU +# SPDX-License-Identifier: GPL-2.0 + +set -e + +output=3D$(mktemp /tmp/perf.drm_pmu.XXXXXX.txt) + +cleanup() { + rm -f "${output}" + + trap - EXIT TERM INT +} + +trap_cleanup() { + echo "Unexpected signal in ${FUNCNAME[1]}" + cleanup + exit 1 +} +trap trap_cleanup EXIT TERM INT + +# Array to store file descriptors and device names +declare -A device_fds + +# Open all devices and store file descriptors. Opening the device will cre= ate a +# /proc/$$/fdinfo file containing the DRM statistics. +fd_count=3D3 # Start with file descriptor 3 +for device in /dev/dri/* +do + if [[ ! -c "$device" ]] + then + continue + fi + major=3D$(stat -c "%Hr" "$device") + if [[ "$major" !=3D 226 ]] + then + continue + fi + echo "Opening $device" + eval "exec $fd_count<\"$device\"" + echo "fdinfo for: $device (FD: $fd_count)" + cat "/proc/$$/fdinfo/$fd_count" + echo + device_fds["$device"]=3D"$fd_count" + fd_count=3D$((fd_count + 1)) +done + +if [[ ${#device_fds[@]} -eq 0 ]] +then + echo "No DRM devices found [Skip]" + cleanup + exit 2 +fi + +# For each DRM event +err=3D0 +for p in $(perf list --raw-dump drm-) +do + echo -n "Testing perf stat of $p. " + perf stat -e "$p" --pid=3D$$ true > "$output" 2>&1 + if ! grep -q "$p" "$output" + then + echo "Missing DRM event in: [Failed]" + cat "$output" + err=3D1 + else + echo "[OK]" + fi +done + +# Close all file descriptors +for fd in "${device_fds[@]}"; do + eval "exec $fd<&-" +done + +# Finished +cleanup +exit $err --=20 2.50.0.727.gbf7dc18ff4-goog