From nobody Mon May 25 03:33:07 2026 Received: from mail-dl1-f73.google.com (mail-dl1-f73.google.com [74.125.82.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8757E311C2D for ; Tue, 19 May 2026 05:02:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779166933; cv=none; b=bY9adBJY0j8/H8ZpQ3PRmdIGNGyR0rLgzajaUE9MaVAfQrzZaLTHvDD6I0BHwhMa/xep1lGNn4c4FNaDm0fyTq9+i3oBmpU2K5X9wp2AbEVMxaSvxT6kMuUMU5qsUDI2SjbtXvYoTmQVimibvb5RTpS2Sa6clNvGA7i0xlrGhTg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779166933; c=relaxed/simple; bh=Dm1dRXMcj8LHggSUomlhbdBytPSZTwFogeXZK3/CgVE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gEGnYtt78XXfkkfS2yO+JHz8+FGHsZKMAIHFZUIXtGBnAt/aVdnFDqKgSBzKj5OfEeDKtvJ9FnFk0o4O935l5UacoekK8kYIN/Kym8eIRr/D2byxq6rZxvu0aqbXX+UM7N3U0bKqhksnn58nsiyBNmmKwwR5KvhmIBsCm2GDqjA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BgdugCJh; arc=none smtp.client-ip=74.125.82.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BgdugCJh" Received: by mail-dl1-f73.google.com with SMTP id a92af1059eb24-12dba1e866dso2175372c88.1 for ; Mon, 18 May 2026 22:02:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779166932; x=1779771732; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rpNAlzgW5SSa1ZsBC1HKHkIDD3KLaNZAgkWCJbzVU4w=; b=BgdugCJhpvIOA867XUU95b14UnKvWvo7prRnNNR4sO2jIxEU366vqp9ds+Sayg7u6T ovIKMNfaB/UU0PaqEjhahm+s8y6chsWmfr1iRJrrYOpjONAK8Xq7W02+Pobraz0z5cN5 WXEjLUACldl1PSO9Att16jCyxLeTBoFm3XfcuhuA7spqU1b3bl6fqJkfDIJ1LAK8pccO glMADBToNT5LSIMISVZVpgclUB6Qd4vF8vqmM9mOyjFYU7I6V8tnDljWzyB8cpH9dSaA PbrgRkUeJUVuQpup6bgjzSfEgpmAixIzcBdjzQ42vKr36z4pavzU3r4EDM9RHe784RpY rKQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779166932; x=1779771732; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rpNAlzgW5SSa1ZsBC1HKHkIDD3KLaNZAgkWCJbzVU4w=; b=Y8mljfIrTOC1+zUHLd9dBBE6muKajKFTEBLdg0r3fC43dhQiFLaLmaRNvTImb9M5sT BeGvzKOnJDdbwIh+wQW245rIqci+5FfLymOADVeAvKEe/UHFHuogCKeP+qvz0PwXV0X+ qpEhDU5/Q6xg7Y4NrC28ONKo7fw45o3qnyhG6p+Zhnx65AtkhpnyOisFVI6PYEvPFxfA m/8t3+MXdnHm9hFIO+HrmvtczGC59OLfSFwk/CMvnfmWtAdlq9LqEuIm/Qmu3gpUJn7K U86/rIjegBYOCWXmUbV4whzvCnQoHh12ziA7V4rOtUGEcdmYPvhNyWNnpTJ12sgKLfAR 3Bkw== X-Forwarded-Encrypted: i=1; AFNElJ9/f/MylVk2IQWPQAjtPFV6wejT3UsvpifYoBE5NSeEK0VzDf+ulcAt3vR6EtlaTWJqNXV1yUSFA0B6FPc=@vger.kernel.org X-Gm-Message-State: AOJu0Yxytr+WufHPQvxat1kRAHwO3p1DnNX1vmQcAGCbZkrKvppSP+NR dgvU+398rUDjVHf4QLOjaPn6HYlrhtDBZSQxv+zSkCJg4gDhS+CyhEtuBCHncT7jFe7xw8mCyzF +NQlzXJcknw== X-Received: from dlbrh6.prod.google.com ([2002:a05:7022:f306:b0:12c:1f83:7b4c]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:7022:6b8f:b0:135:3983:2131 with SMTP id a92af1059eb24-135398321e5mr4565827c88.21.1779166931554; Mon, 18 May 2026 22:02:11 -0700 (PDT) Date: Mon, 18 May 2026 22:01:49 -0700 In-Reply-To: <20260519050150.3144907-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260519015108.3094513-1-irogers@google.com> <20260519050150.3144907-1-irogers@google.com> X-Mailer: git-send-email 2.54.0.631.ge1b05301d1-goog Message-ID: <20260519050150.3144907-2-irogers@google.com> Subject: [PATCH v3 1/2] perf stat: Propagate supported flag to follower cgroup BPF events From: Ian Rogers To: irogers@google.com, acme@kernel.org, namhyung@kernel.org Cc: adrian.hunter@intel.com, james.clark@linaro.org, jolsa@kernel.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, skanev@google.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When using BPF counters with cgroups, follower events (for cgroups other than the first one) are not opened. Because they are not opened, their `supported` flag was left as `false`. During metric calculation, `prepare_metric` checks if the event is supported. If it is not supported (like the follower events), it explicitly sets the value to `NAN`, which eventually causes the metric to be reported as `nan %`. Fix this by propagating the `supported` flag from the "leader" events (the ones opened for the first cgroup) to the "follower" events. Reported-by: Svilen Kanev Assisted-by: Antigravity:gemini-3-flash Signed-off-by: Ian Rogers Acked-by: Namhyung Kim --- tools/perf/util/bpf_counter_cgroup.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/tools/perf/util/bpf_counter_cgroup.c b/tools/perf/util/bpf_cou= nter_cgroup.c index 519fee3dc3d0..dd1851634087 100644 --- a/tools/perf/util/bpf_counter_cgroup.c +++ b/tools/perf/util/bpf_counter_cgroup.c @@ -186,6 +186,21 @@ static int bperf_load_program(struct evlist *evlist) i++; } =20 + /* + * Propagate supported flag from leaders to followers. Follower events + * are not opened, so their supported flag remains false. + */ + { + struct evsel *leader; + int num_events =3D evlist->core.nr_entries / nr_cgroups; + + evlist__for_each_entry(evlist, evsel) { + leader =3D evlist__find_evsel(evlist, evsel->core.idx % num_events); + if (leader) + evsel->supported =3D leader->supported; + } + } + /* * bperf uses BPF_PROG_TEST_RUN to get accurate reading. Check * whether the kernel support it --=20 2.54.0.631.ge1b05301d1-goog From nobody Mon May 25 03:33:07 2026 Received: from mail-dl1-f74.google.com (mail-dl1-f74.google.com [74.125.82.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EE983164BA for ; Tue, 19 May 2026 05:02:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779166936; cv=none; b=AWeFuilifG2RGrPu9z1xc5Qy729AlGcqAGa0uJATmWiRXnv1M4mZtnONWZiUY/QhfLVZiKaMF1Hen3pZIBHjdKLfEe9UCuLsWFgkUHeOSsGMJ1Q9Dmu4bXZ4MM5G28RNjuMp0D1auOWL+c7sPK7cfYBGtJLoSFYG1c51VkIhDKk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779166936; c=relaxed/simple; bh=duIREe4HG2vRqUYgC/o42t17tX3jdFwLjdjffAKPCQA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cMyzEfxb/1hLuays6IZIoUGyNBNblRM6uMO8PWxMyfppvVnaJFdV5I9Lb0Yal6ZvWJXz9ybUUcYeNhdJVDuVH5KMMoapKT7IxOArThPkKLfyoeh6C83JXJ7pKX3H2JtcS8FgYET3GuqmMdpsyReC/PVmYkklrbuYPUQizTtT/eA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NgJeWqrl; arc=none smtp.client-ip=74.125.82.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NgJeWqrl" Received: by mail-dl1-f74.google.com with SMTP id a92af1059eb24-134cf343c58so6166583c88.1 for ; Mon, 18 May 2026 22:02:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779166934; x=1779771734; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0aoZoCjQzbjQWWo0j3ADTrlwfj8g1NrK/ci8EJbhSMQ=; b=NgJeWqrlQXKftaWrKhSKpDkWzwh4O9777gb3gwPz80Iub7VjV84OPTUbQ34YuT/sn6 Cf7j3tiJWimMV5fCC9ozj+vkDJVAI3Q2kOZiCOG703LA2jextOTu+J+oae0E+uS6HPfJ o2RoBUldEpgZQQJC+IFHF62WbJtQNML6sp5kWhDcTWuLJXIHwhsi2DRrhZktFZqg/8dM c4AEMUpjtiYUGmlI4nOV02F1yLmrNasXMOkAUr29Lmt8pYLRrAoJ779e8TEA2lo6O8rd gee1JXB+Gd3yZwLbgYlG0TM/JFZT86ktHQuhNrTTwD4SYGumcBg4UFMtOXmSHcMJ9QwO bL0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779166934; x=1779771734; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0aoZoCjQzbjQWWo0j3ADTrlwfj8g1NrK/ci8EJbhSMQ=; b=ss7DQtFATNs02un4ov9xx2J0f58U+YQ78hU0lBVsT/+gINA4RjSFt/x1jaNY9+prBz vAvM10gP7pNwlsClnOQsdQqOLdMv6N5Zs2TzjYxe4BGy/+GoS7pjZpQS+KJx3uZNSs8h 8HBlXxWWWgMB77zgOyadDzMuZ8zdaHMNryKDLeMj5ut01pREQITUnpaLRQDwm8CgrFAC 8MoyzyNz7kvPfgZByE1ZZdFifZ3TrEEKvAby+1UZO0WpI8LboHLlpjSv7DX27koS/m1H 6QRku4U/fvda6tE5+fhV0yJgg43ikecf/QVf8kcaNN6lP6cAVpycNNQHPUukpReqLvy4 1cpQ== X-Forwarded-Encrypted: i=1; AFNElJ+bVBLuu26DDD8azzonbwkHrnf3kfcAJU3TtsFYjsK7x4EUQspIDN8T9RDHSBs/biMlFCegngpYdgqPgkU=@vger.kernel.org X-Gm-Message-State: AOJu0Yw5ZflWzQ8rgv+F6Amef9WeHqTePMrLRpcXR/nad8BsBYNVa0xv hhbmB4++6uHurDkyhoWWE6hszCtQtm/lPIgCjCcXHp110HpEn/GJ8wl3m61VgXrsqiSXWcAa6oQ GfwofISqgEA== X-Received: from dlzz26.prod.google.com ([2002:a05:7022:49a:b0:135:b9d2:2b2a]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:7022:128c:b0:134:def6:e751 with SMTP id a92af1059eb24-1350484238emr7611746c88.30.1779166933465; Mon, 18 May 2026 22:02:13 -0700 (PDT) Date: Mon, 18 May 2026 22:01:50 -0700 In-Reply-To: <20260519050150.3144907-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260519015108.3094513-1-irogers@google.com> <20260519050150.3144907-1-irogers@google.com> X-Mailer: git-send-email 2.54.0.631.ge1b05301d1-goog Message-ID: <20260519050150.3144907-3-irogers@google.com> Subject: [PATCH v3 2/2] perf test: Add stat metrics --for-each-cgroup test From: Ian Rogers To: irogers@google.com, acme@kernel.org, namhyung@kernel.org Cc: adrian.hunter@intel.com, james.clark@linaro.org, jolsa@kernel.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, skanev@google.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a new shell test `stat_metrics_cgrp.sh` to verify metric reporting with `--for-each-cgroup`, both with and without `--bpf-counters`. The test: - Checks if system-wide monitoring is supported (skips if not). - Finds cgroups to test. - Runs `perf stat` with `insn_per_cycle` metric and verifies that the metric is reported for each cgroup and does not contain invalid values (like `nan` or `nested`) or empty values when counts are present. - Tests both standard mode and BPF counters mode (if supported). Assisted-by: Antigravity:gemini-3-flash Signed-off-by: Ian Rogers Reported-by: Svilen Kanev --- tools/perf/tests/shell/stat_metrics_cgrp.sh | 174 ++++++++++++++++++++ 1 file changed, 174 insertions(+) create mode 100755 tools/perf/tests/shell/stat_metrics_cgrp.sh diff --git a/tools/perf/tests/shell/stat_metrics_cgrp.sh b/tools/perf/tests= /shell/stat_metrics_cgrp.sh new file mode 100755 index 000000000000..b413849193a0 --- /dev/null +++ b/tools/perf/tests/shell/stat_metrics_cgrp.sh @@ -0,0 +1,174 @@ +#!/bin/bash +# perf stat metrics --for-each-cgroup test +# SPDX-License-Identifier: GPL-2.0 + +set -e + +test_cgroups=3D + +log_verbose() { + echo "$1" +} + +is_numeric_and_non_zero() { + local val=3D"$1" + if [[ "${val}" =3D~ ^[0-9]+$ ]] && [ "${val}" -gt 0 ] + then + return 0 # True + fi + return 1 # False +} + +# skip if system-wide is not supported +check_system_wide() +{ + log_verbose "Checking system-wide..." + if ! perf stat -a --metrics=3Dinsn_per_cycle sleep 0.01 > /dev/null 2>&1 + then + log_verbose "Skipping: system-wide monitoring not supported" + exit 2 + fi +} + +# find two cgroups to measure +find_cgroups() +{ + log_verbose "Finding cgroups..." + # try usual systemd slices first + if [ -d /sys/fs/cgroup/system.slice ] && [ -d /sys/fs/cgroup/user.slice ] + then + test_cgroups=3D"system.slice,user.slice" + log_verbose "Found cgroups: ${test_cgroups}" + return + fi + + # try root and self cgroups + find_cgroups_self_cgrp=3D$(grep perf_event /proc/self/cgroup | cut -d: -f= 3) + if [ -z "${find_cgroups_self_cgrp}" ] + then + # cgroup v2 doesn't specify perf_event + find_cgroups_self_cgrp=3D$(grep ^0: /proc/self/cgroup | cut -d: -f3) + fi + + if [ -z "${find_cgroups_self_cgrp}" ] + then + test_cgroups=3D"/" + else + test_cgroups=3D"/,${find_cgroups_self_cgrp}" + fi + log_verbose "Found cgroups: ${test_cgroups}" +} + +# Check if metric is reported for each cgroup +# $1: extra options (e.g. --bpf-counters) +check_metric_reported() +{ + local opts=3D"$1" + local output + + log_verbose "Running check_metric_reported with opts '${opts}'..." + # Run perf stat + if ! output=3D$(perf stat -a ${opts} \ + --metrics=3Dinsn_per_cycle \ + --for-each-cgroup "${test_cgroups}" \ + -x, sleep 0.1 2>&1) + then + echo "FAIL: perf stat failed with exit code $?" + echo "Output: ${output}" + exit 1 + fi + + log_verbose "perf stat output:" + log_verbose "${output}" + + # Split test_cgroups by comma + IFS=3D',' read -r -a cgrps <<< "${test_cgroups}" + + for cgrp in "${cgrps[@]}"; do + # Find metric lines for this cgroup + # We use exact cgroup match with surrounding commas + local cgrp_lines + cgrp_lines=3D$(echo "${output}" | grep -F ",${cgrp}," | grep "insn_per_c= ycle" || true) + + if [ -z "${cgrp_lines}" ] + then + echo "FAIL: No metric lines found for cgroup '${cgrp}'" + exit 1 + fi + + # Parse each metric line + while read -r line; do + [ -z "${line}" ] && continue + + local cycles_val + cycles_val=3D$(echo "${line}" | cut -d, -f1) + + local event_name + event_name=3D$(echo "${line}" | cut -d, -f3) + + # Find corresponding instructions event line + local inst_event_name + inst_event_name=3D"${event_name/cpu-cycles/instructions}" + inst_event_name=3D"${inst_event_name/cycles/instructions}" + + local inst_line + inst_line=3D$(echo "${output}" | \ + grep -F ",${cgrp}," | \ + grep -F "${inst_event_name}" || true) + + local inst_val + inst_val=3D$(echo "${inst_line}" | cut -d, -f1) + + log_verbose "Cgroup '${cgrp}': event '${event_name}' \ +val '${cycles_val}', inst event '${inst_event_name}' val '${inst_val}'" + + # Only enforce metric check if both cycles and + # instructions have non-zero numeric counts + if is_numeric_and_non_zero "${cycles_val}" && \ + is_numeric_and_non_zero "${inst_val}" + then + log_verbose "Enforcing metric check for cgroup '${cgrp}' \ +event '${event_name}'" + # Check for nan or nested in the metric value (7th field) + # Actually we can just check the whole line for simplicity + if echo "${line}" | grep -q -i -E ",nan,|,nested," + then + echo "FAIL: Invalid metric value (nan/nested) \ +for cgroup '${cgrp}'" + echo "Line: ${line}" + exit 1 + fi + # Check for empty metric value (2 consecutive + # commas before the unit) + if echo "${line}" | grep -q -E ",,[[:space:]]*[^,]*insn_per_cycle" + then + echo "FAIL: Empty metric value for cgroup '${cgrp}'" + echo "Line: ${line}" + exit 1 + fi + else + log_verbose "Skipping metric check for cgroup '${cgrp}' \ +event '${event_name}' (idle or not counted)" + fi + done <<< "${cgrp_lines}" + done + log_verbose "check_metric_reported passed for opts '${opts}'" +} + +check_system_wide +find_cgroups + +# Test 1: Without BPF counters +check_metric_reported "" + +# Test 2: With BPF counters (if supported) +log_verbose "Checking BPF support..." +if perf stat -a --bpf-counters --for-each-cgroup / true > /dev/null 2>&1 +then + log_verbose "BPF supported, running Test 2..." + check_metric_reported "--bpf-counters" +else + log_verbose "BPF not supported, skipping Test 2" +fi + +exit 0 --=20 2.54.0.631.ge1b05301d1-goog