From nobody Mon May 25 01:58:03 2026 Received: from mail-dy1-f202.google.com (mail-dy1-f202.google.com [74.125.82.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6E5D408025 for ; Tue, 19 May 2026 15:27:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779204451; cv=none; b=oY1JTReHR0zdebVg/Te2Fi14sCqQZvk5NUTkUEDpJoR+Sa8wiP7nSZM6VcyVpDr2Bem79ECsumwSDSHHDK6yj1RjXJ5wuRAON+2URsNTJa6LQ43VdS/ial+9xCNPs4JjVv+8oWLyMcceLPn5C+yLFJ7+hLIPUFqxEYADT3YZlmc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779204451; c=relaxed/simple; bh=tZ6+tK1cele7SlSUkxxqDApfB/RID9mJwcFOhcOwAM0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pYyypQGPh7Q7qGJ+MOVMU7qGPTG5WHsLFVcNdolBUOQYQKZDF97zZde+HFYhNLf057jIDIKgvxeJsS8qyGzBfP8I544zQ7Le0eTqaYC3fIFVP0gMwmESl8OB/0RdUHNXkOGgvJBGJT66LuomzuLRpVVYKqH2HvHy2+PH4sLxeyQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=k/5atBZU; arc=none smtp.client-ip=74.125.82.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="k/5atBZU" Received: by mail-dy1-f202.google.com with SMTP id 5a478bee46e88-2f2d983d109so15678947eec.0 for ; Tue, 19 May 2026 08:27:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779204449; x=1779809249; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=4fIflKPf+pFDRZswdcVI1JmNYXbeE5Y4CNZteMwLexA=; b=k/5atBZUtdj00pJqMbCy370idpIPN4ZndUKpRgzoQvp9tiqRt0/3kb5De7UtSt5x3p QKvD1d4LpkSLqhwsCqy1yOfDMbatTfFGmrIbpOznNnkbLs993Xj+QOb2jIQzo3fSdogQ SD98h3IExdsv1cPx8g99ljJlWej/yGDm7elbun/eFmEQG6yzXW/IMRT83uI6UgeDrfC+ PmXx30Kq1bvH3t6Ee2lhIADck/fjpz/CA0KmG3hxaTNYYW6QuXvnTHqhLGqX8/qCJdkl yCcg9d5giLKZM6zyIsndR7px89pkqM/zIiofG2reMAaGh+jBeF4ie1cb285VL0LM54G2 nv1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779204449; x=1779809249; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4fIflKPf+pFDRZswdcVI1JmNYXbeE5Y4CNZteMwLexA=; b=pbpseIMfQ0//qEk/c8XalqI6/ydGBAaMMAfBQAyZ3lOEqcg18vupbcPFHCbsSBsZkw LGsFlT87CNzjGR3CtbDsOszbKazYCK8zQp0f23OpQiVls795eIik8oMgHkbn1SiA4iZK EheIxsnGvVbQhWDfY5+aS9KOEJvYC5wG0dzds7z8XTzo1SQyLiitFdtFFRiVDThx5qiH iQIUj42RsuFHbpfvm+W/gwt4+Nz3lA/FeBNbZGOyOY5zD5ZqJrxhjmWVAK6bxVN2rN3E pzTpATekj0fT2j8jxoSqTTBgjOxaISszihbxgqNTQAGN7LF7Q/0MpW51C5tPC8IL7hmL DB6A== X-Forwarded-Encrypted: i=1; AFNElJ8X3eDuohixcDA+8kIIpYNQdq2NsVAXDPY84KPZWvbkhXXmf0d5wPX3QalYiU6kICNyV8DioM2QM34KcZA=@vger.kernel.org X-Gm-Message-State: AOJu0Yx6KRYyU7uGpd2mY5m57YeVKrQ4L5kmLP9TMhwJ6m2dZRGYRlxY cvTWyAs5CtT9ll7G8v1KilQudy3xewQ3SuXK2I4VY6p1rNNE3ddQYwRCiuUjkoXgwmVGAZbx8+a 4sF168/7LmA== X-Received: from dyr13.prod.google.com ([2002:a05:693c:400d:b0:2ee:4745:8c0a]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:7300:be17:b0:2d9:6373:ad1d with SMTP id 5a478bee46e88-30398625eb1mr8624350eec.20.1779204448744; Tue, 19 May 2026 08:27:28 -0700 (PDT) Date: Tue, 19 May 2026 08:27:15 -0700 In-Reply-To: <20260519152716.3464144-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260519050150.3144907-1-irogers@google.com> <20260519152716.3464144-1-irogers@google.com> X-Mailer: git-send-email 2.54.0.631.ge1b05301d1-goog Message-ID: <20260519152716.3464144-2-irogers@google.com> Subject: [PATCH v4 1/2] perf stat: Propagate supported flag to follower cgroup BPF events From: Ian Rogers To: irogers@google.com, acme@kernel.org, namhyung@kernel.org Cc: adrian.hunter@intel.com, james.clark@linaro.org, jolsa@kernel.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, skanev@google.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When using BPF counters with cgroups, follower events (for cgroups other than the first one) are not opened. Because they are not opened, their `supported` flag was left as `false`. During metric calculation, `prepare_metric` checks if the event is supported. If it is not supported (like the follower events), it explicitly sets the value to `NAN`, which eventually causes the metric to be reported as `nan %`. Fix this by propagating the `supported` flag from the "leader" events (the ones opened for the first cgroup) to the "follower" events. Also add a validation check to `bperf_load_program` to ensure `nr_cgroups` is not zero and the number of events is a multiple of `nr_cgroups`, preventing a potential division-by-zero (SIGFPE) exception when `num_events` evaluates to 0 (e.g., with a trailing comma in cgroups list). Reported-by: Svilen Kanev Acked-by: Namhyung Kim Assisted-by: Antigravity:gemini-3-flash Signed-off-by: Ian Rogers --- tools/perf/util/bpf_counter_cgroup.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/tools/perf/util/bpf_counter_cgroup.c b/tools/perf/util/bpf_cou= nter_cgroup.c index 519fee3dc3d0..e1ce5aa3b957 100644 --- a/tools/perf/util/bpf_counter_cgroup.c +++ b/tools/perf/util/bpf_counter_cgroup.c @@ -104,6 +104,11 @@ static int bperf_load_program(struct evlist *evlist) =20 set_max_rlimit(); =20 + if (nr_cgroups =3D=3D 0 || evlist->core.nr_entries % nr_cgroups !=3D 0) { + pr_err("Invalid cgroup or event count\n"); + return -EINVAL; + } + test_max_events_program_load(); =20 skel =3D bperf_cgroup_bpf__open(); @@ -186,6 +191,21 @@ static int bperf_load_program(struct evlist *evlist) i++; } =20 + /* + * Propagate supported flag from leaders to followers. Follower events + * are not opened, so their supported flag remains false. + */ + { + struct evsel *leader; + int num_events =3D evlist->core.nr_entries / nr_cgroups; + + evlist__for_each_entry(evlist, evsel) { + leader =3D evlist__find_evsel(evlist, evsel->core.idx % num_events); + if (leader) + evsel->supported =3D leader->supported; + } + } + /* * bperf uses BPF_PROG_TEST_RUN to get accurate reading. Check * whether the kernel support it --=20 2.54.0.631.ge1b05301d1-goog From nobody Mon May 25 01:58:03 2026 Received: from mail-dy1-f201.google.com (mail-dy1-f201.google.com [74.125.82.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D926934E760 for ; Tue, 19 May 2026 15:27:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779204453; cv=none; b=AuatgGZCQzXxp65tkInLvu+e+agShQbSE6tQulrHaeZBNYdQ+fsx86ZZy8KDnHGeF/vDDgndZIu7hVC7tk+hF39D8li83hVNsrz+ELGLGFPSlQBw/G/Zp9tdW4JF6JhnOJQerkPbRI0LIFtfvA22Wd5ny54oTtNr2p/PcckrlrA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779204453; c=relaxed/simple; bh=hdPQLbELoyYXCHOx/RYBPidYuoOvW8qQm7CvBoVMcZo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=FwfmUIlxWl3mVVmoQQWldyRW9H1S0VcIhJ/NxjUbRrkinAsef1Hpu3FsCoeDyQ6ui1eghHaePsIkdM59cRaboEiqMWFZBZGDLOC+BuGvTCuMaNy9UmnFqPTWFcvjoDXBXJAdwNg1uK3u4ELkAFe9ekJunF0wIlMXPV+0gVwMNsA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cOY6a7nB; arc=none smtp.client-ip=74.125.82.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cOY6a7nB" Received: by mail-dy1-f201.google.com with SMTP id 5a478bee46e88-2f485961555so18151140eec.1 for ; Tue, 19 May 2026 08:27:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779204451; x=1779809251; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mK1jYuMDB5WeEbtxhwS5vhYLwEQjF9HCTJl/UtbPre4=; b=cOY6a7nBP4QNbthX8rwU/k24CrqcIgsHv5VWfQrWT1Vhce1sxO1htdkgFNq58nqMye lf9V4QS4MIdnEmp/vH/Gq8wJxv5I/7WH3cEDTAFeRCs1bjMQH1RcjXC6rGaE5PSXERpx B6jvj81E5TxhrSd6QDu/aEDSF7SJ7awgoJ8GKsae2ocLWff+n7+hplMb9wijLm/MgolN SVNUCbVS/1xEDpJrQdcwvHnBzI6Utqv/mGDD7bOCgkDvWrLXbkMhp1g4iaVNqX5Nr8Zz rD29A+7n4R8mwDD3mMvD8ePrpVD004dpcfm2CEYuQGJiiZf7lb2cc70FYNPSnQEi0dk9 BWKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779204451; x=1779809251; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mK1jYuMDB5WeEbtxhwS5vhYLwEQjF9HCTJl/UtbPre4=; b=iMCk8lA0bZRtc7oxioNZ8MPTMqaOvhX8nqmAOhocYYisLiZxD6dZvb6rTHOUwKuTV/ AOUboKPVSFte1K0UXOdZLhCyzaeE9vsNTRaEQOE6l72bwXXfGu71DP4Ybx5mohYQJ6FR dALuQEY+qpiQ/7XruA3frrIDZowjamOhEbIF3PiTjyQRmtioiMX6TsZEsHePwr6TGJrY /N5PuwA3IO0nzYHJoZjk9qHaZcfkG/hc54kAxSS3FvMOJdmSmu53D4rxw7ze7Aid/2BZ Gz4lWKqihNG8kDTfDu/4z0msN6Z1XhCbfUj5zBcOZkHokqx5Qt4LlNvFxDFY5r72Q+oJ uo7Q== X-Forwarded-Encrypted: i=1; AFNElJ9kl1zkveTaWbbOVipHLfqmJZGB5SbX9HR2SIeELRoa8CcxgmyTekuxlnpfbq85gg/Jj7TjFpBC2IgQRpM=@vger.kernel.org X-Gm-Message-State: AOJu0Yzwddw+nu9/vosF4bFaLsvknrath7Cm3FmXOMBfMte7OcLeiLWu 0SZdlXlTFy98CxLrNgRlkpHna9hLMEFUSCOS4xSImbT/m+XgdVU2tjjdtbAjIWYow6V/L8IQnkA g91C4Z7eGMA== X-Received: from dyco28.prod.google.com ([2002:a05:7300:51dc:b0:303:2247:37b]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:7300:a903:b0:2d1:d434:cfe3 with SMTP id 5a478bee46e88-30397b7de6dmr9020627eec.0.1779204450792; Tue, 19 May 2026 08:27:30 -0700 (PDT) Date: Tue, 19 May 2026 08:27:16 -0700 In-Reply-To: <20260519152716.3464144-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260519050150.3144907-1-irogers@google.com> <20260519152716.3464144-1-irogers@google.com> X-Mailer: git-send-email 2.54.0.631.ge1b05301d1-goog Message-ID: <20260519152716.3464144-3-irogers@google.com> Subject: [PATCH v4 2/2] perf test: Add stat metrics --for-each-cgroup test From: Ian Rogers To: irogers@google.com, acme@kernel.org, namhyung@kernel.org Cc: adrian.hunter@intel.com, james.clark@linaro.org, jolsa@kernel.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, skanev@google.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a new shell test `stat_metrics_cgrp.sh` to verify metric reporting with `--for-each-cgroup`, both with and without `--bpf-counters`. The test: - Checks if system-wide monitoring is supported (skips if not). - Finds cgroups to test. - Runs `perf stat` with `insn_per_cycle` metric and verifies that the metric is reported for each cgroup. - Dynamically pairs and verifies instructions and cycles counts to avoid false failures on idle cgroups. - Tests both standard mode and BPF counters mode (if supported). Assisted-by: Antigravity:gemini-3-flash Signed-off-by: Ian Rogers Reported-by: Svilen Kanev --- tools/perf/tests/shell/stat_metrics_cgrp.sh | 200 ++++++++++++++++++++ 1 file changed, 200 insertions(+) create mode 100755 tools/perf/tests/shell/stat_metrics_cgrp.sh diff --git a/tools/perf/tests/shell/stat_metrics_cgrp.sh b/tools/perf/tests= /shell/stat_metrics_cgrp.sh new file mode 100755 index 000000000000..d4226ee0ae98 --- /dev/null +++ b/tools/perf/tests/shell/stat_metrics_cgrp.sh @@ -0,0 +1,200 @@ +#!/bin/bash +# perf stat metrics --for-each-cgroup test +# SPDX-License-Identifier: GPL-2.0 + +set -e + +test_cgroups=3D + +log_verbose() { + echo "$1" +} + +is_numeric_and_non_zero() { + local val=3D"$1" + if [[ "${val}" =3D~ ^[0-9]+$ ]] && [ "${val}" -gt 0 ] + then + return 0 # True + fi + return 1 # False +} + +# skip if system-wide is not supported +check_system_wide() +{ + log_verbose "Checking system-wide..." + if ! perf stat -a --metrics=3Dinsn_per_cycle sleep 0.01 > /dev/null 2>&1 + then + log_verbose "Skipping: system-wide monitoring not supported" + exit 2 + fi +} + +# find two cgroups to measure +find_cgroups() +{ + log_verbose "Finding cgroups..." + # try usual systemd slices first + if [ -d /sys/fs/cgroup/system.slice ] && [ -d /sys/fs/cgroup/user.slice ] + then + test_cgroups=3D"system.slice,user.slice" + log_verbose "Found cgroups: ${test_cgroups}" + return + fi + + # try root and self cgroups + find_cgroups_self_cgrp=3D$(grep perf_event /proc/self/cgroup | cut -d: -f= 3) + if [ -z "${find_cgroups_self_cgrp}" ] + then + # cgroup v2 doesn't specify perf_event + find_cgroups_self_cgrp=3D$(grep ^0: /proc/self/cgroup | cut -d: -f3) + fi + + if [ -z "${find_cgroups_self_cgrp}" ] + then + test_cgroups=3D"/" + else + test_cgroups=3D"/,${find_cgroups_self_cgrp}" + fi + log_verbose "Found cgroups: ${test_cgroups}" +} + +# Check if metric is reported for each cgroup +# $1: extra options (e.g. --bpf-counters) +check_metric_reported() +{ + local opts=3D"$1" + local output + + log_verbose "Running check_metric_reported with opts '${opts}'..." + # Run perf stat + if ! output=3D$(perf stat -a ${opts} \ + --metrics=3Dinsn_per_cycle \ + --for-each-cgroup "${test_cgroups}" \ + -x, sleep 0.1 2>&1) + then + echo "FAIL: perf stat failed with exit code $?" + echo "Output: ${output}" + exit 1 + fi + + log_verbose "perf stat output:" + log_verbose "${output}" + + # Split test_cgroups by comma + IFS=3D',' read -r -a cgrps <<< "${test_cgroups}" + + for cgrp in "${cgrps[@]}"; do + # Find metric lines for this cgroup + # We use exact cgroup match with surrounding commas + local cgrp_lines + cgrp_lines=3D$(echo "${output}" | grep -F ",${cgrp}," | grep "insn_per_c= ycle" || true) + + if [ -z "${cgrp_lines}" ] + then + echo "FAIL: No metric lines found for cgroup '${cgrp}'" + exit 1 + fi + + # Parse each metric line + while read -r line; do + [ -z "${line}" ] && continue + + local val1 + val1=3D$(echo "${line}" | cut -d, -f1) + + local event_name + event_name=3D$(echo "${line}" | cut -d, -f3) + + local cycles_val=3D"" + local inst_val=3D"" + + if echo "${event_name}" | grep -q -i "cycles" + then + cycles_val=3D"${val1}" + # Find corresponding instructions event + local inst_event_name + inst_event_name=3D"${event_name/cpu-cycles/instructions}" + inst_event_name=3D"${inst_event_name/cycles/instructions}" + + local inst_line + inst_line=3D$(echo "${output}" | \ + grep -F ",${cgrp}," | \ + grep -F "${inst_event_name}" || true) + inst_val=3D$(echo "${inst_line}" | cut -d, -f1) + elif echo "${event_name}" | grep -q -i "instructions" + then + inst_val=3D"${val1}" + # Find corresponding cycles event (try cpu-cycles + # first, then cycles) + local cycles_event_name + cycles_event_name=3D"${event_name/instructions/cpu-cycles}" + local cycles_line + cycles_line=3D$(echo "${output}" | \ + grep -F ",${cgrp}," | \ + grep -F "${cycles_event_name}" || true) + + if [ -z "${cycles_line}" ] + then + # Try "cycles" instead of "cpu-cycles" + cycles_event_name=3D"${event_name/instructions/cycles}" + cycles_line=3D$(echo "${output}" | \ + grep -F ",${cgrp}," | \ + grep -F "${cycles_event_name}" || true) + fi + cycles_val=3D$(echo "${cycles_line}" | cut -d, -f1) + fi + + log_verbose "Cgroup '${cgrp}': event '${event_name}' \ +val '${cycles_val}', inst val '${inst_val}'" + + # Only enforce metric check if both cycles and + # instructions have non-zero numeric counts + if is_numeric_and_non_zero "${cycles_val}" && \ + is_numeric_and_non_zero "${inst_val}" + then + log_verbose "Enforcing metric check for cgroup '${cgrp}' \ +event '${event_name}'" + # Check for nan or nested in the metric value (7th field) + # Actually we can just check the whole line for simplicity + if echo "${line}" | grep -q -i -E ",nan,|,nested," + then + echo "FAIL: Invalid metric value (nan/nested) \ +for cgroup '${cgrp}'" + echo "Line: ${line}" + exit 1 + fi + # Check for empty metric value (2 consecutive + # commas before the unit) + if echo "${line}" | grep -q -E ",,[[:space:]]*[^,]*insn_per_cycle" + then + echo "FAIL: Empty metric value for cgroup '${cgrp}'" + echo "Line: ${line}" + exit 1 + fi + else + log_verbose "Skipping metric check for cgroup '${cgrp}' \ +event '${event_name}' (idle or not counted)" + fi + done <<< "${cgrp_lines}" + done + log_verbose "check_metric_reported passed for opts '${opts}'" +} + +check_system_wide +find_cgroups + +# Test 1: Without BPF counters +check_metric_reported "" + +# Test 2: With BPF counters (if supported) +log_verbose "Checking BPF support..." +if perf stat -a --bpf-counters --for-each-cgroup / true > /dev/null 2>&1 +then + log_verbose "BPF supported, running Test 2..." + check_metric_reported "--bpf-counters" +else + log_verbose "BPF not supported, skipping Test 2" +fi + +exit 0 --=20 2.54.0.631.ge1b05301d1-goog