From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6CFC1CD0C; Thu, 1 May 2025 02:28:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066496; cv=none; b=lKwLTdP8kH4losHoFgI9n23DvPpDPNcBj6+tATykRVHk7D9XrRf08vWuGamyGaPPrxnHN2CBaf5ZzzGnL/kxJrWEE0U2Fr9CTtop1R3gU1bCNmDf0WrufiR531sLGllNCGPCbzkEAS+nzM3fx2rWLSswhVg4uSlCwG1hKoIXqQE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066496; c=relaxed/simple; bh=WeNQ/P1lo7UWLgx5V8UqqLjs6z1+EhkpAyKNoReZE6E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KerTW2+zgtjP2bLwaBryHHlDZaJBiUoo1C2tmMO0A+FPPdP81zr8Q3yhKsTQVHh4UtvfaBfn85w+syIorky5a8kJ6jmVd0mWrMykIm7PFCHRtAFtwv3yNqu/cTBw6fm1r61VapD745rNJgl3zJVIPybn7LPPLjTQI94wqYyob1s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=R8YqzTC0; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="R8YqzTC0" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-739b3fe7ce8so569579b3a.0; Wed, 30 Apr 2025 19:28:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066494; x=1746671294; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=12hNC2maesUW+julZVveceim9osLUhYsROQkLQc04oc=; b=R8YqzTC02oysUKbpALiol2XKmBhIOpV5zRMS8Nui2N9ErUilYQf1sMr8r3gBF4NrPp 4GapoIVKZr+sb1OJVVXqjkpxLtJ0zj8D3YU4t3rds3JcZ5XnVj7brfPUSwGjzoHAzgXK 9xsllY35Py69Lr4yYo4UFkvXy0h+V32oV01CGxJS+dhcPFqJkXTtHplpJMAeLP3H6TC+ zjeiYbddsrfKhnjFHe0+WLLe7Hbe91WCnrPyofIpy90xZSz4ArtPiUfHHnyGZ7IJOO95 rDDnCvshzsyHQ5h2ezlTGa3zuTcuMsOQtdGnov9sF4yPzfCGxDKC7AKt3Z9RCH88ooV9 P+mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066494; x=1746671294; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=12hNC2maesUW+julZVveceim9osLUhYsROQkLQc04oc=; b=uDgiLZOYXFgj3el3cYl6Dd3sntU/AVf0qzQrrGbmRgpVP6TJpHMXztQDn2nTATuHtb 9b2fSqQZIfz34wimUsDubjp/sm6GAvRHp9U2ElrRfMbrX0vgH30r6SwAW2/hYxGkrO3+ A+dyXXiCHzT7tLTeWvRf58RxVIpSVF5slkvG/qzc7Azileg6fa7OnHTGCF4KN26JSUWm 4ZJTOLQj0EdUyv82xIJFdoYnl6pSfonlPIk5DA1orflybzBG1+yifICp41VVzHo8l4jO IHRvLlubAMm9rjyYJeavCI77fg+l9FEM3bOBOnHSwSGvNKMKrtTiMi/gB1RF44dBORBL Z4lA== X-Forwarded-Encrypted: i=1; AJvYcCV7QxjwBbD8GvCEpOTh7osQRXbdLHlanslzjJBnbsYJll+ui+ZRhbsTY61ia5SBoNS7SfjgI3vDpmyoPks=@vger.kernel.org, AJvYcCVPHLKNTqXs/+K733ELWT/sunkBCCSzc8/FtYX1xwBCV5UHsjT5Wq8PIXEYTGBt74h8CZ7GhR7FLyq6fRVFzIkl0w==@vger.kernel.org X-Gm-Message-State: AOJu0Yy8HRZcx8Pxg0HFDBHCIsbprZ0WelVvECf9Hfrf83Gk3sxOfhW8 UZAEYXseUQ12AF91lmbaIhIntETuMHfKMsfaGh5MkR45Iwwjccsh X-Gm-Gg: ASbGncufRq/pgQ8k0SZGPhgMf6Bx1uANIG5q6uSufPi64ceMt7uK+iPXZY2BTwgJBO8 QMy07PfTykkqsIUtICfS1PPvjeKZ40s1V+V9haR3yYXWUmrvd62YkWWCbbbbW3EYIb20rVLXL1o dJVILKnJ/UxGQvbWtc1Wx5nlL3XghsI1Jk9gU/8AY2ZLjDijZ9fVD8141b4eVMMTfcZm/tHSzR2 S4nD/GNG/5HVqQ2HqEfzzjDsEfFvT98F6T+f2w2CAx8HKPH2Uu7hJL0bb+FIne+kh7U4VQPA3u4 FHWOfs9jhEmyvzmMdoMIIxravNLskKJ93+24f/yC1JeD+oFh8Cy7bwepTFS0qKRPewrz+biil7M AlklKlMH2Gg== X-Google-Smtp-Source: AGHT+IGi7N5e90qJn3IHMOD45SIBkEJCakMr53jcFWv6tBFJxDpiqJFsA1JKY1ksJ4eRMmmXznH/Qw== X-Received: by 2002:aa7:888c:0:b0:73d:ff02:8d83 with SMTP id d2e1a72fcca58-74038959430mr8855574b3a.3.1746066493942; Wed, 30 Apr 2025 19:28:13 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:13 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu , James Clark , Arnaldo Carvalho de Melo Subject: [PATCH v16 01/10] perf evsel: Expose evsel__is_offcpu_event() for future use Date: Wed, 30 Apr 2025 19:27:59 -0700 Message-ID: <20250501022809.449767-2-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose evsel__is_offcpu_event() so it can be used in off_cpu_config(), evsel__parse_sample(), and perf script. Tested-by: Gautam Menghani Acked-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-3-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Ian Rogers --- tools/perf/util/evsel.c | 2 +- tools/perf/util/evsel.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index 1d79ffecd41f..2559943d199e 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -1275,7 +1275,7 @@ static void evsel__set_default_freq_period(struct rec= ord_opts *opts, } } =20 -static bool evsel__is_offcpu_event(struct evsel *evsel) +bool evsel__is_offcpu_event(struct evsel *evsel) { return evsel__is_bpf_output(evsel) && evsel__name_is(evsel, OFFCPU_EVENT); } diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h index 42dcadfef8ce..3d47d9318d92 100644 --- a/tools/perf/util/evsel.h +++ b/tools/perf/util/evsel.h @@ -563,4 +563,6 @@ u64 evsel__bitfield_swap_branch_flags(u64 value); void evsel__set_config_if_unset(struct perf_pmu *pmu, struct evsel *evsel, const char *config_name, u64 val); =20 +bool evsel__is_offcpu_event(struct evsel *evsel); + #endif /* __PERF_EVSEL_H */ --=20 2.45.2 From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B7A2230BCE; Thu, 1 May 2025 02:28:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066500; cv=none; b=bFrEZFF0u3YOSwnf5s3n6CfJWac/jnU5oaQjN/gsa/V4tIJJTejD5D5tJpQpORAkMkSGJtM/G6an2xlGVdsgcYW4rsp3WTFQsB0WWVR6P+5krq5I5IwVKvobP+//xMbN3F25a3QDmC/qpDxEG4EAgkqXUBojmQGkdh7K+5SCuhk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066500; c=relaxed/simple; bh=7BWU0fhhXrKCxr5Gwg8jIyFJNzq/a4Ofz3xmf40pdoc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sa1QK8/xm3Em2PUw0BAwssFby30g14F2SJWmH5uMKwzZXdY4niwuSgWruHtfRWThvmj2yo66/VWBNZMc0Q1Wp2Y8fvDoNC9QQ22bQAW3UW6uInC+8rcLpFwxte3wEKV9MhseC2NOi++rzh1NMTfYkzrqEKhpEYBQQQdG3YauC8o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=clXZpFaF; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="clXZpFaF" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-736b350a22cso455579b3a.1; Wed, 30 Apr 2025 19:28:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066496; x=1746671296; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G3DkBn0xSoja9kYhz3QjHqlls2QdLj28ENeq+GUQ7u0=; b=clXZpFaFK8UcKiKvh4SipwUliv4snQrtd5wcbb0U3wLELeJm/66SxEpx9zHqRypOtb aK9fbchfx5587aTG8cRId1QyLhrdSQBUnb8m7V2yRxXY5u/MSR2qhZ3VDWShGGSr1fme ypErSLk+RoBSmrHfhzMA1PLvQpj12oVNm6QcqZOtxRXUOuCt77Oht/rL+/VoVX91Yx43 37ZW5TB4ycry/RanM2AYt5yZqwMCGk2nGWKKCwpXL7FLABnQ/gZS8YIiFYhXggYtEqbZ zml5tepVAODX77xQA8vDaxb9yBHPGr1u8kLctb7cWsB32NISpgBiO501JOub5yIICYHv f+Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066496; x=1746671296; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G3DkBn0xSoja9kYhz3QjHqlls2QdLj28ENeq+GUQ7u0=; b=X/4vkHeaw/RQ769teDfvxX75wfqNB5QNQvz9Or4uAiPC1PiQHzzP0CGwOHfn+l93Sl i7ziGWTYwykWpAgDQzBRt8ldtnXQ640wb/cXei4VAxYM2q+ft6kbvhXFucCwniuMcqD3 Oz4gjgIP4jAuaXJAt/3+2UwTQIg/x0IcsaSL7j9I7rQbg9QLxswGNYR4Csb7WwhTUizI Xku0Ol9bDzBjICxF0gU2cUtfBYmEOXMUMac07dgMjXtIsubpiXzI6rAByLvcr5k5uqpU pTSyRbkRtn2jGPYY+stlzOS9Nupgc0ISy0q9R8i/BlwahoGp6LATmBqu4Y0nJf1eFA3n Z5rA== X-Forwarded-Encrypted: i=1; AJvYcCU0fx8kwbhmeGL4dgNieBo9QPHkwhLuzjKF7hKKmlcnbG8N4VU94Rge0NC8+KfyJ/JKB9gYEkYfK0yeMDiC+KStxg==@vger.kernel.org, AJvYcCUBBvyC4vO6jZL/uYuZ+tH1uz6bUYm6tLchJVNCLxYaqKQG+ek5E78WeMUlEhbmfevqzexukerr8WKX8hk=@vger.kernel.org X-Gm-Message-State: AOJu0YwW7bUoMW25Hup34I8Ld+s4Z4UoOJOw0/MImXfStRb/O5Z2+7qs VxUF2TDmJIUlt+Lx3oj03P65SLxYj3aOIju6HVe3Yc1yCYNqN7lL X-Gm-Gg: ASbGncviVsimyHexSbqoUyyc1WguF/vg2HjvaKLH+ZOx6xiTf47ZF8dwZh+5GcPacii 0y/KlZxOPZCJoowXt6mcA3xPbFv49/BwMz1zdG0RbdYIGJkoAKnx94zCvQKLBKhv1oUiMus/G1C Ag2P6op4pZDjSLFGQhAM0ZJeAO0nfRV8zR6g2P7ETYHk5Oww3pH3Ehys97WrIQld8DWDK0e0gjy Q4Erzt99y4TXRr/a9z1QjzbnJZ6qvAGDIwtyOjbBFr4nvosxN03WEJN1lCI8xAlLdVYJ10cLPyI yBOX0eioADNezymgD1IvOBr1rKZXryPxSWkU9k/MX2/Fj3ClimlAcNvvrQxCY7GTdfLShItt65C 4DP1+ZaIcjZQlAWlM7CEM X-Google-Smtp-Source: AGHT+IFfrssPHRMX/Aghe6lNX8WIG5prSK3wvERdQwFt9TNE4n5M6amx3AbdEcEJpfXUZXKzmqXipg== X-Received: by 2002:a05:6a21:3414:b0:1ee:a914:1d64 with SMTP id adf61e73a8af0-20bd82525cfmr1151519637.28.1746066495751; Wed, 30 Apr 2025 19:28:15 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:15 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu , James Clark , Arnaldo Carvalho de Melo Subject: [PATCH v16 02/10] perf record --off-cpu: Parse off-cpu event Date: Wed, 30 Apr 2025 19:28:00 -0700 Message-ID: <20250501022809.449767-3-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Parse the off-cpu event using parse_event(), as bpf-output. Call evlist__enable_evsel() on off-cpu event. This fixes the inability to collect direct off-cpu samples on a workload, as reported by Arnaldo Carvalho de Melo . The reason being, workload sets enable_on_exec instead of calling evlist__enable(), but off-cpu event does not attach to an executable and execve won't be called, so the fds from perf_event_open() are not enabled. no-inherit should be set to 1, here's the reason: We update the BPF perf_event map for direct off-cpu sample dumping (in following patches), it executes as follows: bpf_map_update_value() bpf_fd_array_map_update_elem() perf_event_fd_array_get_ptr() perf_event_read_local() In perf_event_read_local(), there is: int perf_event_read_local(struct perf_event *event, u64 *value, u64 *enabled, u64 *running) { ... /* * It must not be an event with inherit set, we cannot read * all child counters from atomic context. */ if (event->attr.inherit) { ret =3D -EOPNOTSUPP; goto out; } Which means no-inherit has to be true for updating the BPF perf_event map. Moreover, for bpf-output events, we primarily want a system-wide event instead of a per-task event. The reason is that in BPF's bpf_perf_event_output(), BPF uses the CPU index to retrieve the perf_event file descriptor it outputs to. Making a bpf-output event system-wide naturally satisfies this requirement by mapping CPU appropriately. Tested-by: Gautam Menghani Acked-by: Namhyung Kim Suggested-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-4-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Ian Rogers --- tools/perf/builtin-record.c | 7 +++++++ tools/perf/util/bpf_off_cpu.c | 33 +++++++++++---------------------- tools/perf/util/evsel.c | 4 +++- 3 files changed, 21 insertions(+), 23 deletions(-) diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index ba20bf7c011d..da28c52af464 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -2568,6 +2568,13 @@ static int __cmd_record(struct record *rec, int argc= , const char **argv) if (!target__none(&opts->target) && !opts->target.initial_delay) evlist__enable(rec->evlist); =20 + /* + * offcpu-time does not call execve, so enable_on_exe wouldn't work + * when recording a workload, do it manually + */ + if (rec->off_cpu) + evlist__enable_evsel(rec->evlist, (char *)OFFCPU_EVENT); + /* * Let the child rip */ diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index 4269b41d1771..2101aa2b7c42 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -38,32 +38,21 @@ union off_cpu_data { =20 static int off_cpu_config(struct evlist *evlist) { + char off_cpu_event[64]; struct evsel *evsel; - struct perf_event_attr attr =3D { - .type =3D PERF_TYPE_SOFTWARE, - .config =3D PERF_COUNT_SW_BPF_OUTPUT, - .size =3D sizeof(attr), /* to capture ABI version */ - }; - char *evname =3D strdup(OFFCPU_EVENT); - - if (evname =3D=3D NULL) - return -ENOMEM; =20 - evsel =3D evsel__new(&attr); - if (!evsel) { - free(evname); - return -ENOMEM; + scnprintf(off_cpu_event, sizeof(off_cpu_event), "bpf-output/name=3D%s/", = OFFCPU_EVENT); + if (parse_event(evlist, off_cpu_event)) { + pr_err("Failed to open off-cpu event\n"); + return -1; } =20 - evsel->core.attr.freq =3D 1; - evsel->core.attr.sample_period =3D 1; - /* off-cpu analysis depends on stack trace */ - evsel->core.attr.sample_type =3D PERF_SAMPLE_CALLCHAIN; - - evlist__add(evlist, evsel); - - free(evsel->name); - evsel->name =3D evname; + evlist__for_each_entry(evlist, evsel) { + if (evsel__is_offcpu_event(evsel)) { + evsel->core.system_wide =3D true; + break; + } + } =20 return 0; } diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index 2559943d199e..ea6b1ff16e6d 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -1555,8 +1555,10 @@ void evsel__config(struct evsel *evsel, struct recor= d_opts *opts, if (evsel__is_dummy_event(evsel)) evsel__reset_sample_bit(evsel, BRANCH_STACK); =20 - if (evsel__is_offcpu_event(evsel)) + if (evsel__is_offcpu_event(evsel)) { evsel->core.attr.sample_type &=3D OFFCPU_SAMPLE_TYPES; + attr->inherit =3D 0; + } =20 arch__post_evsel_config(evsel, attr); } --=20 2.45.2 From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FF0E230BE9; Thu, 1 May 2025 02:28:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066500; cv=none; b=bhpcR+6JktkfGAQb2IZ2m2Vk03LxUqCNzxCr/vD/S2rLi1miJULmACoe7hzjAjNIrbj/5VaJGQdPXpqI9Ru2EES0V2+RVB2pfKjnfr371nwcLBcpyUh+20oafkSXwZMRCzrTDwaPog2Vq4WyiRrq5Q7Y74Eks0mB2cF3RTRP2mY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066500; c=relaxed/simple; bh=VFBcBi6kgTg4RgBDcmOg2i1bLJ9QJGSKr0nnYNEwcMI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fMqd7rioZ8SRH+2n+YWSBW55RrB2sEKoSti6ZXJbDw9GE/Wiq9YUneonoxbOfv0NuYcx+17KQ8GQzQ5e4EeTlNRlg/yHXi2OK0zc/ffNqsdhSrKhjcNWp9h88fVhJUbSpV7+wWUs0CGdIO02S07TwuNlXACRUx+mzX7ZkWgTMFc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=M3lvAy3G; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="M3lvAy3G" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-74019695377so462998b3a.3; Wed, 30 Apr 2025 19:28:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066497; x=1746671297; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uIdt8T44pUg9ZI5PqJY71nYgZCT1z4VuC2192Gm2TOU=; b=M3lvAy3GDMj9nCPF99O1+pZQPPNFpBQ7gQOSDK7Hr/14gU4a1YRsiYSwavY0sppP4l TPvWdAMMtl9xYdgLwwiCiMk/yNhy50xAnOQJWJXiTVBffuFm2rdUGYfuSAKxPS0/w0Fb Mk5NgAkCgG8xk2EMBdU8Sx+SDAEjYoFKRJVX3+VQ7D76H8ZeAWAx1Cp34whe+sLg5qx2 pEHSmqrYBtY1L9I2hI+ViR536ScweUSpMo6alQ37Qi/eBi2PNk64v3pIZ3C99M+UTBQ3 J2u2QrL8cKJgRRZxN9sElgSPj9EAo5onZjdvHmgoWCovdCZ5nXu+pL89Fm1RhUxB0kRn Tf9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066497; x=1746671297; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uIdt8T44pUg9ZI5PqJY71nYgZCT1z4VuC2192Gm2TOU=; b=DEpngoVBIMRJebPYVepOuTv7+fi4QGCuBaPOfesNbm1m+cmImfKx3Oq9pecLu6Olit WtEIccHcAHW6I5I84+Qyk9YfQfwYPBJXn6AKyUKYd7b4CxeY74gXolaxkM4hfMcTL4dE IY9EJrFIpF5aj1Zr0VYsqoAB3eFiDBnp0DcQ8et054H5KqhyZ1/XM9NmdwAmZ66PiWSp m7DcTPeFDBkfZyZvBmllLbhMpZ55G07ql0u2t6R3Uv9tPvjxczM12WjWm2CNijLJ5LUs Vc2BeF6eFAvd2wB22wi6Ds8HIhIzRBWpbRpqoUkhfB/8oo8HhCjcCkPk/rpH83RqMeXX DyNw== X-Forwarded-Encrypted: i=1; AJvYcCUHVi2fIffNNGt92SKYKGjGsucAk8o0ZOe6Q2DwZemXrfGg+fSit/461vI5FRqOHd6BIqku4JWB7U6pUug=@vger.kernel.org, AJvYcCWbSuLps8YWY56WlbF+FJPLuPGP3SZV4Wf75eFP6tb1fzK/ee2UAKqQvH3lUoXVDcYoJHTHaTimpKl+ry1DstMn8Q==@vger.kernel.org X-Gm-Message-State: AOJu0YxMhEii/ARj4IDWURoshmZUzZJhr1mpmlieo3fLQ2K5iZB+1y3w /sGH4p0aGJmBFt92jLSlUzAcBqfXPvuheTQCR5YA+PPO6VK3izeA X-Gm-Gg: ASbGncvoDG8Zz5UbokNCJUnhyIADjxmRNtaOGO07H2m+9sPSf9DT4oMR5vG2ejGrmrR cHuHaKBRhVgGYQkSAVINOjAhzbPbVKx0Fvs3C63FgeFTVmRZcMP8Oonfa0s6aUqUBwVmOXGVyMj zod3+mDuEiH2094og84FUGyB92ao+yh5vlrVFxc0ntPndS3A4ABJ73RqPa9PnvKoHc+HB7HofbT 5dK6djTDfoNsM87B/3hOdPT0ykicAPNttPwCW6APz4LPt8O5UjBVIF2Q/qwmpPc0flyOORJ0wC+ zSHnrBn+a6BqDGPIVLXwxaKu5ZmMnw5M2Zc2kovEYpOWXYL6OMKwhlZlW64dxBF3RqYAuWPShrC noYZDIavkljsooqshZw/y X-Google-Smtp-Source: AGHT+IEUB68AE0so/EbVx6QyFjubOz1tQGHLljnIt6qiiMZG1XrfHKq8vApAf57wCaVvn4EaqTWUeA== X-Received: by 2002:a05:6a00:1383:b0:739:3f55:b23f with SMTP id d2e1a72fcca58-740491f63aemr1145658b3a.14.1746066497366; Wed, 30 Apr 2025 19:28:17 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:17 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu , James Clark , Arnaldo Carvalho de Melo Subject: [PATCH v16 03/10] perf record --off-cpu: Preparation of off-cpu BPF program Date: Wed, 30 Apr 2025 19:28:01 -0700 Message-ID: <20250501022809.449767-4-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set the perf_event map in BPF for dumping off-cpu samples, and set the offcpu_thresh to specify the threshold. Tested-by: Gautam Menghani Acked-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-5-howardchu95@gmail.= com [ Added some missing iteration variables to off_cpu_config() and fixed up a manually edited patch hunk line boundary line ] Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Ian Rogers --- tools/perf/util/bpf_off_cpu.c | 24 ++++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 9 +++++++++ 2 files changed, 33 insertions(+) diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index 2101aa2b7c42..1975a02c1683 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -13,6 +13,7 @@ #include "util/cgroup.h" #include "util/strlist.h" #include +#include =20 #include "bpf_skel/off_cpu.skel.h" =20 @@ -60,6 +61,9 @@ static int off_cpu_config(struct evlist *evlist) static void off_cpu_start(void *arg) { struct evlist *evlist =3D arg; + struct evsel *evsel; + struct perf_cpu pcpu; + int i; =20 /* update task filter for the given workload */ if (skel->rodata->has_task && skel->rodata->uses_tgid && @@ -73,6 +77,26 @@ static void off_cpu_start(void *arg) bpf_map_update_elem(fd, &pid, &val, BPF_ANY); } =20 + /* update BPF perf_event map */ + evsel =3D evlist__find_evsel_by_str(evlist, OFFCPU_EVENT); + if (evsel =3D=3D NULL) { + pr_err("%s evsel not found\n", OFFCPU_EVENT); + return; + } + + perf_cpu_map__for_each_cpu(pcpu, i, evsel->core.cpus) { + int err; + int cpu_nr =3D pcpu.cpu; + + err =3D bpf_map__update_elem(skel->maps.offcpu_output, &cpu_nr, sizeof(i= nt), + xyarray__entry(evsel->core.fd, cpu_nr, 0), + sizeof(int), BPF_ANY); + if (err) { + pr_err("Failed to update perf event map for direct off-cpu dumping\n"); + return; + } + } + skel->bss->enabled =3D 1; } =20 diff --git a/tools/perf/util/bpf_skel/off_cpu.bpf.c b/tools/perf/util/bpf_s= kel/off_cpu.bpf.c index c152116df72f..a5002e25b259 100644 --- a/tools/perf/util/bpf_skel/off_cpu.bpf.c +++ b/tools/perf/util/bpf_skel/off_cpu.bpf.c @@ -18,6 +18,8 @@ #define MAX_STACKS 32 #define MAX_ENTRIES 102400 =20 +#define MAX_CPUS 4096 + struct tstamp_data { __u32 stack_id; __u32 state; @@ -39,6 +41,13 @@ struct { __uint(max_entries, MAX_ENTRIES); } stacks SEC(".maps"); =20 +struct { + __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); + __uint(key_size, sizeof(int)); + __uint(value_size, sizeof(int)); + __uint(max_entries, MAX_CPUS); +} offcpu_output SEC(".maps"); + struct { __uint(type, BPF_MAP_TYPE_TASK_STORAGE); __uint(map_flags, BPF_F_NO_PREALLOC); --=20 2.45.2 From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD3011CD0C; Thu, 1 May 2025 02:28:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066501; cv=none; b=Yb+xlN2LOKBTWERV1ZFQBbdJauwG5bvtdZ5ky2c5+OyEQzFg2fvTDkn6oRL+JsHiWBF6HU8GjHLo/M8JH/eM+++UG8DZmgeS1W+7LmoEqVlL+h3Dbjk7Lx42d6t71UKN9eprVYf4oWCGDKdZLiW0usHRjaOpZV8BU6WGl7wwbBE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066501; c=relaxed/simple; bh=maCxAVlgYLrPtTMDiRXoouVhL1n+mDRVFX081dzkkkI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XGOeB13FDzI1iTEkeMLobXJUzpXbxo7UwqmfIo9ape1oCo933UvQU0c1qj8eBcN1rTbdWSv3T4Z9foiMQqFindzAU6M5e7ta9MkLP38ovtqz7WbPQrkuJ0DXy48WtLR/7RF7RzlyqZIy1crS4nHKJ+V5ot65O1xiAVx86NX84lc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=krAmhMUq; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="krAmhMUq" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-7370a2d1981so447755b3a.2; Wed, 30 Apr 2025 19:28:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066499; x=1746671299; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=99+pPmvSZ94HNiM6U5VflkxaawFrODtXZOw1+c9x/Ww=; b=krAmhMUqkXJT9Gy1AXN1VVkaOfM62sDv5Efu09nwcGHURwsZBmAd+kmC2kbFNIX69L b5eYH0UaIth4MSKcN75uhJ+1boV5QyOxyqS2CHJSUqKTSH/kdUgDfgTrKqcYTvParfoy Hh950MK5opcNLxhQrMXs8daFo4E6o3dpPAK4vH2LgiNQ3NgjDykTM3secrvsNgOx0fpD 6dqNjqSnhc0PRIyghcjn5lFd9U1xr8IMmp3U0NHbLoxbuQbOh7CcCnemvzv8n20vPYHW eakFobxjKYy6Q7zmbrjOdhZa/PPyR6r1NEr9r53+qsHxUejpPmWdaIgsSs/WWQ0P7HtP SaPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066499; x=1746671299; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=99+pPmvSZ94HNiM6U5VflkxaawFrODtXZOw1+c9x/Ww=; b=jDIZrfPssBgmTKDkmaNdXSdugml/6nRkSix3VZFtmdLseV8zxIq0wu1y/7h9pMYwPg ob68bAho9d+rOh/mkgrIGbxwU5UZMairNclQ81QHvp/qKgG8o6c9a4Awa7h/Nb0l0GbM kO9wnB0Vp3829KtWWauggQBjDbPGQuxt4ROs7GjY4aXwbBRXnEdLRz5iZi9cxOynigUQ kKnvucJkGPlmLb11Ub/+PTloRWKjkv839Dqnzht5OTXG3tqLwPykeWQjGIm/1UQ7nrFl 8Hr/SRToYEkt2AIVoWP3H0vkCTTrB3jNXnRYxti38WlQyknKPrmQzWGPUzdA50CeWOBr 0nSQ== X-Forwarded-Encrypted: i=1; AJvYcCUxS8qL3tSLkLrrSg1jJBNSRxT2aG2zyqJ7PYIr7HavJFnRPZoHxOdfEdTHWSUCfKQeXpTekSyN53EDw/QKNeppJg==@vger.kernel.org, AJvYcCVqF70xDhy2GosdHOqg2cT1yO8tbqW2CPeiblT5XwjjziyG4JBB89UbkMxTrY9CyXG9S9CvjgqCuz9Vl6E=@vger.kernel.org X-Gm-Message-State: AOJu0YyzrrEwr79uSkgMKDmmn4kg3eDLkC1imeM+i4FAzWDAyZ1YdKly sSQXAUD845po6wXlphRsCjrCbtMAQe6BwnTx1ifknHbe26ehh1PI X-Gm-Gg: ASbGncsroS8Z9oT8ChV3blBp0tCBl8H4/pam2PFNczuszVAif/vZ9aEIOeKwVfjQavU KAQcWA3AvQN/wBtV0YKyhfvd9GAAxVboOQjCUu2Q4Gv41M8W6YNqoqyW+TgY+is6x20HseaaNuL BTUHItmtzYMkAVPlvbb6PPEYGoiy8+E1urNEv3w98j1wNNKAgp8hRQkHylQ8OIrccrbUxIeofor J35Smvy5uvhi5t2o94b7jaxB+/YxfGC8aBpvnFOiMLELl3RkUJETeRR/6mr20+vcHMT1zNcYY3U tJAxFKbnmSunb3rrpSdMztntzopSBNY3I8HUSctrVO8C6FMEQ23t2bMnGyN5lklejdlEf/m1jgz +EU1+a1q//g== X-Google-Smtp-Source: AGHT+IFOC1GJ6lS+g+xIUeme5KfR/5i6w/pzEEXr4APwC7p/sKfCc1ON5b3O9tGlMZoutDbvIhXSZA== X-Received: by 2002:a05:6a00:3006:b0:740:34cd:498f with SMTP id d2e1a72fcca58-74047764a8bmr1695756b3a.5.1746066499217; Wed, 30 Apr 2025 19:28:19 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:18 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu , James Clark , Arnaldo Carvalho de Melo Subject: [PATCH v16 04/10] perf record --off-cpu: Dump off-cpu samples in BPF Date: Wed, 30 Apr 2025 19:28:02 -0700 Message-ID: <20250501022809.449767-5-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Collect tid, period, callchain, and cgroup id and dump them when off-cpu time threshold is reached. We don't collect the off-cpu time twice (the delta), it's either in direct samples, or accumulated samples that are dumped at the end of perf.data. Tested-by: Gautam Menghani Acked-by: Namhyung Kim Reviewed-by: Ian Rogers Suggested-by: Namhyung Kim Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-6-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Ian Rogers --- tools/perf/util/bpf_skel/off_cpu.bpf.c | 89 ++++++++++++++++++++++++-- 1 file changed, 84 insertions(+), 5 deletions(-) diff --git a/tools/perf/util/bpf_skel/off_cpu.bpf.c b/tools/perf/util/bpf_s= kel/off_cpu.bpf.c index a5002e25b259..14cd8881f8bb 100644 --- a/tools/perf/util/bpf_skel/off_cpu.bpf.c +++ b/tools/perf/util/bpf_skel/off_cpu.bpf.c @@ -19,11 +19,18 @@ #define MAX_ENTRIES 102400 =20 #define MAX_CPUS 4096 +#define MAX_OFFCPU_LEN 37 + +// We have a 'struct stack' in vmlinux.h when building with GEN_VMLINUX_H= =3D1 +struct __stack { + u64 array[MAX_STACKS]; +}; =20 struct tstamp_data { __u32 stack_id; __u32 state; __u64 timestamp; + struct __stack stack; }; =20 struct offcpu_key { @@ -41,6 +48,10 @@ struct { __uint(max_entries, MAX_ENTRIES); } stacks SEC(".maps"); =20 +struct offcpu_data { + u64 array[MAX_OFFCPU_LEN]; +}; + struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); __uint(key_size, sizeof(int)); @@ -48,6 +59,13 @@ struct { __uint(max_entries, MAX_CPUS); } offcpu_output SEC(".maps"); =20 +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(struct offcpu_data)); + __uint(max_entries, 1); +} offcpu_payload SEC(".maps"); + struct { __uint(type, BPF_MAP_TYPE_TASK_STORAGE); __uint(map_flags, BPF_F_NO_PREALLOC); @@ -106,6 +124,8 @@ const volatile bool uses_cgroup_v1 =3D false; =20 int perf_subsys_id =3D -1; =20 +__u64 offcpu_thresh_ns =3D 500000000ull; + /* * Old kernel used to call it task_struct->state and now it's '__state'. * Use BPF CO-RE "ignored suffix rule" to deal with it like below: @@ -192,6 +212,47 @@ static inline int can_record(struct task_struct *t, in= t state) return 1; } =20 +static inline int copy_stack(struct __stack *from, struct offcpu_data *to,= int n) +{ + int len =3D 0; + + for (int i =3D 0; i < MAX_STACKS && from->array[i]; ++i, ++len) + to->array[n + 2 + i] =3D from->array[i]; + + return len; +} + +/** + * off_cpu_dump - dump off-cpu samples to ring buffer + * @data: payload for dumping off-cpu samples + * @key: off-cpu data + * @stack: stack trace of the task before being scheduled out + * + * If the threshold of off-cpu time is reached, acquire tid, period, callc= hain, and cgroup id + * information of the task, and dump it as a raw sample to perf ring buffer + */ +static int off_cpu_dump(void *ctx, struct offcpu_data *data, struct offcpu= _key *key, + struct __stack *stack, __u64 delta) +{ + int n =3D 0, len =3D 0; + + data->array[n++] =3D (u64)key->tgid << 32 | key->pid; + data->array[n++] =3D delta; + + /* data->array[n] is callchain->nr (updated later) */ + data->array[n + 1] =3D PERF_CONTEXT_USER; + data->array[n + 2] =3D 0; + len =3D copy_stack(stack, data, n); + + /* update length of callchain */ + data->array[n] =3D len + 1; + n +=3D len + 2; + + data->array[n++] =3D key->cgroup_id; + + return bpf_perf_event_output(ctx, &offcpu_output, BPF_F_CURRENT_CPU, data= , n * sizeof(u64)); +} + static int off_cpu_stat(u64 *ctx, struct task_struct *prev, struct task_struct *next, int state) { @@ -216,6 +277,16 @@ static int off_cpu_stat(u64 *ctx, struct task_struct *= prev, pelem->state =3D state; pelem->stack_id =3D stack_id; =20 + /* + * If stacks are successfully collected by bpf_get_stackid(), collect the= m once more + * in task_storage for direct off-cpu sample dumping + */ + if (stack_id > 0 && bpf_get_stack(ctx, &pelem->stack, MAX_STACKS * sizeof= (u64), BPF_F_USER_STACK)) { + /* + * This empty if block is used to avoid 'result unused warning' from bpf= _get_stack(). + * If the collection fails, continue with the logic for the next task. + */ + } next: pelem =3D bpf_task_storage_get(&tstamp, next, NULL, 0); =20 @@ -230,11 +301,19 @@ static int off_cpu_stat(u64 *ctx, struct task_struct = *prev, __u64 delta =3D ts - pelem->timestamp; __u64 *total; =20 - total =3D bpf_map_lookup_elem(&off_cpu, &key); - if (total) - *total +=3D delta; - else - bpf_map_update_elem(&off_cpu, &key, &delta, BPF_ANY); + if (delta >=3D offcpu_thresh_ns) { + int zero =3D 0; + struct offcpu_data *data =3D bpf_map_lookup_elem(&offcpu_payload, &zero= ); + + if (data) + off_cpu_dump(ctx, data, &key, &pelem->stack, delta); + } else { + total =3D bpf_map_lookup_elem(&off_cpu, &key); + if (total) + *total +=3D delta; + else + bpf_map_update_elem(&off_cpu, &key, &delta, BPF_ANY); + } =20 /* prevent to reuse the timestamp later */ pelem->timestamp =3D 0; --=20 2.45.2 From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FE8B231856; Thu, 1 May 2025 02:28:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066503; cv=none; b=cZQHk45zntApOyj/XZMqAeT7mfiDT5yAV93/0IQ81EQV+l1XUUr2H/TB10v/kI3sq+STarCxvEVJwuJ4iA74p2ymAsfcsMftWD3ZOCvB8hTXuqWj3Z7Zoapzc8ZCjPz+mNNo6nFjTTVMa/4MpPTViKbTirGGgIrhQxuJwMcXL9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066503; c=relaxed/simple; bh=DOtmYZ08nWeqolCBw1DBGPoZGcMaiKoo2RsbESyGlPU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QA5cwb/pFvQpMH40rXFAiE7VhB0fD40pyNbCjHxMGJUMf8eCJ6OMn8Og2ofu0uNAxbRqnjTHrE/NBFglVdUZT/TGfLEths7N4PkhxDA1TP6KzkuPONRuAvkAmfEFe5kfK41fxjO04ICL66JqIONWDlHJ1j8pAZe0r5vM+2lDQ2Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=K38yyngw; arc=none smtp.client-ip=209.85.210.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="K38yyngw" Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-736aa9d0f2aso804069b3a.0; Wed, 30 Apr 2025 19:28:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066501; x=1746671301; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BOLcMhRBa8687KTEG0Rm1QPK2LiDeZ9ivT+5m0+6uLw=; b=K38yyngwzTak/tfcWkWH26gvZ4vONWgE/6F+Y8zD+3n9KNDzcVozgHhydkqe6qN8yJ TAbV7XxUbEwaYGOsb7xgCO6jzZKgFrZ4iz9u+3TviehVSWrqwwM1g4EOVpAq7Kbgycgm 0+cPFKgXSd8xZpXA1fojkTfZ5slOSdPM15CaHxPdWgmZMFm49Xs1PH9n4An/7lPfxtLN m6c71QFRdJNGeceGr7lqNbbVRz+SLSYUNnpQyDlyWTvnnB+sybEzpPBdOavtrX3w9tnK QZwZaJ4sWrsuUq22Fj3o7kXzO6zOdOf82k6UrgkWf3AKowr7zu5cmei8hJZpBiZnwakI WuCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066501; x=1746671301; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BOLcMhRBa8687KTEG0Rm1QPK2LiDeZ9ivT+5m0+6uLw=; b=JSAVZMbpsOqkV+0Iai1shir7y2ZLj2SqTot5VAKuyQFdMBTKQbqqL6k3qhcLI2+grQ zlD5RJHdzrltcy6e0AD2EDMVdkMB74QxtEXOAr2LY6cQ1pYeWs/qG4ANRD9+Al0VroY1 jYdbDY/IIGcnE+KTPRsT3dSVZp7jZ13k51rBZgOXnBwllElALLFrKsIiw9lRSfGOsfQn lM6+c/gGXpfniTBgyZ7gJgWJrnK0ZR5M0q0fX0R3QBEp5HYJMgqY8aIxLKBnT/VVRt6w gl96mWDzgd503sqdIJvaGJOiO6DZUdVOU2BSeg1PsNsefEMMrTu35rRs3kcJe/TLP8ME avMA== X-Forwarded-Encrypted: i=1; AJvYcCVrtmjzXuBvnVd2cy5+XibiyPZieC97HFHAavK4CVzBW0NmRUYeprMQewYRsIw6rnO/TcTeyEU+x8nYcqg=@vger.kernel.org, AJvYcCXRoKl/Ckg3eysGNu7/xxe/NvvZN0O/sI1nL0zjq+f9I6kaIhrcJgBqNGp92vxZJthy+juGAYKRO7jkAvIMN3dllA==@vger.kernel.org X-Gm-Message-State: AOJu0Yzst7YhFTgNTMr5Y3Co4DMJozGNXDBd6+54M+lt5A70p62FEmn8 7kAx1RI2ri1N3r6pC7cseHex8+xxXfKtVAbAwLiFMHRselrojsHsLUeI8m7Ogq4= X-Gm-Gg: ASbGncvhQwQOUr+Est0jXunllwWMdSjKgCMttxLbxPrjat64HDGu2XzIwyTsRZI9z+a uTNUPnyg5DlpM3EOoScaoVseDouJRFNegj+2n2BjCrj2Ks0zuSM8z9PcWCWc1m8YaNrkE6EFshh mw8CCVYgjoIaDSci6QC1EEWdG53dC+KoFtNGKzkg5nPsAZ0O7zuIprl0WlW3lXwqNKuUnNw47pw ZcKAOuh96c4NXeUFlqkrJks3QQ4XnsLZ7C8ph8YTq2vHxy3Bck3KZzI5qTqEjxnYtrmA0JHXK5Y xKtYl3CcvIKCnQwyux4CLi7U2viwdyanIondbM+Q3WHPrUL9IxMvjFoNWSMpjopBqBbsLRA5uWC qPpo48+teuQ== X-Google-Smtp-Source: AGHT+IGwoFq1spxsTGSX9ojzlcdbQ7wa+yn2JHsmbQO8RCRg+bgIYAPQPuy0Fo2mPoZTnruJNEN6gA== X-Received: by 2002:a05:6a00:3a09:b0:736:3e50:bfec with SMTP id d2e1a72fcca58-7404777ecd5mr2154034b3a.8.1746066501236; Wed, 30 Apr 2025 19:28:21 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:20 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu , James Clark , Arnaldo Carvalho de Melo Subject: [PATCH v16 05/10] perf evsel: Assemble offcpu samples Date: Wed, 30 Apr 2025 19:28:03 -0700 Message-ID: <20250501022809.449767-6-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the data in bpf-output samples, to assemble offcpu samples. In evsel__is_offcpu_event(), Check if sample_type is PERF_SAMPLE_RAW to support off-cpu sample data created by an older version of perf. Testing compatibility on offcpu samples collected by perf before this patch= series: See below, the sample_type still uses PERF_SAMPLE_CALLCHAIN $ perf script --header -i ./perf.data.ptn | grep "event : name =3D offcpu-t= ime" # event : name =3D offcpu-time, , id =3D { 237917, 237918, 237919, 237920 = }, type =3D 1 (software), size =3D 136, config =3D 0xa (PERF_COUNT_SW_BPF_O= UTPUT), { sample_period, sample_freq } =3D 1, sample_type =3D IP|TID|TIME|C= ALLCHAIN|CPU|PERIOD|IDENTIFIER, read_format =3D ID|LOST, disabled =3D 1, fr= eq =3D 1, sample_id_all =3D 1 The output is correct. $ perf script -i ./perf.data.ptn | grep offcpu-time gmain 2173 [000] 18446744069.414584: 100102015 offcpu-time: NetworkManager 901 [000] 18446744069.414584: 5603579 offcpu-time: Web Content 1183550 [000] 18446744069.414584: 46278 offcpu-time: gnome-control-c 2200559 [000] 18446744069.414584: 11998247014 offcpu-time: And after this patch series: $ perf script --header -i ./perf.data.off-cpu-v9 | grep "event : name =3D o= ffcpu-time" # event : name =3D offcpu-time, , id =3D { 237959, 237960, 237961, 237962 = }, type =3D 1 (software), size =3D 136, config =3D 0xa (PERF_COUNT_SW_BPF_O= UTPUT), { sample_period, sample_freq } =3D 1, sample_type =3D IP|TID|TIME|C= PU|PERIOD|RAW|IDENTIFIER, read_format =3D ID|LOST, disabled =3D 1, freq =3D= 1, sample_id_all =3D 1 perf $ ./perf script -i ./perf.data.off-cpu-v9 | grep offcpu-time gnome-shell 1875 [001] 4789616.361225: 100097057 offcpu-time: gnome-shell 1875 [001] 4789616.461419: 100107463 offcpu-time: firefox 2206821 [002] 4789616.475690: 255257245 offcpu-time: Tested-by: Gautam Menghani Acked-by: Namhyung Kim Suggested-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-7-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Ian Rogers --- tools/perf/util/evsel.c | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index ea6b1ff16e6d..22a1c5b26fb0 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -1277,7 +1277,8 @@ static void evsel__set_default_freq_period(struct rec= ord_opts *opts, =20 bool evsel__is_offcpu_event(struct evsel *evsel) { - return evsel__is_bpf_output(evsel) && evsel__name_is(evsel, OFFCPU_EVENT); + return evsel__is_bpf_output(evsel) && evsel__name_is(evsel, OFFCPU_EVENT)= && + evsel->core.attr.sample_type & PERF_SAMPLE_RAW; } =20 /* @@ -2923,6 +2924,35 @@ static inline bool evsel__has_branch_counters(const = struct evsel *evsel) return false; } =20 +static int __set_offcpu_sample(struct perf_sample *data) +{ + u64 *array =3D data->raw_data; + u32 max_size =3D data->raw_size, *p32; + const void *endp =3D (void *)array + max_size; + + if (array =3D=3D NULL) + return -EFAULT; + + OVERFLOW_CHECK_u64(array); + p32 =3D (void *)array++; + data->pid =3D p32[0]; + data->tid =3D p32[1]; + + OVERFLOW_CHECK_u64(array); + data->period =3D *array++; + + OVERFLOW_CHECK_u64(array); + data->callchain =3D (struct ip_callchain *)array++; + OVERFLOW_CHECK(array, data->callchain->nr * sizeof(u64), max_size); + data->ip =3D data->callchain->ips[1]; + array +=3D data->callchain->nr; + + OVERFLOW_CHECK_u64(array); + data->cgroup =3D *array; + + return 0; +} + int evsel__parse_sample(struct evsel *evsel, union perf_event *event, struct perf_sample *data) { @@ -3277,6 +3307,9 @@ int evsel__parse_sample(struct evsel *evsel, union pe= rf_event *event, array =3D (void *)array + sz; } =20 + if (evsel__is_offcpu_event(evsel)) + return __set_offcpu_sample(data); + return 0; } =20 --=20 2.45.2 From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E70172356DE; Thu, 1 May 2025 02:28:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066505; cv=none; b=Uf+4+SmBPoPbE8A+gc1J0uVi3Quf6DxYT7+bkpVfvg0pnIZR14qr9+EmnIlwnPCa7X+/3DUesOem4HH+5xyMiNeErEdTRSFNcVhJvkpNAZ6KH9vyLdYybNmei4Lh4GG2jt0ctha+bsPKwAvze0Ra6gJiQu/hpdQDN+QTq2+ausg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066505; c=relaxed/simple; bh=Grz3UB4vnKUWYIaF0oX6/68Fsz+qYXnAONgbVsmT5L4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GnP9VCIHKOmg7O+VpYaBu5yeu3CB3apEmUTIWkpgu0OzR9Jj2/M3gDxguGpemLTmWuVFPQQqBWiDkZsTene0EnaG9MR82pTB2Vj6AC+7+nReQyneZwcnjRCzatOQ635FkRD3Hqq0BPh8JPd9KYreHo67TLn7Ytd/563HMGlC1gQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hL0miSHS; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hL0miSHS" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-736aa9d0f2aso804096b3a.0; Wed, 30 Apr 2025 19:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066503; x=1746671303; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Pp9bvoYC1HVNo7F+IfC04BEtYRP27VaZRbTXVY0Kr+Q=; b=hL0miSHSHmHkUjP7VquRDpIOJPrYFLVVETbxrw8wPAB/QxLUzEF99blCepSbt+iCS2 V3pYhybuITOyA1cmcJno5/eNJvNtaoQyBLmg5JL8zTCb/Js00UyLNeXBxkp7r2epSzRr 7nwybmBBKNnkm450iJ0YQYvkHEEkW07D/ii/Q88xEuwalXfi3NX8SCndgfnWrcVX9baa bOCvQNWLroXvnxlj/UzI1UOJdmFnK68fv8csO1AVO+qR83Ot2Z1sXRrOcc/tz8A2Z4Dl pYC3+o1UyttnrjmaVjgf8CAzeGXiGwEPujVw+z/xWAZxYqLpYa/6WiQbYHWR/hd7T37O f1/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066503; x=1746671303; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Pp9bvoYC1HVNo7F+IfC04BEtYRP27VaZRbTXVY0Kr+Q=; b=mwBygwJu499oXgVOUd3CnW7JfEdJbnNgRIEHUduwyhYpgG58NWSsW7ZN5/13288tJV qANabn79DwvdVQKQZ2YtVSR9/aCkCCiFJAcDEIX7gmDeL6jZhzTqbhF33121+Fn23gmE 0ilRQFA3tNOTk5L6x2Ri02ky/Wscd054gQUUxE5O+Y8rSOiJ/B8iJhaVihxGrtGiCQlj YD3L3BkstelEgTNUxvl2z8d1Z1pAxsBb2jOTyRwHajXJ9c9EgaXKqiIOLwj186RcIaHE nuVlwNrFVP7n2pqEFLOCDVGqxm/LyCZ0YFbwdgussjc23/WsjsQfSooTUv6eyY41VKjW 3FKQ== X-Forwarded-Encrypted: i=1; AJvYcCUZ7wZjZyqeh9Mlu1US3MZiES7XLjC+VzTAvLfhP7tpacKGX4gJ4TPQbN2VCkkY1S+m3rW9OVa7kE6eyc2bWOZfYg==@vger.kernel.org, AJvYcCXnyJoHg0IqMMOoLyP9OcR5MV0Fk27gP7Lm5Ot0wuKF4wQseRUTpt4hrHAZjUDj10Nc80qYnAQ5lh8OqpI=@vger.kernel.org X-Gm-Message-State: AOJu0YyL2mXzA66zUGlrHYIbaNgC7qmbz2k5f0NQj5e2dNyoVLomVPYV uzAe3JodA06L4MNTtCfvjUAs+ZTEWhMQzz5nmijPJYHzd+crivTI X-Gm-Gg: ASbGncurTqCu6kxfyOP6duVT/jZvPFXLnNrIja/EuxSh2gGZZS188MbzljJVZYPDJJR hrhKIXCWlCJ6NAjFAeLX/PSAzbcsM7hYKHhXYFwhieYQsBhAGTwa4ae6vgx3kOaMoh5aq0qn4YH rWn97DjSrt+0rbhv+z2JGLqaS4+DQuBehakW3vQMkKPZgXWOYMbso/mWkc4z9jzv5fiHKB7cN8U zPAABYeAd9alIYtsOSkiP9mshzSOvKggDlWQIPI5nQyvMUuhE594UpEq8Yqn3aE16dKpLiidk5M YuAJqCSl4kRIjX12u17tpYiOjZjnGCissaHAReml3WrRUsUm7nRbrxlw3VTUpnrL2fLRERr5a4r Wz+5QQHB5jw== X-Google-Smtp-Source: AGHT+IGC15hbaC94Tt6bIjlRvVzNyq6MOLBhUHtckC7dTNYpg5EQ7HoxtJSOCnczvFU9fclrkx2dWQ== X-Received: by 2002:a05:6a00:2348:b0:736:4644:86ee with SMTP id d2e1a72fcca58-740477a3273mr1806201b3a.14.1746066503122; Wed, 30 Apr 2025 19:28:23 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:22 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu , James Clark , Arnaldo Carvalho de Melo Subject: [PATCH v16 06/10] perf record --off-cpu: Disable perf_event's callchain collection Date: Wed, 30 Apr 2025 19:28:04 -0700 Message-ID: <20250501022809.449767-7-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" There is a check in evsel.c that does this: if (evsel__is_offcpu_event(evsel)) evsel->core.attr.sample_type &=3D OFFCPU_SAMPLE_TYPES; This along with: #define OFFCPU_SAMPLE_TYPES (PERF_SAMPLE_IDENTIFIER | PERF_SAMPLE_IP | \ PERF_SAMPLE_TID | PERF_SAMPLE_TIME | \ PERF_SAMPLE_ID | PERF_SAMPLE_CPU | \ PERF_SAMPLE_PERIOD | PERF_SAMPLE_CALLCHAIN | \ PERF_SAMPLE_CGROUP) will tell perf_event to collect callchain. We don't need the callchain from perf_event when collecting off-cpu samples, because it's prev's callchain, not next's callchain. (perf_event) (task_storage) (needed) prev next | | ---sched_switch----> Tested-by: Gautam Menghani Acked-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-8-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Ian Rogers --- tools/perf/util/off_cpu.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/perf/util/off_cpu.h b/tools/perf/util/off_cpu.h index 2dd67c60f211..2a4b7f9b2c4c 100644 --- a/tools/perf/util/off_cpu.h +++ b/tools/perf/util/off_cpu.h @@ -13,7 +13,7 @@ struct record_opts; #define OFFCPU_SAMPLE_TYPES (PERF_SAMPLE_IDENTIFIER | PERF_SAMPLE_IP | \ PERF_SAMPLE_TID | PERF_SAMPLE_TIME | \ PERF_SAMPLE_ID | PERF_SAMPLE_CPU | \ - PERF_SAMPLE_PERIOD | PERF_SAMPLE_CALLCHAIN | \ + PERF_SAMPLE_PERIOD | PERF_SAMPLE_RAW | \ PERF_SAMPLE_CGROUP) =20 =20 --=20 2.45.2 From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DBCC22FAE1; Thu, 1 May 2025 02:28:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066507; cv=none; b=X02UsXpjj1ZajyvNx1rH+28vUOtWAxzXrPgRmz4ZteYbrSEEwU4T4uqbLmL37+lZf+8j6WSVMAfHBKeXjMtg6UVnyaB2MGh89M5RXRHwXiQV8/+o0I8Qv2MLAyYBMiB+Z3Nt6zTJgFOUX+yhwTFweHXcMTD7m1+BTfKk1W6xc1k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066507; c=relaxed/simple; bh=fkqYzF36h6AgRaLeSgkPKbTgXMvVxYruaKPs0gzkDHA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Chne+xTB/nqZSKAK6Z5aLZvXWccg2cn4J+Df3Wi8obBDK+U+0wwVHxZGRpBNvyvmw6ykiufGh994HvMA65aCywIBb/ondceq9AN0um6kKUx6f0RXTLfjUMpFWxkbTT28AzC9bFrTJNwfMuLXbDxn+5D1QIbrLXKYyVBL5m0o5go= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ikL+KCkv; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ikL+KCkv" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-736c3e7b390so547489b3a.2; Wed, 30 Apr 2025 19:28:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066505; x=1746671305; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GTjiJWBJ0ei7byXhRRPp/TXdgUBSDbCjGFLqGPTVJoA=; b=ikL+KCkvbJRMynXiq60RK0Q/zfmxJ9uO2rIVKz8W9Tz/CtbHO6WPKtsG1DjuwAKfAc RPlif/xtfFYyoP2kt9YLMz0JZ40JOghCk6x6pUAjxzoJ9uDChLGAuywCtliYlu4+iqux p/M/S1n491j4x49FOqy2lWFA5RvG4YPESFtE7gZeQOHaG4wHhIZ9q2HbA12uJqw9kyg9 unHsAN8GpqTHn2vRj0L2xbuIQDyOLHqtB0cUVVyzY0FW5YbqdEFvY1Rzqix6rTmC56wv fhHcapav6RgXinSyYH9qFzq9LQO62A3nkMhgjzptlIVztmV5f8T2eImEkI+KVgSuxCda 3e4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066505; x=1746671305; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GTjiJWBJ0ei7byXhRRPp/TXdgUBSDbCjGFLqGPTVJoA=; b=t0ddHu4iyiDK3JKlhSfObB9ESZ7JakL+RBczVk8+mKDp7pjRZ/v16Mj6bvBwE+F0ZD XIrHz/z7thJZIQk/fO2q0Cjk3Nu/7CNqrAXc8Fta4UYY1E2X0bqvHFOj4QG92b+l8Qzq wqzWCjVhAbgG1brzOW61aG72jTJ9cTcYRF2Bh6dtV+zVqUjGQr2qW8bDnyK4Sp0W/0Ma hdy6a5zUta/aza7wKgGpee0CCIOvMB2vKoxi2qWi3llFpjp+psxwrcBK1J5LWnwI9JGh SUO91+nBuC0RXZGTDbn5xBsOCAjIicAt26ATlSHmH2hlRgDiw5QNkc8ZyZ3I6Jd4I3sK 0R6A== X-Forwarded-Encrypted: i=1; AJvYcCUEWTJNYKtRVC9kIGkxzwp2iHR/8ISZliaZ88HS/ZP5sCWtQLtXekOmqk7iAyKJuTr4QC1cI5YaReRcUF2RJvhanQ==@vger.kernel.org, AJvYcCVBrrlrBZJQJiWgxVeeufWyuXU6zemzm4YPP6Dwp2ViRsBYq0cEbZ7r5AogXgmiWsyX+apQQ1cWrarc3ec=@vger.kernel.org X-Gm-Message-State: AOJu0Yy48+O02hSfx/wmrjKmy5Z5pKUDiDA1tD6T4TDYhiu4QsSic0W7 MKLijh3Ahy0yYxCgGQhgDtvcbHfLCcs/IkvEj1NXjVJKcNT1gwFL X-Gm-Gg: ASbGncs9Vb2G3PuAbVbUOst/+i/Giso9RCoqCsho4Zw+89HpcCpi9LcmtXVxeNNUOVh 5y9bNzQlcpn6cTdgcu6g+CJ/oxh/iNGQnSxD6CVuDVIdKtzTWbiaeAi94kYMQlcaPnka2ToxgQP xeAxFZcAPeiZF4PKADM9UmItO027OBYjrDFOi53acd8UaN2iscHLAr/6jfKPuBQ6DDF2sAdboRn Ix3zFoy80G7j59sZSZxqOy5uzcCypwl6b9+mtQXHVNcmTAa+cK6rj/YzWissaeqicfxpUqMS2HQ JHBUXgNwOmR7LnC22pwp3XlVbOYGJbzHnh1gNnfG/O4AmKMsAf2u89Q8DBh3fDnFiPnwDBZMQPR 0GXgdQFT1VA== X-Google-Smtp-Source: AGHT+IGXNTTLCqWQqzs3MU3R+DswRoGpbzJFNCcnJdqEEJJ1A4VgFf3QRRQJFdqonQq9NkCkyHDktw== X-Received: by 2002:a05:6a21:1585:b0:1f5:889c:3cdb with SMTP id adf61e73a8af0-20bd65580a0mr1249766637.8.1746066504677; Wed, 30 Apr 2025 19:28:24 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:24 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu , James Clark , Arnaldo Carvalho de Melo Subject: [PATCH v16 07/10] perf script: Display off-cpu samples correctly Date: Wed, 30 Apr 2025 19:28:05 -0700 Message-ID: <20250501022809.449767-8-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable No PERF_SAMPLE_CALLCHAIN in sample_type, but I need perf script to display a callchain, have to specify manually. Also, prefer displaying a callchain: gvfs-afc-volume 2267 [001] 3829232.955656: 1001115340 offcpu-time: 77f05292603f __pselect+0xbf (/usr/lib/x86_64-linux-gnu/libc.so.= 6) 77f052a1801c [unknown] (/usr/lib/x86_64-linux-gnu/libusbmuxd-2.= 0.so.6.0.0) 77f052a18d45 [unknown] (/usr/lib/x86_64-linux-gnu/libusbmuxd-2.= 0.so.6.0.0) 77f05289ca94 start_thread+0x384 (/usr/lib/x86_64-linux-gnu/libc= .so.6) 77f052929c3c clone3+0x2c (/usr/lib/x86_64-linux-gnu/libc.so.6) to a raw binary BPF output: BPF output: 0000: dd 08 00 00 db 08 00 00
...... 0008: cc ce ab 3b 00 00 00 00 =CE=AB;.... 0010: 06 00 00 00 00 00 00 00 ........ 0018: 00 fe ff ff ff ff ff ff . 0020: 3f 60 92 52 f0 77 00 00 ?`.Rw.. 0028: 1c 80 a1 52 f0 77 00 00 ..Rw.. 0030: 45 8d a1 52 f0 77 00 00 E.Rw.. 0038: 94 ca 89 52 f0 77 00 00 ..Rw.. 0040: 3c 9c 92 52 f0 77 00 00 <..Rw.. 0048: 00 00 00 00 00 00 00 00 ........ 0050: 00 00 00 00 .... Tested-by: Gautam Menghani Acked-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-9-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Ian Rogers --- tools/perf/builtin-script.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c index 9b16df881af8..6c3bf74dd78c 100644 --- a/tools/perf/builtin-script.c +++ b/tools/perf/builtin-script.c @@ -680,7 +680,7 @@ static int perf_session__check_output_opt(struct perf_s= ession *session) =20 evlist__for_each_entry(session->evlist, evsel) { not_pipe =3D true; - if (evsel__has_callchain(evsel)) { + if (evsel__has_callchain(evsel) || evsel__is_offcpu_event(evsel)) { use_callchain =3D true; break; } @@ -2295,7 +2295,7 @@ static void process_event(struct perf_script *script, else if (PRINT_FIELD(BRSTACKOFF)) perf_sample__fprintf_brstackoff(sample, thread, evsel, fp); =20 - if (evsel__is_bpf_output(evsel) && PRINT_FIELD(BPF_OUTPUT)) + if (evsel__is_bpf_output(evsel) && !evsel__is_offcpu_event(evsel) && PRIN= T_FIELD(BPF_OUTPUT)) perf_sample__fprintf_bpf_output(sample, fp); perf_sample__fprintf_insn(sample, evsel, attr, thread, machine, fp, al); =20 --=20 2.45.2 From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34BD3239082; Thu, 1 May 2025 02:28:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066508; cv=none; b=T1VrWLLukdZz2VAJzcqPUqGskl9wQoJCIjdxq3quIn3ZWHJJZ3Li6IvGQGIaicmfXIwHJaSD4PEUhMMAm0Qm1jaKbpc9fkCroMJfxxKOve6JRCDsBAgHbwFtdmrnAqyJPm2Xn8psTohUxmPydsXNOOxhGkZ6KFMbd3GOw6JuEuU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066508; c=relaxed/simple; bh=EjZkhDjLf8FF0DjSXesgOJ/arnnm6yfJ+pBD7u5LPM8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=q6Sa5snAxAuYED1yC9cno8pEzl2tIYbWYgfgF6bREeNhvmkYzYhK7ZU/Yi8Rqog4P7l2AFUaXuYQi6GXOBJpI6yUdRH2B7l1SYBJh7OEJzywLczcK0fET/nwsOyqLpRlVKhaRg2Gclm53YJ0il5ZZCvqZ5eI1+wJMOGbppn42C4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XVI0VS0b; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XVI0VS0b" Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-739b3fe7ce8so569658b3a.0; Wed, 30 Apr 2025 19:28:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066506; x=1746671306; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tGiokA3zKaSxSjRFI8xP0E2QGWuB8HpmpdNiCB3BS38=; b=XVI0VS0bbSCO0dgSecD5GhuUuC0ztL7SD4KmW05pJUYz/+NWZ2XWPEOGNOW77qhKxx O09tVkVhHR8EZ6hm4adf5YygwL+FxbKVt9ehDETKJTy2HO5woV3RGC5BHVa+onSnUMER BpR329kSgbAb4o3Gef2/hC/JXveZC1JEoZEE5PGRK8B4QQm+00v5jOoU7rpY/mywiC5x ZNLEKPtC9aQ4UWORhcx756klLPkPbdRFybWDCguMctHyN4iGtNLw4CvR1HDwPgrcf3HQ vhIH68A2rJj9Xxa+AwbaRauPwe7fbkJVwVAz1CiAEnc4n4VpL4seF685uesFyTTGb++J gUwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066506; x=1746671306; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tGiokA3zKaSxSjRFI8xP0E2QGWuB8HpmpdNiCB3BS38=; b=JNhjUXgkfmggOQdR3ucSPZuiEPfUKCwMRUrTEhTN7i6v9eYgli1Z+2ePU1CCH7SdX/ oymZ1WdK82DOCITHLYt6wrTvO57oivp+xBDBYvQZjl0uQpSE1t5jSNZ+XhvxEse4qHZo E5bsbIq2+O4voO50il7shSdah6Wq9hcZLcIQjMSM8qMn41MGRPpKOHFLAFxt+vvka0BZ bmX2D9HcWqHRuEKaLfsT+0l5sLrG/mkzdblIgI8nB5HwbDiQ3oCzVgotoLROoMTBGNED 7ZfqzxXRO3XmZdTei+TE3lB+K377Xuy2BEzII4KJNRUpmyLYpYGiZHEMnZOBoq/JlRnk I4aQ== X-Forwarded-Encrypted: i=1; AJvYcCU2I9RAPO/iojKSFVyhFb+cfr2oNjnLNbKaDHQsKwtOl2AX1yl7VrjJebw/PvaSahpWMlKTR+22WvJkQHY=@vger.kernel.org, AJvYcCWjH0M1L8MIX9zM93jMymdStq3pI8dDGBTapxMHx5A1MC4l3K2cAqUobCwakBjZ7C3/r55PXtPCuhD/1etzUFK4Ng==@vger.kernel.org X-Gm-Message-State: AOJu0YxD4Uk3dsTCaHYdaM3oz85hAK3c7pCENOv+sl9T6W1rXZTnVusS XHB3RZ8mIuEuUplgaUoXgQ2nM7wVOvb92ER10wVvZvBfmQoN8Giz X-Gm-Gg: ASbGncvBnpU7qVDcl7OUmTTgiai7sD2WsGgpBV7yV8tpc5PPUf0UXY7aDvfvRoQVIvP aqTWysu4xFxitMTMrP+SXo7jCq+08U2OjqL5CsCqVaXoem5ybxvEPpLNSirp0DRVXfUwvQMJnoM A9TF+7+k8kvZKnX3ZLBPqygJMSLRMRC2Vmacz49pa9Mp5G1CNGTP9gRAcyfSH1o+p6/VrRuROPM 6Ii9Tvk4X7AwCcJ6C0lasfyW/lZ3tUZwqvVQV+W1Xd6HIQeja7S0bVdu1lS5YS7G5U6n8zNiewD Fa5sNrwqY3A+vyRtTX9LqBzghyA202ErCLzH2Y9xawugs9zfF0+KmiuTRD7Tiosvz4rIR0V8adg EnUtVNbjl2w== X-Google-Smtp-Source: AGHT+IEipX1gYi7PaCzq9l2JDUbdhHcMGSYYfKkXqngEzcvHihjzaGEQt99sELSTwa2BOZTr8+2AEg== X-Received: by 2002:a05:6a20:e687:b0:1f5:9961:c40 with SMTP id adf61e73a8af0-20a876446d4mr7314157637.8.1746066506266; Wed, 30 Apr 2025 19:28:26 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:25 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu , James Clark , Arnaldo Carvalho de Melo Subject: [PATCH v16 08/10] perf record --off-cpu: Dump the remaining samples in BPF's stack trace map Date: Wed, 30 Apr 2025 19:28:06 -0700 Message-ID: <20250501022809.449767-9-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Dump the remaining samples, as if it is dumping a direct sample. Put the stack trace, tid, off-cpu time and cgroup id into the raw_data section, just like a direct off-cpu sample coming from BPF's bpf_perf_event_output(). This ensures that evsel__parse_sample() correctly parses both direct samples and accumulated samples. Tested-by: Gautam Menghani Acked-by: Namhyung Kim Suggested-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-10-howardchu95@gmail= .com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Ian Rogers --- tools/perf/util/bpf_off_cpu.c | 59 +++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 24 deletions(-) diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index 1975a02c1683..c7fde66bb8f9 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -37,6 +37,8 @@ union off_cpu_data { u64 array[1024 / sizeof(u64)]; }; =20 +u64 off_cpu_raw[MAX_STACKS + 5]; + static int off_cpu_config(struct evlist *evlist) { char off_cpu_event[64]; @@ -313,6 +315,7 @@ int off_cpu_write(struct perf_session *session) { int bytes =3D 0, size; int fd, stack; + u32 raw_size; u64 sample_type, val, sid =3D 0; struct evsel *evsel; struct perf_data_file *file =3D &session->data->file; @@ -352,46 +355,54 @@ int off_cpu_write(struct perf_session *session) =20 while (!bpf_map_get_next_key(fd, &prev, &key)) { int n =3D 1; /* start from perf_event_header */ - int ip_pos =3D -1; =20 bpf_map_lookup_elem(fd, &key, &val); =20 + /* zero-fill some of the fields, will be overwritten by raw_data when pa= rsing */ if (sample_type & PERF_SAMPLE_IDENTIFIER) data.array[n++] =3D sid; - if (sample_type & PERF_SAMPLE_IP) { - ip_pos =3D n; + if (sample_type & PERF_SAMPLE_IP) data.array[n++] =3D 0; /* will be updated */ - } if (sample_type & PERF_SAMPLE_TID) - data.array[n++] =3D (u64)key.pid << 32 | key.tgid; + data.array[n++] =3D 0; if (sample_type & PERF_SAMPLE_TIME) data.array[n++] =3D tstamp; - if (sample_type & PERF_SAMPLE_ID) - data.array[n++] =3D sid; if (sample_type & PERF_SAMPLE_CPU) data.array[n++] =3D 0; if (sample_type & PERF_SAMPLE_PERIOD) - data.array[n++] =3D val; - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - int len =3D 0; - - /* data.array[n] is callchain->nr (updated later) */ - data.array[n + 1] =3D PERF_CONTEXT_USER; - data.array[n + 2] =3D 0; - - bpf_map_lookup_elem(stack, &key.stack_id, &data.array[n + 2]); - while (data.array[n + 2 + len]) + data.array[n++] =3D 0; + if (sample_type & PERF_SAMPLE_RAW) { + /* + * [ size ][ data ] + * [ data ] + * [ data ] + * [ data ] + * [ data ][ empty] + */ + int len =3D 0, i =3D 0; + void *raw_data =3D (void *)data.array + n * sizeof(u64); + + off_cpu_raw[i++] =3D (u64)key.pid << 32 | key.tgid; + off_cpu_raw[i++] =3D val; + + /* off_cpu_raw[i] is callchain->nr (updated later) */ + off_cpu_raw[i + 1] =3D PERF_CONTEXT_USER; + off_cpu_raw[i + 2] =3D 0; + + bpf_map_lookup_elem(stack, &key.stack_id, &off_cpu_raw[i + 2]); + while (off_cpu_raw[i + 2 + len]) len++; =20 - /* update length of callchain */ - data.array[n] =3D len + 1; + off_cpu_raw[i] =3D len + 1; + i +=3D len + 2; + + off_cpu_raw[i++] =3D key.cgroup_id; =20 - /* update sample ip with the first callchain entry */ - if (ip_pos >=3D 0) - data.array[ip_pos] =3D data.array[n + 2]; + raw_size =3D i * sizeof(u64) + sizeof(u32); /* 4 bytes for alignment */ + memcpy(raw_data, &raw_size, sizeof(raw_size)); + memcpy(raw_data + sizeof(u32), off_cpu_raw, i * sizeof(u64)); =20 - /* calculate sample callchain data array length */ - n +=3D len + 2; + n +=3D i + 1; } if (sample_type & PERF_SAMPLE_CGROUP) data.array[n++] =3D key.cgroup_id; --=20 2.45.2 From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97F2D22FE06; Thu, 1 May 2025 02:28:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066510; cv=none; b=MxgdgmSPcm00JmBB8zYnS98aMWJlOzNCLj3IiXu7dh1yFGDs2o7MuJ2SwBaAM9cnMzcdQwlWVkW7nhyoc3135eSh8GnSr4CNxkdYHvQIo/Pg9G5D5xXkwENq0b6i5Q6V4g0XT+nRXoK/KLR7rd2itp7tHisEz+jqKo6BGlLE2y4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066510; c=relaxed/simple; bh=UBEORm636/9ymi4rsoYE3mXyHcRxH6DVHX1aMMbqXN0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EGai+CwubYV12V/XM1vGBMtBHqJheBMEqK2YLWt3/uSlrO2+7woeg9yQJwtqp3JrRT+lG1BLThKPxeuMzpt6+TICFqzmii8d2bB+N1zm9/ldgp3kIW2+FSoCJ1S2WijnZqpARPWKLAEVPkaNg6ch20qYhSZfgVQbR6PLU1FB88A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Si6bPtgi; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Si6bPtgi" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-736b0c68092so523721b3a.0; Wed, 30 Apr 2025 19:28:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066508; x=1746671308; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cE7k5mUPZGsaEEvfnaUZGMWrp21zZwEJVPsVMxHJ10g=; b=Si6bPtgiQxwX4BlDMEM6dH6Ob0lV4zCpoAkCeJrtQAx8/8T4vMkWU+WCle5czWgxZk 1XpWs3gwwUnuMCqFa0ze0vXVvP8VC2JXUIfFZE/63ig9t8RPXZJnY4kWyQALuHaKJkPF uVqB2+fh7ruRJmnsR4QkqaD0qPixUQKIj5PadU2S6GUCey34MhC5oF9x/oGax0RYQ+Gl /CQQmDtBXBKJqKwhcpvPq+PKDzJRrFa1juO3zyS+UIxInXoe55kpr2f17VMY5Xq6KU96 yD7pBSNN76vjlpjawofaKXPyXCHDu/n7fGOm3gitRvXyiIk2PkCI2u5jz/QyC32GCIP0 uBLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066508; x=1746671308; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cE7k5mUPZGsaEEvfnaUZGMWrp21zZwEJVPsVMxHJ10g=; b=nrDMdruO+5B3A4UdxwuggpgY8vl5WoCoeK3cRGD1Ynaku0Y6qmC3efUEiO6eJopTW2 7wnQDBPVzapKwXjEvx1Efs6Iykc47JyTQN3O6UNrmX5Ma6U6/rjf0w7RvAJiXjp8m/9B 4TCzv3laGFFTZfiOQryNtXj8eLjrJxxUqfhmLvXmsKcXICUoXfdSZjjQvi7sIrUin0Yz lBR9vo8xWpZx4WzxBorWU6Ch11zmu3/al6r2W6vjG9ummVfDklKCgmqzaYyoqGEADnCM d9EzhZDw90nshO+hYme4IYvs0q854BwbbM/6i1XCp8j0SO7CJ/wNXjIholuMuyHsp2Tr 46fQ== X-Forwarded-Encrypted: i=1; AJvYcCXNrDbSU07G545YVc+Q+kTB8cp5Z0793cI6D9RTw7J15q2bUxAcXCCc9PGJaXg6N3Yr6qoCQjlNOShksgU=@vger.kernel.org, AJvYcCXYHgmb/ZDk9X57DvTb8TQWt9c0dFNBqKN2Vw2XiBKUuqnQoH+KXMiuzkyAqjSQA/b+G/hV69DYZquKOfC86oYqLQ==@vger.kernel.org X-Gm-Message-State: AOJu0YwpS7sPsh3PHnx66Dzb4MF6nDvOUBYHoFC3ZEY+HFvzzdNk3vbn ovgqcPzPmtWX0EH8XzE/Ip6nt5mkrDZ8ZDg+uOLnz0bimpIxNfyw X-Gm-Gg: ASbGncs/Glct5g1/qKv5DfXzcAedYnlnQAWAyhtklDjVPI+X7ODum4IiEEu/Hp4mSGi MiiD349CAK322voPW9e/k8+DaGSEAs/O0FmQCoMC52PIgoAwucfPCYalCEllyfeFvmMCh0KLaqx WshobvQqLF4GuQTy2zooKVbFaccIUcq7rqoiSqKn4cIrTm+kgsS5FBaka2AWyt/1AubElbuaMw4 rvhuHNvIDKAfoHpLsvw3EfRqo11A7+XinORiYU7gCUO1LScF5t2E0f3A9cqAJvf3KyoltI5pTs+ 2XUZVKNt4rpgsneqPu49NjqhUOiZI2tljOOyShdp54X4gJvMAjBUlN/PvQQCpJ77iuYo+VKOX4n otaCazWZkm7UoqbCcH4WJ X-Google-Smtp-Source: AGHT+IF7qt+/r8Rp0+kzvQvqTnA0A3Pxuei84gotGox2TjH+PinMYF7hSy+NC2a+KF+l2X1kTo/ksg== X-Received: by 2002:a05:6a00:2185:b0:730:79bf:c893 with SMTP id d2e1a72fcca58-74049184970mr1136510b3a.4.1746066507975; Wed, 30 Apr 2025 19:28:27 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:27 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu , Arnaldo Carvalho de Melo , James Clark Subject: [PATCH v16 09/10] perf record --off-cpu: Add --off-cpu-thresh option Date: Wed, 30 Apr 2025 19:28:07 -0700 Message-ID: <20250501022809.449767-10-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Specify the threshold for dumping offcpu samples with --off-cpu-thresh, the unit is milliseconds. Default value is 500ms. Example: perf record --off-cpu --off-cpu-thresh 824 The example above collects direct off-cpu samples where the off-cpu time is longer than 824ms Tested-by: Gautam Menghani Acked-by: Namhyung Kim Suggested-by: Ian Rogers Suggested-by: Namhyung Kim Suggested-by: Arnaldo Carvalho de Melo Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-2-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Ian Rogers --- tools/perf/Documentation/perf-record.txt | 9 ++++++++ tools/perf/builtin-record.c | 26 ++++++++++++++++++++++++ tools/perf/util/bpf_off_cpu.c | 3 +++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 2 +- tools/perf/util/off_cpu.h | 1 + tools/perf/util/record.h | 1 + 6 files changed, 41 insertions(+), 1 deletion(-) diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Document= ation/perf-record.txt index c7fc1ba265e2..d84f7a0af05b 100644 --- a/tools/perf/Documentation/perf-record.txt +++ b/tools/perf/Documentation/perf-record.txt @@ -837,6 +837,15 @@ filtered through the mask provided by -C option. only, as of now. So the applications built without the frame pointer might see bogus addresses. =20 + off-cpu profiling consists two types of samples: direct samples, which + share the same behavior as regular samples, and the accumulated + samples, stored in BPF stack trace map, presented after all the regular + samples. + +--off-cpu-thresh:: + Once a task's off-cpu time reaches this threshold (in milliseconds), it + generates a direct off-cpu sample. The default is 500ms. + --setup-filter=3D:: Prepare BPF filter to be used by regular users. The action should be either "pin" or "unpin". The filter can be used after it's pinned. diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index da28c52af464..d3ddf933d7a5 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -3162,6 +3162,28 @@ static int record__parse_mmap_pages(const struct opt= ion *opt, return ret; } =20 +static int record__parse_off_cpu_thresh(const struct option *opt, + const char *str, + int unset __maybe_unused) +{ + struct record_opts *opts =3D opt->value; + char *endptr; + u64 off_cpu_thresh_ms; + + if (!str) + return -EINVAL; + + off_cpu_thresh_ms =3D strtoull(str, &endptr, 10); + + /* the threshold isn't string "0", yet strtoull() returns 0, parsing fail= ed */ + if (*endptr || (off_cpu_thresh_ms =3D=3D 0 && strcmp(str, "0"))) + return -EINVAL; + else + opts->off_cpu_thresh_ns =3D off_cpu_thresh_ms * NSEC_PER_MSEC; + + return 0; +} + void __weak arch__add_leaf_frame_record_opts(struct record_opts *opts __ma= ybe_unused) { } @@ -3355,6 +3377,7 @@ static struct record record =3D { .ctl_fd =3D -1, .ctl_fd_ack =3D -1, .synth =3D PERF_SYNTH_ALL, + .off_cpu_thresh_ns =3D OFFCPU_THRESH, }, }; =20 @@ -3580,6 +3603,9 @@ static struct option __record_options[] =3D { OPT_BOOLEAN(0, "off-cpu", &record.off_cpu, "Enable off-cpu analysis"), OPT_STRING(0, "setup-filter", &record.filter_action, "pin|unpin", "BPF filter action"), + OPT_CALLBACK(0, "off-cpu-thresh", &record.opts, "ms", + "Dump off-cpu samples if off-cpu time exceeds this threshold (in mi= lliseconds). (Default: 500ms)", + record__parse_off_cpu_thresh), OPT_END() }; =20 diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index c7fde66bb8f9..c367fefe6ecb 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -14,6 +14,7 @@ #include "util/strlist.h" #include #include +#include =20 #include "bpf_skel/off_cpu.skel.h" =20 @@ -292,6 +293,8 @@ int off_cpu_prepare(struct evlist *evlist, struct targe= t *target, } } =20 + skel->bss->offcpu_thresh_ns =3D opts->off_cpu_thresh_ns; + err =3D off_cpu_bpf__attach(skel); if (err) { pr_err("Failed to attach off-cpu BPF skeleton\n"); diff --git a/tools/perf/util/bpf_skel/off_cpu.bpf.c b/tools/perf/util/bpf_s= kel/off_cpu.bpf.c index 14cd8881f8bb..72763bb8d1de 100644 --- a/tools/perf/util/bpf_skel/off_cpu.bpf.c +++ b/tools/perf/util/bpf_skel/off_cpu.bpf.c @@ -124,7 +124,7 @@ const volatile bool uses_cgroup_v1 =3D false; =20 int perf_subsys_id =3D -1; =20 -__u64 offcpu_thresh_ns =3D 500000000ull; +__u64 offcpu_thresh_ns; =20 /* * Old kernel used to call it task_struct->state and now it's '__state'. diff --git a/tools/perf/util/off_cpu.h b/tools/perf/util/off_cpu.h index 2a4b7f9b2c4c..64bf763ddf50 100644 --- a/tools/perf/util/off_cpu.h +++ b/tools/perf/util/off_cpu.h @@ -16,6 +16,7 @@ struct record_opts; PERF_SAMPLE_PERIOD | PERF_SAMPLE_RAW | \ PERF_SAMPLE_CGROUP) =20 +#define OFFCPU_THRESH 500000000ULL =20 #ifdef HAVE_BPF_SKEL int off_cpu_prepare(struct evlist *evlist, struct target *target, diff --git a/tools/perf/util/record.h b/tools/perf/util/record.h index a6566134e09e..c82db4833b0a 100644 --- a/tools/perf/util/record.h +++ b/tools/perf/util/record.h @@ -79,6 +79,7 @@ struct record_opts { int synth; int threads_spec; const char *threads_user_spec; + u64 off_cpu_thresh_ns; }; =20 extern const char * const *record_usage; --=20 2.45.2 From nobody Sun Dec 14 19:11:43 2025 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FCAB23C4E9; Thu, 1 May 2025 02:28:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066511; cv=none; b=GrTpVwL9zmmPJzFgQqVKoVsqlb+9CcTZrha4GZSfVqtolpaGCYfUURh6bVm304tsEngOjseB6ChypqGKty2hetEg1xneJYLBe+ne2hk6XZbsGQGRhEKgjEklZ4gSU/cBMkGbmvTBQEtc4+duHdsr99nvs9rGbs2RtN6Orh+RT7k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746066511; c=relaxed/simple; bh=88Fl2EiHB2cjDAVZPyhgr0clBqVOitL8l2L37ibdLfE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KONFvneXBT8mTIdraR6kn3fWZuQ1oaWQvUEKkNxUAigj0G9uak+nXQHE+E+OXFAAl1cDMSfTiiJ8eemTVq7wOX3ghyLAEF/uK8NbQy3ZGgRmPqW+JarVVAGWY8TMr4BYkmWUPEzffN/TCDLhfJiOT+DNSNAlrD7Dkjrf1mTz4L8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ApY/8GuB; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ApY/8GuB" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-74019695377so463075b3a.3; Wed, 30 Apr 2025 19:28:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746066509; x=1746671309; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UD84FLOguCUPK491YqR0LpWQbCrSSziD1nV34s4DcTM=; b=ApY/8GuBfst77z6SkA6iwijCcDVYoTAbGTV86afGpOUZ2r9x11q7IM0WHMNjm/pGET PDY7FZvLX1vKBYQ7ftOd7O7qy6udWKhiWW8uyzAHDSijrRAE06Uyvg+X6l5FZzvBpPEW 3dSsjFhvtIb1bmizDQpbF9CQtHKqmNt0L2xrd83Iem+CGuBZMWNKIFPdE8B0MM54DSeq ZF4HSa7+3lcU8Bhl+Cwwjae1Bn5tzgdwDiorUUTS7211zTtUM5xifHgVJ3BdkoL0hB47 coWv51L7VVHEQljAIQx4ysvY+C4vYDsMb2uTmhhXnIuWeKfiQPXlFApvXew+xUyBpRtk B+Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746066509; x=1746671309; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UD84FLOguCUPK491YqR0LpWQbCrSSziD1nV34s4DcTM=; b=Rxb42zM1tX7XKldQS6cV6wW6xOzqCb8w7lZoGrYgx1xMXHs+MQ8M+kAVO2DxI7aXFb 2ixREpW6ni20SwIy0KXqo9DczhjQt0CNzv+2yuGrPwGGR9D3yCzPcaGdufUZnnPMMMA6 ets2WeeDEPRwctrNlhNoKNejjGyTBvw9k72TUpZYSZiOqXHp+ysITnoy3m1OThgKKrWp s14AmTIRcEHTmpqzhXbeJCdUIy/dayZ7x6yFOJklBER2UNBtbXO/nM0wFQ6ySwQXMS+u 3hNPdXrpW6ERj4zSOyoVJcz8A+gmj19QbC9Jiy2Hp3986Iv4t0+nizMJTbUGwSVxQUWc rm3g== X-Forwarded-Encrypted: i=1; AJvYcCUWVQ7qYWDyoT6BOIlryELANjgGNtExUVEadj9SFsRE+Upw5cD/F/5p5bkotPF8SyTF8C5INgW0MI4qipWI9cWb8Q==@vger.kernel.org, AJvYcCWzv4RX5UTxidxU7GLZz0uqekfkgFNlUlbQwDnzd60JthfxQGKtp/EJD8v4uwmQq1WnMwKPOf0j9shsRJk=@vger.kernel.org X-Gm-Message-State: AOJu0YxhjJ22YDbtBk1P5J4Rml0fgwLovxIUeylA8Kq1fz4Bz7TQ9+YN IWm7FyQDQIdJcAsFslZDrAuOix+n1p1Og1GzdFJ7Eh1j0XbVO5Ob9bDxH83pAcE= X-Gm-Gg: ASbGnctV8W5z0i0Is4pthg9GCMECmNVMZb2vrJNiYiryKczzoZ2KLXvYb4wYz+gWHUC vwWJu4Wt8nVig/P9wHDHR7I9mu8Wa1Hsnq49HVQ5o9GiWi/gjdBPY1BrEmS7S6MNhzLngsjqtHC feP77bVmsWeu6x88UsBcpF2qeorAy0sMAJMdNHdhbkwi587U4eVs0DvWTQVnsE4qdlq5ViKacEx bn87VTIwXRQLkUiHjuqOydIzgi0CICKZEe7t/6FJyhScFVltVOLgEtupiI2hPBimgR42K3/akWi zsBdqe4lMwfvFbjLvCJkjLofaISwA8+j6VkxAxDfrPlAWJ1e4bBSeEvxb2f1T4z4rxnsynMzk4B eCtBpcwDvYg== X-Google-Smtp-Source: AGHT+IHUl/RIG+xtTeDx0MHsPJ86xGEI/v/ocRMn1FSmJ5Z/rhqw/mhisX97wA5jyrbu7YM47GRV2Q== X-Received: by 2002:a05:6a00:2189:b0:739:56c2:b661 with SMTP id d2e1a72fcca58-740491eb4fdmr1233728b3a.12.1746066509458; Wed, 30 Apr 2025 19:28:29 -0700 (PDT) Received: from howard-ubuntu.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a8fbe7sm2459599b3a.160.2025.04.30.19.28.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Apr 2025 19:28:29 -0700 (PDT) From: Howard Chu To: acme@kernel.org Cc: mingo@redhat.com, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, peterz@infradead.org, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, gautam@linux.ibm.com, Howard Chu Subject: [PATCH v16 10/10] perf test: Add direct off-cpu tests Date: Wed, 30 Apr 2025 19:28:08 -0700 Message-ID: <20250501022809.449767-11-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250501022809.449767-1-howardchu95@gmail.com> References: <20250501022809.449767-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since we added --off-cpu-thresh, add tests for when a sample's off-cpu time is above the threshold, and when it's below the threshold. Note that the basic test performed in test_offcpu_basic() collects a direct sample now, since sleep 1 has duration of 1000ms, higher than the default value of --off-cpu-thresh of 500ms, resulting in a direct sample. An example: $ sudo perf test offcpu 124: perf record offcpu profiling tests : Ok Tested-by: Gautam Menghani Acked-by: Namhyung Kim Reviewed-by: Ian Rogers Suggested-by: Namhyung Kim Signed-off-by: Howard Chu Tested-by: Ian Rogers --- tools/perf/tests/shell/record_offcpu.sh | 71 +++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/tools/perf/tests/shell/record_offcpu.sh b/tools/perf/tests/she= ll/record_offcpu.sh index 678947fe69ee..21a22efe08f5 100755 --- a/tools/perf/tests/shell/record_offcpu.sh +++ b/tools/perf/tests/shell/record_offcpu.sh @@ -7,6 +7,9 @@ set -e err=3D0 perfdata=3D$(mktemp /tmp/__perf_test.perf.data.XXXXX) =20 +ts=3D$(printf "%u" $((~0 << 32))) # OFF_CPU_TIMESTAMP +dummy_timestamp=3D${ts%???} # remove the last 3 digits to match perf script + cleanup() { rm -f ${perfdata} rm -f ${perfdata}.old @@ -19,6 +22,9 @@ trap_cleanup() { } trap trap_cleanup EXIT TERM INT =20 +test_above_thresh=3D"Threshold test (above threshold)" +test_below_thresh=3D"Threshold test (below threshold)" + test_offcpu_priv() { echo "Checking off-cpu privilege" =20 @@ -88,6 +94,63 @@ test_offcpu_child() { echo "Child task off-cpu test [Success]" } =20 +# task blocks longer than the --off-cpu-thresh, perf should collect a dire= ct sample +test_offcpu_above_thresh() { + echo "${test_above_thresh}" + + # collect direct off-cpu samples for tasks blocked for more than 999ms + if ! perf record -e dummy --off-cpu --off-cpu-thresh 999 -o ${perfdata} = -- sleep 1 2> /dev/null + then + echo "${test_above_thresh} [Failed record]" + err=3D1 + return + fi + # direct sample's timestamp should be lower than the dummy_timestamp of = the at-the-end sample + # check if a direct sample exists + if ! perf script --time "0, ${dummy_timestamp}" -i ${perfdata} -F event = | grep -q "offcpu-time" + then + echo "${test_above_thresh} [Failed missing direct samples]" + err=3D1 + return + fi + # there should only be one direct sample, and its period should be highe= r than off-cpu-thresh + if ! perf script --time "0, ${dummy_timestamp}" -i ${perfdata} -F period= | \ + awk '{ if (int($1) > 999000000) exit 0; else exit 1; }' + then + echo "${test_above_thresh} [Failed off-cpu time too short]" + err=3D1 + return + fi + echo "${test_above_thresh} [Success]" +} + +# task blocks shorter than the --off-cpu-thresh, perf should collect an at= -the-end sample +test_offcpu_below_thresh() { + echo "${test_below_thresh}" + + # collect direct off-cpu samples for tasks blocked for more than 1.2s + if ! perf record -e dummy --off-cpu --off-cpu-thresh 1200 -o ${perfdata}= -- sleep 1 2> /dev/null + then + echo "${test_below_thresh} [Failed record]" + err=3D1 + return + fi + # see if there's an at-the-end sample + if ! perf script --time "${dummy_timestamp}," -i ${perfdata} -F event | = grep -q 'offcpu-time' + then + echo "${test_below_thresh} [Failed at-the-end samples cannot be found]" + err=3D1 + return + fi + # plus there shouldn't be any direct samples + if perf script --time "0, ${dummy_timestamp}" -i ${perfdata} -F event | = grep -q 'offcpu-time' + then + echo "${test_below_thresh} [Failed direct samples are found when they = shouldn't be]" + err=3D1 + return + fi + echo "${test_below_thresh} [Success]" +} =20 test_offcpu_priv =20 @@ -99,5 +162,13 @@ if [ $err =3D 0 ]; then test_offcpu_child fi =20 +if [ $err =3D 0 ]; then + test_offcpu_above_thresh +fi + +if [ $err =3D 0 ]; then + test_offcpu_below_thresh +fi + cleanup exit $err --=20 2.45.2