From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 003A022D7A0; Thu, 13 Feb 2025 23:00:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487616; cv=none; b=ixa9q3nGjF6guhrML9Uo3/UmIJkZR8ZP/KFOXxJISPCgIKM8ea9xU3B2S8zZbkYfVKd7J3vEW8n4RF4ceEfKg7q06thk24RMNFWSvwP4cglC7pmV4YUeoFFuM72Q33r7icQEhcFHZixpHl3MinnmdtKPfHvqFcrwTW2ZcULAZVs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487616; c=relaxed/simple; bh=xud1vkpx5QBShBiEAnxqUAouFYf1iqY9potr05uwQ48=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HzbHYLICHPeyFuGitOK49mlsZz+3KKpHawlM61pOzRjjZPiIOXPoPLIWDuzZr9emFhLCcOipdTJj7kjAUKZlzEuRjLwOnqHSajv9kgQDLJAkTCfnzmmRd5s43vOeH7APZ+YircINk0rC+qlacQACgJNSGhX13lV8dYbM5fmBxMY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=B964/z8H; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B964/z8H" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-220d39a5627so20179235ad.1; Thu, 13 Feb 2025 15:00:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487614; x=1740092414; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DHmbob/EuNLj2y3XD/x7wmk2Skfmv18lfRqvGZo51aQ=; b=B964/z8HDPGTsOgWLIIpmlXxhbpzHfWpueNxf6/sGL+Sudr71lG06Q45FqcLGbkUiw HJF7JbwotMk7a78F05zpvlk3XhAwJS093gHYVfi3jmmY33wbxTLFyVimq2KXS1WeqehA +PopZQQ/noNrhi1P+xMEppE99ehtSdJ7J4dULz2UTdnQTSJMEWuCf2bNe410Yy/cCQ3f eBPDz4Ry8nPAmmrWwy9Bat6zYfmoaQaq2EmQPHXaEE+B6hD1wKDJnjm5KMt8QFkvAPyW QJ98MW2VzR0Ywm4uxghsF7FZ5CuXj5CUWtNMT7Rub8/fq8lYBh2GK0sPmsGPB+0x81AO iVNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487614; x=1740092414; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DHmbob/EuNLj2y3XD/x7wmk2Skfmv18lfRqvGZo51aQ=; b=sdFiAjdvuNYiA6mD4Yuq6tcLCszUaxjfyTY0E9SdqsJ/lE8HhtdHWl/GY9Ukpekoct QmdwdPtCuqc/Nd69Otr2/gaKansscYgoWySKADncMapqapGyanMqQ+/axn6fUCyBpvRd /2q9ztaFCN2IxSsYXHA8LseQM3oaxThCsnC6Vly9NLfPjiiWIzWh9PpoZDv40wAAGkwp tkBnUXEk+y2XTOm7b+W5j5zO8puTSskFdXIiV2s3TExaP/b9PP4txKpsJdDmJSWFTock 5C5L9rRGg5VrPKbE5XFeDYiHt5ZA9pTI0HKIxRC2oZkWWxVwelhDz/tm/rgbYYu5yttv xO4w== X-Forwarded-Encrypted: i=1; AJvYcCVP6Qe0a7zBcCwITMJSe1+nBCFdzdHC593NeeIVeLdf+QHHjNeIm/wn6PB+EIEMGUfOXqEPgeWWKSM/cN8JfeU6qQ==@vger.kernel.org, AJvYcCXl4Yz7KkFv5SHjftffE1qI9wWiUj9GnDIfWZYskqFSpMl5ARkoo9r4GWNoqPV56+OaOW7CXevZxp+XYYk=@vger.kernel.org X-Gm-Message-State: AOJu0YypC9WjxuoR91shRQiuietnMjlbJUtuXLejjvC0c8MrenV2FYdt +C672B3bhx5wZfgtFo2qhJvzmt85U8CrW+JNkZ0volfdrH3zNAIt X-Gm-Gg: ASbGncvfkv1cjNZTkebx24uqc/pdQbdSxx8vN1B/pXKI/7b5vjeNSRAEVyK9JghnVrD 80UqxfwgLUS1Txj4GxC0a2yGTCy5CpvsALMCnejDD15EIauEnDNw1w1L7UgLmxoex4dty4siZga bBlkSORlyZWCS2Xs7onyP3CvJ2BEJupnMvrCPp2CIwTLeB5J83mWS1NObNFgSHdbKFrHmDWWPTm 7X5lQRm+Mttf50lMyudV/liC7Nbf7Mv0c0TAakX6c57mnRzafB86md7+Gc6x1hJ3gwLjtS8zyt1 jfNuEBHmJ1Bb3enF4NEICwRxhWpJ/e0LvkLG9NQNZ3629TU= X-Google-Smtp-Source: AGHT+IE6kXadUoKzU/LoFKYjA6Imroxpr8JrHoRV6BPSzyCVoPVGQjJmiB9t+qktmg9gyHdRNzCbxQ== X-Received: by 2002:a17:903:124e:b0:212:996:3536 with SMTP id d9443c01a7336-220bdedcf55mr148246935ad.10.1739487614257; Thu, 13 Feb 2025 15:00:14 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:14 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra , Arnaldo Carvalho de Melo Subject: [PATCH v15 01/10] perf evsel: Expose evsel__is_offcpu_event() for future use Date: Thu, 13 Feb 2025 15:00:00 -0800 Message-ID: <20250213230009.1450907-2-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose evsel__is_offcpu_event() so it can be used in off_cpu_config(), evsel__parse_sample(), and perf script. Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-3-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/util/evsel.c | 2 +- tools/perf/util/evsel.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index 4a0ef095db92..d5519ab25996 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -1261,7 +1261,7 @@ static void evsel__set_default_freq_period(struct rec= ord_opts *opts, } } =20 -static bool evsel__is_offcpu_event(struct evsel *evsel) +bool evsel__is_offcpu_event(struct evsel *evsel) { return evsel__is_bpf_output(evsel) && evsel__name_is(evsel, OFFCPU_EVENT); } diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h index aae431d63d64..e58de60210a4 100644 --- a/tools/perf/util/evsel.h +++ b/tools/perf/util/evsel.h @@ -557,4 +557,6 @@ u64 evsel__bitfield_swap_branch_flags(u64 value); void evsel__set_config_if_unset(struct perf_pmu *pmu, struct evsel *evsel, const char *config_name, u64 val); =20 +bool evsel__is_offcpu_event(struct evsel *evsel); + #endif /* __PERF_EVSEL_H */ --=20 2.45.2 From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CD3322D7AE; Thu, 13 Feb 2025 23:00:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487619; cv=none; b=lwqq2/gH/KzzM/O5iRDERxSB8WK3LXnursbtDWktmKtWbYTbrjHR92+MS4lTZ6BpDEgJYyb7pN0H9oGdSPv/BXRlwIG3UY+D9UII5jHoZzDUNUfYFKgJYVuspXY3RgiDnqflU7LA5XKvf3OchZcwLVCjgcRoWnnh4Cp5wxcsBSA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487619; c=relaxed/simple; bh=+LIyydtsbkVbytSVWuF0ihlrP6GR1uKO1gkergpXWcs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZGaP6CXJXsA14DtKE/rXv76FIwOnpvrSygBwPUxrOarzIEGq41j929KJEe1D7Qy90QlttKB84zFBv5P1qhKQIMQKn+w6wOCem5fCLC3+CrrosOLALNVDpbCQoblTKZN2b+QlH5XXhoJ18zm14pQ5nbiNo3JJQxe9b9BWaUIDPgs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bieEvJPv; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bieEvJPv" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-220c8f38febso25329225ad.2; Thu, 13 Feb 2025 15:00:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487616; x=1740092416; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U/cB+5rb2VYgNBJmJ429bfHfSZADSnkjTZQmABHfwMw=; b=bieEvJPvb5qUs++EVOeZYkYYB7WGy62g2aCRF+/z8ZI+GDz0YNoVJqkjZm39Axap8u YUclvREBOwNVpoF8L3mAV9dGz0L0Pk4xKHPRbg4cxS6TKGJ9CIoOTsly6/QWTusfXUWE 3j22lDnQyP2A7UvOqRq2bzk4TVpaa6RWmRtwU+E3vz5xRGJKceGdn8rWXzRDeU6HFwXz b6yXcYquM1DodGw/TfqHrcOxRj7KLbNhKAXUgtnGuSaVEG2YVBX4JTMQaaDsZ4KGj6Ck PnWNORelBf/gf0DIiUb6VHx2q1WUYl/10ItExZEvqJcQz71bjisc3PJpwjyOmRJ1EBOB Gdiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487616; x=1740092416; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U/cB+5rb2VYgNBJmJ429bfHfSZADSnkjTZQmABHfwMw=; b=D8Phh+LwcxVUJxAW/A6KF2RQOsQBCgE08WqeKBHf6V/D3+7v2dOE0f0Hjlkn5P1wzS dEJtddDERDXcZo3GK/zQji/k0uwajdIb/OQ/Jr6LUz3cQIbVEhYV5Vo8Isfw8PeblsxZ 8XU4t3ph4XsHgDtwJFlg+WvgDl4737NA0x60RoHiptEMIJE4aAX9Q1RsYyLELWd7nCqJ 3Tynhk6MbrhRJu4bzo6YWx8+bXhHgg7Ry3p5w3f0nr8o8NZRwPP41kElavhH21qnYaKP tx35+QddVIsmDsNVBWmop8NQ5vr3vpF19HxRq0e+yHBxVwkhCQ1/5E6twrhg0TyZO+3v vi/Q== X-Forwarded-Encrypted: i=1; AJvYcCU1AfHLw5Gz7lMWYcy1JmjdjYRU1vg0tCwRdtGh7n4uUlmUDQBICsbkhJI+mK1E68RndBXb7UIfqVcKNDQLetvfgw==@vger.kernel.org, AJvYcCXziQuDuViw6iVatgCeZz09sdr3hE5hOIlDzV2RcIMuaaQPQBfaq1I/dabXxIpaLeRhBfOpdQJKZCnB1S8=@vger.kernel.org X-Gm-Message-State: AOJu0YztJweqK3ta+u/iMDzkFoIIl1XcLqvkgw+kdtaTgE77fdacJ5K+ H2jxZVE4bLNJNGKSJxrHF+e0ckjPoiHOJ30oEC9nJEEAPvZrOEZ3 X-Gm-Gg: ASbGncuuHJYdtOrogkvRaoXkdDRy0wC9BBcw+XkilmLOoazyMjai21AvLIVpibDMb8g LHQLJMaNbKLj4j0hxJtDKKqBq3fmYwyIYW9PxjXQ9geADXtLabA6Vb2t7F2Kmecl10JqE2qfhKO V4fZRVHAYt4tj5Jfuu0Q78RhljWc8ewILuLomeAOqLqOCUO1H8PuGxGqj8q8BMbYJH6ODTJqih1 6UUEhUP3t26fDINKQ6USV7UcwXliX0DjhRxdx7l5YRytJQRUf311Q5uJkP+OxCNWW0WVe6uyRUN FSimfK7FUVvZbrqtLiEcMKWHJJtgxW1sMPuo4t4sX975yNY= X-Google-Smtp-Source: AGHT+IG9wYfRSE14QBm/ju7dOQfI+9wjtovE5c2SVC5nlKZUWruxX4FmqvEpobUPYslbKNttbXZwQg== X-Received: by 2002:a17:902:d481:b0:215:89a0:416f with SMTP id d9443c01a7336-220bdf5f556mr124533075ad.30.1739487615701; Thu, 13 Feb 2025 15:00:15 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:15 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra , Arnaldo Carvalho de Melo Subject: [PATCH v15 02/10] perf record --off-cpu: Parse off-cpu event Date: Thu, 13 Feb 2025 15:00:01 -0800 Message-ID: <20250213230009.1450907-3-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Parse the off-cpu event using parse_event(), as bpf-output. Call evlist__enable_evsel() on off-cpu event. This fixes the inability to collect direct off-cpu samples on a workload, as reported by Arnaldo Carvalho de Melo . The reason being, workload sets enable_on_exec instead of calling evlist__enable(), but off-cpu event does not attach to an executable and execve won't be called, so the fds from perf_event_open() are not enabled. no-inherit should be set to 1, here's the reason: We update the BPF perf_event map for direct off-cpu sample dumping (in following patches), it executes as follows: bpf_map_update_value() bpf_fd_array_map_update_elem() perf_event_fd_array_get_ptr() perf_event_read_local() In perf_event_read_local(), there is: int perf_event_read_local(struct perf_event *event, u64 *value, u64 *enabled, u64 *running) { ... /* * It must not be an event with inherit set, we cannot read * all child counters from atomic context. */ if (event->attr.inherit) { ret =3D -EOPNOTSUPP; goto out; } Which means no-inherit has to be true for updating the BPF perf_event map. Moreover, for bpf-output events, we primarily want a system-wide event instead of a per-task event. The reason is that in BPF's bpf_perf_event_output(), BPF uses the CPU index to retrieve the perf_event file descriptor it outputs to. Making a bpf-output event system-wide naturally satisfies this requirement by mapping CPU appropriately. Suggested-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-4-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/builtin-record.c | 7 +++++++ tools/perf/util/bpf_off_cpu.c | 33 +++++++++++---------------------- tools/perf/util/evsel.c | 4 +++- 3 files changed, 21 insertions(+), 23 deletions(-) diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index cda7e6a7b45d..f3e5f856f4a4 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -2564,6 +2564,13 @@ static int __cmd_record(struct record *rec, int argc= , const char **argv) if (!target__none(&opts->target) && !opts->target.initial_delay) evlist__enable(rec->evlist); =20 + /* + * offcpu-time does not call execve, so enable_on_exe wouldn't work + * when recording a workload, do it manually + */ + if (rec->off_cpu) + evlist__enable_evsel(rec->evlist, (char *)OFFCPU_EVENT); + /* * Let the child rip */ diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index 4269b41d1771..2101aa2b7c42 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -38,32 +38,21 @@ union off_cpu_data { =20 static int off_cpu_config(struct evlist *evlist) { + char off_cpu_event[64]; struct evsel *evsel; - struct perf_event_attr attr =3D { - .type =3D PERF_TYPE_SOFTWARE, - .config =3D PERF_COUNT_SW_BPF_OUTPUT, - .size =3D sizeof(attr), /* to capture ABI version */ - }; - char *evname =3D strdup(OFFCPU_EVENT); - - if (evname =3D=3D NULL) - return -ENOMEM; =20 - evsel =3D evsel__new(&attr); - if (!evsel) { - free(evname); - return -ENOMEM; + scnprintf(off_cpu_event, sizeof(off_cpu_event), "bpf-output/name=3D%s/", = OFFCPU_EVENT); + if (parse_event(evlist, off_cpu_event)) { + pr_err("Failed to open off-cpu event\n"); + return -1; } =20 - evsel->core.attr.freq =3D 1; - evsel->core.attr.sample_period =3D 1; - /* off-cpu analysis depends on stack trace */ - evsel->core.attr.sample_type =3D PERF_SAMPLE_CALLCHAIN; - - evlist__add(evlist, evsel); - - free(evsel->name); - evsel->name =3D evname; + evlist__for_each_entry(evlist, evsel) { + if (evsel__is_offcpu_event(evsel)) { + evsel->core.system_wide =3D true; + break; + } + } =20 return 0; } diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index d5519ab25996..f45d4b44d70d 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -1540,8 +1540,10 @@ void evsel__config(struct evsel *evsel, struct recor= d_opts *opts, if (evsel__is_dummy_event(evsel)) evsel__reset_sample_bit(evsel, BRANCH_STACK); =20 - if (evsel__is_offcpu_event(evsel)) + if (evsel__is_offcpu_event(evsel)) { evsel->core.attr.sample_type &=3D OFFCPU_SAMPLE_TYPES; + attr->inherit =3D 0; + } =20 arch__post_evsel_config(evsel, attr); } --=20 2.45.2 From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F04F22D7B2; Thu, 13 Feb 2025 23:00:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487620; cv=none; b=aChRDqi+is3BqTniASlaXpzNH4zFBlcDjGPzJLYWO0xUcsf+U8VHfpRwXca67ho2af3Jxl3K4aACWdG/3Ua8K/OwCkVKgg00fz6lCNOM1Nd0URYgLAuQ4qMLlekRLMxbgy+bN7er7dRspZg/r1RLYnpz1ITbOOUNSG3474awjzM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487620; c=relaxed/simple; bh=B7pJ83661aYJVk1jhA2ZmEebrYQLC7i67R/mekIQ7/o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rTI4YKyBKOKHsf2ZhkQSjfQvntVB52VV81KS/7y8/xaT0Hg/7oNtPtp/MGY77b1y3FViLm7c2sdRy4fUd6fBeLjMsQToeXBRC2Ac7WvKvlmwyHuMuewGzfytyf/CwSLZIGhiJcrZpKTExw7p61MgYDuUwyZfAaOp5N3aaRin4ls= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JZVcOijp; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JZVcOijp" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-21f5660c2fdso29103295ad.2; Thu, 13 Feb 2025 15:00:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487617; x=1740092417; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IcIDgG6uFwVdSQFSE2lqNj2d3UfP3qCKN5H2yvCiHY8=; b=JZVcOijpJAxJtitZt7KkYaCWx9A16DKgiBQ8fwJjOJjyVRbccRzjXon/V6rs8X821F 387lbzQFCrEWEvz48rOCuC9YzAXtS/kAIiHMbJEB8E4L6vBhs60N126IFboiUlYDMP5y jNVKp+oqa9JOmUs/DwxaDcnyXF+3zaV2t57dG3b7zWskbsenLeZ01z5ShbXcpSuq3dHA 7jwPKWMvzQkr+qsVeTOjy0jK5EeXGJzOkugHAQ7IAPN0qXFM78b4yd5H9j4UlDK7TTYQ 16kwQwEZPig/uUI66N6EH1tDyXwROvQQHNtnYmUsVP3ALI2XizVgaJuptpt9j5HWuBae P63Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487617; x=1740092417; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IcIDgG6uFwVdSQFSE2lqNj2d3UfP3qCKN5H2yvCiHY8=; b=rj56koEr0miSzxe1tLXUWr0IcYBFiADtl9OXA26lzWnNPcavg3r4AX0Y4ZNPZvpgh/ xIVbhFA4nRlAzlusP8yPsZrGzr4pblQHqinY9QvUyWAXuQbMZGGm6D2nDxUqqEenVAKV QA3ptob+a4odCng977e8thAyKQveKSm9ISyg4jkxWUHX08v3DOfPogIf4gw3ZDw3MsQa IcvT8+vGTFSfdGZ120TWQmg2VCT2vcY+AZ+fD5ZAQZJXMKUvjPtjlvXDSIsG1iqnPMZG EGPAKPHTLIH29Hw6FRwCs9zdqj6qz7i2QF0MKx8MSrqjG+Gkh3HpjF49iez/mKvrmMmW CDBg== X-Forwarded-Encrypted: i=1; AJvYcCUCc+4Lg2rcuZY2vbgKbaw4Iula1YcFx4iFqDOk5V3viN3jiDMbGu2uLoFvKqmepq0Qce7QO44IsoizYAw=@vger.kernel.org, AJvYcCX/5OEvRXVNHuWjZFTyQyWY2k9gf3REKuY7QJMCkVCX8btY65p0CUuLjeRO70rVZIN4zy2SkodYOIRPi+7r8VTelQ==@vger.kernel.org X-Gm-Message-State: AOJu0YwdkOV+ioVGNB0qhOZ6cNvfRi4qLTnhEmoG4kynsd1Dm9r5RT5l 9WsZbJzDwE9lgCWqAcgdq9eJBcBVBK/iID+PDleKOK8zhSK3Npjj X-Gm-Gg: ASbGncvpwHAnGwBJnvZEA/5ZujnGR47qk5DEBT/K4H88UwmpLmFkhWsruxkFook5v2n aRxp5+QEoQJ3kElcePgy8Qx8eKox9b+2EDRyOx2LkOCcRX4ExSjQaUq0EYOmO7uKoNv1ZVis2AO kSG1XmFBecw69+lT4EkxaIU9H3j39OK1Og8lOOQoLV64lDil4KqRCqxyk3Ja0t+kd4jyoFYXFFZ XfuR8DncFlZOInV4sebkkLiAbC2Db8u8+QNTs/DLFSHZcGnlHkmYqpYWAsME0Dvu/USoK+6zEPk PgTf9gmNmc6QqdpayraBTI3ugju8t17yquASZyx1z09tpTo= X-Google-Smtp-Source: AGHT+IGL/ijyFjjZMK5uS6WK7zyW0nMkN/WJlEJdDy/BS2iFi4C+qqMS96u38AnPURIGvF9F+9xOtw== X-Received: by 2002:a17:902:d508:b0:215:e685:fa25 with SMTP id d9443c01a7336-220d1eddf52mr74858945ad.20.1739487617191; Thu, 13 Feb 2025 15:00:17 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:16 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra , Arnaldo Carvalho de Melo Subject: [PATCH v15 03/10] perf record --off-cpu: Preparation of off-cpu BPF program Date: Thu, 13 Feb 2025 15:00:02 -0800 Message-ID: <20250213230009.1450907-4-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set the perf_event map in BPF for dumping off-cpu samples. Set the offcpu_thresh to specify the threshold. Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-5-howardchu95@gmail.= com [ Added some missing iteration variables to off_cpu_config() and fixed up a manually edited patch hunk line boundary line ] Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/util/bpf_off_cpu.c | 23 +++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 9 +++++++++ 2 files changed, 32 insertions(+) diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index 2101aa2b7c42..de71ff7a80d0 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -13,6 +13,7 @@ #include "util/cgroup.h" #include "util/strlist.h" #include +#include =20 #include "bpf_skel/off_cpu.skel.h" =20 @@ -60,6 +61,9 @@ static int off_cpu_config(struct evlist *evlist) static void off_cpu_start(void *arg) { struct evlist *evlist =3D arg; + struct evsel *evsel; + struct perf_cpu pcpu; + int i; =20 /* update task filter for the given workload */ if (skel->rodata->has_task && skel->rodata->uses_tgid && @@ -73,6 +77,25 @@ static void off_cpu_start(void *arg) bpf_map_update_elem(fd, &pid, &val, BPF_ANY); } =20 + /* update BPF perf_event map */ + evsel =3D evlist__find_evsel_by_str(evlist, OFFCPU_EVENT); + if (evsel =3D=3D NULL) { + pr_err("%s evsel not found\n", OFFCPU_EVENT); + return; + } + + perf_cpu_map__for_each_cpu(pcpu, i, evsel->core.cpus) { + int err; + + err =3D bpf_map__update_elem(skel->maps.offcpu_output, &pcpu.cpu, sizeof= (__u32), + xyarray__entry(evsel->core.fd, i, 0), + sizeof(__u32), BPF_ANY); + if (err) { + pr_err("Failed to update perf event map for direct off-cpu dumping\n"); + return; + } + } + skel->bss->enabled =3D 1; } =20 diff --git a/tools/perf/util/bpf_skel/off_cpu.bpf.c b/tools/perf/util/bpf_s= kel/off_cpu.bpf.c index c152116df72f..1cdd4d63ea92 100644 --- a/tools/perf/util/bpf_skel/off_cpu.bpf.c +++ b/tools/perf/util/bpf_skel/off_cpu.bpf.c @@ -18,6 +18,8 @@ #define MAX_STACKS 32 #define MAX_ENTRIES 102400 =20 +#define MAX_CPUS 4096 + struct tstamp_data { __u32 stack_id; __u32 state; @@ -39,6 +41,13 @@ struct { __uint(max_entries, MAX_ENTRIES); } stacks SEC(".maps"); =20 +struct { + __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(__u32)); + __uint(max_entries, MAX_CPUS); +} offcpu_output SEC(".maps"); + struct { __uint(type, BPF_MAP_TYPE_TASK_STORAGE); __uint(map_flags, BPF_F_NO_PREALLOC); --=20 2.45.2 From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08A45245B01; Thu, 13 Feb 2025 23:00:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487621; cv=none; b=I+zeJWlWZSA3DpB7Z1ksRA6MthSrtSjnI3MWoETZKZbwvxDC5g0eGsxtBHaEVMOIastbhCHTsFgwJ7qmdXySEq4ZAHvUC+a9lhWpNtI5mXNd3lXv+RBAdzxqNzdhuuCdyispcTJjgYWIyxVYrL/FmkIHqtPps233pvXeK8RsNd0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487621; c=relaxed/simple; bh=zkA9NoqxuTJ9143G+fHdqorubFcg2Pyh1R5efL11ql8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j954x1EP88B/Qwwwl5VoEo7Ui0fuAJI5wOa68/KT4OwqTY5PijkYa9mL5AvjueWZLiH5iK+QR/Fjg+9uty8GsA/uJoki1B1SpcEGj6wjqUoJJmGFyhA3kNYyxe8i/wyFE0LAeYm5RHyzI9TFFO+r1EBs8vNGeZNEibNGRSFhO0o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=R1Qz/JS0; arc=none smtp.client-ip=209.85.216.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="R1Qz/JS0" Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-2fbfc9ff0b9so2343411a91.2; Thu, 13 Feb 2025 15:00:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487619; x=1740092419; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7omCjmTFle31Ye/Qq9MO6Xsp5wPVSFLbffxm3Uj/s40=; b=R1Qz/JS0t+crKKK15BPl/sSr4C0NSEVJY+4ktxheW+lDVgqC8jJlPIz7E3989oGete MENS3nh+Bg+sZ+yV9c56G01pwTYi3+n9a33gVW6b0fWwx1PhhXNq0d59hxar4ocW5Dyl s2rmEJ4DYXkYECbU3dbaUl9YRAMzftqvSYt5n9AViV2J2qGWyZhIGW+v1GJJsSY/ZYbb yAm+aODbufleDtT3tUDHzc4NvsdqKU9HTDdGcuYjV89jsafTSzAgUzNYlqZY9Qtj2k3x 3Dinr7SSU20/UIO+ti7b3XZp5P+2nHMoi2iZ27Z075e6c45pq+1FlTUr0dUqmwUJENev r/rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487619; x=1740092419; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7omCjmTFle31Ye/Qq9MO6Xsp5wPVSFLbffxm3Uj/s40=; b=LnUdBlMVKLYXiNQfwGZ+DuNl9MdlLEbo2h8kiAxUKk3oYDnpXUhkNnb/QT2FbYgWLr a/QxyJCWjx4A8tbIDACkSUW716xvMir/+RAyNABgEzIOP3tLpw8GhdoOZFAeHZmC02fh 0aPywmUVX5u/9buXLIsMr67dw5Ju+reONBFzWiHOW9xrAi4sXgmYoLqYtYlVAcOxVZQQ ujf9PDrqZ38U+bxAZMZHMCLpGWCCWq2Srg8xXOBkzb8mRk44eCR1sgsea0MYJ5RGiL8k 7j9lN9Rfm6V6FZhMj9h/IgLHicLb8lL2mlH+PTgqhXIpyiEPoQyGBL86k9+AidIQV6MK mn5g== X-Forwarded-Encrypted: i=1; AJvYcCWw8FgHVbPykzHi4n8e7Ge91waCBG0n89zNndnhB1tU9iGLg7xLlAyH+Oc47QoLLwD4RIkya/IGsezACpc=@vger.kernel.org, AJvYcCX9ek+VDbnVCQfeZZgeyl1mJogSdseVUMmMARmxV9GcMiDfTNq2Wir/8ygDN1I59OVQbpLx/cZcf61yYmBJFXhY7A==@vger.kernel.org X-Gm-Message-State: AOJu0YwbilXCZdTCQfSmyr2zKc7qDvdKoqxkDXoSKzK16gD5gB9FvqiO XmLJAYhH7LAuyQGNKg8A1SugKf7j9nphKlRdFjjKd0awUc0ZYF5A X-Gm-Gg: ASbGncshxwn9YGRqMHqNSa/vFpFzaUXSv05kohAn7974FqjWY/vS77NC4GsvT1VtUl3 8meA0qcz/Sqh0WXfC4KVJb0zyQcQQ9AJkcCirbGGVMIYdA98czxsWXm4/X6pjucisP5eeKtqrOX NPxTon1xK1w7O7vlgQ12WSG2s3RzKAOUXDFcTAIEFQPhnYhX60WhfDDRP1fYJjhTN4VYujsQqzA uUoonVc2FxAmePq0qi4uxmJYl1GEGVTrE816EmCxrEJ7v9YM8bttNx2EyTX4Sl2Y/XCQlGT0Q83 UXtnj2u7Q3ojD3Wyj2D4kktUv7MacIUaihTWchPsATuoD98= X-Google-Smtp-Source: AGHT+IHQkqj2tuGm2/iP1In6kDFCvUkCoGVTR8ApKhEJwPXCq9/+A/FrzymyX6JtyclRG3ZO0r6Vkw== X-Received: by 2002:a17:90b:38c6:b0:2ee:f80c:6884 with SMTP id 98e67ed59e1d1-2fc0f0db50dmr7986893a91.33.1739487618650; Thu, 13 Feb 2025 15:00:18 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:18 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra , Arnaldo Carvalho de Melo Subject: [PATCH v15 04/10] perf record --off-cpu: Dump off-cpu samples in BPF Date: Thu, 13 Feb 2025 15:00:03 -0800 Message-ID: <20250213230009.1450907-5-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Collect tid, period, callchain, and cgroup id and dump them when off-cpu time threshold is reached. We don't collect the off-cpu time twice (the delta), it's either in direct samples, or accumulated samples that are dumped at the end of perf.data. Suggested-by: Namhyung Kim Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-6-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/util/bpf_skel/off_cpu.bpf.c | 89 ++++++++++++++++++++++++-- 1 file changed, 84 insertions(+), 5 deletions(-) diff --git a/tools/perf/util/bpf_skel/off_cpu.bpf.c b/tools/perf/util/bpf_s= kel/off_cpu.bpf.c index 1cdd4d63ea92..c15b69586723 100644 --- a/tools/perf/util/bpf_skel/off_cpu.bpf.c +++ b/tools/perf/util/bpf_skel/off_cpu.bpf.c @@ -19,11 +19,18 @@ #define MAX_ENTRIES 102400 =20 #define MAX_CPUS 4096 +#define MAX_OFFCPU_LEN 37 + +// We have a 'struct stack' in vmlinux.h when building with GEN_VMLINUX_H= =3D1 +struct __stack { + u64 array[MAX_STACKS]; +}; =20 struct tstamp_data { __u32 stack_id; __u32 state; __u64 timestamp; + struct __stack stack; }; =20 struct offcpu_key { @@ -41,6 +48,10 @@ struct { __uint(max_entries, MAX_ENTRIES); } stacks SEC(".maps"); =20 +struct offcpu_data { + u64 array[MAX_OFFCPU_LEN]; +}; + struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); __uint(key_size, sizeof(__u32)); @@ -48,6 +59,13 @@ struct { __uint(max_entries, MAX_CPUS); } offcpu_output SEC(".maps"); =20 +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(struct offcpu_data)); + __uint(max_entries, 1); +} offcpu_payload SEC(".maps"); + struct { __uint(type, BPF_MAP_TYPE_TASK_STORAGE); __uint(map_flags, BPF_F_NO_PREALLOC); @@ -106,6 +124,8 @@ const volatile bool uses_cgroup_v1 =3D false; =20 int perf_subsys_id =3D -1; =20 +__u64 offcpu_thresh_ns =3D 500000000ull; + /* * Old kernel used to call it task_struct->state and now it's '__state'. * Use BPF CO-RE "ignored suffix rule" to deal with it like below: @@ -192,6 +212,47 @@ static inline int can_record(struct task_struct *t, in= t state) return 1; } =20 +static inline int copy_stack(struct __stack *from, struct offcpu_data *to,= int n) +{ + int len =3D 0; + + for (int i =3D 0; i < MAX_STACKS && from->array[i]; ++i, ++len) + to->array[n + 2 + i] =3D from->array[i]; + + return len; +} + +/** + * off_cpu_dump - dump off-cpu samples to ring buffer + * @data: payload for dumping off-cpu samples + * @key: off-cpu data + * @stack: stack trace of the task before being scheduled out + * + * If the threshold of off-cpu time is reached, acquire tid, period, callc= hain, and cgroup id + * information of the task, and dump it as a raw sample to perf ring buffer + */ +static int off_cpu_dump(void *ctx, struct offcpu_data *data, struct offcpu= _key *key, + struct __stack *stack, __u64 delta) +{ + int n =3D 0, len =3D 0; + + data->array[n++] =3D (u64)key->tgid << 32 | key->pid; + data->array[n++] =3D delta; + + /* data->array[n] is callchain->nr (updated later) */ + data->array[n + 1] =3D PERF_CONTEXT_USER; + data->array[n + 2] =3D 0; + len =3D copy_stack(stack, data, n); + + /* update length of callchain */ + data->array[n] =3D len + 1; + n +=3D len + 2; + + data->array[n++] =3D key->cgroup_id; + + return bpf_perf_event_output(ctx, &offcpu_output, BPF_F_CURRENT_CPU, data= , n * sizeof(u64)); +} + static int off_cpu_stat(u64 *ctx, struct task_struct *prev, struct task_struct *next, int state) { @@ -216,6 +277,16 @@ static int off_cpu_stat(u64 *ctx, struct task_struct *= prev, pelem->state =3D state; pelem->stack_id =3D stack_id; =20 + /* + * If stacks are successfully collected by bpf_get_stackid(), collect the= m once more + * in task_storage for direct off-cpu sample dumping + */ + if (stack_id > 0 && bpf_get_stack(ctx, &pelem->stack, MAX_STACKS * sizeof= (u64), BPF_F_USER_STACK)) { + /* + * This empty if block is used to avoid 'result unused warning' from bpf= _get_stack(). + * If the collection fails, continue with the logic for the next task. + */ + } next: pelem =3D bpf_task_storage_get(&tstamp, next, NULL, 0); =20 @@ -230,11 +301,19 @@ static int off_cpu_stat(u64 *ctx, struct task_struct = *prev, __u64 delta =3D ts - pelem->timestamp; __u64 *total; =20 - total =3D bpf_map_lookup_elem(&off_cpu, &key); - if (total) - *total +=3D delta; - else - bpf_map_update_elem(&off_cpu, &key, &delta, BPF_ANY); + if (delta >=3D offcpu_thresh_ns) { + int zero =3D 0; + struct offcpu_data *data =3D bpf_map_lookup_elem(&offcpu_payload, &zero= ); + + if (data) + off_cpu_dump(ctx, data, &key, &pelem->stack, delta); + } else { + total =3D bpf_map_lookup_elem(&off_cpu, &key); + if (total) + *total +=3D delta; + else + bpf_map_update_elem(&off_cpu, &key, &delta, BPF_ANY); + } =20 /* prevent to reuse the timestamp later */ pelem->timestamp =3D 0; --=20 2.45.2 From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1E24266B5A; Thu, 13 Feb 2025 23:00:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487622; cv=none; b=UJuI4C2XRv7KViydsk3QvokFw7EQ6VjsDOVT+Lsc9MgjAsk/CerTKG81YxyPquBXaVmzELmj6PJb+7FW9Q7tacdol6jJhJ/uPEuEZ7bcdkZSlI+dIds9r9gQ3hMa6b3Q0fLmIIpidY/k/dgt2k+84WKT+MNnrMxZ4e0sOEDhwNA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487622; c=relaxed/simple; bh=A8XyHQ5YgqWUiyI6TKjmZSVKVDEzl5SRC14UPMPAJJM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CHH2wxqxrBqmw1qKu6l97usFIB4KctiTb4ESCR5ma2WDUMBqf7P95uAyQzx1nAjhIOLFB78KVHZJJGXIFntMwJKjDesOLkmeQCtzjggldton8Xx9c9pT+GAgBrlbLxzUk2/zXuaoUkdUbut9TIEc77H6qoDwlFYIwUIjXbyUuu8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dXfMJ7xR; arc=none smtp.client-ip=209.85.216.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dXfMJ7xR" Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2fa8ac56891so2123083a91.2; Thu, 13 Feb 2025 15:00:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487620; x=1740092420; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eu4dGM1SJKv24SUYwHSLe7ZuUX9YrvzrAdTqGotWDLE=; b=dXfMJ7xRQK/rMTeCefjdy6m+3lFQ7WLbRzV2Ni3tkrey0QHV7CrC5CRhwdGG7tyADX GvXwbA7UdjJG4VRyAhSVk5jNCYZQyebqJ17WUebaWjnTSUDiM19hh8I9rJAN8ztGrVJq G1uEUv9kjAdO0YiJ43jBuzwS4PWVgItSzQUzryVjYzXBEXFhFMxJZs7+LNwSxir1ubMa jKTZlOzNe1cMBpmEAcDoYxMCJIqNwMg9kVpqdI1XLW8ueeA6sK3USvBLsfrEc3brzkfx JKSfYH3emJ738OoBAPZbKiDGtFanHJqiDUHpUUAcC7V24mAQvgus7TLAjMK7l8NtEt89 IHOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487620; x=1740092420; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eu4dGM1SJKv24SUYwHSLe7ZuUX9YrvzrAdTqGotWDLE=; b=WPP3fq8oznvNXR53tr/qEyG7jVQEYrzl4tNALAdHjfmI2YNiTMqp8LUmbUwM6806TW 2njmOglSAeCfz543aYkF7Gc+zVbzZWcciRqpFMsIlpUbY/6ua32PU6XQH1s7EEttVDdY f+KdeS/BuffRP0y3zG69ki+jcpPDD2QpdVR2ZtI8Q54GPvdmC4F9uHu8N9acr8cCd4M+ SulWcQGETVShHSFs7LGVXqt1wH4zqe0rFjX3/2FlA9/woCKxGhEpXBghqTMuF2YmTWxr B5h+I32cwVoKbAQschwrrvom1tSzNO8T+JhLwgwXnMPeFw6YpgL4lQzfxPpneAvfQz12 lp+A== X-Forwarded-Encrypted: i=1; AJvYcCUtoNXHF8dakJjSrsFXsH6Rd75pUZgJYVm6liP33+81GCVF1jgKt+Bx54ToX891OSFO00MGUvbPqWNb/5E=@vger.kernel.org, AJvYcCWjPqRBX8yWXFl2ls7o+JG4uVi9xTQYTenZlwwBeiuVvBxv+pNVEm1bMh4b+gquRx+3CbW/okxEXQFGHBrmfq9I7g==@vger.kernel.org X-Gm-Message-State: AOJu0Yzye4wfVxgKNKTpTd5Rp3t6AGJtD5AhTdpOQc6ou0tpg7zxkqoO /eO4lbWSna0rthYUkA76orq0KrXaRWxqiGC1MNIFDqc7k7zos04Q X-Gm-Gg: ASbGncsPxTnz64FEH4E5aM0Idh2JM7jw9QicIOQWNsfw7zcDRjq8ad5Q2/KkmfwsnTP onwg2bDSZDiJ4z027xO6V0ylLu3sFMFVHcOlatEJAqF1pPINKCdXifoopwUKvWp2sCwL6vU4ZAk eWDTNsQVGS9B4nXqqeDwFfBfvjAST+SNgEsLchTaCQ5S32T8/OZZWRcuU+092uvodzhNYHVqsPd WpnxpVVchjR8Oz6OsC/VSofulZS24sdos9+Hjc8ZFeqjou3i2KAkG9Epv3hED/otlsCWI0BoeGB xCpRZF6T31cite4ClSlqd9dehM3nCe76joVd5SonJQkO2U4= X-Google-Smtp-Source: AGHT+IGW+BaY+KT0cT3H87owpFFd6R5pLw+7uhEWVUtlZCUAPKjS/GVgvV5ChblqDu6QVqaVg9eNig== X-Received: by 2002:a17:90b:1847:b0:2ee:a583:e616 with SMTP id 98e67ed59e1d1-2fc0e1c6256mr7122076a91.9.1739487620128; Thu, 13 Feb 2025 15:00:20 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:19 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra , Arnaldo Carvalho de Melo Subject: [PATCH v15 05/10] perf evsel: Assemble offcpu samples Date: Thu, 13 Feb 2025 15:00:04 -0800 Message-ID: <20250213230009.1450907-6-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the data in bpf-output samples, to assemble offcpu samples. In evsel__is_offcpu_event(), Check if sample_type is PERF_SAMPLE_RAW to support off-cpu sample data created by an older version of perf. Testing compatibility on offcpu samples collected by perf before this patch= series: See below, the sample_type still uses PERF_SAMPLE_CALLCHAIN $ perf script --header -i ./perf.data.ptn | grep "event : name =3D offcpu-t= ime" # event : name =3D offcpu-time, , id =3D { 237917, 237918, 237919, 237920 = }, type =3D 1 (software), size =3D 136, config =3D 0xa (PERF_COUNT_SW_BPF_O= UTPUT), { sample_period, sample_freq } =3D 1, sample_type =3D IP|TID|TIME|C= ALLCHAIN|CPU|PERIOD|IDENTIFIER, read_format =3D ID|LOST, disabled =3D 1, fr= eq =3D 1, sample_id_all =3D 1 The output is correct. $ perf script -i ./perf.data.ptn | grep offcpu-time gmain 2173 [000] 18446744069.414584: 100102015 offcpu-time: NetworkManager 901 [000] 18446744069.414584: 5603579 offcpu-time: Web Content 1183550 [000] 18446744069.414584: 46278 offcpu-time: gnome-control-c 2200559 [000] 18446744069.414584: 11998247014 offcpu-time: And after this patch series: $ perf script --header -i ./perf.data.off-cpu-v9 | grep "event : name =3D o= ffcpu-time" # event : name =3D offcpu-time, , id =3D { 237959, 237960, 237961, 237962 = }, type =3D 1 (software), size =3D 136, config =3D 0xa (PERF_COUNT_SW_BPF_O= UTPUT), { sample_period, sample_freq } =3D 1, sample_type =3D IP|TID|TIME|C= PU|PERIOD|RAW|IDENTIFIER, read_format =3D ID|LOST, disabled =3D 1, freq =3D= 1, sample_id_all =3D 1 perf $ ./perf script -i ./perf.data.off-cpu-v9 | grep offcpu-time gnome-shell 1875 [001] 4789616.361225: 100097057 offcpu-time: gnome-shell 1875 [001] 4789616.461419: 100107463 offcpu-time: firefox 2206821 [002] 4789616.475690: 255257245 offcpu-time: Suggested-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-7-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/util/evsel.c | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index f45d4b44d70d..fcd49f407f52 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -1263,7 +1263,8 @@ static void evsel__set_default_freq_period(struct rec= ord_opts *opts, =20 bool evsel__is_offcpu_event(struct evsel *evsel) { - return evsel__is_bpf_output(evsel) && evsel__name_is(evsel, OFFCPU_EVENT); + return evsel__is_bpf_output(evsel) && evsel__name_is(evsel, OFFCPU_EVENT)= && + evsel->core.attr.sample_type & PERF_SAMPLE_RAW; } =20 /* @@ -2933,6 +2934,35 @@ static inline bool evsel__has_branch_counters(const = struct evsel *evsel) return false; } =20 +static int __set_offcpu_sample(struct perf_sample *data) +{ + u64 *array =3D data->raw_data; + u32 max_size =3D data->raw_size, *p32; + const void *endp =3D (void *)array + max_size; + + if (array =3D=3D NULL) + return -EFAULT; + + OVERFLOW_CHECK_u64(array); + p32 =3D (void *)array++; + data->pid =3D p32[0]; + data->tid =3D p32[1]; + + OVERFLOW_CHECK_u64(array); + data->period =3D *array++; + + OVERFLOW_CHECK_u64(array); + data->callchain =3D (struct ip_callchain *)array++; + OVERFLOW_CHECK(array, data->callchain->nr * sizeof(u64), max_size); + data->ip =3D data->callchain->ips[1]; + array +=3D data->callchain->nr; + + OVERFLOW_CHECK_u64(array); + data->cgroup =3D *array; + + return 0; +} + int evsel__parse_sample(struct evsel *evsel, union perf_event *event, struct perf_sample *data) { @@ -3287,6 +3317,9 @@ int evsel__parse_sample(struct evsel *evsel, union pe= rf_event *event, array =3D (void *)array + sz; } =20 + if (evsel__is_offcpu_event(evsel)) + return __set_offcpu_sample(data); + return 0; } =20 --=20 2.45.2 From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F5A026FA6A; Thu, 13 Feb 2025 23:00:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487625; cv=none; b=CjXdViQC3oc1lheocqH/zakpDvOrJFiRPwxe72GWUgyUMiuDLrcVe46izHCNZpO4EUqQp0B95wixYTWV5wVA8XUks+o+UnAh5KzVAQlWfA8Fk/DDrTRhkn2XZ6F5tDxxZnzMeZ2rBNqE9Uj3unB36TlwCZ4pId6zn1lqX6uWnE0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487625; c=relaxed/simple; bh=Pqez/aGnMBPJahMHse06GPvbqP7rG0cCe15F1SQqc0o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MEgAIzGyc9605wJOTH9yLN/o6Ac4qxiDOrwUmkKTnoNUoVDATV6SrHfsXfo/vNeUmDz4wy5qOk7WUjVpqporPV5IU+UlJgB0CDR9dLNXNNRZen/9ESsjgw3gRWfTO63rWjxtKqfzFOhx7yS30NWbiclHa7QjFXmloYGstLcMxKg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NcDBNVg+; arc=none smtp.client-ip=209.85.216.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NcDBNVg+" Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-2fa1fb3c445so2148532a91.2; Thu, 13 Feb 2025 15:00:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487622; x=1740092422; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mrDkT78AbLp193RSVHymTnt8PVvEiWM3gXo16d10Ee8=; b=NcDBNVg+GfwxOmVPnC6oLo5x/etqWs6gSf981lAl3Z4XdA8ntvyviVpsRdsLfYUctY JkXuoM1Jqn7kKnGm+Epbu7UCUGChULjLdhp8mEQnl4aRxB/8L84t8jnUns6/Jgl5HTeE QhhTzJtuyQL6XdA5xOq2W0ZbCfkgTAFddAbJ5T1Zuac/TxWzxVqALdEOOgEAJXryd3uN sePv3QGu3S8MBoAw4QqJ+OBHzAHaYcMIaFeZYiABjuqB3TM7L/PU2iPEbKddIjHSfzxa IjmZOYpR4cWyzemRSYBjcubVTU0CeTvHnROxDbiju6WiBNKzXmpoGYA/pPvQUppRUwj6 sTnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487622; x=1740092422; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mrDkT78AbLp193RSVHymTnt8PVvEiWM3gXo16d10Ee8=; b=t+rAzntDMez9dR2L/Mhkjt57a6dSuczUYJz+EsgurNxFcaaf4oxaPO4igsBVdxUsQO cOjXot0ZDUR3ZPqJX5WgpYC9rKmMrocgh8SpufjrSg0lf6T/yMjxsjifqzA5Mougv9Q4 NORP1AH2QDdBfjEN7xp0duxJ5J49ndsR3ZzH8oZ04deax5g3qlXQnJwm1I+hTbBzhI+F xz1v1uDRwvlpaMyhttaAJsHJCFsDPkJY8am2i3vaqzM6NApFk5kLmV87gLf0FkoPh03/ S1pD7QTgGZkyvO6IPbFpYcxstyM/bSZt5Yb6AMj73nrLiSn+UbnFt9tbKQtfFdn5xHwm Lsag== X-Forwarded-Encrypted: i=1; AJvYcCVBO2mLiDTnNUHMjBFy5l0UDAQ16uNHZYos96AUwPTq+M0OWDfaf7LThTIYJXmfyAR8DBLFUYLzd+jYqyE=@vger.kernel.org, AJvYcCVPU/ki0r2L8QH7+Amy/mGRzOquUiGdTwrF+1wVHpd1tQ/whwUZ2KCfntj/bGODejvLbx5DGXZTbn53M7otJwfjew==@vger.kernel.org X-Gm-Message-State: AOJu0YwGVbNOk5STEynEBjNtj6KCxN+8GHJsh3pREpoXX2NJ3MhWOUzX WY3suXYTlZmrbhwJLpkOvPziFQHvy9/eVyqtG6nMzmAr/JyWJHiK X-Gm-Gg: ASbGncvxmV6T7OYJ6Aut2GpNWaHQC+hQYRI2CcoMaJu2DA8U06Swbn3bUTm/U31VGoO QzlfyqPtSRF28aO7+8n8fH/gD/LcZXjvX4ldecr72fVA8BB04a0SH+2sR7YqbarXTEBQ8Mfd3D/ 5LqfK46K+0LYlfY8dOEwXoIhp+hFZ8njdEPvCHjcj8WEYNlDW+oSZzPczI6uMzUPm6YldjT8EyX PlvQj35/RcF37ZMwiwemTxZ2OGWbGOzjuCXsjpsVTHoqt7KoBYWNXWJ/TSAdOY3rda/0EjZBdMZ ZpsIW6rc7nxgFEOApaQY5wbtaqZVXkdrM0JY1PX2TxQVZyk= X-Google-Smtp-Source: AGHT+IGVG4jPUE2Vy3GKr7ThKUQyZply+qxvizrvUmFRuNzLEG7JPuoxpuF3oziSYxBhOn3eRPUBwg== X-Received: by 2002:a17:90b:4a10:b0:2ee:b66d:6576 with SMTP id 98e67ed59e1d1-2fbf9134584mr11216916a91.30.1739487621585; Thu, 13 Feb 2025 15:00:21 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:21 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra , Arnaldo Carvalho de Melo Subject: [PATCH v15 06/10] perf record --off-cpu: Disable perf_event's callchain collection Date: Thu, 13 Feb 2025 15:00:05 -0800 Message-ID: <20250213230009.1450907-7-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" There is a check in evsel.c that does this: if (evsel__is_offcpu_event(evsel)) evsel->core.attr.sample_type &=3D OFFCPU_SAMPLE_TYPES; This along with: #define OFFCPU_SAMPLE_TYPES (PERF_SAMPLE_IDENTIFIER | PERF_SAMPLE_IP | \ PERF_SAMPLE_TID | PERF_SAMPLE_TIME | \ PERF_SAMPLE_ID | PERF_SAMPLE_CPU | \ PERF_SAMPLE_PERIOD | PERF_SAMPLE_CALLCHAIN | \ PERF_SAMPLE_CGROUP) will tell perf_event to collect callchain. We don't need the callchain from perf_event when collecting off-cpu samples, because it's prev's callchain, not next's callchain. (perf_event) (task_storage) (needed) prev next | | ---sched_switch----> Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-8-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/util/off_cpu.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/perf/util/off_cpu.h b/tools/perf/util/off_cpu.h index 2dd67c60f211..2a4b7f9b2c4c 100644 --- a/tools/perf/util/off_cpu.h +++ b/tools/perf/util/off_cpu.h @@ -13,7 +13,7 @@ struct record_opts; #define OFFCPU_SAMPLE_TYPES (PERF_SAMPLE_IDENTIFIER | PERF_SAMPLE_IP | \ PERF_SAMPLE_TID | PERF_SAMPLE_TIME | \ PERF_SAMPLE_ID | PERF_SAMPLE_CPU | \ - PERF_SAMPLE_PERIOD | PERF_SAMPLE_CALLCHAIN | \ + PERF_SAMPLE_PERIOD | PERF_SAMPLE_RAW | \ PERF_SAMPLE_CGROUP) =20 =20 --=20 2.45.2 From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FFA122D781; Thu, 13 Feb 2025 23:00:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487625; cv=none; b=KZCMj5m9Ah7B2vKdD8Sdh9XE8/uOhq0kW0M52grLiYZtsRbMzmXe3dIsH/gDveHLFGctNH55FORMIWbrLe+03Tu7MZiky/0tM+qVJjnk3GnVx8wzKYF7WZd/oNLbWsOn9ZC1w5tFr85jl7AvTI/YoWwNwMKUl1wmVeGuUclrpBQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487625; c=relaxed/simple; bh=UjjR7OdU1VMuMu0dk05ePpEpKY020R6nckVERK33lt8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Wgqgd7swF11V22h0vYR2nGqVvSOeaJ1ujh/UVZOkklm6ogWTq1+3A6SksIFgxUuXZpyj578TQ9BfUO3fRYK8u2veS6Citquxc1YYKXjzq23f+uxe9k8GxVgboGO56MvYJbJmZJKXSB7qeTJtF5gAODtxHL2860lIHFExxLhWZtY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZnLfDSLJ; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZnLfDSLJ" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-220c8eb195aso27246095ad.0; Thu, 13 Feb 2025 15:00:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487623; x=1740092423; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1aTgCNo0b7SsW3yuyjGgod0qmPvKeo0A7oK4bUI9QpQ=; b=ZnLfDSLJUTrLTzO024/NrINjNLAin+idcfXYO8z0oyAxjezPdfN0BmyUrBglimLcg4 mC6PVzasXuhX26ABywWwawPNUkxiL/8eRAoO7+NxSFROCTBRSLcLQvEhcM090hiud3tD s5/hdCAqfFrK9hvhpeg/R45D7TVhs9aTnv9hcKecpY8dbDpY8TGuUVzozbDQ4B4Q4NgT a41X7CGTcQzHT/J2TtW732D1A4OkZZfM741V101/HyQ1cvYsr4BDJqNk7NcruOfkSjQy 8NHhAA71Lbl3Pkx/KoeMN5j0NZhmWearR8kDff0SqnaVic6tpCiO74vdUKSDFFu+BiFF UxIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487623; x=1740092423; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1aTgCNo0b7SsW3yuyjGgod0qmPvKeo0A7oK4bUI9QpQ=; b=jZ5yB3RrIT6xjxmWF8fSGuuEu9JMG6BheUsnCYbTRp1gtfvy9dMwY+IMK+v1ZaZo8N pvELC4htxtK39wxsO+EXUr9JPOZbS7SFbtUidC9oSTmblDVdAQxOdnt7UNv9nGUqqHsZ 5mjJsqYBjK2mj1lfD7My+WTOBLSO0PjCmTmU40FzzrIfkTBWpnO3CEyP4XPm3Sbybcla hN0mimiIIccV3aO9eJZSJg5f3sAJ4M0OgkeciqEmqkqaLfaD24EjVic4H0I50Tfrl/6c n3yXitdreIyhv0vjq8tNt5XGQtdYrJqFLidLw8f6Y2b95m73FI0zkiFOpNYhs8X7keI4 VLHA== X-Forwarded-Encrypted: i=1; AJvYcCUeRQ4k5dx/SMJlTWLdJotoTlULOlFhMA5XVROaj6j+RvHcvS59WD8ZahM0tWLDMFdZhWjgxJIZwTvouyF2YGFTrQ==@vger.kernel.org, AJvYcCUgrpqlT9t1kLl43XFswcVhnVCmdBr2wbdbToJdKdVMoNGw4Q+0pqr+RwhqRP65ZgVbmgM+2OdoTPHfHaE=@vger.kernel.org X-Gm-Message-State: AOJu0YygpyOD/hQWwYIh10PigF6yC+RnCNla3tf4xhFzllNbxrNip+uZ ya4/uYBqk6/HjawEYX3nCfBGTE0H5Al6UjfQh0+NYEPn5eQhR0Gw X-Gm-Gg: ASbGncsJgNO0JqwPqp5rMBYB2E6ndHpLiFm9Gmih5lANJB91WRnZf1aPQA460d6B8IX tkrvcNXQzNDIhksJxW0rnmLx09MpPqtCD08BWmf+xOgmy8jCXIteV7ix2W2fVgDFDWJNsBWp6vR ugGlWkvUYUYURVWs3bJWi2N7JEobiVrabDIHuUgwVMnXXdzAzANXynKqy+m62lO5LC1T8Omnytx HjBymbSR2OMBa2os2hrpSccT/6MknyQvrE9zXnLdWfYg8E6f0SnX+8akl8KBMRJfNhXcbs8tbRq oYOfzu5Kvy7nHOzVXYK0B0pZ9Pvia0cXqialZAIA/646V0g= X-Google-Smtp-Source: AGHT+IFr+vj96n7kaUSAg360rDSVM3RSUvO0RDfO4cwkPjYnQqKwJRbpt1kdSySImQrQuX5ER2YwMA== X-Received: by 2002:a17:902:f64c:b0:21f:164d:93fe with SMTP id d9443c01a7336-220bbc8ba82mr175017795ad.53.1739487623151; Thu, 13 Feb 2025 15:00:23 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:22 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra , Arnaldo Carvalho de Melo Subject: [PATCH v15 07/10] perf script: Display off-cpu samples correctly Date: Thu, 13 Feb 2025 15:00:06 -0800 Message-ID: <20250213230009.1450907-8-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable No PERF_SAMPLE_CALLCHAIN in sample_type, but I need perf script to display a callchain, have to specify manually. Also, prefer displaying a callchain: gvfs-afc-volume 2267 [001] 3829232.955656: 1001115340 offcpu-time: 77f05292603f __pselect+0xbf (/usr/lib/x86_64-linux-gnu/libc.so.= 6) 77f052a1801c [unknown] (/usr/lib/x86_64-linux-gnu/libusbmuxd-2.= 0.so.6.0.0) 77f052a18d45 [unknown] (/usr/lib/x86_64-linux-gnu/libusbmuxd-2.= 0.so.6.0.0) 77f05289ca94 start_thread+0x384 (/usr/lib/x86_64-linux-gnu/libc= .so.6) 77f052929c3c clone3+0x2c (/usr/lib/x86_64-linux-gnu/libc.so.6) to a raw binary BPF output: BPF output: 0000: dd 08 00 00 db 08 00 00
...... 0008: cc ce ab 3b 00 00 00 00 =CE=AB;.... 0010: 06 00 00 00 00 00 00 00 ........ 0018: 00 fe ff ff ff ff ff ff . 0020: 3f 60 92 52 f0 77 00 00 ?`.Rw.. 0028: 1c 80 a1 52 f0 77 00 00 ..Rw.. 0030: 45 8d a1 52 f0 77 00 00 E.Rw.. 0038: 94 ca 89 52 f0 77 00 00 ..Rw.. 0040: 3c 9c 92 52 f0 77 00 00 <..Rw.. 0048: 00 00 00 00 00 00 00 00 ........ 0050: 00 00 00 00 .... Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-9-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/builtin-script.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c index d797cec4f054..16b500b23417 100644 --- a/tools/perf/builtin-script.c +++ b/tools/perf/builtin-script.c @@ -670,7 +670,7 @@ static int perf_session__check_output_opt(struct perf_s= ession *session) =20 evlist__for_each_entry(session->evlist, evsel) { not_pipe =3D true; - if (evsel__has_callchain(evsel)) { + if (evsel__has_callchain(evsel) || evsel__is_offcpu_event(evsel)) { use_callchain =3D true; break; } @@ -2274,7 +2274,7 @@ static void process_event(struct perf_script *script, else if (PRINT_FIELD(BRSTACKOFF)) perf_sample__fprintf_brstackoff(sample, thread, evsel, fp); =20 - if (evsel__is_bpf_output(evsel) && PRINT_FIELD(BPF_OUTPUT)) + if (evsel__is_bpf_output(evsel) && !evsel__is_offcpu_event(evsel) && PRIN= T_FIELD(BPF_OUTPUT)) perf_sample__fprintf_bpf_output(sample, fp); perf_sample__fprintf_insn(sample, evsel, attr, thread, machine, fp, al); =20 --=20 2.45.2 From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6778E270EC8; Thu, 13 Feb 2025 23:00:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487627; cv=none; b=WK+hqYBlDJyKdq2xTFUi0k5gqPu4Yz33mls3iyctKbZpxe9Fr6l41RWWI7iFcEq4VAH/BF9/DoA0wt18KwEhpYPaJg4o/XhxHVaqO+VgBddtVFtlaBljgdoJ0V0Mf0cIbNJ01haNFifrJmCMWP1ruXbsKDMk0TyGj2dmaSQySmE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487627; c=relaxed/simple; bh=5iyF3ehQWFPe6nTiSzDQN4FmBZhfvZFLZQvIVUC4BZY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WPmTQlHDNheYNKpTw+M+ZfWpaNrsmqS7bO54Ay1Y9UXPGk/2DWl8wErf0C8dzWZSONDE9chuGs4bSEjmnNpzaFh/iSkgCE5+M2ZHuFKt84Y+jEuc/YHYsUX4vCvvadE0wy0idS3cLt8+R7TJCuNuYqDk7fDL72Z7uKpI4d/Be+0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YbdHz0XJ; arc=none smtp.client-ip=209.85.216.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YbdHz0XJ" Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2f9b9c0088fso2272760a91.0; Thu, 13 Feb 2025 15:00:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487625; x=1740092425; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NVFzboUXL9aW366CZRys/UMk8WcR5XGdl84CfzinmhE=; b=YbdHz0XJelENNiVkRkXsJjuW14shscNvqjYx1MVlVBZ9dMo95EHucmlBcbZhu0ldDV JyGqtLnusM3ahUaimB/Ef53sU6S5fftDjp+FYINpvY7iBlu1SOGnOaZ7TYnAP5iqte/s PmC8TDunEHtK6Kx6UH8/ufZ0Pte5xUUJPs8ISp6FdbK+TDQsltTRvu8mHdCeBfU6vxjX EKrL0gp8bRo1ZS84PS0ekJjyZULbQLKVx3JPQoilgc30VJFh6IDsoPeec+w/GOdmoRCH PV+PRDlct3WKne2RAK0nNDXnz6xOQ3mlTtvvPLBCSzDd/as2trjvrV1rpP1ohqXyjMqX F+gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487625; x=1740092425; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NVFzboUXL9aW366CZRys/UMk8WcR5XGdl84CfzinmhE=; b=Oea/S4Si0TOnx7oKCAJrAvzYmnTXEy8hjgrzLFBZ5aHr65mUUF0gTmxqXPpP9XjTPY iyT8/fHqR0iX+YBP/WNw5TsDuDrELGe05INofEstsj5gZpJ9EPiV5ylxNZoa96s2QH6I BskqZC82iETgV5H5V/Xw25LJqbUMnZRa1VSvsxb5B5nkT5/pL62omAp4ityws2jPzieZ a+OaflPcHn0tWzt2dHzGjRIrfvTpEHwrcR1xHY9yiq0G1WFpvaQfraQPP2MTbxY0YGg6 fUM9TaA8HQlxBSfCJ0N6hshc2XvpZHUMN496iD7Knm/1PF9LJFA1m0oBIgY8d3PS1CIs DyYw== X-Forwarded-Encrypted: i=1; AJvYcCWM/ao0OULB6KJrNK9UWoT3k7iqUrel//F4paYobiJZTt0rPkCDDSetdLBl0hI2oV8wEz8qxkaMmLoPHlc=@vger.kernel.org, AJvYcCXUlB2TzZDiiLB6ztiUEbNJUjEHX2Wu2qETFrQTyubOngm9ZwW/uQ2Xs0FagtIEaXUMfImwEJgguAAt4SC+sj2paQ==@vger.kernel.org X-Gm-Message-State: AOJu0YzjLK77MFzXk6HvINKMNWJTQDvE7xE3TxK70Al6hlT/UuE5NSaf 1MA/naHpcnIiUHFStKQpIlBtKYDW038+8rvxpO/zglEe58Puyl9v X-Gm-Gg: ASbGncsbCxZJtnTJwK3spO9yEGyseg+tFV2pQ+SCxqNME1CEbZEVS+dQmiRP8+fhjlX E9lCgQkxqhJgD6kj2rl/SBr+2yPgioSEZ2QAv+OBMGRXad928gOGaRfzz32bhnPoSk/Q4uYk6yQ W0+/h89pcP2tCMGiCQIWO5PDVBiYTtWMKSEBk/REJEhBTsZ6iw3yTYia4nj/VS23zDBQZTPoBCZ LxamFQefxbobHWDhrb2H5ApD38X/43fKRSAq9p4f8cfPNUy2nWdZMKDJUPhsJYB8MApZFy1lpK1 gkx06+4XqaRFBpQqP0nrEuV7hAzmyQCh7HbD39XOUknBMrw= X-Google-Smtp-Source: AGHT+IGtFiztEAjpNU0miFV/6OmgPWklJsqw92dDFR2Pjph8R0ElKCr9s7N9z9rklcQlkb7p8Rbdkg== X-Received: by 2002:a17:90b:394f:b0:2fa:2252:f436 with SMTP id 98e67ed59e1d1-2fc0fbca207mr6855717a91.3.1739487624648; Thu, 13 Feb 2025 15:00:24 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:24 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra , Arnaldo Carvalho de Melo Subject: [PATCH v15 08/10] perf record --off-cpu: Dump the remaining samples in BPF's stack trace map Date: Thu, 13 Feb 2025 15:00:07 -0800 Message-ID: <20250213230009.1450907-9-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Dump the remaining samples, as if it is dumping a direct sample. Put the stack trace, tid, off-cpu time and cgroup id into the raw_data section, just like a direct off-cpu sample coming from BPF's bpf_perf_event_output(). This ensures that evsel__parse_sample() correctly parses both direct samples and accumulated samples. Suggested-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-10-howardchu95@gmail= .com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/util/bpf_off_cpu.c | 59 +++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 24 deletions(-) diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index de71ff7a80d0..e55693bcbf08 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -37,6 +37,8 @@ union off_cpu_data { u64 array[1024 / sizeof(u64)]; }; =20 +u64 off_cpu_raw[MAX_STACKS + 5]; + static int off_cpu_config(struct evlist *evlist) { char off_cpu_event[64]; @@ -312,6 +314,7 @@ int off_cpu_write(struct perf_session *session) { int bytes =3D 0, size; int fd, stack; + u32 raw_size; u64 sample_type, val, sid =3D 0; struct evsel *evsel; struct perf_data_file *file =3D &session->data->file; @@ -351,46 +354,54 @@ int off_cpu_write(struct perf_session *session) =20 while (!bpf_map_get_next_key(fd, &prev, &key)) { int n =3D 1; /* start from perf_event_header */ - int ip_pos =3D -1; =20 bpf_map_lookup_elem(fd, &key, &val); =20 + /* zero-fill some of the fields, will be overwritten by raw_data when pa= rsing */ if (sample_type & PERF_SAMPLE_IDENTIFIER) data.array[n++] =3D sid; - if (sample_type & PERF_SAMPLE_IP) { - ip_pos =3D n; + if (sample_type & PERF_SAMPLE_IP) data.array[n++] =3D 0; /* will be updated */ - } if (sample_type & PERF_SAMPLE_TID) - data.array[n++] =3D (u64)key.pid << 32 | key.tgid; + data.array[n++] =3D 0; if (sample_type & PERF_SAMPLE_TIME) data.array[n++] =3D tstamp; - if (sample_type & PERF_SAMPLE_ID) - data.array[n++] =3D sid; if (sample_type & PERF_SAMPLE_CPU) data.array[n++] =3D 0; if (sample_type & PERF_SAMPLE_PERIOD) - data.array[n++] =3D val; - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - int len =3D 0; - - /* data.array[n] is callchain->nr (updated later) */ - data.array[n + 1] =3D PERF_CONTEXT_USER; - data.array[n + 2] =3D 0; - - bpf_map_lookup_elem(stack, &key.stack_id, &data.array[n + 2]); - while (data.array[n + 2 + len]) + data.array[n++] =3D 0; + if (sample_type & PERF_SAMPLE_RAW) { + /* + * [ size ][ data ] + * [ data ] + * [ data ] + * [ data ] + * [ data ][ empty] + */ + int len =3D 0, i =3D 0; + void *raw_data =3D (void *)data.array + n * sizeof(u64); + + off_cpu_raw[i++] =3D (u64)key.pid << 32 | key.tgid; + off_cpu_raw[i++] =3D val; + + /* off_cpu_raw[i] is callchain->nr (updated later) */ + off_cpu_raw[i + 1] =3D PERF_CONTEXT_USER; + off_cpu_raw[i + 2] =3D 0; + + bpf_map_lookup_elem(stack, &key.stack_id, &off_cpu_raw[i + 2]); + while (off_cpu_raw[i + 2 + len]) len++; =20 - /* update length of callchain */ - data.array[n] =3D len + 1; + off_cpu_raw[i] =3D len + 1; + i +=3D len + 2; + + off_cpu_raw[i++] =3D key.cgroup_id; =20 - /* update sample ip with the first callchain entry */ - if (ip_pos >=3D 0) - data.array[ip_pos] =3D data.array[n + 2]; + raw_size =3D i * sizeof(u64) + sizeof(u32); /* 4 bytes for alignment */ + memcpy(raw_data, &raw_size, sizeof(raw_size)); + memcpy(raw_data + sizeof(u32), off_cpu_raw, i * sizeof(u64)); =20 - /* calculate sample callchain data array length */ - n +=3D len + 2; + n +=3D i + 1; } if (sample_type & PERF_SAMPLE_CGROUP) data.array[n++] =3D key.cgroup_id; --=20 2.45.2 From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 056F5271278; Thu, 13 Feb 2025 23:00:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487628; cv=none; b=Ly7jLiR4j2vgpfKb/mMy3AU6WdokN2h02L4Su1f0j+o56OY9HiQ2dYZbcCgb4p1qSiUQmUVL6KKYBiIibR7YO56Dd6It6NnVsjn3M0mC51eEmxaa5AwQYopdiamJbO9pRGmahj5tgmwaes6axVdozVgc8el5CBkzmZTOlaKpgTM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487628; c=relaxed/simple; bh=GPD6HDKgqeAnoPrMX7rRtPfeuzzlgM8kDH13INdoZR0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nDDKEiNExFum65h4MEaFmmGuuwK5mALUaLMhhwlUV3wi81VTh1mF4ZVnT6/dMNg4j9FGUe6jxc4FycQzbtcCqzptpzTj/BoO213WIKuzZG3t8SAAwi81BLdCnxnKZRc0idAKsN/Kx0KbK7AC6YR6Rlas5X5IB9VUe+NMGU6hrYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Yle24aW3; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Yle24aW3" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-2fa7465baceso2933916a91.0; Thu, 13 Feb 2025 15:00:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487626; x=1740092426; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fg1NcBAxZgPijirr2zNVBcLz8NdtPXgGW3roTpjnlrk=; b=Yle24aW3ZM5pvwlRDuVYl92yNa3Y9FjYeboRF2VQICowBLjCoNBQi+gn3QU8ITNqDA 7fatfDSbcz9BOHQwotsF2KfqnZb6fBcLktHo7CR2ZU9qXvDydNCJxggML6pCNu+rGpt3 ECGxx0bN/p3xY0RjUSJslB2gtKcdVvAZDiV/hW/F7z6FnFUMRK9piGyyBAFY5RMof3kf Waq7F03gODOV4h7Ekl3Wszat/k+o85lqPLzkgw+8/e6+QghKetR3yKtMVxwbhq7c3XOF cbffR34ttnMpViV5IS1gez/Bw7jYmIbDVM7YeZWT1Lzs6WNvZFCVqdqA+uokaZEtOgVM ywJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487626; x=1740092426; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fg1NcBAxZgPijirr2zNVBcLz8NdtPXgGW3roTpjnlrk=; b=QRazVrPU7MpO+HQ0pCKvg6AdM8/Iy5wGdxvnDpNw7UpIlDJ0bl+LYWf5vbVIm9yS2a DLT/9a3XljVMwVvxtddF4iRXLHSGep6aGATtu8Xb8oSKQqfed74S4tXyBHuLvwCMiSze OekF0OMPHEZyvKfa5pJz4d55uELxWL/EnjC9pCJUJJixASXk2H+0K2hAwsRu/Ht2z1s1 pOw4vBLAQV4TvOdy5MDUmbF42dgrp59Hyk7i4QHs0924pD8GbVR2ILuJJx6e+9aPTzzA By6067QWP5O+A7ytGumZ9YnAhxJFkNncaDBucbfpS52YV11jQx/Ohmo/ku8vm++7igfC QltQ== X-Forwarded-Encrypted: i=1; AJvYcCUBCZIzmeA52sPEQGtrLdJRR9+M/JCqedCUilcon8p5hVHh/Y+xAxxMAL+XpPvWVyO+jjW803nOJrpJpQYcuLC1Vg==@vger.kernel.org, AJvYcCWhXPh+Ohd2uZhrElxkjaRtBam5eIyIc91h9GP4pB7UekTQyFKHbtvQ7ji6BeDuCr5rnGIPuB90/dddRq4=@vger.kernel.org X-Gm-Message-State: AOJu0YwD04i/A7jx4lwcV06AcV/LnR1nNMND/aHnG94iOm+mBESIt67O 9s/ceuLznx6CumPxuxKUNxWDSZgmTB99OIuGZJU4Eu/6FZAtMVJO X-Gm-Gg: ASbGnct3UOXy/G1d6TggEvkCpHwvGPriK3Q5TQ3KmHBj9e4QsBF9kpiHJhFXZMl5ejk C/q49uNH7/I1twVbivE/Bj+mOi69JlbcQFlY9ZObmIE1RZHjfkbzVCSBsSVhHNUQ03Ly3SPtvAT x/jS84OMRBaSrMAyPUKem98JeMdNel1cyDjpbBBjbjq4b2ld8RliAlUkT/SEXlhwn9TcqSuSkMe bAi5tATMNk+qfCRMX3kHHSHcsnWGYmgOutCN1slt6e6RWo2+VvNDm+Ar6DiFBxLMmGqjj3q9EvG khdlqFgtHkJhhN39JxspopNRykW+FKwL+BT07hxKgfoxgt0= X-Google-Smtp-Source: AGHT+IFsHQahU8g/tgoe9IgIJg9ph+LpQRNRdynO2o45MxVtooFtulDIsAz9hzwQxu8IoEaBaTh1hw== X-Received: by 2002:a17:90b:4b8e:b0:2f8:49ad:4079 with SMTP id 98e67ed59e1d1-2fc0dfc650amr6273603a91.6.1739487626127; Thu, 13 Feb 2025 15:00:26 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:25 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Arnaldo Carvalho de Melo , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra Subject: [PATCH v15 09/10] perf record --off-cpu: Add --off-cpu-thresh option Date: Thu, 13 Feb 2025 15:00:08 -0800 Message-ID: <20250213230009.1450907-10-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Specify the threshold for dumping offcpu samples with --off-cpu-thresh, the unit is milliseconds. Default value is 500ms. Example: perf record --off-cpu --off-cpu-thresh 824 The example above collects off-cpu samples where the off-cpu time is longer than 824ms Suggested-by: Ian Rogers Suggested-by: Namhyung Kim Suggested-by: Arnaldo Carvalho de Melo Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-2-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/Documentation/perf-record.txt | 9 ++++++++ tools/perf/builtin-record.c | 26 ++++++++++++++++++++++++ tools/perf/util/bpf_off_cpu.c | 3 +++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 2 +- tools/perf/util/off_cpu.h | 1 + tools/perf/util/record.h | 1 + 6 files changed, 41 insertions(+), 1 deletion(-) diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Document= ation/perf-record.txt index 80686d590de2..3a87e635f52c 100644 --- a/tools/perf/Documentation/perf-record.txt +++ b/tools/perf/Documentation/perf-record.txt @@ -833,6 +833,15 @@ filtered through the mask provided by -C option. only, as of now. So the applications built without the frame pointer might see bogus addresses. =20 + off-cpu profiling consists two types of samples: direct samples, which + share the same behavior as regular samples, and the accumulated + samples, stored in BPF stack trace map, presented after all the regular + samples. + +--off-cpu-thresh:: + Once a task's off-cpu time reaches this threshold (in milliseconds), it + generates a direct off-cpu sample. The default is 500ms. + --setup-filter=3D:: Prepare BPF filter to be used by regular users. The action should be either "pin" or "unpin". The filter can be used after it's pinned. diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index f3e5f856f4a4..4bdc7a0111ef 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -3158,6 +3158,28 @@ static int record__parse_mmap_pages(const struct opt= ion *opt, return ret; } =20 +static int record__parse_off_cpu_thresh(const struct option *opt, + const char *str, + int unset __maybe_unused) +{ + struct record_opts *opts =3D opt->value; + char *endptr; + u64 off_cpu_thresh_ms; + + if (!str) + return -EINVAL; + + off_cpu_thresh_ms =3D strtoull(str, &endptr, 10); + + /* the threshold isn't string "0", yet strtoull() returns 0, parsing fail= ed */ + if (*endptr || (off_cpu_thresh_ms =3D=3D 0 && strcmp(str, "0"))) + return -EINVAL; + else + opts->off_cpu_thresh_ns =3D off_cpu_thresh_ms * NSEC_PER_MSEC; + + return 0; +} + void __weak arch__add_leaf_frame_record_opts(struct record_opts *opts __ma= ybe_unused) { } @@ -3351,6 +3373,7 @@ static struct record record =3D { .ctl_fd =3D -1, .ctl_fd_ack =3D -1, .synth =3D PERF_SYNTH_ALL, + .off_cpu_thresh_ns =3D OFFCPU_THRESH, }, }; =20 @@ -3573,6 +3596,9 @@ static struct option __record_options[] =3D { OPT_BOOLEAN(0, "off-cpu", &record.off_cpu, "Enable off-cpu analysis"), OPT_STRING(0, "setup-filter", &record.filter_action, "pin|unpin", "BPF filter action"), + OPT_CALLBACK(0, "off-cpu-thresh", &record.opts, "ms", + "Dump off-cpu samples if off-cpu time exceeds this threshold (in mi= lliseconds). (Default: 500ms)", + record__parse_off_cpu_thresh), OPT_END() }; =20 diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index e55693bcbf08..00736fb2678c 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -14,6 +14,7 @@ #include "util/strlist.h" #include #include +#include =20 #include "bpf_skel/off_cpu.skel.h" =20 @@ -291,6 +292,8 @@ int off_cpu_prepare(struct evlist *evlist, struct targe= t *target, } } =20 + skel->bss->offcpu_thresh_ns =3D opts->off_cpu_thresh_ns; + err =3D off_cpu_bpf__attach(skel); if (err) { pr_err("Failed to attach off-cpu BPF skeleton\n"); diff --git a/tools/perf/util/bpf_skel/off_cpu.bpf.c b/tools/perf/util/bpf_s= kel/off_cpu.bpf.c index c15b69586723..8df35541141b 100644 --- a/tools/perf/util/bpf_skel/off_cpu.bpf.c +++ b/tools/perf/util/bpf_skel/off_cpu.bpf.c @@ -124,7 +124,7 @@ const volatile bool uses_cgroup_v1 =3D false; =20 int perf_subsys_id =3D -1; =20 -__u64 offcpu_thresh_ns =3D 500000000ull; +__u64 offcpu_thresh_ns; =20 /* * Old kernel used to call it task_struct->state and now it's '__state'. diff --git a/tools/perf/util/off_cpu.h b/tools/perf/util/off_cpu.h index 2a4b7f9b2c4c..64bf763ddf50 100644 --- a/tools/perf/util/off_cpu.h +++ b/tools/perf/util/off_cpu.h @@ -16,6 +16,7 @@ struct record_opts; PERF_SAMPLE_PERIOD | PERF_SAMPLE_RAW | \ PERF_SAMPLE_CGROUP) =20 +#define OFFCPU_THRESH 500000000ULL =20 #ifdef HAVE_BPF_SKEL int off_cpu_prepare(struct evlist *evlist, struct target *target, diff --git a/tools/perf/util/record.h b/tools/perf/util/record.h index a6566134e09e..c82db4833b0a 100644 --- a/tools/perf/util/record.h +++ b/tools/perf/util/record.h @@ -79,6 +79,7 @@ struct record_opts { int synth; int threads_spec; const char *threads_user_spec; + u64 off_cpu_thresh_ns; }; =20 extern const char * const *record_usage; --=20 2.45.2 From nobody Thu Dec 18 20:16:15 2025 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAF0E274248; Thu, 13 Feb 2025 23:00:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487630; cv=none; b=QRn/3wy+bcv3YE88e2rj+4nJOClQS+geNPHW4gGxpeP6Sti8FVM3Quxft79DCptJSiTSRXRXeZOVt+r/MILMY6F9eLHCODmeO0Zf2xyt1LRsudtgh1RkcV+mkWNFyZ+aq4zqo3nwiHkwJVslHKw3tZ7tGWJskcCqM+n5mwD4vY0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739487630; c=relaxed/simple; bh=ikG1TN6rDUOxIhapEuldyAS8weMydskL6gYwyosfnGI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jqEiTvSlPaDrQwTzMn7xAd3WEai3wFTZh2DNKSgirlHxDuEBlAr8GkaPe8/N4nO6jvnGctvFrLiocoUnBD0Va0pPVK9p+dHiDGh0lJzMPIEnIjqqqEix3j8k8Z2Tr+GJh4shGQmejky0zN0uXidnNJX2HeJcEd5wLwv7CMxuNzU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=N9JsSv3M; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="N9JsSv3M" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-220bff984a0so24867525ad.3; Thu, 13 Feb 2025 15:00:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739487628; x=1740092428; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jEFFMAJ36x+7ZOWQGJAZsnY5Yqg+IxQ7p+2U41q9RK4=; b=N9JsSv3MDUKQFp35g6f8TvawB89UuBaxIUnMiKvrlYoAzhCxZ1nsXe5pl2qRmEv7F0 CmsCaIYkStjpA+HZwMpeJGivVwgt/PZILyalvmjk4de8CL3gjRZqdlrBL6vu5SwN6vwA B98O5l1TheA9RP8IWuyudGmv2VfSXEmZ5pUM6Q9bNs1YNuAOVWg9nmAtglLQuJkYXnon wCsZk7Z1trGc+TAoy6vE2EMeFO1MW1jblAx3nzN41wNN6CiH8/FCtqYuKfLCg5yGurgE jSmynDl5XnsmT+HcfcvHEkSbtxfWHkFclPVNCeCyI9yUdAxuitvEo4bsjCqxk1jllKO4 Vg3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739487628; x=1740092428; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jEFFMAJ36x+7ZOWQGJAZsnY5Yqg+IxQ7p+2U41q9RK4=; b=UjyihSzyXqF6D4Jp/i+s+2iu/08w7ArEs1kOXTgkqcfUJ+6cSl9jp7uiRO9Zao3nN2 O/bxEjoUaDGO2fme9NUngXnfQ6fTIWY7qcD9bq0JZGB8QR1TEPXszGPB5XQTmdWkSO1G 1T60AWuj/e8moRke3fWczUNo46cDR7vkZey4Gy2C7wnMFlq0JWFWPUHiET5YnzszLjaw mWQC58qJkf7V4vgiMA3CuNLtGId+i8pHm2pdq9QtX/WoCdA8US7uS3eTXXn1Mpcu5s2t 5UbHdmwwEKgwx7uFyQ5xRq3qns+y4YG4DhL8ZZMBLaUkhyHMfuokUBZdJmPdEOTgTofq lZAA== X-Forwarded-Encrypted: i=1; AJvYcCVWZw6m7nBsmh43O7613T5+n5hH1gMDOJbFSDC2aHzk7uTBD0AO5q6WaX92u4hgkTNdNm6f6zPrPxw7EtM8eZ1/vg==@vger.kernel.org, AJvYcCWXRAtQJCN5Jiov/CX9CovIIBOu14g3JEq1aP0LojsCNR5VMzLw9hEwGCKt6Fn7yyV5xg8NjvMJn9F55jA=@vger.kernel.org X-Gm-Message-State: AOJu0Yw1eu4cQVr3bghmvXCcp3WR05z29ct4t7T3+2QeLGGu38ArloSv p/QWP73mBkp5qM2AHin5W1Uvfyjrej5Xdd+qwOq8LOmQA6Q9gDJe X-Gm-Gg: ASbGncsWULIiC15JrE2mLGWipvESs08uwZWOxpw3oFhbKbqUGLQGtFTi35HJmvWQUZ1 wQH23kFKZkwVTV8Ydb9fWRjx/fg/mA+NNwDxWX7pO0nLLLpd9ULmj1XAhDM4aRVU4TKgjsVCRxG CEkRXiF+B4afA2MZuTNyi1G/TeFSY605X7bdxh2Sth+1QVExR32aEPwHwv7aQgo6WUImaIYbkZq PAoDP36sOn0SG0rSSYY3jOf79yisHi2qL6NvB8aWOSzH1EYKHTMqNxFSQ0GUpDSo0jhtY9ik6AE AQBbWaL1fMBLdb92Uoewaqy1ODQvE2Hh4cNiKp6favgXp8E= X-Google-Smtp-Source: AGHT+IFohfSbEB1liEUcnA4N3fAQDj2EifKHmSJglbrdlBIoeSc+y8q6nOwXo2yUfPiX8FmEQftthw== X-Received: by 2002:a17:903:1cc:b0:220:e9f5:263d with SMTP id d9443c01a7336-220e9f5288cmr21523915ad.7.1739487627614; Thu, 13 Feb 2025 15:00:27 -0800 (PST) Received: from mbp.lan (c-73-202-46-50.hsd1.ca.comcast.net. [73.202.46.50]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d5349210sm17641735ad.11.2025.02.13.15.00.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Feb 2025 15:00:27 -0800 (PST) From: Howard Chu To: acme@kernel.org Cc: namhyung@kernel.org, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , Ingo Molnar , James Clark , Kan Liang , Mark Rutland , Peter Zijlstra , Arnaldo Carvalho de Melo Subject: [PATCH v15 10/10] perf test: Add direct off-cpu test Date: Thu, 13 Feb 2025 15:00:09 -0800 Message-ID: <20250213230009.1450907-11-howardchu95@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250213230009.1450907-1-howardchu95@gmail.com> References: <20250213230009.1450907-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Why is there a --off-cpu-thresh 2000? We collect an off-cpu period __ONLY ONCE__, either in direct sample form, or in accumulated form (in BPF stack trace map). If I don't add --off-cpu-thresh 2000, the sample in the original test goes into the ring buffer instead of the BPF stack trace map. Additionally, when using -e dummy, the ring buffer is not open, causing us to lose a sample. Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-11-howardchu95@gmail= .com Signed-off-by: Arnaldo Carvalho de Melo Tested-by: Gautam Menghani --- tools/perf/tests/shell/record_offcpu.sh | 71 +++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/tools/perf/tests/shell/record_offcpu.sh b/tools/perf/tests/she= ll/record_offcpu.sh index 678947fe69ee..c5d6cae94c65 100755 --- a/tools/perf/tests/shell/record_offcpu.sh +++ b/tools/perf/tests/shell/record_offcpu.sh @@ -7,6 +7,9 @@ set -e err=3D0 perfdata=3D$(mktemp /tmp/__perf_test.perf.data.XXXXX) =20 +ts=3D$(printf "%u" $((~0 << 32))) # OFF_CPU_TIMESTAMP +dummy_timestamp=3D${ts%???} # remove the last 3 digits to match perf script + cleanup() { rm -f ${perfdata} rm -f ${perfdata}.old @@ -19,6 +22,9 @@ trap_cleanup() { } trap trap_cleanup EXIT TERM INT =20 +test_over_thresh=3D"Threshold test (over threshold)" +test_below_thresh=3D"Threshold test (below threshold)" + test_offcpu_priv() { echo "Checking off-cpu privilege" =20 @@ -88,6 +94,63 @@ test_offcpu_child() { echo "Child task off-cpu test [Success]" } =20 +# task blocks longer than the --off-cpu-thresh, perf should collect a dire= ct sample +test_offcpu_over_thresh() { + echo "${test_over_thresh}" + + # collect direct off-cpu samples for tasks blocked for more than 999ms + if ! perf record -e dummy --off-cpu --off-cpu-thresh 999 -o ${perfdata} = -- sleep 1 2> /dev/null + then + echo "${test_over_thresh} [Failed record]" + err=3D1 + return + fi + # direct sample's timestamp should be lower than the dummy_timestamp of = the at-the-end sample + # check if a direct sample exists + if ! perf script --time "0, ${dummy_timestamp}" -i ${perfdata} -F event = | grep -q "offcpu-time" + then + echo "${test_over_thresh} [Failed missing direct samples]" + err=3D1 + return + fi + # there should only be one direct sample, and its period should be highe= r than off-cpu-thresh + if ! perf script --time "0, ${dummy_timestamp}" -i ${perfdata} -F period= | \ + awk '{ if (int($1) > 999000000) exit 0; else exit 1; }' + then + echo "${test_over_thresh} [Failed off-cpu time too short]" + err=3D1 + return + fi + echo "${test_over_thresh} [Success]" +} + +# task blocks shorter than the --off-cpu-thresh, perf should collect an at= -the-end sample +test_offcpu_below_thresh() { + echo "${test_below_thresh}" + + # collect direct off-cpu samples for tasks blocked for more than 1.2s + if ! perf record -e dummy --off-cpu --off-cpu-thresh 12000 -o ${perfdata= } -- sleep 1 2> /dev/null + then + echo "${test_below_thresh} [Failed record]" + err=3D1 + return + fi + # see if there's an at-the-end sample + if ! perf script --time "${dummy_timestamp}," -i ${perfdata} -F event | = grep -q 'offcpu-time' + then + echo "${test_below_thresh} [Failed at-the-end samples cannot be found]" + err=3D1 + return + fi + # plus there shouldn't be any direct samples + if perf script --time "0, ${dummy_timestamp}" -i ${perfdata} -F event | = grep -q 'offcpu-time' + then + echo "${test_below_thresh} [Failed direct samples are found when they = shouldn't be]" + err=3D1 + return + fi + echo "${test_below_thresh} [Success]" +} =20 test_offcpu_priv =20 @@ -99,5 +162,13 @@ if [ $err =3D 0 ]; then test_offcpu_child fi =20 +if [ $err =3D 0 ]; then + test_offcpu_over_thresh +fi + +if [ $err =3D 0 ]; then + test_offcpu_below_thresh +fi + cleanup exit $err --=20 2.45.2