From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 024445661; Wed, 13 Nov 2024 00:28:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457705; cv=none; b=dOmZOpksz7L4egZmmBJCalTz//ArLPqV7vmOeVTWPMH4eMmCPjVaA7BhMz1mlI/KuZBXKzPrKr3Wwc/bke8MkAHMIYKbn67M2f7oTAAJZQBPc0kg8wGDtnwMEp6BDLE3b3RTL63v67AYborXuWMRkdgu4sGMzrMvX0p43ZAzLtE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457705; c=relaxed/simple; bh=3vwXpoLqH4tgEDR/6BEKnB6KYg193amWFuZtazpLuE4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WXEESjmp51L3wasapUi+dhv76NUq1pAUUoVrGp6dBxcSRRxT6+n7s5yg+OdWKsgVkmPniwFwPtePjMv9rlb3ASmbFp/RhecmlXQHDin+sGtmK+EjIokcDYTDqWNKME1oICdZxEEBhivY5JdRRwM8EZoo4EOv/UaMjURmrcAH0Pk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bzRlOe2B; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bzRlOe2B" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2114214c63eso53698265ad.3; Tue, 12 Nov 2024 16:28:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457703; x=1732062503; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NKsdbnvRUk9U6O1O9cwYT4JrIhseWm27eIb09OJkB1Y=; b=bzRlOe2BMD6kyyOAcS+dnlBsPmUfGAI7oFdt9qJWWXOeKCWKGXtFOjZrdIXCMK7r6J HUu/dvvBmi0TFGlRdZg5F0UeTOwJcBC7tvMrayTjaNrxazCxK/YgE+YQReS6eaJrTYBs 9VKBqauqM0WQWQg0Pjk6ctFHx66DtxyUUx/S+yuUmwim/tsy3JvEO7m1MSwoUIH8G2xu USb243deZajorRbbedxnAsxWPQk3FTcPkdPPBGOBArhwTxGe/yyY0cajoh5mryB8nLYv 31rlZhP1Dwlss9GB4YoyvFwTyy/2mdrn+rof9a4OdF33dGssSc4+JKz3bIkHVhhIiKar tDzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457703; x=1732062503; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NKsdbnvRUk9U6O1O9cwYT4JrIhseWm27eIb09OJkB1Y=; b=JH3p1eJSQVnfbPScwq0z3sRB2l8aiegbe25aGDSu9KOQPRNr2tWT5pKvjr+qCnqJyB Er25KDgJUjijCaw9vVbV2mU7kIN0zuGVPZQ/9rXqOSciEQGUtn7/9dR8i8AlskTnX4kj /TllSX8wfJ6ru3hl/RyPD7o0mIToE2q9Xf/JEDwWkAljaPC4RYmt7rIaTSd/ZNX/SaPw Z8ptXLdlMlswguRC/aFMnPauNXuJHBrG3jl07Iej+AV0wvRBzBrMrYyHp9k4xVDH0/Bm 1jBARsUPcxJ478WKhIu/NkqD2vYbtaoQJHzrB2UuRKYU44uTyTFS/HluhC0ebi2wnjGv UKLg== X-Forwarded-Encrypted: i=1; AJvYcCXdM5XpaNlAq7dYXAf8S/Sf4hHFz9QfvHxyBmtYfoKE9VTbpmzmywxk8b6D38fY9MNjPl5itqKecOF0DDc=@vger.kernel.org, AJvYcCXrZfOfZfeni2NsyKcydf4RpINICaGbcClIXD31LLAaoL1hgb0XP4Qm0TsfJAZQBGvHJaYfLmAtLtqxMrfAjBtyRQ==@vger.kernel.org X-Gm-Message-State: AOJu0Ywc8tsjKYkvmBwZSzz4LJpaPKNeLw6SNu/M1Hyj+86UuIlbZh8A RYZaLq5QRy6EQJABLVUYdyedjxs25ELHfQXABxt922K9afOJXb0W X-Google-Smtp-Source: AGHT+IHi16NKqwb2xu6avMOIeZm6r90zqsOQDvvkgiPYntKLaBAPO29Q1hJ/6CQnIJRI4rT6B3QcIg== X-Received: by 2002:a17:903:1cb:b0:20c:bffe:e1e5 with SMTP id d9443c01a7336-21183521d84mr268992785ad.19.1731457703105; Tue, 12 Nov 2024 16:28:23 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:22 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 01/10] perf record --off-cpu: Add --off-cpu-thresh option Date: Tue, 12 Nov 2024 16:28:09 -0800 Message-ID: <20241113002818.3578645-2-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Specify the threshold for dumping offcpu samples with --off-cpu-thresh, the unit is us (microsecond). Default value is 500000us (500ms, 0.5s). Suggested-by: Ian Rogers Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-2-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/builtin-record.c | 26 ++++++++++++++++++++++++++ tools/perf/util/off_cpu.h | 1 + tools/perf/util/record.h | 1 + 3 files changed, 28 insertions(+) diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index f83252472921..c069000efe5c 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -3149,6 +3149,28 @@ static int record__parse_mmap_pages(const struct opt= ion *opt, return ret; } =20 +static int record__parse_off_cpu_thresh(const struct option *opt, + const char *str, + int unset __maybe_unused) +{ + struct record_opts *opts =3D opt->value; + char *endptr; + u64 off_cpu_thresh_us; + + if (!str) + return -EINVAL; + + off_cpu_thresh_us =3D strtoull(str, &endptr, 10); + + /* threshold isn't string "0", yet strtoull() returns 0, parsing failed */ + if (*endptr || (off_cpu_thresh_us =3D=3D 0 && strcmp(str, "0"))) + return -EINVAL; + else + opts->off_cpu_thresh_us =3D off_cpu_thresh_us; + + return 0; +} + void __weak arch__add_leaf_frame_record_opts(struct record_opts *opts __ma= ybe_unused) { } @@ -3342,6 +3364,7 @@ static struct record record =3D { .ctl_fd =3D -1, .ctl_fd_ack =3D -1, .synth =3D PERF_SYNTH_ALL, + .off_cpu_thresh_us =3D OFFCPU_THRESH, }, }; =20 @@ -3564,6 +3587,9 @@ static struct option __record_options[] =3D { OPT_BOOLEAN(0, "off-cpu", &record.off_cpu, "Enable off-cpu analysis"), OPT_STRING(0, "setup-filter", &record.filter_action, "pin|unpin", "BPF filter action"), + OPT_CALLBACK(0, "off-cpu-thresh", &record.opts, "us", + "Dump off-cpu samples if off-cpu time reaches this threshold. The u= nit is microsecond (default: 500000)", + record__parse_off_cpu_thresh), OPT_END() }; =20 diff --git a/tools/perf/util/off_cpu.h b/tools/perf/util/off_cpu.h index 2dd67c60f211..c6edc0f7c40d 100644 --- a/tools/perf/util/off_cpu.h +++ b/tools/perf/util/off_cpu.h @@ -16,6 +16,7 @@ struct record_opts; PERF_SAMPLE_PERIOD | PERF_SAMPLE_CALLCHAIN | \ PERF_SAMPLE_CGROUP) =20 +#define OFFCPU_THRESH 500000ull =20 #ifdef HAVE_BPF_SKEL int off_cpu_prepare(struct evlist *evlist, struct target *target, diff --git a/tools/perf/util/record.h b/tools/perf/util/record.h index a6566134e09e..2ca74add26c0 100644 --- a/tools/perf/util/record.h +++ b/tools/perf/util/record.h @@ -79,6 +79,7 @@ struct record_opts { int synth; int threads_spec; const char *threads_user_spec; + u64 off_cpu_thresh_us; }; =20 extern const char * const *record_usage; --=20 2.43.0 From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4FBC18F6C; Wed, 13 Nov 2024 00:28:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457706; cv=none; b=eupcL6qJ4TZakBovCs6NEP5sY9uRSeZPOJC9kYeBs5C0yRouIQjEiaBAdfx7Vn4bUN4QZxrJgRT0jSou+gH5xXeiDb4U3pqV8EwA3ooOUPvTcjRwgvwwitxhizxEO/Lkr3lEF6HcR/DGskBnupJ4jLaasQ6tGl1oV8w760g4WSI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457706; c=relaxed/simple; bh=k1MvbZqv805gektgE0goJNI/HxV1JhVVKX8JGAJXSvo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=POlRzLFdERuA27nwDMiu2ltYcmMRQntXl55o1WyId9XTrsVtIjdUVXEelkDdXhus9xv2F0KKjTp6J/L56JtEOD0mtLlbXIt1z97dpeUjZ/IsbykIAqpce/YaUOkuwCjh9SVaeuvTCI/5TihottPd/4RkAiog2F5A5Pp6Iz6PSUg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CQOfA5+Q; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CQOfA5+Q" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-20e6981ca77so72870955ad.2; Tue, 12 Nov 2024 16:28:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457705; x=1732062505; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IpYScqwzO6WE7u35hrRfQ2TMuWp9RDhpSIjI9CQVyMs=; b=CQOfA5+QkLaUk/xKlL1zC/4vJG0eXzojCclRnHweYoBqz6s+uawCm++t/L5jJ6SuW4 wKl93dHSLPUoG/BkFXR8pL25jdsqM3T0KjQldpYZzStIzPIsb23HGpy9PQSPiE4stp6r wpaV+SDOCAJGgY34869ehS1g53lvDXgREi+qAwCttEsvib6jMh4Yti46NOQaAj/9m5fo /crd7yFVKCKAiHDA/oIJOavAANQbd3UGgPMRjRBNuwxTJfZweGP9kgHsdKMf4SJTkyUj WuQwkvxMNKhhGSN/yNqd3v+QhCnPivtBSWmyaq96fjyr55Hnw3lTlPKKm6CLyVhcBUxH uzxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457705; x=1732062505; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IpYScqwzO6WE7u35hrRfQ2TMuWp9RDhpSIjI9CQVyMs=; b=QT4z+41uYEf2ortJZyEDzpYis8gBTkB7ahMTxtFMqT494nWcR37SYdFT44aGMfBpuy hbAV5J3yqyM4C1Lbq3wY65A+sOf1Pi2FIvHB+NTG1MNU/DGX0tlYOUQBjXa0mI5EazQY 767W6C/A7R/O5dB5W21rbT2BBNNGSuRf9GZ3xq23jbFl8EGm/G+57dmCzjc5AItj5lGw /jpaTDIsfeVS6kA77g15NQE2AQYczAheH4Koc7SW4GkExqwj0gpqNUsxJbPOyT4ODDwr 1TrMvy+7Q0UctRM0VqtJFwVxpqhCCLVic3IpdDlJpbLU70u8vR3UNLtgeQe3yE6+TEi6 PAJQ== X-Forwarded-Encrypted: i=1; AJvYcCUklQtQlJlsyAdvyPFed8GJ/isXA9KM3adIEUbSQyPv7zsK4sI0iyhmQpZAY/p4lITxmxUPhws6+goJl5Xm4pB6vA==@vger.kernel.org, AJvYcCWRHaKh9HopXV5EAP6T0poP0AqkOtjBnfV80h8UilxK7wi7x+6iQHBxP9VccBvhpAGPzwIFNX3pjR1mtGg=@vger.kernel.org X-Gm-Message-State: AOJu0YwC6BovGPXVRLLdxb8ez2JhJIAVpBU7XJdKOQxak3Z9e9scXewH kgUMzcSmBz3s7KVK9wzcZFQ+Df4JAiZ/H8jUeZfkJtp5PkCz90ZQ X-Google-Smtp-Source: AGHT+IHyOviXKH9EpfB9u/+bkxMSW6FybtaQ5PzQ61gBZaCc5XxYEQpGGKGAFuJtQqik9f600yFanQ== X-Received: by 2002:a17:902:f691:b0:20b:ce30:878d with SMTP id d9443c01a7336-211b5c7be75mr9432745ad.23.1731457704622; Tue, 12 Nov 2024 16:28:24 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:24 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 02/10] perf evsel: Expose evsel__is_offcpu_event() for future use Date: Tue, 12 Nov 2024 16:28:10 -0800 Message-ID: <20241113002818.3578645-3-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose evsel__is_offcpu_event() so it can be used in off_cpu_config(), evsel__parse_sample(), and perf script. Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-3-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/evsel.c | 2 +- tools/perf/util/evsel.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index f745723d486b..8d0308f62484 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -1194,7 +1194,7 @@ static void evsel__set_default_freq_period(struct rec= ord_opts *opts, } } =20 -static bool evsel__is_offcpu_event(struct evsel *evsel) +bool evsel__is_offcpu_event(struct evsel *evsel) { return evsel__is_bpf_output(evsel) && evsel__name_is(evsel, OFFCPU_EVENT); } diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h index 04934a7af174..7f68004507d8 100644 --- a/tools/perf/util/evsel.h +++ b/tools/perf/util/evsel.h @@ -554,4 +554,6 @@ u64 evsel__bitfield_swap_branch_flags(u64 value); void evsel__set_config_if_unset(struct perf_pmu *pmu, struct evsel *evsel, const char *config_name, u64 val); =20 +bool evsel__is_offcpu_event(struct evsel *evsel); + #endif /* __PERF_EVSEL_H */ --=20 2.43.0 From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F3D5282FA; Wed, 13 Nov 2024 00:28:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457711; cv=none; b=a7G6SRnTdUNqZb5rYNAIRK1lYqRJtaszaUelieLU7yDqYEgL5OZPm9ogjsvZsJWHYWCJOPgfS//CxhcY8eFvU5ACcOrEzQXGRE4HjPlFtleCaKWLETi/bz7eWVS4qbj/Vmzj4/Vlaxpr0coa1M6TVkIJXAaUrrp16tobskTm0kk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457711; c=relaxed/simple; bh=ZqaG7qQsHv1GLxTzONUQia8uX9AiQpjHtvv5aJZS8x4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SeiIvfWLGT6zoFf+CTJPEffEE9w8P6IBDZgb8LZ4I3xj+PD6TOJ8xSxiA3/fJQd8JsKQKfK/Tcs6XluquS/r6L+oORxze07kF+e0H7zXGVWCop4bc1Q5B3UXW8DHUzTTF8NUWRiY9tyqWVc0frmpjNv4Yiz55J7MYzGycjJ3xMU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MBVPHY4M; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MBVPHY4M" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-21145812538so58161075ad.0; Tue, 12 Nov 2024 16:28:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457706; x=1732062506; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jJ4iGWhCCKarEcyzjNazQFkqawuKP38CgPp1DBRk+2w=; b=MBVPHY4MkKJHROrso1lv2oblvZrMJkrewvRTrdbvhPC9Fssxkl+HMawfyaPCKJmB6z XLdNZ6DSfiMVfWSezbm6ki0n23MKqhr9WR6bwKnHlws/hUjnwF4HuF83suJiHxf8uo9j i7VCKmpDBRwegOeC/l+PClVVDWxXOAcQiDv8T+pgdwig4DPKYUhaWlODNrzVO9R/hJ8K b7YoogHzjh3QRb3T0We7lq7bJuxVJCwPYoWyH4tA+uSHhR1Z0od7T4N5PCOcK1wTTQ0S Vhrn48I0NPRzb1HChu3wZuEhoJ0a76y7ueFHzlt3Fzv+eB0QytVEUcV3stNpbKOxsD75 iXHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457706; x=1732062506; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jJ4iGWhCCKarEcyzjNazQFkqawuKP38CgPp1DBRk+2w=; b=nrA01TEVMGt+e0UPUvNJitUGGB/Zd3g5PLXzj8TKZ4FROjezKzSnMlpbbvJK0hhV6P dXeYnC3JjKBQvVBiGoMh6ufcnOhkKCnAOCBfRyeEuHKqHxe92LsiEX2f8MV7ZbhznKav fk1qaEIuSenAYAaGoS48ad+ZJS4fns4QwZNeIZGpryYjy3HypytajDpC5outXOxXG9+b 4WxYU6XLWNrc4NOYNIxTlXCXdJM1XPKV3nXZx9/CLi33HUViOYY//Bwg9mloDvzpeRfa AgcG838B1ClPzQyJfYnUdgkRO4tIhMjZSwy5saRzXWqltoKEid6cEVtSjpob1K64FQo3 l2Sw== X-Forwarded-Encrypted: i=1; AJvYcCUGxFE3xj29cjBN1ib116SYDl/pP5KdekAT4Dly6quuZO3snSUKdlohMVsa98WHRslnA2Hx68xrNZylp2q6dirrXw==@vger.kernel.org, AJvYcCVHfii5V7mMmuzhNWDvJ/MaPwgqs/MCT6PVHuf0C9d8sHf8WMlSEvCJpnh1VI37oDujT72XFBRlfGL7GYI=@vger.kernel.org X-Gm-Message-State: AOJu0Yy7xnNJ7U+Y2zA9Fnj4CsWRNq0Td+41Ugq77TYeJ00LiIzSYXLL OlT468nW0eJAJcl/t989pxzAzJ5Pc4g8/c4HRZu0BmecW1Z+gKlr X-Google-Smtp-Source: AGHT+IEyu7idpdRNyFr5ItESM+KXN0ZmDU9eiRnk2BYsGMAkANAg8BnuqI9XWimjEa/V5fC7KZUXgA== X-Received: by 2002:a17:902:d2c8:b0:211:6b21:2370 with SMTP id d9443c01a7336-21183ced53dmr251729095ad.29.1731457706121; Tue, 12 Nov 2024 16:28:26 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:25 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 03/10] perf record --off-cpu: Parse off-cpu event Date: Tue, 12 Nov 2024 16:28:11 -0800 Message-ID: <20241113002818.3578645-4-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Parse the off-cpu event using parse_event(), as bpf-output. no-inherit should be set to 1, here's the reason: We update the BPF perf_event map for direct off-cpu sample dumping (in following patches), it executes as follows: bpf_map_update_value() bpf_fd_array_map_update_elem() perf_event_fd_array_get_ptr() perf_event_read_local() In perf_event_read_local(), there is: int perf_event_read_local(struct perf_event *event, u64 *value, u64 *enabled, u64 *running) { ... /* * It must not be an event with inherit set, we cannot read * all child counters from atomic context. */ if (event->attr.inherit) { ret =3D -EOPNOTSUPP; goto out; } Which means no-inherit has to be true for updating the BPF perf_event map. Moreover, for bpf-output events, we primarily want a system-wide event instead of a per-task event. The reason is that in BPF's bpf_perf_event_output(), BPF uses the CPU index to retrieve the perf_event file descriptor it outputs to. Making a bpf-output event system-wide naturally satisfies this requirement by mapping CPU appropriately. Suggested-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-4-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/bpf_off_cpu.c | 33 +++++++++++---------------------- 1 file changed, 11 insertions(+), 22 deletions(-) diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index a590a8ac1f9d..558c5e5c2dc3 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -38,32 +38,21 @@ union off_cpu_data { =20 static int off_cpu_config(struct evlist *evlist) { + char off_cpu_event[64]; struct evsel *evsel; - struct perf_event_attr attr =3D { - .type =3D PERF_TYPE_SOFTWARE, - .config =3D PERF_COUNT_SW_BPF_OUTPUT, - .size =3D sizeof(attr), /* to capture ABI version */ - }; - char *evname =3D strdup(OFFCPU_EVENT); - - if (evname =3D=3D NULL) - return -ENOMEM; =20 - evsel =3D evsel__new(&attr); - if (!evsel) { - free(evname); - return -ENOMEM; + scnprintf(off_cpu_event, sizeof(off_cpu_event), "bpf-output/no-inherit=3D= 1,name=3D%s/", OFFCPU_EVENT); + if (parse_event(evlist, off_cpu_event)) { + pr_err("Failed to open off-cpu event\n"); + return -1; } =20 - evsel->core.attr.freq =3D 1; - evsel->core.attr.sample_period =3D 1; - /* off-cpu analysis depends on stack trace */ - evsel->core.attr.sample_type =3D PERF_SAMPLE_CALLCHAIN; - - evlist__add(evlist, evsel); - - free(evsel->name); - evsel->name =3D evname; + evlist__for_each_entry(evlist, evsel) { + if (evsel__is_offcpu_event(evsel)) { + evsel->core.system_wide =3D true; + break; + } + } =20 return 0; } --=20 2.43.0 From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F40B28E37; Wed, 13 Nov 2024 00:28:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457709; cv=none; b=rVPs29ZTrWh53NqScyz3cJxjucp+dGUYJHAOgvdeIjXeu9o8kNj9H9pMQ90j6yFPIlhcc/ZQ58F9zev/dvCmfNoRpe3BwgEIz0p1yal03XU9Vy6chZCDTjBB1Cdv1f4/ySAJLgRlMXhxkRNdfcMHyLi/rkN1WMgDQWz3rrmtGEE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457709; c=relaxed/simple; bh=FnN0JefpK/XJbE40rMOuzyObGxpocXm8qyCa6PsXe3s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LR4s59jdgTWiBJN9ffzo0isqydhY/U4RXx4Cp+EZo0OyFo8h8D4mR3fPyFAgdxq26PneLMzDb/DYquowzvLdbugk8I82/LRwXUOw3BMmC+BXG49TTUfQjIeLVgqJAPL7AZ8ZLfaXxPFBMwJeZMsSW6dPDJIAomKEoucaNY5gRyU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bAAJK3Bg; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bAAJK3Bg" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-20c805a0753so62265925ad.0; Tue, 12 Nov 2024 16:28:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457708; x=1732062508; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HqhOub90lKje0JDDIfka68HwdQxfOlgwod/Qhwqzi9c=; b=bAAJK3BgQmNuNkOnVi4kaQk5Pnnoqi5OLtcmY3H2ZBmUkfbW2D3e5iSEJ4ukKqtNPf 4oFq3i7GL57JIdhU3/8DWZIgfQG9Gfd5UGmU3T5w4pgJJWR3Aq+j/SzroOsJpXgys7Nc J/0qmyabwnuxrJJk4v9aZD7bH5smqwDRfba+rU9/SkGyW3eYbW0Ukd7lQNOdS8eEczdu gZ/bceWY9I82VqfrhGaxUTNaeryKiqBAnFbYwykgGJYoHEt54ynB5ArHW3F7bCZcYGVM 88mVZ3mZuKfWRypmWvJyfRb6KizhgcwcmUOznHmMG8BzgHj8aX7ERP7gl4frdQuYN/hc VyYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457708; x=1732062508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HqhOub90lKje0JDDIfka68HwdQxfOlgwod/Qhwqzi9c=; b=JJpSmuX3Kio2/b82CbV+kSts7eZpzErlbBxlrYrcX1ELLUe+htvDsOOL16JRjbfB3d FFvm/HW6qpkNVv5mHyblSa096bCQu2f63O9/h7jn2YNBh64SZ1gPldhc1zzJGJQz6bAN 6sFjMbAXDpSvfKalitRDLWCL8yUdAG10sdbjm9sGjKm/6Jp32hKYEhFw48sLXPqwaLG/ FtfxNKMNneqFnbGIAQuEp77FYbfzMH7VGZZh/xCRV9iRsPfix1EElVatWdFFd2kLvAsR Czxqf6AIkxcztK/tXwJY1a/4Z53StqIM9j+ikXE4dE8gQ/sslXZkVNCG1zpVAE4Pf1Gs jMDA== X-Forwarded-Encrypted: i=1; AJvYcCXYwUYOFzB6FIQ2X9hUimbUM9wZQKPjJD5DBK4w56Q0BDWlPTtDjNkU7Mu9KJl63gFAK7TBmyXvvUZwOFU=@vger.kernel.org, AJvYcCXnBWRktNXc4lCSBjwSTR4Z2fW7lJIrWL4P6qThCKnesgdI48pdEFitYKXyDfqLXHw5eyV1GNt0VZgWDPtNXvJnYw==@vger.kernel.org X-Gm-Message-State: AOJu0Ywu1H+43EXsSzeFCxrhc9OrTQgyr6cZ0NVI3cO9fuV0Y1pNuP5k M5KrfYAxImr/SUByECfMj/C/0pqNeIygFr78i0j/Y/JdGF6fJpbb X-Google-Smtp-Source: AGHT+IGQY45Hat16INkJ6kDUhh9NygDODyIXVVIZS9gwbOUKG2dAC/O8EB7uMEN1h5cQXdOdOg2uyw== X-Received: by 2002:a17:903:1104:b0:20c:da66:3875 with SMTP id d9443c01a7336-21183d52b6dmr221292675ad.24.1731457707644; Tue, 12 Nov 2024 16:28:27 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:27 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 04/10] perf record --off-cpu: Preparation of off-cpu BPF program Date: Tue, 12 Nov 2024 16:28:12 -0800 Message-ID: <20241113002818.3578645-5-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set the perf_event map in BPF for dumping off-cpu samples. Set the offcpu_thresh to specify the threshold. Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-5-howardchu95@gmail.= com [ Added some missing iteration variables to off_cpu_config() and fixed up a manually edited patch hunk line boundary line ] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/bpf_off_cpu.c | 25 +++++++++++++++++++++++++ tools/perf/util/bpf_skel/off_cpu.bpf.c | 11 +++++++++++ 2 files changed, 36 insertions(+) diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index 558c5e5c2dc3..61729a65b529 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -13,6 +13,7 @@ #include "util/cgroup.h" #include "util/strlist.h" #include +#include =20 #include "bpf_skel/off_cpu.skel.h" =20 @@ -60,6 +61,9 @@ static int off_cpu_config(struct evlist *evlist) static void off_cpu_start(void *arg) { struct evlist *evlist =3D arg; + struct evsel *evsel; + struct perf_cpu pcpu; + int i; =20 /* update task filter for the given workload */ if (skel->rodata->has_task && skel->rodata->uses_tgid && @@ -73,6 +77,25 @@ static void off_cpu_start(void *arg) bpf_map_update_elem(fd, &pid, &val, BPF_ANY); } =20 + /* update BPF perf_event map */ + evsel =3D evlist__find_evsel_by_str(evlist, OFFCPU_EVENT); + if (evsel =3D=3D NULL) { + pr_err("%s evsel not found\n", OFFCPU_EVENT); + return; + } + + perf_cpu_map__for_each_cpu(pcpu, i, evsel->core.cpus) { + int err; + + err =3D bpf_map__update_elem(skel->maps.offcpu_output, &pcpu.cpu, sizeof= (__u32), + xyarray__entry(evsel->core.fd, i, 0), + sizeof(__u32), BPF_ANY); + if (err) { + pr_err("Failed to update perf event map for direct off-cpu dumping\n"); + return; + } + } + skel->bss->enabled =3D 1; } =20 @@ -261,6 +284,8 @@ int off_cpu_prepare(struct evlist *evlist, struct targe= t *target, } } =20 + skel->bss->offcpu_thresh_ns =3D opts->off_cpu_thresh_us * 1000; + err =3D off_cpu_bpf__attach(skel); if (err) { pr_err("Failed to attach off-cpu BPF skeleton\n"); diff --git a/tools/perf/util/bpf_skel/off_cpu.bpf.c b/tools/perf/util/bpf_s= kel/off_cpu.bpf.c index c152116df72f..c87132e01eb3 100644 --- a/tools/perf/util/bpf_skel/off_cpu.bpf.c +++ b/tools/perf/util/bpf_skel/off_cpu.bpf.c @@ -18,6 +18,8 @@ #define MAX_STACKS 32 #define MAX_ENTRIES 102400 =20 +#define MAX_CPUS 4096 + struct tstamp_data { __u32 stack_id; __u32 state; @@ -39,6 +41,13 @@ struct { __uint(max_entries, MAX_ENTRIES); } stacks SEC(".maps"); =20 +struct { + __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(__u32)); + __uint(max_entries, MAX_CPUS); +} offcpu_output SEC(".maps"); + struct { __uint(type, BPF_MAP_TYPE_TASK_STORAGE); __uint(map_flags, BPF_F_NO_PREALLOC); @@ -97,6 +106,8 @@ const volatile bool uses_cgroup_v1 =3D false; =20 int perf_subsys_id =3D -1; =20 +__u64 offcpu_thresh_ns; + /* * Old kernel used to call it task_struct->state and now it's '__state'. * Use BPF CO-RE "ignored suffix rule" to deal with it like below: --=20 2.43.0 From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E73B82B9BC; Wed, 13 Nov 2024 00:28:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457711; cv=none; b=ig3CA8CBFNAZ/GrZpt6yZU5//OWIw/mP2pXiEpI7upx4HDBw15KjXNtX9LwyEJDcO1sqYAxixGQgNppnMhC9zxROBrfBNJZVCgI/CIRkl6X/zWJANikcYJOq33O48PSsTJxRrzfGBxWH83ZsyXleR4dsfzwJiDsN9UPgY94Htak= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457711; c=relaxed/simple; bh=FhYBFcBipaHcJVExkC+3V5vt7nolG31xdwTCXFsCPv8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tpXJuaryyDhELyckCkGSWuLWkFb+IJUZiQIctRm0gqaAW9z7w1nHcXzSUBZIxE+2cFAiECbwi7xZnZE8XyADsJ7zmVGspltV3BkQVciwv0Oi8LwJwpPdQXoSF8Xciq1SpyqWYbCBFYTRmutfP5T2ByGUY5QBTi/jWLGtW+a/s2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WuI6N243; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WuI6N243" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-20c803787abso1708815ad.0; Tue, 12 Nov 2024 16:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457709; x=1732062509; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HM89nxEPCN1/mXqArlw/4nxHHrqhLHtRHulGS1q1v6g=; b=WuI6N2435/69kIR6J7rIdeOga1jvQOevLCE4na9Gq5OmBUzLppiKed+Dbzym2rhKnG MKHfUDDvvK/OAxyMTGCLD7lBANsOdGhVtvgOvq+MRh3e14oZTKGl1IahWDxrpcpHZPey Vu7kY73hsApzEBycUXKuQiFHxdLh0mzLQFmToEXNm/n97QqedSJ9B5Gmteep1PPiwXK1 eJkZHRTdjHIlkXLfoMVQYFhyK91Y4AkSbKR6vmI3mwJWEibSu6y/rmLpwytZLhM0iGCF biW7GW2i4z8Ba1XIElE82Sv96LOP1W95voXyEdrR7ViD6GR1PWs/xGx6DUu/Q2d9/xhD KW0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457709; x=1732062509; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HM89nxEPCN1/mXqArlw/4nxHHrqhLHtRHulGS1q1v6g=; b=Oj1el4HbmQpmkgYVVO+zp/56VmHJj2dr+NL+lMIFdr6szvGE0UcJwpGO2vhF1NJNvJ kn1vUJFE6J1PMSmboTkQhMwRZQPJ1kqbckVHk7zJ6vOv7SFeC80fXRqwqAFh6aTkJk/C EnypHPLFrfajE3eScAsArKdqT5XwH20ejtD3+kf/xsqS63SkG0jJK12JVGJzuZAEqRMP lOSO/R5fEMTOYKq7Agq3YoaLCby/m+bjycjjHSFSrPNiH2V2TwUszeUc2mvAXlwAgWRg 9MhIV4s7uE30Ovew8zHsyHSn81Tws0nwhLp/fBrzl1Ncv/fbIP8ZC7xgJUuKj4CGyFQ7 oOPA== X-Forwarded-Encrypted: i=1; AJvYcCUr+r3sJC+kf/HyjD2jbeP34pa1Ze6lUam/cIl7n7xiiS5E3WefWVPUZXfLWuCXp8vcXIkfp3JvcNnTP9aiNHgBqA==@vger.kernel.org, AJvYcCVmolAdfucU0Hh1hYvrNu6zLOHSUDWzLbQDWy64Yy3ZnyE1J8D9zIJ9ZuuNdT7/2PSOTd/k3ZSfSFSFxWo=@vger.kernel.org X-Gm-Message-State: AOJu0YyJZH2opZuZuSjC7utR0QbWS21ZX3CC1uJ7PgpDM/Ro9fZ2Ldhi /eCNXPOW7dou+z/C2ZBYOxfbqUp8F0eJ5GwCjQbJOYQy2kfqqKpE X-Google-Smtp-Source: AGHT+IHrQ5cn+T07aVsaHLPMLxQxSg2+xjL27GuTAYLQtp8AH1pRu3KFuF1Ma8D8cAC4R6z4z5e+qg== X-Received: by 2002:a17:902:c412:b0:20c:da7c:6e8c with SMTP id d9443c01a7336-2118215cc97mr223373435ad.3.1731457709183; Tue, 12 Nov 2024 16:28:29 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:28 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 05/10] perf record --off-cpu: Dump off-cpu samples in BPF Date: Tue, 12 Nov 2024 16:28:13 -0800 Message-ID: <20241113002818.3578645-6-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Collect tid, period, callchain, and cgroup id and dump them when off-cpu time threshold is reached. We don't collect the off-cpu time twice (the delta), it's either in direct samples, or accumulated samples that are dumped at the end of perf.data. Suggested-by: Namhyung Kim Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-6-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Reviewed-by: Ian Rogers --- tools/perf/util/bpf_skel/off_cpu.bpf.c | 86 ++++++++++++++++++++++++-- 1 file changed, 81 insertions(+), 5 deletions(-) diff --git a/tools/perf/util/bpf_skel/off_cpu.bpf.c b/tools/perf/util/bpf_s= kel/off_cpu.bpf.c index c87132e01eb3..aae63d999abb 100644 --- a/tools/perf/util/bpf_skel/off_cpu.bpf.c +++ b/tools/perf/util/bpf_skel/off_cpu.bpf.c @@ -19,11 +19,17 @@ #define MAX_ENTRIES 102400 =20 #define MAX_CPUS 4096 +#define MAX_OFFCPU_LEN 37 + +struct stack { + u64 array[MAX_STACKS]; +}; =20 struct tstamp_data { __u32 stack_id; __u32 state; __u64 timestamp; + struct stack stack; }; =20 struct offcpu_key { @@ -41,6 +47,10 @@ struct { __uint(max_entries, MAX_ENTRIES); } stacks SEC(".maps"); =20 +struct offcpu_data { + u64 array[MAX_OFFCPU_LEN]; +}; + struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); __uint(key_size, sizeof(__u32)); @@ -48,6 +58,13 @@ struct { __uint(max_entries, MAX_CPUS); } offcpu_output SEC(".maps"); =20 +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(struct offcpu_data)); + __uint(max_entries, 1); +} offcpu_payload SEC(".maps"); + struct { __uint(type, BPF_MAP_TYPE_TASK_STORAGE); __uint(map_flags, BPF_F_NO_PREALLOC); @@ -194,6 +211,47 @@ static inline int can_record(struct task_struct *t, in= t state) return 1; } =20 +static inline int copy_stack(struct stack *from, struct offcpu_data *to, i= nt n) +{ + int len =3D 0; + + for (int i =3D 0; i < MAX_STACKS && from->array[i]; ++i, ++len) + to->array[n + 2 + i] =3D from->array[i]; + + return len; +} + +/** + * off_cpu_dump - dump off-cpu samples to ring buffer + * @data: payload for dumping off-cpu samples + * @key: off-cpu data + * @stack: stack trace of the task before being scheduled out + * + * If the threshold of off-cpu time is reached, acquire tid, period, callc= hain, and cgroup id + * information of the task, and dump it as a raw sample to perf ring buffer + */ +static int off_cpu_dump(void *ctx, struct offcpu_data *data, struct offcpu= _key *key, + struct stack *stack, __u64 delta) +{ + int n =3D 0, len =3D 0; + + data->array[n++] =3D (u64)key->tgid << 32 | key->pid; + data->array[n++] =3D delta; + + /* data->array[n] is callchain->nr (updated later) */ + data->array[n + 1] =3D PERF_CONTEXT_USER; + data->array[n + 2] =3D 0; + len =3D copy_stack(stack, data, n); + + /* update length of callchain */ + data->array[n] =3D len + 1; + n +=3D len + 2; + + data->array[n++] =3D key->cgroup_id; + + return bpf_perf_event_output(ctx, &offcpu_output, BPF_F_CURRENT_CPU, data= , n * sizeof(u64)); +} + static int off_cpu_stat(u64 *ctx, struct task_struct *prev, struct task_struct *next, int state) { @@ -218,6 +276,16 @@ static int off_cpu_stat(u64 *ctx, struct task_struct *= prev, pelem->state =3D state; pelem->stack_id =3D stack_id; =20 + /* + * If stacks are successfully collected by bpf_get_stackid(), collect the= m once more + * in task_storage for direct off-cpu sample dumping + */ + if (stack_id > 0 && bpf_get_stack(ctx, &pelem->stack, MAX_STACKS * sizeof= (u64), BPF_F_USER_STACK)) { + /* + * This empty if block is used to avoid 'result unused warning' from bpf= _get_stack(). + * If the collection fails, continue with the logic for the next task. + */ + } next: pelem =3D bpf_task_storage_get(&tstamp, next, NULL, 0); =20 @@ -232,11 +300,19 @@ static int off_cpu_stat(u64 *ctx, struct task_struct = *prev, __u64 delta =3D ts - pelem->timestamp; __u64 *total; =20 - total =3D bpf_map_lookup_elem(&off_cpu, &key); - if (total) - *total +=3D delta; - else - bpf_map_update_elem(&off_cpu, &key, &delta, BPF_ANY); + if (delta >=3D offcpu_thresh_ns) { + int zero =3D 0; + struct offcpu_data *data =3D bpf_map_lookup_elem(&offcpu_payload, &zero= ); + + if (data) + off_cpu_dump(ctx, data, &key, &pelem->stack, delta); + } else { + total =3D bpf_map_lookup_elem(&off_cpu, &key); + if (total) + *total +=3D delta; + else + bpf_map_update_elem(&off_cpu, &key, &delta, BPF_ANY); + } =20 /* prevent to reuse the timestamp later */ pelem->timestamp =3D 0; --=20 2.43.0 From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C52945473C; Wed, 13 Nov 2024 00:28:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457714; cv=none; b=OSTQPtmTtdDl+h4eY8BJrzVAt0/Am+gAbKIk/4vqyOFUelRmnV1SIkGKMPdnycygJVxflWxVyu1Ezg4VfnKXEAyf+yfJ20VKPq2pKnKd7j2dnL09F+QAmIL5F68+lECSxrZ3NHSsWjNCN69MrR8TiO3Q+hLoM7B/ml3gzSFpXmA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457714; c=relaxed/simple; bh=zCl0xnl89SvnrshuskE96re8cOP94+xKShT4NmxSfSA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bkopoF6UPX6gI1jBu7saAyibpve8kP12IFknl2UM4mBPsVHz9Nvfr49TacVXaHw/hFyoHWNYXrleCUyZbg/AQEcfDkshVu/LiOkdDd9N6ObNGVxsVcgGu5MVdeXP6qZnvO9/nYMplDQZJiPQT9noRoKi+sZorqOsD53QSbXK6hg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=c58sxkHN; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="c58sxkHN" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-20c714cd9c8so64657805ad.0; Tue, 12 Nov 2024 16:28:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457712; x=1732062512; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sGntfCGquRPrSkUE9BoFC9ltLxK8sMXbqjMOtZuSP1w=; b=c58sxkHNbztmVmQR6XJ8WrV97wFP/rLQyC43WK5oa9qO0J3fo+RVWTuZCM2KC5tgX2 jXk+/Yliy65wvih9B6uIkyJ8HU5XQnOrDCHIZRDRT2I5AQlAhyGMy1VhnjihnN9RGts5 3FeO9aDAU5whYzSc5t81vLQRJ2tfMBKDSB0wqNT3vzLHcf2QxKAM5tjkHqUQ6MGt988X 6KXo6sU+8HiJ8MB+S/LJFErTJU0UtyfQIH5XXREg9GmPdRtRfK99+Vh4cZRjzESjxupk fGaSz+ozhLXiWlkwBVloYUIhWUC3A+GKteguDYMXP0fdS5TAfSsZEWIlLUZjfV1nzRwy mvIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457712; x=1732062512; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sGntfCGquRPrSkUE9BoFC9ltLxK8sMXbqjMOtZuSP1w=; b=wb7ickdanAogMewLLEES/8/EiNumxaaUXuDraASt5YFjA/iVbMreWzTlpP9FKtvM8A YBBwCOfaYKYq2xR8cQhkzbmemocSHzqz+Ff+6gqXg/gNaamf7l7JUqEOeIALomKa/APY 9lVXQ/5FXTa980I8QYyyLhnvJCdz7bEFH3ZctLOsCNGItrFH9ZFU4m/esRaMjDHdAkru 5ZheSr0SMypXNYzHsLiJH4OXSuRLrcD0dsn/vqbI7OSLqCElvNvrG3+I4V3xI1J5g3SG hjHLofwnl1GQx9JUKbKq2OSCQ2Vs/TwklPV1onuQY8irSFY3ulhJKl8fvV5wHgmvIfjc Eaqw== X-Forwarded-Encrypted: i=1; AJvYcCW0CbDXxPeKi23OSNi6rwvOC0QUA9xItQXz7gpP+nOdpDkn64g16eo3gQpUT3hUyBirHI7lYqDGeFss5Bzqho8kUQ==@vger.kernel.org, AJvYcCWZ1UMth3IcuXSykdiWWWvyVPmTLbmNSq4GdNWeloF7chsFpfmvV8l43lNhhd8l2Uwub3Ys4naK0o5LVwY=@vger.kernel.org X-Gm-Message-State: AOJu0Yx0x008jawjGjKtND1m/k7qsLyoHo5f4NEC5uTVtKPuh2IbIadT CV5Np/5Dg3EzQf6tPh+AdCGQ2YqBxjqTwALVc4G8NREb21SzTbSx X-Google-Smtp-Source: AGHT+IGq42LbyfeKhm5l3kMgRH4KezRUf2lapAMDVAc86sftYZ+f18bQal2lOEtwuZdx3aUI7gP2Kw== X-Received: by 2002:a17:902:cf12:b0:20c:528d:7063 with SMTP id d9443c01a7336-21183d4baa1mr270807775ad.19.1731457710719; Tue, 12 Nov 2024 16:28:30 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:30 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 06/10] perf evsel: Assemble offcpu samples Date: Tue, 12 Nov 2024 16:28:14 -0800 Message-ID: <20241113002818.3578645-7-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the data in bpf-output samples, to assemble offcpu samples. Suggested-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-7-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/evsel.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index 8d0308f62484..654fb5196ecf 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -2792,6 +2792,35 @@ static inline bool evsel__has_branch_counters(const = struct evsel *evsel) return false; } =20 +static int __set_offcpu_sample(struct perf_sample *data) +{ + u64 *array =3D data->raw_data; + u32 max_size =3D data->raw_size, *p32; + const void *endp =3D (void *)array + max_size; + + if (array =3D=3D NULL) + return -EFAULT; + + OVERFLOW_CHECK_u64(array); + p32 =3D (void *)array++; + data->pid =3D p32[0]; + data->tid =3D p32[1]; + + OVERFLOW_CHECK_u64(array); + data->period =3D *array++; + + OVERFLOW_CHECK_u64(array); + data->callchain =3D (struct ip_callchain *)array++; + OVERFLOW_CHECK(array, data->callchain->nr * sizeof(u64), max_size); + data->ip =3D data->callchain->ips[1]; + array +=3D data->callchain->nr; + + OVERFLOW_CHECK_u64(array); + data->cgroup =3D *array; + + return 0; +} + int evsel__parse_sample(struct evsel *evsel, union perf_event *event, struct perf_sample *data) { @@ -3143,6 +3172,9 @@ int evsel__parse_sample(struct evsel *evsel, union pe= rf_event *event, array =3D (void *)array + sz; } =20 + if (evsel__is_offcpu_event(evsel)) + return __set_offcpu_sample(data); + return 0; } =20 --=20 2.43.0 From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD28861FD8; Wed, 13 Nov 2024 00:28:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457715; cv=none; b=IKjhqHFbEWHvewlZyZP+l4LNenWHeJbuFRmr6agecFZrneNby6Z3wxlu8HO6s/Dt8f5U491FTp0Zd4dhaVpiacTAPA3MJKf01yXV1athN9eKllowq4lNAFzdFq/Bi+b2Z/QEgFvFX/GROtU90B4Jjo9rCB8NK05wM4w2b6EttMk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457715; c=relaxed/simple; bh=vccFHYr8ZR9cgyM4HGgwFgDtARhDnJAfANCV1e9IKOY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cafM8LNK2ekFedqjiCxSERjbzTS9ZQ0vJr7zOmhoNmhySYaNkg2uawpKBTHuG41yClQToJjmkp8A+AhhRQ1FFMKA/vfydKeRqhurlCIz1gkpwhNegsWVM2XKjY8xfyLzPe9YRbUhVCB4JAz1Dbylj6G5bVKNbTW/sicbi8y9EI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=N8PWPOKY; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="N8PWPOKY" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-20cb47387ceso65543915ad.1; Tue, 12 Nov 2024 16:28:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457712; x=1732062512; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hBlhrPHwwByaR2iQ3j+uYlOmJcTMAeh7R4Wsq+6yt50=; b=N8PWPOKYLcPDhX4ZIO2waw4OhXLvKiGDtFGDyAQ0ZzUJ5K+MU83ymVKyLkMjQFp+sK dViFyJSfBjTObLODz9f8/FTNbJKjmz/FkBaqqInZ9niSRlAmRlMpwhuGWi1wtMKS2iz4 gQyQntzuSRFd6bmLQHxKPSCsDYbZVBPJACFTioG4sP3nDRAc8hsVzpx7UpS0/rdLKX4A X5iGAbDwiTz9gns9e81Q19TkZjQVqQ3cE5FxXF2/OgPQgMfR9s26f5iuvdcDXMeeBB2M Im6Oz+/WOUGy1Ru1o3fztenajvtPxkuc10c0nxHqqzw7Bx5+9p3tq53i/PvTGhvRwjg+ yX8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457712; x=1732062512; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hBlhrPHwwByaR2iQ3j+uYlOmJcTMAeh7R4Wsq+6yt50=; b=PRS0d9dLgWfwkfyaXWiFQgSYxmMIwM1NpgKZ3OWj8XAw9S1NTNa18mKapra6K5Yv7I BitfiA1iZ6qYccR3Ssw2lZnMq1Lz748E0xk0eKPjvkcqfV9Q9OX1T4qROUdBA/HVmoYX 8mFjtlDtqw7YW2fWTNAEPCKlCA7E+D/7+6qqPQAJ3R+T58OLhWRA6QA3Q4OASOMHhZcE UEQX9SDTYLjLlVNU91sntI0mQ8ItK5LMfJGLwQr1nqv93X0pXesnBIaNbnA7f4aNf08H Lkah3K4J+5fXwakECvAkqjNeSZy31bpwvt+ETrJVVXpE8j/TZVdxssa+cM1a7vVFZaWr n0PA== X-Forwarded-Encrypted: i=1; AJvYcCU8vFYkloJzQrasgRM/wsXrkxzXy0K8Z2G3WxYz6KND3tTTOwf/AFQxE2ZlSo2/c4BeLFYb9TUzduYCz5ERH9AdNw==@vger.kernel.org, AJvYcCWh/fZrfAnkGhAq/P+FU1EyRCLiIXue9yDCc/rfOog1y5Uv8i8RlauA+b2TCiNfElRopp8hY/y2toNps+k=@vger.kernel.org X-Gm-Message-State: AOJu0YwfTljKe6S4cmlFgFXoRLvhj+zGJfixTT0tgJLNnFDI6IJC8H9b MtcfCKfQlckoqBWnkgtOKIVCbuD3BZcna/e8aDA4AwzQ7OCb2Fy0f9HmE8/1MV4= X-Google-Smtp-Source: AGHT+IFewqU5eBEm5zFq7vySqDCIUFDJrjJcHBGEbPxPCUnETye5z/jZ6fo/RWYIg9NbCll/O2EuxA== X-Received: by 2002:a17:902:d4cb:b0:20c:8f98:5dbe with SMTP id d9443c01a7336-2118354bfe9mr260228245ad.33.1731457712202; Tue, 12 Nov 2024 16:28:32 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:31 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 07/10] perf record --off-cpu: Disable perf_event's callchain collection Date: Tue, 12 Nov 2024 16:28:15 -0800 Message-ID: <20241113002818.3578645-8-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" There is a check in evsel.c that does this: if (evsel__is_offcpu_event(evsel)) evsel->core.attr.sample_type &=3D OFFCPU_SAMPLE_TYPES; This along with: #define OFFCPU_SAMPLE_TYPES (PERF_SAMPLE_IDENTIFIER | PERF_SAMPLE_IP | \ PERF_SAMPLE_TID | PERF_SAMPLE_TIME | \ PERF_SAMPLE_ID | PERF_SAMPLE_CPU | \ PERF_SAMPLE_PERIOD | PERF_SAMPLE_CALLCHAIN | \ PERF_SAMPLE_CGROUP) will tell perf_event to collect callchain. We don't need the callchain from perf_event when collecting off-cpu samples, because it's prev's callchain, not next's callchain. (perf_event) (task_storage) (needed) prev next | | ---sched_switch----> Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-8-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo Reviewed-by: Ian Rogers --- tools/perf/util/off_cpu.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/perf/util/off_cpu.h b/tools/perf/util/off_cpu.h index c6edc0f7c40d..f07ab2e36317 100644 --- a/tools/perf/util/off_cpu.h +++ b/tools/perf/util/off_cpu.h @@ -13,7 +13,7 @@ struct record_opts; #define OFFCPU_SAMPLE_TYPES (PERF_SAMPLE_IDENTIFIER | PERF_SAMPLE_IP | \ PERF_SAMPLE_TID | PERF_SAMPLE_TIME | \ PERF_SAMPLE_ID | PERF_SAMPLE_CPU | \ - PERF_SAMPLE_PERIOD | PERF_SAMPLE_CALLCHAIN | \ + PERF_SAMPLE_PERIOD | PERF_SAMPLE_RAW | \ PERF_SAMPLE_CGROUP) =20 #define OFFCPU_THRESH 500000ull --=20 2.43.0 From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B269C13049E; Wed, 13 Nov 2024 00:28:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457717; cv=none; b=B9YtcAtY3mmD1vCyNl+eQG7GfOdnLnAl9g1kgiFSx30dBVVYzNhvk0SUI2BDtxCj2QUyuVZZwxGQaYFJiarjasyWP9CSCb8aWxhUn/P72VgD4t3BdP+FxpQwWxTzTw9TQjdGkcUhxwcweRfdBd5sK+OtL+msMn4HBYesVpDOpbk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457717; c=relaxed/simple; bh=k1Tn5Q4+wHFYp0tK7QptT/pbuAUsO9q/uZ8bPfpXGOY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=YeoXF7dFRKJ7w8O/hNU0lFN2ggllAX0KmY2W3OMvYtlnD7KAHSelIJmPC39a58s6OKqIpNAlpKXh0rE+enCbtml9c8sq7IJk1Vi+gkAcyy0UBDkYqMMb9D/Sk4WlnSi6V9NSnqyjM+cwVzLq9Hb7t8KQgUwvPEnCKUfqlTf272M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cxsjsVlv; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cxsjsVlv" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-20cb7139d9dso59578905ad.1; Tue, 12 Nov 2024 16:28:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457714; x=1732062514; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=o/ufGw+ywqZDjP+CMDcVLPSB5OERM7z6+MoX4VJalRo=; b=cxsjsVlv2eoN07FNd8HYDSak4M4LnraNNkS3CNSpFjYsN6o1mFXSXbwsdAz4kYEAiK zwJqOydg1Ucw8XmPnLmJ76q2bmQNvUFkxytpO0sS74cz/saps96fU4CVz8oV5iS/HwWh 8wvhu/qiE7d+0E8gguOBLn7cmlp2vIBEbSkQOjYcoVsZZ4Gjyo7iVxYtfCiy2zoXkQXy f8jNlywIFkkmOZf0QkoWVQEHNKJvdnqk+biExxHmyIus+RVDsUJJwzmbN9J8/2TfFI7v uRSNaNFNLaZlg9cuwWYn/Vs5s2jT8DFs4YjG92okizN3I8vW4Zb3p45I2QRyxNAjL3u7 VppA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457714; x=1732062514; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o/ufGw+ywqZDjP+CMDcVLPSB5OERM7z6+MoX4VJalRo=; b=j+N12eQ+DoH8npu7X8AsME3enCDR1A0o0jc3LWmKOLER58njMLN1isAANZbYoPIEKJ 1RH692r4qiE+h/b+2VspO+QKQ5UgfJMjISvY8TKxeB/3WCHP6qgso6NEgMSJvcKpntHz SbzswFfcHaGzx6rHwdao3E4WmvUbrbOGxB30vg41rZX3egGO4b1Pq9IhZhMCssPRB5pZ fNNydnyuYP36y5CLFgPdF1mwKHCc9IL7FHv6vBnyp1wsg6TmGKRKMTJAQFeOVjA7I4eI WWRIWkqd+/6F5XuUcmdhq/XmFVL1Ii1phjzgY7oMCOD9SmLyUztKTC6/0FMXfKVDaKpk fdAQ== X-Forwarded-Encrypted: i=1; AJvYcCU5ZBEGv4c1svMfc2fzRUgZt5kjZ/A4cQ9LFppoDVh3hLO5l8zYoSNCWbWG8J6DD3t8oHwjZujABPaW+4w=@vger.kernel.org, AJvYcCWIWzNdWyeMkoEeLQOpOkDP1SFFKmVIYYPHyysv4R4vWTul+wj6KrQ47I0LUGGdOEswnlGRdNsoWEE22HAcJUXQUA==@vger.kernel.org X-Gm-Message-State: AOJu0YyjFCrJ341LNAw4DVdm58XG0qGj9YE1gx1OyraU0cutjv7KJlD/ R6OqPtuRjNRA5N4G87I5sPNY0NE6lQhvBMqXRnqgP7fm5fzjQ1L4 X-Google-Smtp-Source: AGHT+IE9rhRej//kC4pX2pcD6Wh7QLUH124VswuZxJIVkoa1ynZqEutl03UpSHBBUBETFJBYImcUnA== X-Received: by 2002:a17:902:c411:b0:20c:9d79:cf85 with SMTP id d9443c01a7336-21183e5ee46mr223905605ad.54.1731457713752; Tue, 12 Nov 2024 16:28:33 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:33 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 08/10] perf script: Display off-cpu samples correctly Date: Tue, 12 Nov 2024 16:28:16 -0800 Message-ID: <20241113002818.3578645-9-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable No PERF_SAMPLE_CALLCHAIN in sample_type, but I need perf script to display a callchain, have to specify manually. Also, prefer displaying a callchain: gvfs-afc-volume 2267 [001] 3829232.955656: 1001115340 offcpu-time: 77f05292603f __pselect+0xbf (/usr/lib/x86_64-linux-gnu/libc.so.= 6) 77f052a1801c [unknown] (/usr/lib/x86_64-linux-gnu/libusbmuxd-2.= 0.so.6.0.0) 77f052a18d45 [unknown] (/usr/lib/x86_64-linux-gnu/libusbmuxd-2.= 0.so.6.0.0) 77f05289ca94 start_thread+0x384 (/usr/lib/x86_64-linux-gnu/libc= .so.6) 77f052929c3c clone3+0x2c (/usr/lib/x86_64-linux-gnu/libc.so.6) to a raw binary BPF output: BPF output: 0000: dd 08 00 00 db 08 00 00
...... 0008: cc ce ab 3b 00 00 00 00 =CE=AB;.... 0010: 06 00 00 00 00 00 00 00 ........ 0018: 00 fe ff ff ff ff ff ff . 0020: 3f 60 92 52 f0 77 00 00 ?`.Rw.. 0028: 1c 80 a1 52 f0 77 00 00 ..Rw.. 0030: 45 8d a1 52 f0 77 00 00 E.Rw.. 0038: 94 ca 89 52 f0 77 00 00 ..Rw.. 0040: 3c 9c 92 52 f0 77 00 00 <..Rw.. 0048: 00 00 00 00 00 00 00 00 ........ 0050: 00 00 00 00 .... Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-9-howardchu95@gmail.= com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/builtin-script.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c index 9e47905f75a6..dd0f95c51571 100644 --- a/tools/perf/builtin-script.c +++ b/tools/perf/builtin-script.c @@ -662,7 +662,7 @@ static int perf_session__check_output_opt(struct perf_s= ession *session) =20 evlist__for_each_entry(session->evlist, evsel) { not_pipe =3D true; - if (evsel__has_callchain(evsel)) { + if (evsel__has_callchain(evsel) || evsel__is_offcpu_event(evsel)) { use_callchain =3D true; break; } @@ -2353,7 +2353,7 @@ static void process_event(struct perf_script *script, else if (PRINT_FIELD(BRSTACKOFF)) perf_sample__fprintf_brstackoff(sample, thread, attr, fp); =20 - if (evsel__is_bpf_output(evsel) && PRINT_FIELD(BPF_OUTPUT)) + if (evsel__is_bpf_output(evsel) && !evsel__is_offcpu_event(evsel) && PRIN= T_FIELD(BPF_OUTPUT)) perf_sample__fprintf_bpf_output(sample, fp); perf_sample__fprintf_insn(sample, evsel, attr, thread, machine, fp, al); =20 --=20 2.43.0 From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22E9D13635C; Wed, 13 Nov 2024 00:28:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457717; cv=none; b=Sqi9nNURHiV4pCwbvDTZm9sEU7CeHeObE8i4Y6qGMYM/E6eFGgEPWyQE+Y2YyhGq0MJy2lsldHhlFeARQceI02mk9tdwg217w7WaecGgNbMvtjCy2xwgttjiFGX9glwlDonuhkdE8O/1CLkk9PDX46mYG7Ra2nveQI38BgC5m04= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457717; c=relaxed/simple; bh=sb+supRPW/fkpZ9vc7XCX9Jloxm0dJEW/JuJSj/ygOk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BV8oHmbdIfM/0n9c6pVrWYy9g7ewUiiBmwzuh2MYlWSB+UqEN4Ea7ooI/GLe+W8F1qi4gOg8xcHKRj9HDSMjR3YE2xosGNLFCFt3J7PfFJTlMdMYvUOsT6iPugZxhAWMd5rdqy7C/58QstOxqkk6BBpx8NCaQGQRBa07HR0GVb8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Yn9dJZzZ; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Yn9dJZzZ" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-20e6981ca77so72872535ad.2; Tue, 12 Nov 2024 16:28:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457715; x=1732062515; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9mWDHVgN1ZhsnvTOrpwFobfS9+/sponIhrFkhay1424=; b=Yn9dJZzZK9X/0B9zXj7l4q6Z/dYooZxBdraLO/uZPXq/jXt3ZtZMvJKWHHb8hnX73k Dic05zYxICSjKSO7OtzKkBRUrLS2L7Mam3uhhhEQB3X8yQPTqaRjYUIbHxrZc6tWCSHp Am74oKWcJxLAoXf37pW+V9xYNAwBECHJKcIZ+VV8LPCjUAQYBn5JEPdSFuLS7napR8Ts +8Fkaou1oYi3ldTOrZ4BQBdzI7AvQDdrJHdmbr8m2Y8iHjGYwB3wR6G0TAzX61scK2aI kAAKQZYY/cY+SSgXqKLgDY6v+ODvY0HIF3nI+qrFh/0frEtPwdicrqz+o6OSJjxFWDRh YUaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457715; x=1732062515; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9mWDHVgN1ZhsnvTOrpwFobfS9+/sponIhrFkhay1424=; b=XRfEyCTmL5SHpR1Lnu6ZqvTB9H0YUE+P3+vDI9ih79a/kD5jhK5uaLVd1du9lH2SVR DVPmDUtL+n+5XfeUE3bCss1JMGmonL1RPpsapZVWOw5ewmHJsyKPXn49i9u4MYzCsAYd VBGQCjBAFyFmr1BIvy9ZyoRgDB3Y4K7WlsL7hOZtrsEB8g1Nna5noytO4CcUlVef31GL 80ZhjuYuJl6Z1AhaH2MccgjEpAtDVZwzVDz2W/yVcpL/iInG2sJ68oMQmJF2JZ3F99V9 Y4MzEUy0noZUTvt3Lv6CQ5AGbGYUUwzf7B3bJx+KKwbV1iNYDu8WUdWpRlgf7ePwNZBp cTjA== X-Forwarded-Encrypted: i=1; AJvYcCWRt7vroGD5FvywjfGWh+A4zryqmydub8F4SoJPTBJL9WbK3+BPdw16WyUGGSHYl46sMvL/2m+kkkmYWkL2r94AnQ==@vger.kernel.org, AJvYcCWq/FxB7VkTOvUxcCl8LEZ+NQaRWlGLS4IUMmSXMgnD3MbN7vGtBlYc9VvkNCccgxV98BUwZgEyarFGhWg=@vger.kernel.org X-Gm-Message-State: AOJu0YzqczmXEF0HpolEy+TEMtR5ad0q1YequDPv6uZzKjdPqdBboUUV uAsJ37jKRah3/s0SI+aPnJMcYC3R3jqoQDinLxo0U9zK1vDMdMYx X-Google-Smtp-Source: AGHT+IF7GKcD2Q2v101tqIyQO1ocFMlkX5zhmmT47kNxgCSWd7nXOGA/SB55eXP9P/6KlBiH57eE3A== X-Received: by 2002:a17:902:ea02:b0:20b:ab4b:5432 with SMTP id d9443c01a7336-211b5bcc3b0mr14309325ad.12.1731457715324; Tue, 12 Nov 2024 16:28:35 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:35 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 09/10] perf record --off-cpu: Dump the remaining samples in BPF's stack trace map Date: Tue, 12 Nov 2024 16:28:17 -0800 Message-ID: <20241113002818.3578645-10-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Dump the remaining samples, as if it is dumping a direct sample. Put the stack trace, tid, off-cpu time and cgroup id into the raw_data section, just like a direct off-cpu sample coming from BPF's bpf_perf_event_output(). This ensures that evsel__parse_sample() correctly parses both direct samples and accumulated samples. Suggested-by: Namhyung Kim Reviewed-by: Ian Rogers Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-10-howardchu95@gmail= .com Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/util/bpf_off_cpu.c | 59 +++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 24 deletions(-) diff --git a/tools/perf/util/bpf_off_cpu.c b/tools/perf/util/bpf_off_cpu.c index 61729a65b529..16528f8509b7 100644 --- a/tools/perf/util/bpf_off_cpu.c +++ b/tools/perf/util/bpf_off_cpu.c @@ -37,6 +37,8 @@ union off_cpu_data { u64 array[1024 / sizeof(u64)]; }; =20 +u64 off_cpu_raw[MAX_STACKS + 5]; + static int off_cpu_config(struct evlist *evlist) { char off_cpu_event[64]; @@ -309,6 +311,7 @@ int off_cpu_write(struct perf_session *session) { int bytes =3D 0, size; int fd, stack; + u32 raw_size; u64 sample_type, val, sid =3D 0; struct evsel *evsel; struct perf_data_file *file =3D &session->data->file; @@ -348,46 +351,54 @@ int off_cpu_write(struct perf_session *session) =20 while (!bpf_map_get_next_key(fd, &prev, &key)) { int n =3D 1; /* start from perf_event_header */ - int ip_pos =3D -1; =20 bpf_map_lookup_elem(fd, &key, &val); =20 + /* zero-fill some of the fields, will be overwritten by raw_data when pa= rsing */ if (sample_type & PERF_SAMPLE_IDENTIFIER) data.array[n++] =3D sid; - if (sample_type & PERF_SAMPLE_IP) { - ip_pos =3D n; + if (sample_type & PERF_SAMPLE_IP) data.array[n++] =3D 0; /* will be updated */ - } if (sample_type & PERF_SAMPLE_TID) - data.array[n++] =3D (u64)key.pid << 32 | key.tgid; + data.array[n++] =3D 0; if (sample_type & PERF_SAMPLE_TIME) data.array[n++] =3D tstamp; - if (sample_type & PERF_SAMPLE_ID) - data.array[n++] =3D sid; if (sample_type & PERF_SAMPLE_CPU) data.array[n++] =3D 0; if (sample_type & PERF_SAMPLE_PERIOD) - data.array[n++] =3D val; - if (sample_type & PERF_SAMPLE_CALLCHAIN) { - int len =3D 0; - - /* data.array[n] is callchain->nr (updated later) */ - data.array[n + 1] =3D PERF_CONTEXT_USER; - data.array[n + 2] =3D 0; - - bpf_map_lookup_elem(stack, &key.stack_id, &data.array[n + 2]); - while (data.array[n + 2 + len]) + data.array[n++] =3D 0; + if (sample_type & PERF_SAMPLE_RAW) { + /* + * [ size ][ data ] + * [ data ] + * [ data ] + * [ data ] + * [ data ][ empty] + */ + int len =3D 0, i =3D 0; + void *raw_data =3D (void *)data.array + n * sizeof(u64); + + off_cpu_raw[i++] =3D (u64)key.pid << 32 | key.tgid; + off_cpu_raw[i++] =3D val; + + /* off_cpu_raw[i] is callchain->nr (updated later) */ + off_cpu_raw[i + 1] =3D PERF_CONTEXT_USER; + off_cpu_raw[i + 2] =3D 0; + + bpf_map_lookup_elem(stack, &key.stack_id, &off_cpu_raw[i + 2]); + while (off_cpu_raw[i + 2 + len]) len++; =20 - /* update length of callchain */ - data.array[n] =3D len + 1; + off_cpu_raw[i] =3D len + 1; + i +=3D len + 2; + + off_cpu_raw[i++] =3D key.cgroup_id; =20 - /* update sample ip with the first callchain entry */ - if (ip_pos >=3D 0) - data.array[ip_pos] =3D data.array[n + 2]; + raw_size =3D i * sizeof(u64) + sizeof(u32); /* 4 bytes for alignment */ + memcpy(raw_data, &raw_size, sizeof(raw_size)); + memcpy(raw_data + sizeof(u32), off_cpu_raw, i * sizeof(u64)); =20 - /* calculate sample callchain data array length */ - n +=3D len + 2; + n +=3D i + 1; } if (sample_type & PERF_SAMPLE_CGROUP) data.array[n++] =3D key.cgroup_id; --=20 2.43.0 From nobody Sat Nov 23 12:19:36 2024 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8642213B2A5; Wed, 13 Nov 2024 00:28:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457719; cv=none; b=tO8fojNRw66aGYS+reAxu1lAD6yzK32LYr8mEf/AiwmsaNf9jSJMu7XXMcKzdDj+Vo9Hb6K/X+6AYeX3BSqtGlQ7B0imKPmjF/+jBm5ASoiyrEVnekzOavpfx2rFietyQ9nVOLHrOGZVjzeRn1clBmWX1003U5cxNTejAIQTwFs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731457719; c=relaxed/simple; bh=i38KITFoXQxHV5vC+zLrvHnE4A5s7OsP/Virq4w9OTo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KdxHvt8VufC3RzrZeuYvsWVWriq/VGfHX93lail5qsP0OyTiCPakmHNpx+vHxv3/3PS1EBAg4lqd5CEIeU5IEVz8cQhk0TuFFIh39fUXIR7RXFSy6nWB5P7AuKNlZVvVpoQmoKwjy8fFD/DI44jn3oBFTQ5rJOaZihAi42V9Uyw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=LtVt8KDM; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LtVt8KDM" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-20e6981ca77so72872785ad.2; Tue, 12 Nov 2024 16:28:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731457717; x=1732062517; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ghYLYuPX8XvrvfkfC7VIB18q5PzN1p3CvZzBe/JSGjo=; b=LtVt8KDMPn3MAcDOer1YNxjyzrIsl4wpauLPrNYMBX16K2OUtbnhDVToJX72du0BLy +4F3lhOVMUCNc5zoncHGieWWUOpY5+wz7g/Q70Vn3/Q5xzi8yoNM0elJ4h9GIXvbslIg qLgZgSdB+6EgN6VTubjA+efJs0oQpBHGNz31dib1ksD6XjqsA6nnROxt1tbqX/Rup33/ dSJhaptgTPGUG0rEBohyayMM9rCghCyGrQpp+eDQmvGC4hA8kvym4RdyN3g5wnjlyCrh wrThxnLg2byV2fi0wmFvmjNU6bT5N16BDod9ktFpW06p3esFcPLkzv20IMANkx67cc2W pCKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731457717; x=1732062517; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ghYLYuPX8XvrvfkfC7VIB18q5PzN1p3CvZzBe/JSGjo=; b=ZTgb12M1Kcq31T3KW7QEhwf5yerib0gAUo11xubTAshbftq1dLe12x7QIgHZpojkOW U8r0GJwGgFog7fZm4rCRwQajd1GW1A7T/vUDxeTyHU9ef4yBib9euttxF/z0hA2S8CJU 9/gzWB+GkFpghY/XJQD+jyE/4WNGI0+OcDndWQSCC3YahWxQw4u9Pcz7PDkiVa7SUtYm 6fmknKYc+yJdU+eSG0US0wMV1fE/qfkBEZBoy13EzKPtRBBdk0W31olAGHpNal307P/d ZITzArCBXcbTpZghuKkNPqXfok8ud2DORl0qr9G7ih/MJ1sea4+DR8EcqDkKDhpcIq+X Duyg== X-Forwarded-Encrypted: i=1; AJvYcCUKeNzReF+VKfmFyQjvBTT2oAqv7T0t8D6xqRimH/SYGJhmmq5J0IBubMgthgt2l+EJ1UEYTYyikQBxfHgoPsJJhA==@vger.kernel.org, AJvYcCXlMnRTe33uSVsTDcaMUkbv4H4xMY+uUhDrHKd8/Xx4zY+dmmv8/mbpI9VoqpHHBBZEN7lpdrOw0AXPMWM=@vger.kernel.org X-Gm-Message-State: AOJu0Yy7GssVpwTavbMq4ADnjIccHuQ8Pfd8RwsAELFOzrttbCAn9jd8 ZJDzDuZucwaUE3XSfjGaPt9DeRemZ6M63sOgihw4O0e2NlXVS0Vh X-Google-Smtp-Source: AGHT+IFlIs8DtmqJHA1/pZmtXpdKQvE3IgCCH7jggV6P9t/wXULHezqSkIGXqItGieSsloamXvfLCg== X-Received: by 2002:a17:903:2282:b0:20e:552c:53ee with SMTP id d9443c01a7336-211b5c884acmr11159055ad.24.1731457716874; Tue, 12 Nov 2024 16:28:36 -0800 (PST) Received: from mbp.lan (c-67-174-206-244.hsd1.ca.comcast.net. [67.174.206.244]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e458bdsm100158425ad.133.2024.11.12.16.28.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Nov 2024 16:28:36 -0800 (PST) From: Howard Chu To: acme@kernel.org, peterz@infradead.org Cc: namhyung@kernel.org, irogers@google.com, mingo@redhat.com, jolsa@kernel.org, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Howard Chu , Alexander Shishkin , James Clark , Mark Rutland , Arnaldo Carvalho de Melo Subject: [PATCH v8 10/10] perf test: Add direct off-cpu test Date: Tue, 12 Nov 2024 16:28:18 -0800 Message-ID: <20241113002818.3578645-11-howardchu95@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241113002818.3578645-1-howardchu95@gmail.com> References: <20241113002818.3578645-1-howardchu95@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Why is there a --off-cpu-thresh 2000000? We collect an off-cpu period __ONLY ONCE__, either in direct sample form, or in accumulated form (in BPF stack trace map). If I don't add --off-cpu-thresh 200000, the sample in the original test goes into the ring buffer instead of the BPF stack trace map. Additionally, when using -e dummy, the ring buffer is not open, causing us to lose a sample. Signed-off-by: Howard Chu Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20241108204137.2444151-11-howardchu95@gmail= .com Signed-off-by: Arnaldo Carvalho de Melo Reviewed-by: Ian Rogers --- tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/shell/record_offcpu.sh | 35 ++++++++++++++++++++++++- tools/perf/tests/tests.h | 1 + tools/perf/tests/workloads/Build | 1 + tools/perf/tests/workloads/offcpu.c | 16 +++++++++++ 5 files changed, 53 insertions(+), 1 deletion(-) create mode 100644 tools/perf/tests/workloads/offcpu.c diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-tes= t.c index 8dcf74d3c0a3..43dc04075ecb 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -146,6 +146,7 @@ static struct test_workload *workloads[] =3D { &workload__brstack, &workload__datasym, &workload__landlock, + &workload__offcpu, }; =20 #define workloads__for_each(workload) \ diff --git a/tools/perf/tests/shell/record_offcpu.sh b/tools/perf/tests/she= ll/record_offcpu.sh index 678947fe69ee..8719130fbf70 100755 --- a/tools/perf/tests/shell/record_offcpu.sh +++ b/tools/perf/tests/shell/record_offcpu.sh @@ -6,6 +6,10 @@ set -e =20 err=3D0 perfdata=3D$(mktemp /tmp/__perf_test.perf.data.XXXXX) +TEST_PROGRAM=3D"perf test -w offcpu" + +ts=3D$(printf "%u" $((~0 << 32))) # OFF_CPU_TIMESTAMP +dummy_timestamp=3D${ts%???} # remove the last 3 digits to match perf script =20 cleanup() { rm -f ${perfdata} @@ -39,7 +43,11 @@ test_offcpu_priv() { test_offcpu_basic() { echo "Basic off-cpu test" =20 - if ! perf record --off-cpu -e dummy -o ${perfdata} sleep 1 2> /dev/null + # We collect an off-cpu period __ONLY ONCE__, either in direct sample fo= rm, or in accumulated form + # (in BPF stack trace map). Without the --off-cpu-thresh 200000 below, t= he sample will go into the + # ring buffer instead of the BPF stack trace map. Additionally, when usi= ng -e dummy, the ring + # buffer is not enabled, resulting in a lost sample. + if ! perf record --off-cpu --off-cpu-thresh 2000000 -e dummy -o ${perfda= ta} sleep 1 2> /dev/null then echo "Basic off-cpu test [Failed record]" err=3D1 @@ -88,6 +96,27 @@ test_offcpu_child() { echo "Child task off-cpu test [Success]" } =20 +test_offcpu_direct() { + echo "Direct off-cpu test" + + # dump off-cpu samples for task blocked for more than 1.999999s + # -D for initial delay, to enable evlist + if ! perf record -e dummy -D 500 --off-cpu --off-cpu-thresh 1999999 -o $= {perfdata} ${TEST_PROGRAM} 2> /dev/null + then + echo "Direct off-cpu test [Failed record]" + err=3D1 + return + fi + # Direct sample's timestamp should be lower than the dummy_timestamp of = the at-the-end sample. + if ! perf script -i ${perfdata} -F time,period | sed "s/[\.:]//g" | \ + awk "{ if (\$1 < ${dummy_timestamp} && \$2 > 1999999999) exit 0; el= se exit 1; }" + then + echo "Direct off-cpu test [Failed missing direct sample]" + err=3D1 + return + fi + echo "Direct off-cpu test [Success]" +} =20 test_offcpu_priv =20 @@ -99,5 +128,9 @@ if [ $err =3D 0 ]; then test_offcpu_child fi =20 +if [ $err =3D 0 ]; then + test_offcpu_direct +fi + cleanup exit $err diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h index cb58b43aa063..2e655a617b30 100644 --- a/tools/perf/tests/tests.h +++ b/tools/perf/tests/tests.h @@ -217,6 +217,7 @@ DECLARE_WORKLOAD(sqrtloop); DECLARE_WORKLOAD(brstack); DECLARE_WORKLOAD(datasym); DECLARE_WORKLOAD(landlock); +DECLARE_WORKLOAD(offcpu); =20 extern const char *dso_to_test; extern const char *test_objdump_path; diff --git a/tools/perf/tests/workloads/Build b/tools/perf/tests/workloads/= Build index 5af17206f04d..0e78fd01eaf1 100644 --- a/tools/perf/tests/workloads/Build +++ b/tools/perf/tests/workloads/Build @@ -7,6 +7,7 @@ perf-test-y +=3D sqrtloop.o perf-test-y +=3D brstack.o perf-test-y +=3D datasym.o perf-test-y +=3D landlock.o +perf-test-y +=3D offcpu.o =20 CFLAGS_sqrtloop.o =3D -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_leafloop.o =3D -g -O0 -fno-inline -fno-omit-frame-pointer -= U_FORTIFY_SOURCE diff --git a/tools/perf/tests/workloads/offcpu.c b/tools/perf/tests/workloa= ds/offcpu.c new file mode 100644 index 000000000000..57cee201a4c3 --- /dev/null +++ b/tools/perf/tests/workloads/offcpu.c @@ -0,0 +1,16 @@ +#include +#include +#include "../tests.h" + +static int offcpu(int argc __maybe_unused, const char **argv __maybe_unuse= d) +{ + /* get past the initial delay */ + sleep(1); + + /* what we want to collect as a direct sample */ + sleep(2); + + return 0; +} + +DEFINE_WORKLOAD(offcpu); --=20 2.43.0