From nobody Mon Oct 6 13:35:27 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49A2F245008 for ; Mon, 21 Jul 2025 23:11:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753139512; cv=none; b=Tpv1VR7Am3FM8KI/KBlvvkMSt9AxdqJIC2twhdiuGyd6w2/WOjh+KtlKrD7BHNpAb153DQj3SNKV55EG2hvnTqnhAWmwmMIrMfoQFDrXTlRml6wrrPpoqsEDXsMhEeAd8LO43iSE6SIPkEDIPRsaUYskkiTS/bizQuB/fZ5sgLE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753139512; c=relaxed/simple; bh=LVHnQL0MDlysX2xX/2iQYOLlG2OJPiUU5c9ExQc0XXA=; h=Message-ID:Date:From:To:Cc:Subject:References:MIME-Version: Content-Type; b=VWOzlUpkeAy37rdxsjUZK4BOB38IybDHx3VeIBQAsvRJwEwsZ7te77sClO5yuyQuHGjMydQHoCeAk7UlsakxSo58qlHsiXwjP3czh23kUjgAStQyQMPKzvSpCb2xsasRLlHoMEHY2JA1i24vmdGiX8CwY4i91XpAUE0qir0rZ7w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DaudlZIJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DaudlZIJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E32C4C4CEFA; Mon, 21 Jul 2025 23:11:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753139512; bh=LVHnQL0MDlysX2xX/2iQYOLlG2OJPiUU5c9ExQc0XXA=; h=Date:From:To:Cc:Subject:References:From; b=DaudlZIJ9Gy7HnIGdVBCrlO4yQxGbt/X6TswI+uxgicSSRlTMJ0tMBFt5Syjxcn7e w9ZRTrL9T/ervJU0gLglGndn/OQLvWlafzfy131/fLszPpAAanTU24zD6TkGqQBt1Q aawkV9goo0bAmjtk9zRg6uplQVYRh6sIgIAEg2tNEDJa5c/wELu+FDLg6rLWA2FiZK R9fYj+UvmHLzvmJ4jtK2vmi8ZPceLHEqXCDE1hU0NCTvNzGS/Woqtt35prjxo2cCjt O2FY6ZM/SKV1lqFJquGh5ndIF8AZpEcwzBVYjNSD6Nxnlmv824sNsCiL5CaDOR6X2L f8RbgmlkwHSFQ== Received: from rostedt by gandalf with local (Exim 4.98.2) (envelope-from ) id 1udzgF-00000009vJM-0Rxp; Mon, 21 Jul 2025 19:12:23 -0400 Message-ID: <20250721231222.964514612@kernel.org> User-Agent: quilt/0.68 Date: Mon, 21 Jul 2025 19:12:09 -0400 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Tomas Glozar , John Kacur , Luis Goncalves , Arnaldo Carvalho de Melo , Chang Yin , Costa Shulyupin , Crystal Wood , Gabriele Monaco Subject: [for-next][PATCH 5/9] rtla/timerlat: Add action on end feature References: <20250721231204.100737734@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Tomas Glozar Implement actions on end next to actions on threshold. A new option, --on-end is added, parallel to --on-threshold. Instead of being executed whenever a latency threshold is reached, it is executed at the end of the measurement. For example: $ rtla timerlat hist -d 5s --on-end trace will save the trace output at the end. All actions supported by --on-threshold are also supported by --on-end, except for continue, which does nothing with --on-end. Cc: John Kacur Cc: Luis Goncalves Cc: Arnaldo Carvalho de Melo Cc: Chang Yin Cc: Costa Shulyupin Cc: Crystal Wood Cc: Gabriele Monaco Link: https://lore.kernel.org/20250626123405.1496931-6-tglozar@redhat.com Signed-off-by: Tomas Glozar Signed-off-by: Steven Rostedt (Google) --- tools/tracing/rtla/src/timerlat.h | 5 ++- tools/tracing/rtla/src/timerlat_hist.c | 44 +++++++++++++++++-------- tools/tracing/rtla/src/timerlat_top.c | 45 ++++++++++++++++++-------- 3 files changed, 65 insertions(+), 29 deletions(-) diff --git a/tools/tracing/rtla/src/timerlat.h b/tools/tracing/rtla/src/tim= erlat.h index d1fcf9a97621..bc55ed04fc96 100644 --- a/tools/tracing/rtla/src/timerlat.h +++ b/tools/tracing/rtla/src/timerlat.h @@ -48,7 +48,10 @@ struct timerlat_params { struct sched_attr sched_param; struct trace_events *events; enum timerlat_tracing_mode mode; - struct actions actions; + + struct actions threshold_actions; + struct actions end_actions; + union { struct { /* top only */ diff --git a/tools/tracing/rtla/src/timerlat_hist.c b/tools/tracing/rtla/sr= c/timerlat_hist.c index 4f13a8f92711..9baea1b251ed 100644 --- a/tools/tracing/rtla/src/timerlat_hist.c +++ b/tools/tracing/rtla/src/timerlat_hist.c @@ -758,6 +758,7 @@ static void timerlat_hist_usage(char *usage) " --trace-buffer-size kB: set the per-cpu trace buffer size in kB", " --deepest-idle-state n: only go down to idle state n on cpus used= by timerlat to reduce exit from idle latency", " --on-threshold : define action to be executed at latency = threshold, multiple are allowed", + " --on-end : define action to be executed at measurement en= d, multiple are allowed", NULL, }; =20 @@ -793,7 +794,8 @@ static struct timerlat_params if (!params) exit(1); =20 - actions_init(¶ms->actions); + actions_init(¶ms->threshold_actions); + actions_init(¶ms->end_actions); =20 /* disabled by default */ params->dma_latency =3D -1; @@ -846,6 +848,7 @@ static struct timerlat_params {"trace-buffer-size", required_argument, 0, '\3'}, {"deepest-idle-state", required_argument, 0, '\4'}, {"on-threshold", required_argument, 0, '\5'}, + {"on-end", required_argument, 0, '\6'}, {0, 0, 0, 0} }; =20 @@ -1038,7 +1041,14 @@ static struct timerlat_params params->deepest_idle_state =3D get_llong_from_str(optarg); break; case '\5': - retval =3D actions_parse(¶ms->actions, optarg); + retval =3D actions_parse(¶ms->threshold_actions, optarg); + if (retval) { + err_msg("Invalid action %s\n", optarg); + exit(EXIT_FAILURE); + } + break; + case '\6': + retval =3D actions_parse(¶ms->end_actions, optarg); if (retval) { err_msg("Invalid action %s\n", optarg); exit(EXIT_FAILURE); @@ -1050,7 +1060,7 @@ static struct timerlat_params } =20 if (trace_output) - actions_add_trace_output(¶ms->actions, trace_output); + actions_add_trace_output(¶ms->threshold_actions, trace_output); =20 if (geteuid()) { err_msg("rtla needs root permission\n"); @@ -1077,7 +1087,8 @@ static struct timerlat_params * mixed mode */ if (params->mode =3D=3D TRACING_MODE_BPF && - (params->actions.present[ACTION_TRACE_OUTPUT] || !params->no_aa)) + (params->threshold_actions.present[ACTION_TRACE_OUTPUT] || + params->end_actions.present[ACTION_TRACE_OUTPUT] || !params->no_aa)) params->mode =3D TRACING_MODE_MIXED; =20 return params; @@ -1270,13 +1281,15 @@ int timerlat_hist_main(int argc, char *argv[]) } } =20 - if (params->actions.present[ACTION_TRACE_OUTPUT]) { + if (params->threshold_actions.present[ACTION_TRACE_OUTPUT] || + params->end_actions.present[ACTION_TRACE_OUTPUT]) { record =3D osnoise_init_trace_tool("timerlat"); if (!record) { err_msg("Failed to enable the trace instance\n"); goto out_free; } - params->actions.trace_output_inst =3D record->trace.inst; + params->threshold_actions.trace_output_inst =3D record->trace.inst; + params->end_actions.trace_output_inst =3D record->trace.inst; =20 if (params->events) { retval =3D trace_events_enable(&record->trace, params->events); @@ -1342,7 +1355,7 @@ int timerlat_hist_main(int argc, char *argv[]) * tracing while enabling other instances. The trace instance is the * one with most valuable information. */ - if (params->actions.present[ACTION_TRACE_OUTPUT]) + if (record) trace_instance_start(&record->trace); if (!params->no_aa) trace_instance_start(&aa->trace); @@ -1375,14 +1388,14 @@ int timerlat_hist_main(int argc, char *argv[]) } =20 if (osnoise_trace_is_off(tool, record)) { - actions_perform(¶ms->actions); + actions_perform(¶ms->threshold_actions); =20 - if (!params->actions.continue_flag) + if (!params->threshold_actions.continue_flag) /* continue flag not set, break */ break; =20 /* continue action reached, re-enable tracing */ - if (params->actions.present[ACTION_TRACE_OUTPUT]) + if (record) trace_instance_start(&record->trace); if (!params->no_aa) trace_instance_start(&aa->trace); @@ -1403,14 +1416,14 @@ int timerlat_hist_main(int argc, char *argv[]) =20 if (!stop_tracing) { /* Threshold overflow, perform actions on threshold */ - actions_perform(¶ms->actions); + actions_perform(¶ms->threshold_actions); =20 - if (!params->actions.continue_flag) + if (!params->threshold_actions.continue_flag) /* continue flag not set, break */ break; =20 /* continue action reached, re-enable tracing */ - if (params->actions.present[ACTION_TRACE_OUTPUT]) + if (record) trace_instance_start(&record->trace); if (!params->no_aa) trace_instance_start(&aa->trace); @@ -1435,6 +1448,8 @@ int timerlat_hist_main(int argc, char *argv[]) =20 timerlat_print_stats(params, tool); =20 + actions_perform(¶ms->end_actions); + return_value =3D PASSED; =20 if (osnoise_trace_is_off(tool, record) && !stop_tracing) { @@ -1464,7 +1479,8 @@ int timerlat_hist_main(int argc, char *argv[]) osnoise_destroy_tool(aa); osnoise_destroy_tool(record); osnoise_destroy_tool(tool); - actions_destroy(¶ms->actions); + actions_destroy(¶ms->threshold_actions); + actions_destroy(¶ms->end_actions); if (params->mode !=3D TRACING_MODE_TRACEFS) timerlat_bpf_destroy(); free(params); diff --git a/tools/tracing/rtla/src/timerlat_top.c b/tools/tracing/rtla/src= /timerlat_top.c index 60f9c78cb272..c80b81c0b4da 100644 --- a/tools/tracing/rtla/src/timerlat_top.c +++ b/tools/tracing/rtla/src/timerlat_top.c @@ -517,6 +517,7 @@ static void timerlat_top_usage(char *usage) " --trace-buffer-size kB: set the per-cpu trace buffer size in kB", " --deepest-idle-state n: only go down to idle state n on cpus used= by timerlat to reduce exit from idle latency", " --on-threshold : define action to be executed at latency = threshold, multiple are allowed", + " --on-end: define action to be executed at measurement end, multip= le are allowed", NULL, }; =20 @@ -552,7 +553,8 @@ static struct timerlat_params if (!params) exit(1); =20 - actions_init(¶ms->actions); + actions_init(¶ms->threshold_actions); + actions_init(¶ms->end_actions); =20 /* disabled by default */ params->dma_latency =3D -1; @@ -597,6 +599,7 @@ static struct timerlat_params {"trace-buffer-size", required_argument, 0, '7'}, {"deepest-idle-state", required_argument, 0, '8'}, {"on-threshold", required_argument, 0, '9'}, + {"on-end", required_argument, 0, '\1'}, {0, 0, 0, 0} }; =20 @@ -623,6 +626,7 @@ static struct timerlat_params =20 /* set trace */ trace_output =3D "timerlat_trace.txt"; + break; case '5': /* it is here because it is similar to -a */ @@ -776,7 +780,14 @@ static struct timerlat_params params->deepest_idle_state =3D get_llong_from_str(optarg); break; case '9': - retval =3D actions_parse(¶ms->actions, optarg); + retval =3D actions_parse(¶ms->threshold_actions, optarg); + if (retval) { + err_msg("Invalid action %s\n", optarg); + exit(EXIT_FAILURE); + } + break; + case '\1': + retval =3D actions_parse(¶ms->end_actions, optarg); if (retval) { err_msg("Invalid action %s\n", optarg); exit(EXIT_FAILURE); @@ -788,7 +799,7 @@ static struct timerlat_params } =20 if (trace_output) - actions_add_trace_output(¶ms->actions, trace_output); + actions_add_trace_output(¶ms->threshold_actions, trace_output); =20 if (geteuid()) { err_msg("rtla needs root permission\n"); @@ -812,7 +823,8 @@ static struct timerlat_params * mixed mode */ if (params->mode =3D=3D TRACING_MODE_BPF && - (params->actions.present[ACTION_TRACE_OUTPUT] || !params->no_aa)) + (params->threshold_actions.present[ACTION_TRACE_OUTPUT] || + params->end_actions.present[ACTION_TRACE_OUTPUT] || !params->no_aa)) params->mode =3D TRACING_MODE_MIXED; =20 return params; @@ -934,14 +946,14 @@ timerlat_top_main_loop(struct osnoise_tool *top, timerlat_print_stats(params, top); =20 if (osnoise_trace_is_off(top, record)) { - actions_perform(¶ms->actions); + actions_perform(¶ms->threshold_actions); =20 - if (!params->actions.continue_flag) + if (!params->threshold_actions.continue_flag) /* continue flag not set, break */ break; =20 /* continue action reached, re-enable tracing */ - if (params->actions.present[ACTION_TRACE_OUTPUT]) + if (record) trace_instance_start(&record->trace); if (!params->no_aa) trace_instance_start(&aa->trace); @@ -993,14 +1005,14 @@ timerlat_top_bpf_main_loop(struct osnoise_tool *top, =20 if (wait_retval =3D=3D 1) { /* Stopping requested by tracer */ - actions_perform(¶ms->actions); + actions_perform(¶ms->threshold_actions); =20 - if (!params->actions.continue_flag) + if (!params->threshold_actions.continue_flag) /* continue flag not set, break */ break; =20 /* continue action reached, re-enable tracing */ - if (params->actions.present[ACTION_TRACE_OUTPUT]) + if (record) trace_instance_start(&record->trace); if (!params->no_aa) trace_instance_start(&aa->trace); @@ -1128,13 +1140,15 @@ int timerlat_top_main(int argc, char *argv[]) } } =20 - if (params->actions.present[ACTION_TRACE_OUTPUT]) { + if (params->threshold_actions.present[ACTION_TRACE_OUTPUT] || + params->end_actions.present[ACTION_TRACE_OUTPUT]) { record =3D osnoise_init_trace_tool("timerlat"); if (!record) { err_msg("Failed to enable the trace instance\n"); goto out_free; } - params->actions.trace_output_inst =3D record->trace.inst; + params->threshold_actions.trace_output_inst =3D record->trace.inst; + params->end_actions.trace_output_inst =3D record->trace.inst; =20 if (params->events) { retval =3D trace_events_enable(&record->trace, params->events); @@ -1201,7 +1215,7 @@ int timerlat_top_main(int argc, char *argv[]) * tracing while enabling other instances. The trace instance is the * one with most valuable information. */ - if (params->actions.present[ACTION_TRACE_OUTPUT]) + if (record) trace_instance_start(&record->trace); if (!params->no_aa) trace_instance_start(&aa->trace); @@ -1236,6 +1250,8 @@ int timerlat_top_main(int argc, char *argv[]) =20 timerlat_print_stats(params, top); =20 + actions_perform(¶ms->end_actions); + return_value =3D PASSED; =20 if (osnoise_trace_is_off(top, record) && !stop_tracing) { @@ -1276,7 +1292,8 @@ int timerlat_top_main(int argc, char *argv[]) osnoise_destroy_tool(aa); osnoise_destroy_tool(record); osnoise_destroy_tool(top); - actions_destroy(¶ms->actions); + actions_destroy(¶ms->threshold_actions); + actions_destroy(¶ms->end_actions); if (params->mode !=3D TRACING_MODE_TRACEFS) timerlat_bpf_destroy(); free(params); --=20 2.47.2