From nobody Sat Feb 7 17:55:27 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A9C4C001DC for ; Tue, 18 Jul 2023 22:48:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230288AbjGRWsA (ORCPT ); Tue, 18 Jul 2023 18:48:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229518AbjGRWr6 (ORCPT ); Tue, 18 Jul 2023 18:47:58 -0400 Received: from mail-oi1-x22b.google.com (mail-oi1-x22b.google.com [IPv6:2607:f8b0:4864:20::22b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E052DA1; Tue, 18 Jul 2023 15:47:53 -0700 (PDT) Received: by mail-oi1-x22b.google.com with SMTP id 5614622812f47-38c35975545so4839172b6e.1; Tue, 18 Jul 2023 15:47:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689720473; x=1692312473; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=R5YqH3VqKGjLLAoXXHfP8sq49ATZcVTncb1eNoNAhtA=; b=SGae1hx5Ep3cwgFdPAy75Mv3EV96nNYg5p5X+GUs0FZ9jcD2vL3yTAENZV/+qLhaTO 7h75tXKjL9rNTf3Wvg6xHLiRo6V7987kBEmldd52lMhgrjIec8+26EYINE+GYiaAgIDS 9u0xaLpFsX/GBlFsyZPyG2AIiKUXqqGyYwPYW7LgQG2RoRrK+MB4zfHRBuiuQC8WUF9v 7YNqYQ3x0OGKbVYRvw6rhDu+1QNt448ysVd1XO5lnz2sFJRQgQWe1N7nvgvMEaTZ4Nti vPZBgEt45q90u9XTl3nw2pZCP/R6y86byGiltDxypFy/ZWHkU5rsT3gKbHBI+DNPfDkE uXfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689720473; x=1692312473; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=R5YqH3VqKGjLLAoXXHfP8sq49ATZcVTncb1eNoNAhtA=; b=LYO+VMFOI9eT4DNeaDFQvB0bEVCeuqxyeHWgSId4BvFs8PmjoVP2i2DA4GPjqVqMm9 GMpJ5iDyzh72fnnTycZkQ9XMzb9qe/iUPQy/Ee6Xa4qC5V7oVrCDBzKfGnxEwpOeoU3H PuEAtw/21mRutJKC5ngLPH8ieXtwlQyvBm7sEZELqGFf/TtDrr0YPjC12pu/btsDAlgK hcfbeRM/zbTJ4xM6B0xqARpS3cXNtkdI45aBFbNnyK+3lbMvV2YgmSe/874VY137PgB7 YMRu0xPqjcdsP5c7xBKTTbkbE0qPb7JX+KtuuVUjCkM0fxkct7VSs8yf2ugkTQF+5iYD K1bQ== X-Gm-Message-State: ABy/qLbM1Y7ygUh5NS9R0LxdePN3+cxSuKoAbthk+QQjGSgIa88hqIrn A0uYrXzUznRKR0yBoI4OjNU= X-Google-Smtp-Source: APBJJlFgWGatezkp9nPkMa8w08ly5NtSVSXqx51EiWMl8ZUgey5YauYcJG6q8Nkal092t9zMgC/6Ag== X-Received: by 2002:aca:bd04:0:b0:398:2f92:65ca with SMTP id n4-20020acabd04000000b003982f9265camr12531578oif.7.1689720472963; Tue, 18 Jul 2023 15:47:52 -0700 (PDT) Received: from yoga ([2400:1f00:13:be68:c61a:77e0:3624:e4f6]) by smtp.gmail.com with ESMTPSA id w5-20020a17090a380500b0025e7f7b46c3sm86662pjb.25.2023.07.18.15.47.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jul 2023 15:47:52 -0700 (PDT) Date: Wed, 19 Jul 2023 04:17:44 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 1/6] perf scripts python: Add initial script file with usage information Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Added necessary modules, including the Perf-Trace-Util library, and defines the required functions and variables for using perf script python. The perf_trace_context and Core modules for tracing and processing events has been also imported. Also added usage information. Signed-off-by: Anup Sharma Acked-by: Ian Rogers --- .../scripts/python/firefox-gecko-converter.py | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 tools/perf/scripts/python/firefox-gecko-converter.py diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py new file mode 100644 index 000000000000..63ecaf1abee8 --- /dev/null +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -0,0 +1,31 @@ +# firefox-gecko-converter.py - Convert perf record output to Firefox's gec= ko profile format +# SPDX-License-Identifier: GPL-2.0 +# +# The script converts perf.data to Gecko Profile Format, +# which can be read by https://profiler.firefox.com/. +# +# Usage: +# +# perf record -a -g -F 99 sleep 60 +# perf script firefox-gecko-converter.py > output.json + +import os +import sys +from typing import Dict + +# Add the Perf-Trace-Util library to the Python path +sys.path.append(os.environ['PERF_EXEC_PATH'] + \ + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') + +from perf_trace_context import * +from Core import * + +# Uses perf script python interface to parse each +# event and store the data in the thread builder. +def process_event(param_dict: Dict) -> None: + pass + +# Trace_end runs at the end and will be used to aggregate +# the data into the final json object and print it out to stdout. +def trace_end() -> None: + pass --=20 2.34.1 From nobody Sat Feb 7 17:55:27 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1B80C001DC for ; Tue, 18 Jul 2023 22:48:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231168AbjGRWsu (ORCPT ); Tue, 18 Jul 2023 18:48:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229518AbjGRWsr (ORCPT ); Tue, 18 Jul 2023 18:48:47 -0400 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF384EC; Tue, 18 Jul 2023 15:48:34 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id 41be03b00d2f7-55c993e26ffso2433620a12.0; Tue, 18 Jul 2023 15:48:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689720514; x=1692312514; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=1d8swg9Y9ZixZmusx5ZmXbxt99Xqol7znCwCmQSk3ZI=; b=emwJY9ZJECLmBaafe7jMvIHAr4sMp56GWkOkD7dQZL5tAOkKbIVuBc5y7+2hvJD5ZC SScNluQBdVqrXU8XA0q5rXinV8EaBkL4x9F83xoAmn/kx2+LcNhCFghJVdly7VCbGlwN 8HCR4Gdqoy74YVjuRkik+yCbDi9/QQwRkVnUgbv9gA5na23PcLR0YLBCDT1wetrDr1c0 qVNappH1/hxsd6g85MH1Gctgw1CZvKwUKqGwG2LGdBHwN23rNY4pZdI/vEW7MWBpIpWt kTm2Umi602swO+e+Ye72G0Sjoa+2Q5/56UGa6oTsayuOyc9fKa2mTzFxPvQQRQN1czE8 xLeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689720514; x=1692312514; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=1d8swg9Y9ZixZmusx5ZmXbxt99Xqol7znCwCmQSk3ZI=; b=NTUrLlri4JASAN2goQRF1D7ovF9DPpCGdphH7Mx7SNUf0MorAAWdUJzZ+gwjheeynq bXdZA2nBBx1gwc41jNKKbly/PoDw9zeavPOby4nTcTjQFqyCtN3kF1dz4hNzX5Ux4Tp+ /rETyYkMabMxwu+rQn2kjKcZ3qPAAFyGBLLsR2UuEL2Au38DwjtKmBVpGRuyZR1nVB8F GXlZ1vqM1wl/lFEqE7ta2P2Ody2susU7Ak06IsAolU2C3N96Ums5YAt1pue2FnRriaEN xvmza0vAanSTQAHf2WBg+n9JxAQDPuvF/OPqNvM91dudRGt8UPWpsMludXjnwHM6FlMd JaeQ== X-Gm-Message-State: ABy/qLbM6yZm5qugN9rGUVttb/tiyHNT20wtgwkEQLHlQ5SU9Tduh88G TXSHr0uylGmo0lFeneo8x4g= X-Google-Smtp-Source: APBJJlEr9VeGDpzg3TrHyG+9650W/M65MuPWS5iwlfcndUVKOal5WfoaF32Bn8dB3y9BpmI1dyGKSg== X-Received: by 2002:a05:6a20:3955:b0:12f:dce2:b385 with SMTP id r21-20020a056a20395500b0012fdce2b385mr4062572pzg.10.1689720514171; Tue, 18 Jul 2023 15:48:34 -0700 (PDT) Received: from yoga ([2400:1f00:13:be68:c61a:77e0:3624:e4f6]) by smtp.gmail.com with ESMTPSA id u21-20020aa78395000000b0065434edd521sm1959826pfm.196.2023.07.18.15.48.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jul 2023 15:48:33 -0700 (PDT) Date: Wed, 19 Jul 2023 04:18:26 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 2/6] perf scripts python: Extact necessary information from process event Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The script takes in a sample event dictionary(param_dict) and retrieves relevant data such as time stamp, PID, TID, and comm for each event. Also start time is defined as a global variable as it need to be passed to trace_end for gecko meta information field creation. Signed-off-by: Anup Sharma Acked-by: Ian Rogers --- .../perf/scripts/python/firefox-gecko-converter.py | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index 63ecaf1abee8..bfc8c2b026bf 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -20,10 +20,22 @@ sys.path.append(os.environ['PERF_EXEC_PATH'] + \ from perf_trace_context import * from Core import * =20 +# start_time is intialiazed only once for the all event traces. +start_time =3D None + # Uses perf script python interface to parse each # event and store the data in the thread builder. def process_event(param_dict: Dict) -> None: - pass + global start_time + global tid_to_thread + time_stamp =3D (param_dict['sample']['time'] // 1000) / 1000 + pid =3D param_dict['sample']['pid'] + tid =3D param_dict['sample']['tid'] + comm =3D param_dict['comm'] + + # Start time is the time of the first sample + if not start_time: + start_time =3D time_stamp =20 # Trace_end runs at the end and will be used to aggregate # the data into the final json object and print it out to stdout. --=20 2.34.1 From nobody Sat Feb 7 17:55:27 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D24BC001DC for ; Tue, 18 Jul 2023 22:50:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230373AbjGRWu1 (ORCPT ); Tue, 18 Jul 2023 18:50:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229765AbjGRWuZ (ORCPT ); Tue, 18 Jul 2023 18:50:25 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 163B4FD; Tue, 18 Jul 2023 15:49:43 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-262e5e71978so140951a91.1; Tue, 18 Jul 2023 15:49:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689720582; x=1692312582; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=j0vN6LDudj3dFgUyjWuCTJLzuM5gYdF0WTTzcmqzoCU=; b=WSEwnS0RKZHGWcnIJkoVosJhvJSrwQVYhe4nHKSVNS5Sxg1n7Ng3VViJvitlDF6gyE msKvsLR3ARGi9KUbiw1ZWpDQdeol/UGsIeIJm/XiH0rz6Z0BAEfZ50BNFO/QjGU68Tm/ Y/pH7pOdF3e3QOoCPL0lV7seZ+MRUsAzCl1tv5L3YHzidqN2wWCrMLKUggRZEjT8UV5/ exLJteyMKJY8DGnulLuFmbET6gdtS2QdBjwB6Zop2Zx73IYARjhajXZB3wMoNt34hPPy glN3TpdpKaTVBxok3LFHn1UzVYkLGTem2MSVnAtQekxLjS6342rIArfzWyHOLrDTzKy6 as9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689720582; x=1692312582; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=j0vN6LDudj3dFgUyjWuCTJLzuM5gYdF0WTTzcmqzoCU=; b=hEYQp1PizPs05X3UMRO1JDgayfLD7OFHgVKYVpiS4gtCk3LcGtXJDUZHjpvw7cV3sV +HS9KXBVkzOPNr5cEbyt491OBujUKAvOhhRyjnjs2T40HpISJLyh0hvgVq3va53dRqm4 W8zN7mIw0azYisfIJkpOEi13uvhtEp4L9+9bB7CSsoWEolVdqrYX61XLES3pdXIMIx/r oqRfj9f+rKXeS8RV+pw8fXz+D1N77Ypnc61lHWtNm4hHEA8cGaxtSyStg3ve6AKzZ1al Hfx7cvNkBNZ8TmbIwwL/SYr5fcObq06tcycVA8zPHo+wfkn6DmFnojUPlpOXeMQVk1ei 4y9g== X-Gm-Message-State: ABy/qLbHPd5vwUoXHBFr2w2msq89XyvpYt7UR95aftrWJ6XvpGsoZz63 TSkc/SM7SA1sBJIlIBbs8Dc= X-Google-Smtp-Source: APBJJlHDIrXsg5LMUjeK2xSM746FyL+Vrg/R2SmZDvVK8+xHoFff6K+mIggFjV3D1GHjJUpFIQKO5g== X-Received: by 2002:a17:90a:e8f:b0:256:675f:1d49 with SMTP id 15-20020a17090a0e8f00b00256675f1d49mr632064pjx.0.1689720582361; Tue, 18 Jul 2023 15:49:42 -0700 (PDT) Received: from yoga ([2400:1f00:13:be68:c61a:77e0:3624:e4f6]) by smtp.gmail.com with ESMTPSA id 2-20020a17090a004200b00262ff206931sm78374pjb.42.2023.07.18.15.49.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jul 2023 15:49:42 -0700 (PDT) Date: Wed, 19 Jul 2023 04:19:34 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 3/6] perf scripts python: Add classes and conversion functions Message-ID: <1cf78360595ce67c81e027d52144d4dff29d1091.1689718662.git.anupnewsmail@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit introduces new classes and conversion functions to facilitate the representation of Gecko profile information. The new classes Frame, Stack, Sample, and Thread are added to handle specific components of the profile data, also link to the origin docs has been commented out. Additionally, Inside the Thread class _to_json_dict() method has been created that converts the current thread data into the corresponding format expected by the GeckoThread JSON schema, as per the Gecko profile format specification. Signed-off-by: Anup Sharma Acked-by: Ian Rogers --- .../scripts/python/firefox-gecko-converter.py | 134 +++++++++++++++++- 1 file changed, 133 insertions(+), 1 deletion(-) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index bfc8c2b026bf..d9b1ec18997a 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -11,7 +11,8 @@ =20 import os import sys -from typing import Dict +from dataclasses import dataclass, field +from typing import List, Dict, Optional, NamedTuple, Set, Tuple, Any =20 # Add the Perf-Trace-Util library to the Python path sys.path.append(os.environ['PERF_EXEC_PATH'] + \ @@ -20,9 +21,140 @@ sys.path.append(os.environ['PERF_EXEC_PATH'] + \ from perf_trace_context import * from Core import * =20 +StringID =3D int +StackID =3D int +FrameID =3D int +CategoryID =3D int +Milliseconds =3D float + # start_time is intialiazed only once for the all event traces. start_time =3D None =20 +# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7= 457fee1d66cd4e2737/src/types/gecko-profile.js#L156 +class Frame(NamedTuple): + string_id: StringID + relevantForJS: bool + innerWindowID: int + implementation: None + optimizations: None + line: None + column: None + category: CategoryID + subcategory: int + +# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7= 457fee1d66cd4e2737/src/types/gecko-profile.js#L216 +class Stack(NamedTuple): + prefix_id: Optional[StackID] + frame_id: FrameID + +# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7= 457fee1d66cd4e2737/src/types/gecko-profile.js#L90 +class Sample(NamedTuple): + stack_id: Optional[StackID] + time_ms: Milliseconds + responsiveness: int + +@dataclass +class Thread: + """A builder for a profile of the thread. + + Attributes: + comm: Thread command-line (name). + pid: process ID of containing process. + tid: thread ID. + samples: Timeline of profile samples. + frameTable: interned stack frame ID -> stack frame. + stringTable: interned string ID -> string. + stringMap: interned string -> string ID. + stackTable: interned stack ID -> stack. + stackMap: (stack prefix ID, leaf stack frame ID) -> interned Stack ID. + frameMap: Stack Frame string -> interned Frame ID. + comm: str + pid: int + tid: int + samples: List[Sample] =3D field(default_factory=3Dlist) + frameTable: List[Frame] =3D field(default_factory=3Dlist) + stringTable: List[str] =3D field(default_factory=3Dlist) + stringMap: Dict[str, int] =3D field(default_factory=3Ddict) + stackTable: List[Stack] =3D field(default_factory=3Dlist) + stackMap: Dict[Tuple[Optional[int], int], int] =3D field(default_factory= =3Ddict) + frameMap: Dict[str, int] =3D field(default_factory=3Ddict) + """ + comm: str + pid: int + tid: int + samples: List[Sample] =3D field(default_factory=3Dlist) + frameTable: List[Frame] =3D field(default_factory=3Dlist) + stringTable: List[str] =3D field(default_factory=3Dlist) + stringMap: Dict[str, int] =3D field(default_factory=3Ddict) + stackTable: List[Stack] =3D field(default_factory=3Dlist) + stackMap: Dict[Tuple[Optional[int], int], int] =3D field(default_factory= =3Ddict) + frameMap: Dict[str, int] =3D field(default_factory=3Ddict) + + def _to_json_dict(self) -> Dict: + """Converts current Thread to GeckoThread JSON format.""" + # Gecko profile format is row-oriented data as List[List], + # And a schema for interpreting each index. + # Schema: + # https://github.com/firefox-devtools/profiler/blob/main/docs-developer/= gecko-profile-format.md + # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26= d7457fee1d66cd4e2737/src/types/gecko-profile.js#L230 + return { + "tid": self.tid, + "pid": self.pid, + "name": self.comm, + # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e2= 6d7457fee1d66cd4e2737/src/types/gecko-profile.js#L51 + "markers": { + "schema": { + "name": 0, + "startTime": 1, + "endTime": 2, + "phase": 3, + "category": 4, + "data": 5, + }, + "data": [], + }, + + # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e2= 6d7457fee1d66cd4e2737/src/types/gecko-profile.js#L90 + "samples": { + "schema": { + "stack": 0, + "time": 1, + "responsiveness": 2, + }, + "data": self.samples + }, + + # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e2= 6d7457fee1d66cd4e2737/src/types/gecko-profile.js#L156 + "frameTable": { + "schema": { + "location": 0, + "relevantForJS": 1, + "innerWindowID": 2, + "implementation": 3, + "optimizations": 4, + "line": 5, + "column": 6, + "category": 7, + "subcategory": 8, + }, + "data": self.frameTable, + }, + + # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e2= 6d7457fee1d66cd4e2737/src/types/gecko-profile.js#L216 + "stackTable": { + "schema": { + "prefix": 0, + "frame": 1, + }, + "data": self.stackTable, + }, + "stringTable": self.stringTable, + "registerTime": 0, + "unregisterTime": None, + "processType": "default", + } + + # Uses perf script python interface to parse each # event and store the data in the thread builder. def process_event(param_dict: Dict) -> None: --=20 2.34.1 From nobody Sat Feb 7 17:55:27 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27FF7EB64DC for ; Tue, 18 Jul 2023 22:50:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231190AbjGRWus (ORCPT ); Tue, 18 Jul 2023 18:50:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231164AbjGRWup (ORCPT ); Tue, 18 Jul 2023 18:50:45 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DF7110A; Tue, 18 Jul 2023 15:50:16 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1b8ad907ba4so37168925ad.0; Tue, 18 Jul 2023 15:50:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689720616; x=1692312616; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=80aRBWvaVaWLuW8nDpBJWXcRLl3VPc3POw1KKbLSEx4=; b=B2t7nteaMvgwOtKhySgmdV3tsVDETsWZoJAbzd16487TXUVnftXjvzq/BadVWL4cjB emH8NGSmC2fBAidHjs7T9I19ohQXTNEfPxvM6fQSFNTqRuHDJ0clV9fBeAiV0eTdb4ff o1g6zg1z4VEndA5Ko8G3os6TgzzhP5482DI/cQebTX+kQPDFToS9MIRm4pSNwbWDpPTe qsfQD/U20QVnlJUxYE0i/2l5UutIzilga1uKBFwitrmYJ1NZtwmL0OfFlcS9Cuuf2DQc c+NFZ8diSixknyzgOAC9f1qK5n6NxQk7hCXcGALt2mLTKpDSrFI5FBmxiOrixyBKNfm0 r6hA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689720616; x=1692312616; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=80aRBWvaVaWLuW8nDpBJWXcRLl3VPc3POw1KKbLSEx4=; b=kJSRP/nzIhKS9DXeVmVpWh6yVu2hh6mwyyFbvhBU6bYNUV+rFsp531rHqfYLhzrumg ZQWOZPZen6KoQU9xOixqrNed9hQmiQRWjuek04iiiUM9XEzk5/pe3G5PfXAg43FOQGBB ZUxrm1hbnb1Ch3LvvwVa3V76cf0nBSTylEgw1wNKjfRw88A6TK9HNscimf2Vu0vzPHCZ fL8XgZkhlOxZC24+4eyW3R76Gm2Hc8dxCH4XSyg6uakUXNB5bskrhmzorYONfgthdqpl OwV+9Rday6ualcI/uj3C5CfmO27DdPjBPjlGmcvK3J+zAnhpxNG8zVnGLYtSr8LiTBsL eYAQ== X-Gm-Message-State: ABy/qLYGgSkj4d6TeNOi5aXYmlyY0dNcqv3o6l/wi/avn2EcbLlZMdnf jlesq3QPXm/puTwajd1FEss= X-Google-Smtp-Source: APBJJlG29XKMrtZlXN8ZhQ+TIxumqEBXKXA2mzs6U+FOca1FaZtq+TNQ7K/FRaNaVqcMbdtYLicSMw== X-Received: by 2002:a17:902:e744:b0:1b6:7f96:42ca with SMTP id p4-20020a170902e74400b001b67f9642camr15791921plf.66.1689720615804; Tue, 18 Jul 2023 15:50:15 -0700 (PDT) Received: from yoga ([2400:1f00:13:be68:c61a:77e0:3624:e4f6]) by smtp.gmail.com with ESMTPSA id p8-20020a170902b08800b001b9ff5aa2e7sm2372950plr.239.2023.07.18.15.50.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jul 2023 15:50:15 -0700 (PDT) Date: Wed, 19 Jul 2023 04:20:08 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 4/6] perf scripts python: Add trace end processing and PRODUCT and CATEGORIES information Message-ID: <662239f70618982ef659362ae5729422f318db66.1689718662.git.anupnewsmail@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The final output will now be presented in JSON format following the Gecko profile structure. Additionally, the inclusion of PRODUCT allows easy retri= eval of header information for UI. Furthermore, CATEGORIES have been introduced to enable customization of kernel and user colors using input arguments. To facilitate this functional= ity, an argparse-based parser has been implemented. Note that the implementation of threads will be addressed in subsequent com= mits." Signed-off-by: Anup Sharma Acked-by: Ian Rogers --- .../scripts/python/firefox-gecko-converter.py | 64 ++++++++++++++++++- 1 file changed, 63 insertions(+), 1 deletion(-) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index d9b1ec18997a..a0218e2245f2 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -11,6 +11,8 @@ =20 import os import sys +import json +import argparse from dataclasses import dataclass, field from typing import List, Dict, Optional, NamedTuple, Set, Tuple, Any =20 @@ -30,6 +32,13 @@ Milliseconds =3D float # start_time is intialiazed only once for the all event traces. start_time =3D None =20 +# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7= 457fee1d66cd4e2737/src/types/profile.js#L425 +# Follow Brendan Gregg's Flamegraph convention: orange for kernel and yell= ow for user space by default. +CATEGORIES =3D None + +# The product name is used by the profiler UI to show the Operating system= and Processor. +PRODUCT =3D os.popen('uname -op').read().strip() + # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7= 457fee1d66cd4e2737/src/types/gecko-profile.js#L156 class Frame(NamedTuple): string_id: StringID @@ -172,4 +181,57 @@ def process_event(param_dict: Dict) -> None: # Trace_end runs at the end and will be used to aggregate # the data into the final json object and print it out to stdout. def trace_end() -> None: - pass + # Schema: https://github.com/firefox-devtools/profiler/blob/53970305b51b9= b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L305 + gecko_profile_with_meta =3D { + "meta": { + "interval": 1, + "processType": 0, + "product": PRODUCT, + "stackwalk": 1, + "debug": 0, + "gcpoison": 0, + "asyncstack": 1, + "startTime": start_time, + "shutdownTime": None, + "version": 24, + "presymbolicated": True, + "categories": CATEGORIES, + "markerSchema": [], + }, + "libs": [], + "threads": threads, + "processes": [], + "pausedRanges": [], + } + json.dump(gecko_profile_with_meta, sys.stdout, indent=3D2) + +def main() -> None: + global CATEGORIES + parser =3D argparse.ArgumentParser(description=3D"Convert perf.data to Fi= refox\'s Gecko Profile format") + + # Add the command-line options + # Colors must be defined according to this: + # https://github.com/firefox-devtools/profiler/blob/50124adbfa488adba6e26= 74a8f2618cf34b59cd2/res/css/categories.css + parser.add_argument('--user-color', default=3D'yellow', help=3D'Color for= the User category') + parser.add_argument('--kernel-color', default=3D'orange', help=3D'Color f= or the Kernel category') + # Parse the command-line arguments + args =3D parser.parse_args() + # Access the values provided by the user + user_color =3D args.user_color + kernel_color =3D args.kernel_color + + CATEGORIES =3D [ + { + "name": 'User', + "color": user_color, + "subcategories": ['Other'] + }, + { + "name": 'Kernel', + "color": kernel_color, + "subcategories": ['Other'] + }, + ] + +if __name__ =3D=3D '__main__': + main() --=20 2.34.1 From nobody Sat Feb 7 17:55:27 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66383C001DC for ; Tue, 18 Jul 2023 22:51:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230414AbjGRWvx (ORCPT ); Tue, 18 Jul 2023 18:51:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229532AbjGRWvv (ORCPT ); Tue, 18 Jul 2023 18:51:51 -0400 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02958113; Tue, 18 Jul 2023 15:51:00 -0700 (PDT) Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-262d33fa37cso3434069a91.3; Tue, 18 Jul 2023 15:50:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689720659; x=1692312659; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=bAziTTNqkuDxnsVhswEAtYdwMiQKe4Q0DSlep6xN+rY=; b=TG6BttH0y1XtYqBNMg8rULmY5CDgCLLSQfYicSCLKZa00WKkMgXKSZBlzUChvsKO4A 47qSYrFT1ZhmczurwUznvobMAGABpncs7nh339VmTHq92fLtQPjZG86JwikiQY9pvenC q0Tjt5X/ff2Fvsxkgrb++IQsrPhRe5W0PGtnf+b4+tFWrw/rcnzQuLZ+L4AkySf5zV5C h1lm/Uq9VpRMzkR2lyq1vqfRh9V/EeVQLZwY5LjkxKeoSPrdMqyu1uGLBXvurROtFJ0i KoQmawZkYh6EFnmCaMyy5xPTIRoaFS762wuIMdRtccr1aSzm3S737s7bJX3ZHahj2eiC xMOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689720659; x=1692312659; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=bAziTTNqkuDxnsVhswEAtYdwMiQKe4Q0DSlep6xN+rY=; b=HRTAyoqu3Glmk/++aYCyBsFRrOFw36c40NK+1KcJMMMFLk0HOnz3IUXbNRNbg6vCze ek+/vjkwJ3fvPv06lHyujO1gw/XUuC71xT1LGWk1bgQaHsPNIqnfk0lN9D2c4H9WoMk3 4aNeA/sVG5V8eGU/VJPW+bJdqymD9DbRL4AwX7RmfoMRDT24dHsppEs/W+52EPUlO/EA 1rO8pLy8vr12dhjuHGz+CHinA/0NJJ8AwuwZSRln5opYnetKFiZdMu+JG2Nn3KL9opTx 6YnLHM1b6FCTtbKEGmmeC44Ggq1/aeHBHSzuIZusWhgx9urgyuzpnTd30wlzqx0KZ0U2 zpXA== X-Gm-Message-State: ABy/qLbN5KXKDyhaGM/3NQNuna8WWObEau6cdmfUs4KqVWWppdQk1hhH HeykhGMUKujyHNMUdwdjcumEr0bTd3C2mouI X-Google-Smtp-Source: APBJJlE1LxEerg9t/i5ahWNwoPzbIJNTBbMbyjoMt5nbezqaUrOmkjFtf27vc87mxvPig5D29FIKyg== X-Received: by 2002:a17:90a:294b:b0:263:70d5:6d60 with SMTP id x11-20020a17090a294b00b0026370d56d60mr13331369pjf.30.1689720659188; Tue, 18 Jul 2023 15:50:59 -0700 (PDT) Received: from yoga ([2400:1f00:13:be68:c61a:77e0:3624:e4f6]) by smtp.gmail.com with ESMTPSA id gp20-20020a17090adf1400b0025023726fc4sm75420pjb.26.2023.07.18.15.50.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jul 2023 15:50:58 -0700 (PDT) Date: Wed, 19 Jul 2023 04:20:51 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 5/6] perf scripts python: implement internal get or create frame, stack and string function Message-ID: <4cbbbb2b748f29c1a5025d23f0260a395054003a.1689718662.git.anupnewsmail@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The intern_stack function is responsible for retrieving or creating a stack_id based on the provided frame_id and prefix_id. It first generates a key using the frame_id and prefix_id values. If the stack corresponding to the key is found in the stackMap, it is returned. Otherwise, a new stack is created by appending the prefix_id and frame_id to the stackTable. The key and the index of the newly created stack are added to the stackMap for future reference. The _intern_frame function is responsible for retrieving or creating a frame_id based on the provided frame string. If the frame_id corresponding to the frameString is found in the frameMap, it is returned. Otherwise, a new frame is created by appending relevant information to the frameTable and adding the frameString to the string_id through _intern_string. The _intern_string function will gets a matching string, or saves the new string and returns a String ID. Signed-off-by: Anup Sharma Acked-by: Ian Rogers --- .../scripts/python/firefox-gecko-converter.py | 58 ++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index a0218e2245f2..ae69c0a4af13 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -13,6 +13,7 @@ import os import sys import json import argparse +from functools import reduce from dataclasses import dataclass, field from typing import List, Dict, Optional, NamedTuple, Set, Tuple, Any =20 @@ -39,6 +40,13 @@ CATEGORIES =3D None # The product name is used by the profiler UI to show the Operating system= and Processor. PRODUCT =3D os.popen('uname -op').read().strip() =20 +# Here key =3D tid, value =3D Thread +tid_to_thread =3D dict() + +# The category index is used by the profiler UI to show the color of the f= lame graph. +USER_CATEGORY_INDEX =3D 0 +KERNEL_CATEGORY_INDEX =3D 1 + # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7= 457fee1d66cd4e2737/src/types/gecko-profile.js#L156 class Frame(NamedTuple): string_id: StringID @@ -99,6 +107,55 @@ class Thread: stackMap: Dict[Tuple[Optional[int], int], int] =3D field(default_factory= =3Ddict) frameMap: Dict[str, int] =3D field(default_factory=3Ddict) =20 + def _intern_stack(self, frame_id: int, prefix_id: Optional[int]) -> int: + """Gets a matching stack, or saves the new stack. Returns a Stack ID.""" + key =3D f"{frame_id}" if prefix_id is None else f"{frame_id},{prefix_id}" + # key =3D (prefix_id, frame_id) + stack_id =3D self.stackMap.get(key) + if stack_id is None: + # return stack_id + stack_id =3D len(self.stackTable) + self.stackTable.append(Stack(prefix_id=3Dprefix_id, frame_id=3Dframe_id= )) + self.stackMap[key] =3D stack_id + return stack_id + + def _intern_string(self, string: str) -> int: + """Gets a matching string, or saves the new string. Returns a String ID.= """ + string_id =3D self.stringMap.get(string) + if string_id is not None: + return string_id + string_id =3D len(self.stringTable) + self.stringTable.append(string) + self.stringMap[string] =3D string_id + return string_id + + def _intern_frame(self, frame_str: str) -> int: + """Gets a matching stack frame, or saves the new frame. Returns a Frame = ID.""" + frame_id =3D self.frameMap.get(frame_str) + if frame_id is not None: + return frame_id + frame_id =3D len(self.frameTable) + self.frameMap[frame_str] =3D frame_id + string_id =3D self._intern_string(frame_str) + + symbol_name_to_category =3D KERNEL_CATEGORY_INDEX if frame_str.find('kal= lsyms') !=3D -1 \ + or frame_str.find('/vmlinux') !=3D -1 \ + or frame_str.endswith('.ko)') \ + else USER_CATEGORY_INDEX + + self.frameTable.append(Frame( + string_id=3Dstring_id, + relevantForJS=3DFalse, + innerWindowID=3D0, + implementation=3DNone, + optimizations=3DNone, + line=3DNone, + column=3DNone, + category=3Dsymbol_name_to_category, + subcategory=3DNone, + )) + return frame_id + def _to_json_dict(self) -> Dict: """Converts current Thread to GeckoThread JSON format.""" # Gecko profile format is row-oriented data as List[List], @@ -163,7 +220,6 @@ class Thread: "processType": "default", } =20 - # Uses perf script python interface to parse each # event and store the data in the thread builder. def process_event(param_dict: Dict) -> None: --=20 2.34.1 From nobody Sat Feb 7 17:55:27 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9763EB64DD for ; Tue, 18 Jul 2023 22:53:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231384AbjGRWwv (ORCPT ); Tue, 18 Jul 2023 18:52:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229577AbjGRWws (ORCPT ); Tue, 18 Jul 2023 18:52:48 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F6FFEE; Tue, 18 Jul 2023 15:52:22 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-666ed230c81so6299866b3a.0; Tue, 18 Jul 2023 15:52:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689720741; x=1692312741; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=AESNSKu76AwBKDmM6LgRhommsrTxqaO6oADrlXIeM64=; b=gvtye7OYtvyAywIXMhCIK07kjzJzgri2LsA/WZAVczuZuJQjAeR6DmBeDGBw5TGPqw r7RmWVBvLhlinXTGJeSmoyJN9i0604XOOq8vGBK2L8ddyOQ3rkkO8TenTR74Q8I4rWmq 1R1GZzHyuQFQKGeH1ff5EERFFnXn0Hyvnw1x+J6Y+O7BZjwhcmYi+tTyjA5+UY2vtLfC zCKfkdCqrV6vtp9BX7rxe97zAXlbXiUI6oixBEV6EguOEB9Mg7ooV4bw+XUHmittbOCT xExiD/x5gxaPx4/zl20oBsKEQhz+YgWwyrL+MpUUeshIbvST1JFNKXU5WWu28ibEqx3N AxGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689720741; x=1692312741; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=AESNSKu76AwBKDmM6LgRhommsrTxqaO6oADrlXIeM64=; b=hHEcnZ0WR5/GE5oFMp5Y5BUwpTrmDTKDp2gIgNKzuoGMg1N9GI0s8a/BRMndrQ0TZU A8bPag2BjNM7coF3x+hIe+nPhJ15q4LCG0C1CCEcpAgqKbrPeDfImeCkKJzYDjt0yLKW gvxjEZj2xJD/VtVZZKwGK24R7dY5lLzpyKDKSFwPWkWgCdyPoacrE6dPQMuK/sE17cTo tQkx4JBoOOgSHJWZmSqVEhWDzSqpaG7CPUVDcnnRn9iw9O7HDiFvNhfaACNtqUEY9N2g UHs+YiwG/sb3jHSzvWaNtzFz0A4w0DZ48KJon1rLNVoFKp8Efnby/N2orz4SELZ7awF8 JiEw== X-Gm-Message-State: ABy/qLb0BFu9AcSVJiMYhp7GoAB09UsgfJnAEgtVSYXLc3UWHuHNt4fn TJgEVt920CKN9UcwxEAoiG8= X-Google-Smtp-Source: APBJJlFv1a/dZKM9JVmWRbsLvfc2IoXCwZIL/wLdapRaYK1KM4xxmWOn/IR3cQz9bWwJ7Hq0+pK0Pw== X-Received: by 2002:a05:6a20:12c9:b0:12e:c481:7072 with SMTP id v9-20020a056a2012c900b0012ec4817072mr914193pzg.37.1689720741264; Tue, 18 Jul 2023 15:52:21 -0700 (PDT) Received: from yoga ([2400:1f00:13:be68:c61a:77e0:3624:e4f6]) by smtp.gmail.com with ESMTPSA id a11-20020a637f0b000000b005633941a547sm1943769pgd.27.2023.07.18.15.52.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jul 2023 15:52:20 -0700 (PDT) Date: Wed, 19 Jul 2023 04:22:13 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 6/6] perf scripts python: Implement add sample function and thread processing Message-ID: <6771dcadf7507ad8916ba2398d22f873579b0505.1689718662.git.anupnewsmail@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The stack has been created for storing func and dso from the callchain. The sample has been added to a specific thread. It first checks if the thread exists in the Thread class. Then it call _add_sample function which is responsible for appending a new entry to the samples list. Signed-off-by: Anup Sharma Acked-by: Ian Rogers --- .../scripts/python/firefox-gecko-converter.py | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index ae69c0a4af13..9c4393787daa 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -156,6 +156,25 @@ class Thread: )) return frame_id =20 + def _add_sample(self, comm: str, stack: List[str], time_ms: Milliseconds)= -> None: + """Add a timestamped stack trace sample to the thread builder. + Args: + comm: command-line (name) of the thread at this sample + stack: sampled stack frames. Root first, leaf last. + time_ms: timestamp of sample in milliseconds. + """ + # Ihreads may not set their names right after they are created. + # Instead, they might do it later. In such situations, to use the latest= name they have set. + if self.comm !=3D comm: + self.comm =3D comm + + prefix_stack_id =3D reduce(lambda prefix_id, frame: self._intern_stack + (self._intern_frame(frame), prefix_id), stack, None) + if prefix_stack_id is not None: + self.samples.append(Sample(stack_id=3Dprefix_stack_id, + time_ms=3Dtime_ms, + responsiveness=3D0)) + def _to_json_dict(self) -> Dict: """Converts current Thread to GeckoThread JSON format.""" # Gecko profile format is row-oriented data as List[List], @@ -234,9 +253,36 @@ def process_event(param_dict: Dict) -> None: if not start_time: start_time =3D time_stamp =20 + # Parse and append the callchain of the current sample into a stack. + stack =3D [] + if param_dict['callchain']: + for call in param_dict['callchain']: + if 'sym' not in call: + continue + stack.append(f'{call["sym"]["name"]} (in {call["dso"]})') + if len(stack) !=3D 0: + # Reverse the stack, as root come first and the leaf at the end. + stack =3D stack[::-1] + + # During perf record if -g is not used, the callchain is not available. + # In that case, the symbol and dso are available in the event parameters. + else: + func =3D param_dict['symbol'] if 'symbol' in param_dict else '[unknown]' + dso =3D param_dict['dso'] if 'dso' in param_dict else '[unknown]' + stack.append(f'{func} (in {dso})') + + # Add sample to the specific thread. + thread =3D tid_to_thread.get(tid) + if thread is None: + thread =3D Thread(comm=3Dcomm, pid=3Dpid, tid=3Dtid) + tid_to_thread[tid] =3D thread + thread._add_sample(comm=3Dcomm, stack=3Dstack, time_ms=3Dtime_stamp) + # Trace_end runs at the end and will be used to aggregate # the data into the final json object and print it out to stdout. def trace_end() -> None: + threads =3D [thread._to_json_dict() for thread in tid_to_thread.values()] + # Schema: https://github.com/firefox-devtools/profiler/blob/53970305b51b9= b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L305 gecko_profile_with_meta =3D { "meta": { --=20 2.34.1