From nobody Sun Dec 14 02:18:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69C8FEB64D9 for ; Mon, 10 Jul 2023 23:09:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230255AbjGJXJW (ORCPT ); Mon, 10 Jul 2023 19:09:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229635AbjGJXJS (ORCPT ); Mon, 10 Jul 2023 19:09:18 -0400 Received: from mail-oi1-x235.google.com (mail-oi1-x235.google.com [IPv6:2607:f8b0:4864:20::235]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B44CF9; Mon, 10 Jul 2023 16:09:14 -0700 (PDT) Received: by mail-oi1-x235.google.com with SMTP id 5614622812f47-3a38953c928so3827475b6e.1; Mon, 10 Jul 2023 16:09:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689030553; x=1691622553; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=f+MMk4Cs0oW3UJzQ6IJCCiJc/ywnbyZxE3/wgqbegso=; b=cB4U7RfmNBVjkzzjb+1bfAJYUV15wXNhZIG09IYSaeBolI6Fj1Hx4z6M0Y5C6DU3gl fu2zWxli88a2GBK36f3uBwreDdTZSgbCVbg/1+ovmmmBYgXDpB+reTwGik9NWmDo/tI8 OV8NIkWHQdt1B9Fprg6Zr7wnNET5OLVBRLblUwonUk6WBwv6VNCyyt1ZdKYO6knyBqoE VFItMhMt60Pgh3Vcvpe3Qwrde2S18IW6ww0LrI0OVPRkYCHphhearXBQ4pFmYoxekGrr +BeKGyap7AyX+JX2DBOPstz2lJHmicSSkS+VPcO/Fd65YuU6tRlM80Y2qBDZzmscwpyg lrrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689030553; x=1691622553; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=f+MMk4Cs0oW3UJzQ6IJCCiJc/ywnbyZxE3/wgqbegso=; b=THwO9kye7DLB7ykPRDWl633nDQ2SRjYbg+Ez+Vf5Pds2lc6+tPAwrnGyrlASpZEMup 5pTt1HF4RWyoA/GdisjPPIk0FdBeYFmvnu4/GHEGmSQvaPbzeTVKu74sp1qbpKIgnpUd dqtNuz0w+NqSUbfbqDDksN5SPdd157ne5BOGnzesXcwuz001Nfnz/Fd0PV5eeroieVJC RimTNMbm43KQiEdQZcRi+o36AErhHcHnTMSwzt8oDalSvpA3hjAJETm5g9istZDLkwLm PsHfsR2GgXYt1syRXrOpizqPwhIGBfO/w6bJ9XSbRKsD8GbKm6CY/6sAWFksWj3VXUUu YYeQ== X-Gm-Message-State: ABy/qLblgjkcomCud3oZflwWURqYLyibo5K81pMA3TTzv8wUR9yBnu6X gbmt/ibtk4uWt5KqSOqtZ6o= X-Google-Smtp-Source: APBJJlHHaBsPSv7VGng3Q30kxCxCGK4lWOaBLxXha5lDvJ29ByAhHYAgmuQj7yqAtzhpyMNn/HJIbA== X-Received: by 2002:a05:6358:7e06:b0:134:d026:42d2 with SMTP id o6-20020a0563587e0600b00134d02642d2mr9576902rwm.24.1689030553073; Mon, 10 Jul 2023 16:09:13 -0700 (PDT) Received: from yoga ([2400:1f00:13:76f2:868d:bc7d:562b:a5bb]) by smtp.gmail.com with ESMTPSA id b125-20020a633483000000b0054fd46531a1sm236429pga.5.2023.07.10.16.09.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 16:09:12 -0700 (PDT) Date: Tue, 11 Jul 2023 04:39:04 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 1/6] scripts: python: Add initial script file with imports Message-ID: <5d3727bacffa331cd81a059fc3122cfb3f43dabd.1689024635.git.anupnewsmail@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Added the necessary modules, including the Perf-Trace-Util library, and defines the required functions and variables. It leverages the perf_trace_context and Core modules for tracing and processing events. Also added usage information. Signed-off-by: Anup Sharma Acked-by: Ian Rogers --- .../scripts/python/firefox-gecko-converter.py | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 tools/perf/scripts/python/firefox-gecko-converter.py diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py new file mode 100644 index 000000000000..5b342641925c --- /dev/null +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -0,0 +1,28 @@ +# firefox-gecko-converter.py - Convert perf record output to Firefox's gec= ko profile format +# SPDX-License-Identifier: GPL-2.0 +# +# The script converts perf.data to Gecko Profile Format, +# which can be read by https://profiler.firefox.com/. +# +# Usage: +# +# perf record -a -g -F 99 sleep 60 +# perf script firefox-gecko-converter.py > output.json + +import os +import sys +import json +from functools import reduce + +# Add the Perf-Trace-Util library to the Python path +sys.path.append(os.environ['PERF_EXEC_PATH'] + \ + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') + +from perf_trace_context import * +from Core import * + +def trace_end(): + pass + +def process_event(param_dict): + pass --=20 2.34.1 From nobody Sun Dec 14 02:18:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76740EB64DC for ; Mon, 10 Jul 2023 23:10:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230409AbjGJXKw (ORCPT ); Mon, 10 Jul 2023 19:10:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229562AbjGJXKu (ORCPT ); Mon, 10 Jul 2023 19:10:50 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D4A31BC; Mon, 10 Jul 2023 16:10:42 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-26598fc0825so1752305a91.0; Mon, 10 Jul 2023 16:10:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689030642; x=1691622642; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=rNmeeRduS9hujTrUN0M+dmMgGxYGX8uEUfhYcfOYymg=; b=CPziO0/boaqbaHNRFbRAn6svFGoaxS4y3UQaqEdmPdQD100wdIzoZZUBW/OdaFqKyR kVJWKil7yB7MDnWrCkpO0Re0A9fbvIaxU3TXI1RjMy5Pe7Jcwi38EOuuu/2pcGnwsStI 76HznsqwJK3SFEqR94ILTF+sq3Fo8mckSCamxKqiN3y4TPY1sohZZLShFkk63gFjbrLl MLhQxoRaCXrUK8ZSHIcuOA88Y7ZvfquaPuA5CM2R8PXMkPZRbEvTdOfEafvWiJ2EF19s VBWIEGEtAJ40rYhekFltZKyn43g17t3a+qhPx/ylexbrITG9Q2uhyUFh62XodybWS9/v PEsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689030642; x=1691622642; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=rNmeeRduS9hujTrUN0M+dmMgGxYGX8uEUfhYcfOYymg=; b=lDh/+F8ujKNQnvXLQ9KlDo/k2TOHQ+DQd23mQaFbOAn/yu84SrD8VAr0sLrjKJYywL NBPxU6eZjkxlLSc1mM6xpx5ypXAbStIXzWHN2eBx2nB5bFqRT+Px5MpHzcAO8uY5qblw xIiDYZY4onoCj1wX1Gw7aSIRFWkdVqCdT+1EHLsxanjYFIstsN9Hu8dAu9Wtzhasw2r9 ELcDWPZJd2k45L4bvvASbjqGgaQ+fMvzZ5jXhXPjR0oJbCsaJ+aNPUP9Beugn5fZ54Tj ddjpcdlDGrZ6kjjk86GVUkf5N7mnSLq4HrjzNjiQ4hqv1axXjsWJAFDK+CK2EHMi++Jh ddpQ== X-Gm-Message-State: ABy/qLYzOhl0uXBBWy8h0XQ9RkNG3+MgacVhQ471qVtsepS8HhFynxS9 wyBFnKgArCv0oi03iHA9htU= X-Google-Smtp-Source: APBJJlEOEGGbDVkNWLd92CS4nQGfnhewo7Lu4+7IWWAmdosKrgva54gLe97Ro1WymAGaE2Rh3ybmAQ== X-Received: by 2002:a17:90a:8a14:b0:261:219b:13b3 with SMTP id w20-20020a17090a8a1400b00261219b13b3mr20346474pjn.16.1689030641684; Mon, 10 Jul 2023 16:10:41 -0700 (PDT) Received: from yoga ([2400:1f00:13:76f2:868d:bc7d:562b:a5bb]) by smtp.gmail.com with ESMTPSA id 5-20020a17090a018500b0026333ad02c1sm505433pjc.10.2023.07.10.16.10.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 16:10:41 -0700 (PDT) Date: Tue, 11 Jul 2023 04:40:33 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 2/6] scripts: python: Extact necessary information from process event Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The script takes in a sample event dictionary(param_dict) and retrieves relevant data such as time stamp, PID, TID, thread name. Also start time is defined. Signed-off-by: Anup Sharma --- .../perf/scripts/python/firefox-gecko-converter.py | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index 5b342641925c..765f1775cee5 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -21,8 +21,19 @@ sys.path.append(os.environ['PERF_EXEC_PATH'] + \ from perf_trace_context import * from Core import * =20 +start_time =3D None + def trace_end(): pass =20 def process_event(param_dict): - pass + global start_time + # Extract relevant information from the event parameters. The event param= eters + # are in a dictionary: + time_stamp =3D (param_dict['sample']['time'] // 1000) / 1000 + pid =3D param_dict['sample']['pid'] + tid =3D param_dict['sample']['tid'] + thread_name =3D param_dict['comm'] + + # Assume that start time is the time of the first event. + start_time =3D time_stamp if not start_time else start_time --=20 2.34.1 From nobody Sun Dec 14 02:18:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98CA3EB64DC for ; Mon, 10 Jul 2023 23:13:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230460AbjGJXNX (ORCPT ); Mon, 10 Jul 2023 19:13:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230420AbjGJXNT (ORCPT ); Mon, 10 Jul 2023 19:13:19 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 581B99E; Mon, 10 Jul 2023 16:13:18 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-666fb8b1bc8so4468817b3a.1; Mon, 10 Jul 2023 16:13:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689030798; x=1691622798; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=PBGlQsi0NW10Pr3RCzCrk5y26OBsbX2q8LbZgkSF6bI=; b=OXkAr0vN5GVizLDpJNmNvYVQjQp+Qj7fP8tDpQxWGxT6s2HOgYPgsgTnfzDoYHm6dx 4RcrVFIT2gE18YGPJXQKkMiewTuvJOdY4DIM3LJwSW0ABQ2vtDI2xvaVFSE42p84ovd9 PsBXCf9wa1B+ctIubXlKq9Y9NlX4zN4ugjWFYDr0DeSVAIWhktsSkXUiHfPZt8+9VHs4 pRoboiKKOorKqUXkpYYTdoHYiizUxXILf5WugY+KXR8lDumopRYexlOlJpz3xmupdBsa 6wWcyyRSlUMxVyMZBznBLi08YqencKeP9FYkpci2StgBYOOUsD2LH3+rwiZ9s2a9MLHm crbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689030798; x=1691622798; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=PBGlQsi0NW10Pr3RCzCrk5y26OBsbX2q8LbZgkSF6bI=; b=O3Gft5vmZNNvZYCqLUryDh9yK4R4FxnbV23TehCVS/y1VqXIA+F4hhHUKIpXJjXLzI XJ+hCcDdS4ddX22kpO1+Yr4kJ8nLb+kqtbLV6lK8zIOkeTRC3aFYV3JlRI1MGO0Mqaoj zgvPJzyb8W8JQrrrG85JDapQDjs4uReVeGTKPdio7gd9M+a9fFRirF1beO5VYC5vjmr2 FTC42LIS38rkJGdHGMUUm10uOt5JNU1TnqEumql3gJM0jJ++6OebUh9Kjp8l6HA7gSFU Lyc21CQkaOukNeIQLf8rCQUG7I2bSe9u4nQsdtHf699OhPxPnEwuovTZwHP50jIkrxaB QfGw== X-Gm-Message-State: ABy/qLZdLynFaT0wq/ARkt+5iLMALt247F+UE9uMSZxw2lRKPkdRRKVZ W09sIehSBYcAwTYvDCjuGMA= X-Google-Smtp-Source: APBJJlEjrB+QuHD8kR2LCfYOwphuv1tRhHLaQNa/4w/WzsX/3L3Tu+ZiiG4dnZjrlHGek+hTTMltjA== X-Received: by 2002:a05:6a20:1601:b0:10d:5430:c8d6 with SMTP id l1-20020a056a20160100b0010d5430c8d6mr18241619pzj.0.1689030797581; Mon, 10 Jul 2023 16:13:17 -0700 (PDT) Received: from yoga ([2400:1f00:13:76f2:868d:bc7d:562b:a5bb]) by smtp.gmail.com with ESMTPSA id fe23-20020a056a002f1700b0066ebaeb149dsm314797pfb.88.2023.07.10.16.13.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 16:13:17 -0700 (PDT) Date: Tue, 11 Jul 2023 04:43:10 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 3/6] scripts: python: thread sample processing to create thread with schemas Message-ID: <70fd94954a3872caeef5851607b245394149f4a9.1689024635.git.anupnewsmail@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The _addThreadSample function is responsible for adding a sample to a specific thread. It first checks if the thread exists in the thread_map dictionary. The markers structure defines the schema and data for thread markers, including fields such as 'name', 'startTime', 'endTime', 'phase', 'category', and 'data'. The samples structure defines the schema and data for thread samples, including fields such as 'stack', 'time', and 'responsiveness'. The frameTable structure defines the schema and data for frame information, including fields such as 'location', 'relevantForJS', 'innerWindowID', 'implementation', 'optimizations', 'line', 'column', 'category', and 'subcategory'. The purpose of this function is to create a new thread structure These structures provide a framework for storing and organizing information related to thread markers, samples, frame details, and stack information. The call stack is parsed to include function names and the associated DSO, which are requires for creating json schema. Also few libaries has been included which will be used in later commit. Signed-off-by: Anup Sharma --- .../scripts/python/firefox-gecko-converter.py | 70 +++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index 765f1775cee5..0b8a86bdcab1 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -21,6 +21,7 @@ sys.path.append(os.environ['PERF_EXEC_PATH'] + \ from perf_trace_context import * from Core import * =20 +thread_map =3D {} start_time =3D None =20 def trace_end(): @@ -28,6 +29,57 @@ def trace_end(): =20 def process_event(param_dict): global start_time + global thread_map + + def _createThread(name, pid, tid): + markers =3D { + 'schema': { + 'name': 0, + 'startTime': 1, + 'endTime': 2, + 'phase': 3, + 'category': 4, + 'data': 5, + }, + 'data': [], + } + samples =3D { + 'schema': { + 'stack': 0, + 'time': 1, + 'responsiveness': 2, + }, + 'data': [], + } + frameTable =3D { + 'schema': { + 'location': 0, + 'relevantForJS': 1, + 'innerWindowID': 2, + 'implementation': 3, + 'optimizations': 4, + 'line': 5, + 'column': 6, + 'category': 7, + 'subcategory': 8, + }, + 'data': [], + } + stackTable =3D { + 'schema': { + 'prefix': 0, + 'frame': 1, + }, + 'data': [], + } + stringTable =3D [] + + def _addThreadSample(pid, tid, threadName, time_stamp, stack): + thread =3D thread_map.get(tid) + if not thread: + thread =3D _createThread(threadName, pid, tid) + thread_map[tid] =3D thread + # Extract relevant information from the event parameters. The event param= eters # are in a dictionary: time_stamp =3D (param_dict['sample']['time'] // 1000) / 1000 @@ -37,3 +89,21 @@ def process_event(param_dict): =20 # Assume that start time is the time of the first event. start_time =3D time_stamp if not start_time else start_time + + # Parse the callchain of the current sample into a stack array. + if param_dict['callchain']: + stack =3D [] + for call in param_dict['callchain']: + if 'sym' not in call: + continue + stack.append(call['sym']['name'] + f' (in {call["dso"]})') + if len(stack) !=3D 0: + stack =3D stack[::-1] + _addThreadSample(pid, tid, thread_name, time_stamp, stack) + + # During perf record if -g is not used, the callchain is not available. + # In that case, the symbol and dso are available in the event parameters. + else: + func =3D param_dict['symbol'] if 'symbol' in param_dict else '[unknown]' + dso =3D param_dict['dso'] if 'dso' in param_dict else '[unknown]' + _addThreadSample(pid, tid, thread_name, time_stamp, [func + f' (in {dso}= )']) --=20 2.34.1 From nobody Sun Dec 14 02:18:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5468EB64DC for ; Mon, 10 Jul 2023 23:13:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231189AbjGJXN4 (ORCPT ); Mon, 10 Jul 2023 19:13:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230496AbjGJXNt (ORCPT ); Mon, 10 Jul 2023 19:13:49 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7ABB9E; Mon, 10 Jul 2023 16:13:48 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-262e66481c4so2395506a91.1; Mon, 10 Jul 2023 16:13:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689030828; x=1691622828; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=nkq4D9xPcGffxrWboWnEtYoPxH3qm7wLImoo2iq2oVo=; b=TcEJFTldAGYmOwZJM0VtK1sxEAYhc3DH8xF2LIP8d9rXlrXRItrQmp50lOQScm730o J/8WpLWSb+2UkZ8twUnXZwm+eTBe2Rgux6GSI94a87sK9zYCsIRVGaSAVloCh5URJ+pC 2//YV+SyLfo6m+kznHuAJNvfJhxjggbGwtxeKpkKJSLe4XYr/hLs9TWAXnWqWmlD1cRl yk9/5BR6ipwSd1EzLMWtYzF9QCuCPlyVBdpEbAQSZqwI78KTVz70qKa5890aGwG3OaeR ORLY0ktlwb1URqza7X9n/v5h05rMIlzWx4UKOdJH4kqmD4yalUELBGhZOU8UWSIeijv/ AVRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689030828; x=1691622828; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=nkq4D9xPcGffxrWboWnEtYoPxH3qm7wLImoo2iq2oVo=; b=l0ywbY9RD1xDuQEfoncAuQD9wiGDROoaH8I2+YS/wfkASXify066+O0+g+xx1VbhZd 1JYjlArCc7iubvazajHlB7rRxIRQeHm42+VVqTGf4SoFYK2axqDi0qtvqCf5UViXlg6R ij0PfxctLmObOnUmJn4V+WCXytUzv8m/DjQXn0Irm8yctKWgLwSBx17zYK92jgENZYyp ymCTe85DeOe9Cv7031t7nRFRdJJ5JOjMX3ZWdGj29gnev9nOQ5mk3hoTj6qbPeL37/Aq ekyenX3SlhvPHvmcufsDaTIOhShaMHFngcy142w5NoHb1wbVPYIaWaxb6qqFijgiXAFm nyng== X-Gm-Message-State: ABy/qLb3qu6Q/yDU0QFj0IkXUDoD7FmaOtned4Vl749SwduFKBustOd2 aR9sYN36tKuUjdqA1GkoVF8= X-Google-Smtp-Source: APBJJlHzh9FTPyE9kL1HOB2Il09oaQYGfAGYRwKgGdi4LYSXdL7Bhv9GN5zmlgWT6HYc09cDj8Vyfw== X-Received: by 2002:a17:90a:d916:b0:261:631:95c2 with SMTP id c22-20020a17090ad91600b00261063195c2mr10054850pjv.27.1689030828296; Mon, 10 Jul 2023 16:13:48 -0700 (PDT) Received: from yoga ([2400:1f00:13:76f2:868d:bc7d:562b:a5bb]) by smtp.gmail.com with ESMTPSA id 30-20020a17090a191e00b002640b7073cfsm7300485pjg.14.2023.07.10.16.13.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 16:13:48 -0700 (PDT) Date: Tue, 11 Jul 2023 04:43:40 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 4/6] scripts: python: Add trace end processing and JSON output Message-ID: <11543b39f97a7e2f9eac76c1233f1b1caa3f2c48.1689024635.git.anupnewsmail@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Inside the trace end function the final output will be dumped to standard output in JSON gecko format. Additionally, constants such as USER_CATEGORY_INDEX, KERNEL_CATEGORY_INDEX, CATEGORIES, and PRODUCT are defined to provide contextual information. Signed-off-by: Anup Sharma --- .../scripts/python/firefox-gecko-converter.py | 34 ++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index 0b8a86bdcab1..39818a603265 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -24,8 +24,40 @@ from Core import * thread_map =3D {} start_time =3D None =20 +# Follow Brendan Gregg's Flamegraph convention: orange for kernel and yell= ow for user +CATEGORIES =3D [ + {'name': 'User', 'color': 'yellow', 'subcategories': ['Other']}, + {'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']} +] + +# The product name is used by the profiler UI to show the Operating system= and Processor. +PRODUCT =3D os.popen('uname -op').read().strip() + def trace_end(): - pass + thread_array =3D thread_map.values())) + + result =3D { + 'meta': { + 'interval': 1, + 'processType': 0, + 'product': PRODUCT, + 'stackwalk': 1, + 'debug': 0, + 'gcpoison': 0, + 'asyncstack': 1, + 'startTime': start_time, + 'shutdownTime': None, + 'version': 24, + 'presymbolicated': True, + 'categories': CATEGORIES, + 'markerSchema': [] + }, + 'libs': [], + 'threads': thread_array, + 'processes': [], + 'pausedRanges': [] + } + json.dump(result, sys.stdout, indent=3D2) =20 def process_event(param_dict): global start_time --=20 2.34.1 From nobody Sun Dec 14 02:18:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBFC4EB64D9 for ; Mon, 10 Jul 2023 23:14:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231194AbjGJXOf (ORCPT ); Mon, 10 Jul 2023 19:14:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230519AbjGJXOb (ORCPT ); Mon, 10 Jul 2023 19:14:31 -0400 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B127195; Mon, 10 Jul 2023 16:14:30 -0700 (PDT) Received: by mail-pg1-x536.google.com with SMTP id 41be03b00d2f7-51452556acdso3453696a12.2; Mon, 10 Jul 2023 16:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689030869; x=1691622869; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=EIaFszOKIjmn1GPr4vHWBEQHE0KiRcRwYwejxWSEw6Q=; b=ekQZS/WIxWgWBCCc3eYQbd+gCWkoM9tJ6gYrkHQ0yrjLOaMwGNhJlLy8FSrAH/hYKE 22F6BErvLuoFVQOKreCpTGmH2CtgSXiXk9PiTeR2Hr74BXpeSdLF4sPY6x5gaH1kKMIP nBi193IpxPuhGZyO5P8GUgPfYzmlf7wsRjQYtsVBmjVNWY6W8ubw92VjRdgzsIPeffky qfkOQNDg1qfQZ0hUhzpmKWi5xCCO7tDHThABGdvf4jEtOlofC19ukFNlj9C4F1XLqPQl +RkYtwdjNQPIsiiyTiVgerMp/8tXSrK9txrxZ7xJLpHBHPpR7wn1hh8kDq6Z6J5q1Zr4 z4Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689030870; x=1691622870; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=EIaFszOKIjmn1GPr4vHWBEQHE0KiRcRwYwejxWSEw6Q=; b=eW+5qvGh2+rmder9Ma1TeUc5nPZaDQTcLigjkrjaeJ0sarose0/AVGtGqmX313Tj4B 8znFNA6CTKOwefyFNxVCxzWzJAuUoRJtqnFQjVaYDy6nruedP3cEbE8s27HYkZPf8le2 scuy1X/Wju8vBKGq/GYcmEibQohXup76qggdNUSIrkhHpgquSsaJmtOttmcqGVyk8en8 xJmGRyqk4gU82ySe+jK1tCHP7vrJsC2J4qFAN8IR9udh4S0vPQCedTmBFYKCma1Ef2ZZ /uB0w+XrB1A8iWKYioTK894cEMZd4EX+lkCgJ4gpOLOzMd6+az9Y5pUeF+KugAa3HV7B Tenw== X-Gm-Message-State: ABy/qLZcztsaVJpS0SS2Wg6WaRoYclsh3feyDrkOi9RKynalrniB20+d zhRmgOUoI0ssvtfENgvMP/g= X-Google-Smtp-Source: APBJJlGgAvP70h63K/YLCRubROWtnNOywvJJQNDmqcqFLiAhFTt2GlaLDGCF9NIFITC9+YVmX2ZYIw== X-Received: by 2002:a17:902:daca:b0:1b8:9195:1dd8 with SMTP id q10-20020a170902daca00b001b891951dd8mr13057988plx.51.1689030869579; Mon, 10 Jul 2023 16:14:29 -0700 (PDT) Received: from yoga ([2400:1f00:13:76f2:868d:bc7d:562b:a5bb]) by smtp.gmail.com with ESMTPSA id t4-20020a170902bc4400b001b9c960ffeasm428846plz.47.2023.07.10.16.14.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 16:14:29 -0700 (PDT) Date: Tue, 11 Jul 2023 04:44:22 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 5/6] scripts: python: Implement add sample function and return finish Message-ID: <8b80f803ffbed4c84c0c63b3e1dae155a66ac1f2.1689024635.git.anupnewsmail@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The addSample function appends a new entry to the 'samples' data structure. The finish function generates a dictionary containing various profile information such as 'tid', 'pid', 'name', 'markers', 'samples', 'frameTable', 'stackTable', 'stringTable', 'registerTime', 'unregisterTime', and 'processType'. Signed-off-by: Anup Sharma --- .../scripts/python/firefox-gecko-converter.py | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index 39818a603265..6c934de1f608 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -106,11 +106,36 @@ def process_event(param_dict): } stringTable =3D [] =20 + def addSample(threadName, stackArray, time): + responsiveness =3D 0 + samples['data'].append([stack, time, responsiveness]) + + def finish(): + return { + "tid": tid, + "pid": pid, + "name": name, + "markers": markers, + "samples": samples, + "frameTable": frameTable, + "stackTable": stackTable, + "stringTable": stringTable, + "registerTime": 0, + "unregisterTime": None, + "processType": 'default' + } + + return { + "addSample": addSample, + "finish": finish + } + def _addThreadSample(pid, tid, threadName, time_stamp, stack): thread =3D thread_map.get(tid) if not thread: thread =3D _createThread(threadName, pid, tid) thread_map[tid] =3D thread + thread['addSample'](threadName, stack, time_stamp) =20 # Extract relevant information from the event parameters. The event param= eters # are in a dictionary: --=20 2.34.1 From nobody Sun Dec 14 02:18:18 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2060EEB64DA for ; Mon, 10 Jul 2023 23:15:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230409AbjGJXPZ (ORCPT ); Mon, 10 Jul 2023 19:15:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230361AbjGJXPT (ORCPT ); Mon, 10 Jul 2023 19:15:19 -0400 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD9321B7; Mon, 10 Jul 2023 16:15:17 -0700 (PDT) Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-55b1238a013so3633707a12.3; Mon, 10 Jul 2023 16:15:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689030917; x=1691622917; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=dAwiz27o1hNxGAQwrIRT6lxtMelfroQzik/QCJDjd3Q=; b=b0GCU0td2713cSKuiHV/PsI8IVs/WmY2ym5aADJqcjTx3HWC7XNdRQaDOPXjNIRYgc WEnpFPR4gSrPyWkqlStg3rQwvlKROz9w4/WU0yKShDufhQmZC3BCyxgetFfqtU8wNtIS QKiesFEBkm6SxRTGJc8lkWRD/sDTfDpZgEo/QYy4U73TeCaoaqqiHwTbIRFCjyi5rTHT 3+sJxKowFugyM4/TPZSjBML2/CE3P4p00enxTxrL0ab4c/gyDRxpx6LEPqTs7n/oexWl OdXP/MDVFIVkjj1zGsC76AG05Gt5cueT4C3vMoQkdw4BB6BiyHuaLdtY1xVen6ZEH2WB GadA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689030917; x=1691622917; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=dAwiz27o1hNxGAQwrIRT6lxtMelfroQzik/QCJDjd3Q=; b=JTzM+cq9x4Q+ptV2Lx8YLfJx9BaiEiismcM8uVTHc7ChnITAuCSsaOku+Brnv6y2M7 coUY+k40zJeP968xDE8R9HT7GB/IIe7e9Pw1cQnvC5g4E2bxUzfg5xrpCOdfqH+BQcUg vaMJlN4Z5piWF9VE1L/K3nu6So4bIHoTz9uK0L/bpYvyQOCqwyv7A2DNzrx2lwVPxTnS hsaRlPakX1b8THect0TgJ1x/ayVkWwLIIG+KCHwo4BIzBv5xwdXGTyqqB+bt7UQbux+U a9Wp4ddEJ1DUiKnFXBLdVv7lGHFFtC/ZKQ3wy/bvd3D04n1pSMWj/aS1E1N9qB8N0Jpa vODQ== X-Gm-Message-State: ABy/qLZsQXZbVf4a3KCzwY3KrAN3Ues4HBW1nrX8HbFFr9OmhzYXT3LH 6ev/mEQUuIDD148h/g//yXQ= X-Google-Smtp-Source: APBJJlF6C6Tptsh25nmcDwQDB9YW4j1H6BPwVehKw7ItD6HJJIsYlF3lN3rYWXHtoWKCKMoVaY6dzA== X-Received: by 2002:a05:6a20:54a7:b0:131:a21:9f96 with SMTP id i39-20020a056a2054a700b001310a219f96mr11740678pzk.6.1689030917027; Mon, 10 Jul 2023 16:15:17 -0700 (PDT) Received: from yoga ([2400:1f00:13:76f2:868d:bc7d:562b:a5bb]) by smtp.gmail.com with ESMTPSA id k22-20020aa792d6000000b00682868714fdsm325357pfa.95.2023.07.10.16.15.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 16:15:16 -0700 (PDT) Date: Tue, 11 Jul 2023 04:45:10 +0530 From: Anup Sharma To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Anup Sharma , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 6/6] scripts: python: implement get or create frame and stack function Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Complete addSample function, it takes the thread name, stack array, and time as input parameters and if the thread name differs from the current name, it updates the name. The function utilizes the get_or_create_stack and get_or_create_frame methods to construct the stack structure. Finally, it adds the stack, time, and responsiveness values to the 'data' list within 'samples'. The get_or_create_stack function is responsible for retrieving or creating a stack based on the provided frame and prefix. It first generates a key using the frame and prefix values. If the stack corresponding to the key is found in the stackMap, it is returned. Otherwise, a new stack is created by appending the prefix and frame to the stackTable's 'data' array. The key and the index of the newly created stack are added to the stackMap for future reference. The get_or_create_frame function is responsible for retrieving or creating a frame based on the provided frameString. If the frame corresponding to the frameString is found in the frameMap, it is returned. Otherwise, a new frame is created by appending relevant information to the frameTable's 'data' array and adding the frameString to the stringTable. Signed-off-by: Anup Sharma --- .../scripts/python/firefox-gecko-converter.py | 57 ++++++++++++++++++- 1 file changed, 56 insertions(+), 1 deletion(-) diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/p= erf/scripts/python/firefox-gecko-converter.py index 6c934de1f608..97fbe562ee2b 100644 --- a/tools/perf/scripts/python/firefox-gecko-converter.py +++ b/tools/perf/scripts/python/firefox-gecko-converter.py @@ -21,6 +21,8 @@ sys.path.append(os.environ['PERF_EXEC_PATH'] + \ from perf_trace_context import * from Core import * =20 +USER_CATEGORY_INDEX =3D 0 +KERNEL_CATEGORY_INDEX =3D 1 thread_map =3D {} start_time =3D None =20 @@ -34,7 +36,12 @@ CATEGORIES =3D [ PRODUCT =3D os.popen('uname -op').read().strip() =20 def trace_end(): - thread_array =3D thread_map.values())) + thread_array =3D list(map(lambda thread: thread['finish'](), thread_ma= p.values())) + +# Parse the callchain of the current sample into a stack array. + for thread in thread_array: + key =3D thread['samples']['schema']['time'] + thread['samples']['data'].sort(key=3Dlambda data : float(data[key]= )) =20 result =3D { 'meta': { @@ -106,7 +113,55 @@ def process_event(param_dict): } stringTable =3D [] =20 + stackMap =3D dict() + def get_or_create_stack(frame, prefix): + key =3D f"{frame}" if prefix is None else f"{frame},{prefix}" + stack =3D stackMap.get(key) + if stack is None: + stack =3D len(stackTable['data']) + stackTable['data'].append([prefix, frame]) + stackMap[key] =3D stack + return stack + + frameMap =3D dict() + def get_or_create_frame(frameString): + frame =3D frameMap.get(frameString) + if frame is None: + frame =3D len(frameTable['data']) + location =3D len(stringTable) + stringTable.append(frameString) + category =3D KERNEL_CATEGORY_INDEX if frameString.find('kallsyms') != =3D -1 \ + or frameString.find('/vmlinux') !=3D -1 \ + or frameString.endswith('.ko)') \ + else USER_CATEGORY_INDEX + implementation =3D None + optimizations =3D None + line =3D None + relevantForJS =3D False + subcategory =3D None + innerWindowID =3D 0 + column =3D None + + frameTable['data'].append([ + location, + relevantForJS, + innerWindowID, + implementation, + optimizations, + line, + column, + category, + subcategory, + ]) + frameMap[frameString] =3D frame + return frame + def addSample(threadName, stackArray, time): + nonlocal name + if name !=3D threadName: + name =3D threadName + stack =3D reduce(lambda prefix, stackFrame: get_or_create_stack + (get_or_create_frame(stackFrame), prefix), stackArray, None) responsiveness =3D 0 samples['data'].append([stack, time, responsiveness]) =20 --=20 2.34.1