1. Patchew
  2. linux
  3. [PATCH v3 0/6] Add support for Firefox's gecko profile format
  4. View patch

[PATCH v3 4/6] scripts: python: Add trace end processing and JSON output

Anup Sharma posted 6 patches 2 years, 5 months ago
There is a newer version of this series
[PATCH v3 4/6] scripts: python: Add trace end processing and JSON output
Posted by Anup Sharma 2 years, 5 months ago 
Inside the trace end function the final output will be dumped
to standard output in JSON gecko format. Additionally, constants
such as USER_CATEGORY_INDEX, KERNEL_CATEGORY_INDEX, CATEGORIES, and
PRODUCT are defined to provide contextual information.

Signed-off-by: Anup Sharma <anupnewsmail@gmail.com>
---
 .../scripts/python/firefox-gecko-converter.py | 34 ++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
index 0b8a86bdcab1..39818a603265 100644
--- a/tools/perf/scripts/python/firefox-gecko-converter.py
+++ b/tools/perf/scripts/python/firefox-gecko-converter.py
@@ -24,8 +24,40 @@ from Core import *
 thread_map = {}
 start_time = None
 
+# Follow Brendan Gregg's Flamegraph convention: orange for kernel and yellow for user
+CATEGORIES = [
+    {'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
+    {'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
+]
+
+# The product name is used by the profiler UI to show the Operating system and Processor.
+PRODUCT = os.popen('uname -op').read().strip()
+
 def trace_end():
-	pass
+    thread_array = thread_map.values()))
+
+    result = {
+        'meta': {
+            'interval': 1,
+            'processType': 0,
+            'product': PRODUCT,
+            'stackwalk': 1,
+            'debug': 0,
+            'gcpoison': 0,
+            'asyncstack': 1,
+            'startTime': start_time,
+            'shutdownTime': None,
+            'version': 24,
+            'presymbolicated': True,
+            'categories': CATEGORIES,
+            'markerSchema': []
+            },
+        'libs': [],
+        'threads': thread_array,
+        'processes': [],
+        'pausedRanges': []
+    }
+    json.dump(result, sys.stdout, indent=2)
 
 def process_event(param_dict):
 	global start_time
-- 
2.34.1
Re: [PATCH v3 4/6] scripts: python: Add trace end processing and JSON output
Posted by Ian Rogers 2 years, 5 months ago 
On Mon, Jul 10, 2023 at 4:13 PM Anup Sharma <anupnewsmail@gmail.com> wrote:
>
> Inside the trace end function the final output will be dumped
> to standard output in JSON gecko format. Additionally, constants
> such as USER_CATEGORY_INDEX, KERNEL_CATEGORY_INDEX, CATEGORIES, and
> PRODUCT are defined to provide contextual information.
>
> Signed-off-by: Anup Sharma <anupnewsmail@gmail.com>

Acked-by: Ian Rogers <irogers@google.com>

> ---
>  .../scripts/python/firefox-gecko-converter.py | 34 ++++++++++++++++++-
>  1 file changed, 33 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
> index 0b8a86bdcab1..39818a603265 100644
> --- a/tools/perf/scripts/python/firefox-gecko-converter.py
> +++ b/tools/perf/scripts/python/firefox-gecko-converter.py
> @@ -24,8 +24,40 @@ from Core import *
>  thread_map = {}
>  start_time = None
>
> +# Follow Brendan Gregg's Flamegraph convention: orange for kernel and yellow for user
> +CATEGORIES = [
> +    {'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
> +    {'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
> +]

A follow up could be to make these command line options, defaulting to
orange and yellow.

Thanks,
Ian

> +
> +# The product name is used by the profiler UI to show the Operating system and Processor.
> +PRODUCT = os.popen('uname -op').read().strip()
> +
>  def trace_end():
> -       pass
> +    thread_array = thread_map.values()))
> +
> +    result = {
> +        'meta': {
> +            'interval': 1,
> +            'processType': 0,
> +            'product': PRODUCT,
> +            'stackwalk': 1,
> +            'debug': 0,
> +            'gcpoison': 0,
> +            'asyncstack': 1,
> +            'startTime': start_time,
> +            'shutdownTime': None,
> +            'version': 24,
> +            'presymbolicated': True,
> +            'categories': CATEGORIES,
> +            'markerSchema': []
> +            },
> +        'libs': [],
> +        'threads': thread_array,
> +        'processes': [],
> +        'pausedRanges': []
> +    }
> +    json.dump(result, sys.stdout, indent=2)
>
>  def process_event(param_dict):
>         global start_time
> --
> 2.34.1
>
Re: [PATCH v3 4/6] scripts: python: Add trace end processing and JSON output
Posted by Namhyung Kim 2 years, 5 months ago 
Hi Anup and Ian,

On Wed, Jul 12, 2023 at 10:28 AM Ian Rogers <irogers@google.com> wrote:
>
> On Mon, Jul 10, 2023 at 4:13 PM Anup Sharma <anupnewsmail@gmail.com> wrote:
> >
> > Inside the trace end function the final output will be dumped
> > to standard output in JSON gecko format. Additionally, constants
> > such as USER_CATEGORY_INDEX, KERNEL_CATEGORY_INDEX, CATEGORIES, and
> > PRODUCT are defined to provide contextual information.
> >
> > Signed-off-by: Anup Sharma <anupnewsmail@gmail.com>
>
> Acked-by: Ian Rogers <irogers@google.com>

I'm ok with this change too but I think it can be squashed to
patch 1/6 as I think it'd make it more self-contained.  Of course
you might change time and thread to have empty values.

>
> > ---
> >  .../scripts/python/firefox-gecko-converter.py | 34 ++++++++++++++++++-
> >  1 file changed, 33 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
> > index 0b8a86bdcab1..39818a603265 100644
> > --- a/tools/perf/scripts/python/firefox-gecko-converter.py
> > +++ b/tools/perf/scripts/python/firefox-gecko-converter.py
> > @@ -24,8 +24,40 @@ from Core import *
> >  thread_map = {}
> >  start_time = None
> >
> > +# Follow Brendan Gregg's Flamegraph convention: orange for kernel and yellow for user
> > +CATEGORIES = [
> > +    {'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
> > +    {'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
> > +]
>
> A follow up could be to make these command line options, defaulting to
> orange and yellow.

Sounds good.

>
> > +
> > +# The product name is used by the profiler UI to show the Operating system and Processor.
> > +PRODUCT = os.popen('uname -op').read().strip()

I'm not against this but having a command name (or full
command line) of the target process as a title might be better.
But I'm not sure if the python scripting engine exposed the info
(like in perf report --header-only) to the script.

Thanks,
Namhyung


> > +
> >  def trace_end():
> > -       pass
> > +    thread_array = thread_map.values()))
> > +
> > +    result = {
> > +        'meta': {
> > +            'interval': 1,
> > +            'processType': 0,
> > +            'product': PRODUCT,
> > +            'stackwalk': 1,
> > +            'debug': 0,
> > +            'gcpoison': 0,
> > +            'asyncstack': 1,
> > +            'startTime': start_time,
> > +            'shutdownTime': None,
> > +            'version': 24,
> > +            'presymbolicated': True,
> > +            'categories': CATEGORIES,
> > +            'markerSchema': []
> > +            },
> > +        'libs': [],
> > +        'threads': thread_array,
> > +        'processes': [],
> > +        'pausedRanges': []
> > +    }
> > +    json.dump(result, sys.stdout, indent=2)
> >
> >  def process_event(param_dict):
> >         global start_time
> > --
> > 2.34.1
> >
Re: [PATCH v3 4/6] scripts: python: Add trace end processing and JSON output
Posted by Anup Sharma 2 years, 4 months ago 
On Thu, Jul 13, 2023 at 07:31:36PM -0700, Namhyung Kim wrote:
> Hi Anup and Ian,
> 
> On Wed, Jul 12, 2023 at 10:28 AM Ian Rogers <irogers@google.com> wrote:
> >
> > On Mon, Jul 10, 2023 at 4:13 PM Anup Sharma <anupnewsmail@gmail.com> wrote:
> > >
> > > Inside the trace end function the final output will be dumped
> > > to standard output in JSON gecko format. Additionally, constants
> > > such as USER_CATEGORY_INDEX, KERNEL_CATEGORY_INDEX, CATEGORIES, and
> > > PRODUCT are defined to provide contextual information.
> > >
> > > Signed-off-by: Anup Sharma <anupnewsmail@gmail.com>
> >
> > Acked-by: Ian Rogers <irogers@google.com>
> 
> I'm ok with this change too but I think it can be squashed to
> patch 1/6 as I think it'd make it more self-contained.  Of course
> you might change time and thread to have empty values.
> 
> >
> > > ---
> > >  .../scripts/python/firefox-gecko-converter.py | 34 ++++++++++++++++++-
> > >  1 file changed, 33 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
> > > index 0b8a86bdcab1..39818a603265 100644
> > > --- a/tools/perf/scripts/python/firefox-gecko-converter.py
> > > +++ b/tools/perf/scripts/python/firefox-gecko-converter.py
> > > @@ -24,8 +24,40 @@ from Core import *
> > >  thread_map = {}
> > >  start_time = None
> > >
> > > +# Follow Brendan Gregg's Flamegraph convention: orange for kernel and yellow for user
> > > +CATEGORIES = [
> > > +    {'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
> > > +    {'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
> > > +]
> >
> > A follow up could be to make these command line options, defaulting to
> > orange and yellow.
> 
> Sounds good.

Nice Idea, I have added this in the next version of patch.

> >
> > > +
> > > +# The product name is used by the profiler UI to show the Operating system and Processor.
> > > +PRODUCT = os.popen('uname -op').read().strip()
> 
> I'm not against this but having a command name (or full
> command line) of the target process as a title might be better.
> But I'm not sure if the python scripting engine exposed the info
> (like in perf report --header-only) to the script.

I tried to get the command name or any sort of header information but it seems to
be not exposed to the perf-script-python interface. can anyone confirm
if there is any way to get the command name or any header information from the
perf-script-python interface?

> Thanks,
> Namhyung
> 
> 
> > > +
> > >  def trace_end():
> > > -       pass
> > > +    thread_array = thread_map.values()))
> > > +
> > > +    result = {
> > > +        'meta': {
> > > +            'interval': 1,
> > > +            'processType': 0,
> > > +            'product': PRODUCT,
> > > +            'stackwalk': 1,
> > > +            'debug': 0,
> > > +            'gcpoison': 0,
> > > +            'asyncstack': 1,
> > > +            'startTime': start_time,
> > > +            'shutdownTime': None,
> > > +            'version': 24,
> > > +            'presymbolicated': True,
> > > +            'categories': CATEGORIES,
> > > +            'markerSchema': []
> > > +            },
> > > +        'libs': [],
> > > +        'threads': thread_array,
> > > +        'processes': [],
> > > +        'pausedRanges': []
> > > +    }
> > > +    json.dump(result, sys.stdout, indent=2)
> > >
> > >  def process_event(param_dict):
> > >         global start_time
> > > --
> > > 2.34.1
> > >

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.