This patch series adds support for Firefox's gecko profile format.
The format is documented here [1].

I have incorporated several changes based on feedback from the
previous version of the patch.

Changes in v3:
- Implemented object-oriented programming (OOP) approach for Thread and Sample
  to improve code organization and maintainability.
- Enhanced user experience by introducing argparse for changing color settings
  of user and kernel symbols during execution.
- Ensured proper module encapsulation by adding imports wherever necessary.
- Improved code readability by adding descriptive comments and type annotations.

These changes focus on adopting OOP principles, enhancing user interaction with argparse,
and making the code more readable with comments and type information.

TODO:
- use cpu_mode to determine the category instead of finding it from symbol name.
- Write a test.
- add direct execution command for this script under script/python/bin directory.

Committer Testing:
- Tested with a perf.data file generated for single and multiple cpu
  cores enabled.
- Uploaded the stdout into profiler.firefox.com and verified the
  output.
- Verified the output with the output generated by the existing
  script as mentioned here [2].
Method:
- perf record -F 99 -a -g -- sleep 5
- perf script firefox-gecko-converter.py > output.json
- upload output.json to profiler.firefox.com

[1] https://github.com/firefox-devtools/profiler/blob/main/docs-developer/gecko-profile-format.md
[2] https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler

Anup Sharma (6):
  perf scripts python: Add initial script file with usage information
  perf scripts python: Extact necessary information from process event
  perf scripts python: Add classes and conversion functions
  perf scripts python: Add trace end processing and PRODUCT and
    CATEGORIES information
  perf scripts python: implement internal get or create frame, stack and
    string function
  perf scripts python: Implement add sample function and thread
    processing

 .../scripts/python/firefox-gecko-converter.py | 339 ++++++++++++++++++
 1 file changed, 339 insertions(+)
 create mode 100644 tools/perf/scripts/python/firefox-gecko-converter.py

-- 
2.34.1

On Wed, Jul 19, 2023 at 04:15:52AM +0530, Anup Sharma wrote:
> This patch series adds support for Firefox's gecko profile format.
> The format is documented here [1].
> 
> I have incorporated several changes based on feedback from the
> previous version of the patch.
> 
> Changes in v3:
a small typo here. It should be v4 instead of v3 (changes in v4).
> - Implemented object-oriented programming (OOP) approach for Thread and Sample
>   to improve code organization and maintainability.
> - Enhanced user experience by introducing argparse for changing color settings
>   of user and kernel symbols during execution.
> - Ensured proper module encapsulation by adding imports wherever necessary.
> - Improved code readability by adding descriptive comments and type annotations.
> 
> These changes focus on adopting OOP principles, enhancing user interaction with argparse,
> and making the code more readable with comments and type information.
> 
> TODO:
> - use cpu_mode to determine the category instead of finding it from symbol name.
> - Write a test.
> - add direct execution command for this script under script/python/bin directory.
> 
> Committer Testing:
> - Tested with a perf.data file generated for single and multiple cpu
>   cores enabled.
> - Uploaded the stdout into profiler.firefox.com and verified the
>   output.
> - Verified the output with the output generated by the existing
>   script as mentioned here [2].
> Method:
> - perf record -F 99 -a -g -- sleep 5
> - perf script firefox-gecko-converter.py > output.json
> - upload output.json to profiler.firefox.com
> 
> [1] https://github.com/firefox-devtools/profiler/blob/main/docs-developer/gecko-profile-format.md
> [2] https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler
> 
> Anup Sharma (6):
>   perf scripts python: Add initial script file with usage information
>   perf scripts python: Extact necessary information from process event
>   perf scripts python: Add classes and conversion functions
>   perf scripts python: Add trace end processing and PRODUCT and
>     CATEGORIES information
>   perf scripts python: implement internal get or create frame, stack and
>     string function
>   perf scripts python: Implement add sample function and thread
>     processing
> 
>  .../scripts/python/firefox-gecko-converter.py | 339 ++++++++++++++++++
>  1 file changed, 339 insertions(+)
>  create mode 100644 tools/perf/scripts/python/firefox-gecko-converter.py
> 
> -- 
> 2.34.1
>

Em Wed, Jul 19, 2023 at 04:34:28PM +0530, Anup Sharma escreveu:
> On Wed, Jul 19, 2023 at 04:15:52AM +0530, Anup Sharma wrote:
> > This patch series adds support for Firefox's gecko profile format.
> > The format is documented here [1].
> > 
> > I have incorporated several changes based on feedback from the
> > previous version of the patch.
> > 
> > Changes in v3:
> a small typo here. It should be v4 instead of v3 (changes in v4).
> > - Implemented object-oriented programming (OOP) approach for Thread and Sample
> >   to improve code organization and maintainability.
> > - Enhanced user experience by introducing argparse for changing color settings
> >   of user and kernel symbols during execution.
> > - Ensured proper module encapsulation by adding imports wherever necessary.
> > - Improved code readability by adding descriptive comments and type annotations.
> > 
> > These changes focus on adopting OOP principles, enhancing user interaction with argparse,
> > and making the code more readable with comments and type information.
> > 
> > TODO:
> > - use cpu_mode to determine the category instead of finding it from symbol name.
> > - Write a test.
> > - add direct execution command for this script under script/python/bin directory.
> > 
> > Committer Testing:
> > - Tested with a perf.data file generated for single and multiple cpu
> >   cores enabled.
> > - Uploaded the stdout into profiler.firefox.com and verified the
> >   output.
> > - Verified the output with the output generated by the existing
> >   script as mentioned here [2].
> > Method:
> > - perf record -F 99 -a -g -- sleep 5
> > - perf script firefox-gecko-converter.py > output.json
> > - upload output.json to profiler.firefox.com

I see it in the cover letter, will try now.

> > [1] https://github.com/firefox-devtools/profiler/blob/main/docs-developer/gecko-profile-format.md
> > [2] https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler
> > 
> > Anup Sharma (6):
> >   perf scripts python: Add initial script file with usage information
> >   perf scripts python: Extact necessary information from process event
> >   perf scripts python: Add classes and conversion functions
> >   perf scripts python: Add trace end processing and PRODUCT and
> >     CATEGORIES information
> >   perf scripts python: implement internal get or create frame, stack and
> >     string function
> >   perf scripts python: Implement add sample function and thread
> >     processing
> > 
> >  .../scripts/python/firefox-gecko-converter.py | 339 ++++++++++++++++++
> >  1 file changed, 339 insertions(+)
> >  create mode 100644 tools/perf/scripts/python/firefox-gecko-converter.py
> > 
> > -- 
> > 2.34.1
> > 

-- 

- Arnaldo

Em Thu, Jul 20, 2023 at 12:14:10PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Wed, Jul 19, 2023 at 04:34:28PM +0530, Anup Sharma escreveu:
> > On Wed, Jul 19, 2023 at 04:15:52AM +0530, Anup Sharma wrote:
> > > This patch series adds support for Firefox's gecko profile format.
> > > The format is documented here [1].
> > > 
> > > I have incorporated several changes based on feedback from the
> > > previous version of the patch.
> > > 
> > > Changes in v3:
> > a small typo here. It should be v4 instead of v3 (changes in v4).
> > > - Implemented object-oriented programming (OOP) approach for Thread and Sample
> > >   to improve code organization and maintainability.
> > > - Enhanced user experience by introducing argparse for changing color settings
> > >   of user and kernel symbols during execution.
> > > - Ensured proper module encapsulation by adding imports wherever necessary.
> > > - Improved code readability by adding descriptive comments and type annotations.
> > > 
> > > These changes focus on adopting OOP principles, enhancing user interaction with argparse,
> > > and making the code more readable with comments and type information.
> > > 
> > > TODO:
> > > - use cpu_mode to determine the category instead of finding it from symbol name.
> > > - Write a test.
> > > - add direct execution command for this script under script/python/bin directory.
> > > 
> > > Committer Testing:
> > > - Tested with a perf.data file generated for single and multiple cpu
> > >   cores enabled.
> > > - Uploaded the stdout into profiler.firefox.com and verified the
> > >   output.
> > > - Verified the output with the output generated by the existing
> > >   script as mentioned here [2].
> > > Method:
> > > - perf record -F 99 -a -g -- sleep 5
> > > - perf script firefox-gecko-converter.py > output.json
> > > - upload output.json to profiler.firefox.com
> 
> I see it in the cover letter, will try now.

I followed the instructions, uploaded a output.json produced and it
seems to work, good job!

- Arnaldo

On Wed, Jul 19, 2023 at 4:04 AM Anup Sharma <anupnewsmail@gmail.com> wrote:
>
> On Wed, Jul 19, 2023 at 04:15:52AM +0530, Anup Sharma wrote:
> > This patch series adds support for Firefox's gecko profile format.
> > The format is documented here [1].
> >
> > I have incorporated several changes based on feedback from the
> > previous version of the patch.
> >
> > Changes in v3:
> a small typo here. It should be v4 instead of v3 (changes in v4).
> > - Implemented object-oriented programming (OOP) approach for Thread and Sample
> >   to improve code organization and maintainability.
> > - Enhanced user experience by introducing argparse for changing color settings
> >   of user and kernel symbols during execution.
> > - Ensured proper module encapsulation by adding imports wherever necessary.
> > - Improved code readability by adding descriptive comments and type annotations.
> >
> > These changes focus on adopting OOP principles, enhancing user interaction with argparse,
> > and making the code more readable with comments and type information.
> >
> > TODO:
> > - use cpu_mode to determine the category instead of finding it from symbol name.
> > - Write a test.
> > - add direct execution command for this script under script/python/bin directory.
> >

Thanks Anup, is there a minimal TODO so that some code can be merged?
I didn't have any comments for this series so:
Acked-by: Ian Rogers <irogers@google.com>
I think even the test can be follow up as the code isn't on the critical path.

Thanks,
Ian

> > Committer Testing:
> > - Tested with a perf.data file generated for single and multiple cpu
> >   cores enabled.
> > - Uploaded the stdout into profiler.firefox.com and verified the
> >   output.
> > - Verified the output with the output generated by the existing
> >   script as mentioned here [2].
> > Method:
> > - perf record -F 99 -a -g -- sleep 5
> > - perf script firefox-gecko-converter.py > output.json
> > - upload output.json to profiler.firefox.com
> >
> > [1] https://github.com/firefox-devtools/profiler/blob/main/docs-developer/gecko-profile-format.md
> > [2] https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler
> >
> > Anup Sharma (6):
> >   perf scripts python: Add initial script file with usage information
> >   perf scripts python: Extact necessary information from process event
> >   perf scripts python: Add classes and conversion functions
> >   perf scripts python: Add trace end processing and PRODUCT and
> >     CATEGORIES information
> >   perf scripts python: implement internal get or create frame, stack and
> >     string function
> >   perf scripts python: Implement add sample function and thread
> >     processing
> >
> >  .../scripts/python/firefox-gecko-converter.py | 339 ++++++++++++++++++
> >  1 file changed, 339 insertions(+)
> >  create mode 100644 tools/perf/scripts/python/firefox-gecko-converter.py
> >
> > --
> > 2.34.1
> >

On Wed, Jul 19, 2023 at 08:22:24AM -0700, Ian Rogers wrote:
> On Wed, Jul 19, 2023 at 4:04 AM Anup Sharma <anupnewsmail@gmail.com> wrote:
> >
> > On Wed, Jul 19, 2023 at 04:15:52AM +0530, Anup Sharma wrote:
> > > This patch series adds support for Firefox's gecko profile format.
> > > The format is documented here [1].
> > >
> > > I have incorporated several changes based on feedback from the
> > > previous version of the patch.
> > >
> > > Changes in v3:
> > a small typo here. It should be v4 instead of v3 (changes in v4).
> > > - Implemented object-oriented programming (OOP) approach for Thread and Sample
> > >   to improve code organization and maintainability.
> > > - Enhanced user experience by introducing argparse for changing color settings
> > >   of user and kernel symbols during execution.
> > > - Ensured proper module encapsulation by adding imports wherever necessary.
> > > - Improved code readability by adding descriptive comments and type annotations.
> > >
> > > These changes focus on adopting OOP principles, enhancing user interaction with argparse,
> > > and making the code more readable with comments and type information.
> > >
> > > TODO:
> > > - use cpu_mode to determine the category instead of finding it from symbol name.
> > > - Write a test.
> > > - add direct execution command for this script under script/python/bin directory.
> > >
> 
> Thanks Anup, is there a minimal TODO so that some code can be merged?

Yaah, only the first TODO is left, will discuss during office hours.

> I didn't have any comments for this series so:
> Acked-by: Ian Rogers <irogers@google.com>

Thanks for the review.

> I think even the test can be follow up as the code isn't on the critical path.

By the way, I've finished writing the test and will send it very soon.

> Thanks,
> Ian
> 
> > > Committer Testing:
> > > - Tested with a perf.data file generated for single and multiple cpu
> > >   cores enabled.
> > > - Uploaded the stdout into profiler.firefox.com and verified the
> > >   output.
> > > - Verified the output with the output generated by the existing
> > >   script as mentioned here [2].
> > > Method:
> > > - perf record -F 99 -a -g -- sleep 5
> > > - perf script firefox-gecko-converter.py > output.json
> > > - upload output.json to profiler.firefox.com
> > >
> > > [1] https://github.com/firefox-devtools/profiler/blob/main/docs-developer/gecko-profile-format.md
> > > [2] https://perf.wiki.kernel.org/index.php/Tutorial#Firefox_Profiler
> > >
> > > Anup Sharma (6):
> > >   perf scripts python: Add initial script file with usage information
> > >   perf scripts python: Extact necessary information from process event
> > >   perf scripts python: Add classes and conversion functions
> > >   perf scripts python: Add trace end processing and PRODUCT and
> > >     CATEGORIES information
> > >   perf scripts python: implement internal get or create frame, stack and
> > >     string function
> > >   perf scripts python: Implement add sample function and thread
> > >     processing
> > >
> > >  .../scripts/python/firefox-gecko-converter.py | 339 ++++++++++++++++++
> > >  1 file changed, 339 insertions(+)
> > >  create mode 100644 tools/perf/scripts/python/firefox-gecko-converter.py
> > >
> > > --
> > > 2.34.1
> > >