[PATCH 7/9] scripts: python: implement get or create frame function

Anup Sharma posted 9 patches 2 years, 7 months ago
There is a newer version of this series
[PATCH 7/9] scripts: python: implement get or create frame function
Posted by Anup Sharma 2 years, 7 months ago
The CATEGORIES list and the USER_CATEGORY_INDEX and
KERNEL_CATEGORY_INDEX constants has been introduced.

The get_or_create_frame function is responsible for retrieving or
creating a frame based on the provided frameString. If the frame
corresponding to the frameString is found in the frameMap, it is
returned. Otherwise, a new frame is created by appending relevant
information to the frameTable's 'data' array and adding the
frameString to the stringTable.

The index of the newly created frame is added to the frameMap.

Signed-off-by: Anup Sharma <anupnewsmail@gmail.com>
---
 .../scripts/python/firefox-gecko-converter.py | 38 +++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
index 30fc542cfdeb..866751e5d1ce 100644
--- a/tools/perf/scripts/python/firefox-gecko-converter.py
+++ b/tools/perf/scripts/python/firefox-gecko-converter.py
@@ -15,6 +15,13 @@ def isPerfScriptFormat(profile):
     firstLine = profile[:profile.index('\n')]
     return bool(re.match(r'^\S.*?\s+(?:\d+/)?\d+\s+(?:\d+\d+\s+)?[\d.]+:', firstLine))
 
+CATEGORIES = [
+{'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
+{'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
+]
+USER_CATEGORY_INDEX = 0
+KERNEL_CATEGORY_INDEX = 1
+
 def convertPerfScriptProfile(profile):
     def _createtread(name, pid, tid):
         markers = {
@@ -70,6 +77,37 @@ def convertPerfScriptProfile(profile):
                 stackMap[key] = stack
             return stack
 
+        frameMap = dict()
+        def get_or_create_frame(frameString):
+            frame = frameMap.get(frameString)
+            if frame is None:
+                frame = len(frameTable['data'])
+                location = len(stringTable)
+                stringTable.append(frameString)
+            
+                category = KERNEL_CATEGORY_INDEX if frameString.find('kallsyms') != -1 or frameString.find('/vmlinux') != -1 or frameString.endswith('.ko)') else USER_CATEGORY_INDEX
+                implementation = None
+                optimizations = None
+                line = None
+                relevantForJS = False
+                subcategory = None
+                innerWindowID = 0
+                column = None
+
+                frameTable['data'].append([
+                    location,
+                    relevantForJS,
+                    innerWindowID,
+                    implementation,
+                    optimizations,
+                    line,
+                    column,
+                    category,
+                    subcategory,
+                ])
+                frameMap[frameString] = frame
+            return frame
+
         def addSample(threadName, stackArray, time):
             nonlocal name
             if name != threadName:
-- 
2.34.1
Re: [PATCH 7/9] scripts: python: implement get or create frame function
Posted by Namhyung Kim 2 years, 7 months ago
On Wed, Jun 21, 2023 at 12:45 PM Anup Sharma <anupnewsmail@gmail.com> wrote:
>
> The CATEGORIES list and the USER_CATEGORY_INDEX and
> KERNEL_CATEGORY_INDEX constants has been introduced.
>
> The get_or_create_frame function is responsible for retrieving or
> creating a frame based on the provided frameString. If the frame
> corresponding to the frameString is found in the frameMap, it is
> returned. Otherwise, a new frame is created by appending relevant
> information to the frameTable's 'data' array and adding the
> frameString to the stringTable.
>
> The index of the newly created frame is added to the frameMap.
>
> Signed-off-by: Anup Sharma <anupnewsmail@gmail.com>
> ---
>  .../scripts/python/firefox-gecko-converter.py | 38 +++++++++++++++++++
>  1 file changed, 38 insertions(+)
>
> diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
> index 30fc542cfdeb..866751e5d1ce 100644
> --- a/tools/perf/scripts/python/firefox-gecko-converter.py
> +++ b/tools/perf/scripts/python/firefox-gecko-converter.py
> @@ -15,6 +15,13 @@ def isPerfScriptFormat(profile):
>      firstLine = profile[:profile.index('\n')]
>      return bool(re.match(r'^\S.*?\s+(?:\d+/)?\d+\s+(?:\d+\d+\s+)?[\d.]+:', firstLine))
>
> +CATEGORIES = [
> +{'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
> +{'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
> +]
> +USER_CATEGORY_INDEX = 0
> +KERNEL_CATEGORY_INDEX = 1
> +
>  def convertPerfScriptProfile(profile):
>      def _createtread(name, pid, tid):
>          markers = {
> @@ -70,6 +77,37 @@ def convertPerfScriptProfile(profile):
>                  stackMap[key] = stack
>              return stack
>
> +        frameMap = dict()
> +        def get_or_create_frame(frameString):
> +            frame = frameMap.get(frameString)
> +            if frame is None:
> +                frame = len(frameTable['data'])
> +                location = len(stringTable)
> +                stringTable.append(frameString)
> +
> +                category = KERNEL_CATEGORY_INDEX if frameString.find('kallsyms') != -1 or frameString.find('/vmlinux') != -1 or frameString.endswith('.ko)') else USER_CATEGORY_INDEX

This line is too long, we usually don't allow long lines
over 100 characters.

Thanks,
Namhyung


> +                implementation = None
> +                optimizations = None
> +                line = None
> +                relevantForJS = False
> +                subcategory = None
> +                innerWindowID = 0
> +                column = None
> +
> +                frameTable['data'].append([
> +                    location,
> +                    relevantForJS,
> +                    innerWindowID,
> +                    implementation,
> +                    optimizations,
> +                    line,
> +                    column,
> +                    category,
> +                    subcategory,
> +                ])
> +                frameMap[frameString] = frame
> +            return frame
> +
>          def addSample(threadName, stackArray, time):
>              nonlocal name
>              if name != threadName:
> --
> 2.34.1
>
Re: [PATCH 7/9] scripts: python: implement get or create frame function
Posted by Anup Sharma 2 years, 7 months ago
On Fri, Jun 23, 2023 at 05:04:56PM -0700, Namhyung Kim wrote:
> On Wed, Jun 21, 2023 at 12:45 PM Anup Sharma <anupnewsmail@gmail.com> wrote:
> >
> > The CATEGORIES list and the USER_CATEGORY_INDEX and
> > KERNEL_CATEGORY_INDEX constants has been introduced.
> >
> > The get_or_create_frame function is responsible for retrieving or
> > creating a frame based on the provided frameString. If the frame
> > corresponding to the frameString is found in the frameMap, it is
> > returned. Otherwise, a new frame is created by appending relevant
> > information to the frameTable's 'data' array and adding the
> > frameString to the stringTable.
> >
> > The index of the newly created frame is added to the frameMap.
> >
> > Signed-off-by: Anup Sharma <anupnewsmail@gmail.com>
> > ---
> >  .../scripts/python/firefox-gecko-converter.py | 38 +++++++++++++++++++
> >  1 file changed, 38 insertions(+)
> >
> > diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
> > index 30fc542cfdeb..866751e5d1ce 100644
> > --- a/tools/perf/scripts/python/firefox-gecko-converter.py
> > +++ b/tools/perf/scripts/python/firefox-gecko-converter.py
> > @@ -15,6 +15,13 @@ def isPerfScriptFormat(profile):
> >      firstLine = profile[:profile.index('\n')]
> >      return bool(re.match(r'^\S.*?\s+(?:\d+/)?\d+\s+(?:\d+\d+\s+)?[\d.]+:', firstLine))
> >
> > +CATEGORIES = [
> > +{'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
> > +{'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
> > +]
> > +USER_CATEGORY_INDEX = 0
> > +KERNEL_CATEGORY_INDEX = 1
> > +
> >  def convertPerfScriptProfile(profile):
> >      def _createtread(name, pid, tid):
> >          markers = {
> > @@ -70,6 +77,37 @@ def convertPerfScriptProfile(profile):
> >                  stackMap[key] = stack
> >              return stack
> >
> > +        frameMap = dict()
> > +        def get_or_create_frame(frameString):
> > +            frame = frameMap.get(frameString)
> > +            if frame is None:
> > +                frame = len(frameTable['data'])
> > +                location = len(stringTable)
> > +                stringTable.append(frameString)
> > +
> > +                category = KERNEL_CATEGORY_INDEX if frameString.find('kallsyms') != -1 or frameString.find('/vmlinux') != -1 or frameString.endswith('.ko)') else USER_CATEGORY_INDEX
> 
> This line is too long, we usually don't allow long lines
> over 100 characters.

Thanks for your suggestion. I have taken care in latest version.
Is there any way to add such checks in editor itself ? I used checkpatch.pl
scripts, however it didnt catch this.

> Thanks,
> Namhyung
> 
> 
> > +                implementation = None
> > +                optimizations = None
> > +                line = None
> > +                relevantForJS = False
> > +                subcategory = None
> > +                innerWindowID = 0
> > +                column = None
> > +
> > +                frameTable['data'].append([
> > +                    location,
> > +                    relevantForJS,
> > +                    innerWindowID,
> > +                    implementation,
> > +                    optimizations,
> > +                    line,
> > +                    column,
> > +                    category,
> > +                    subcategory,
> > +                ])
> > +                frameMap[frameString] = frame
> > +            return frame
> > +
> >          def addSample(threadName, stackArray, time):
> >              nonlocal name
> >              if name != threadName:
> > --
> > 2.34.1
> >
Re: [PATCH 7/9] scripts: python: implement get or create frame function
Posted by Ian Rogers 2 years, 7 months ago
On Wed, Jul 5, 2023 at 1:01 PM Anup Sharma <anupnewsmail@gmail.com> wrote:
>
> On Fri, Jun 23, 2023 at 05:04:56PM -0700, Namhyung Kim wrote:
> > On Wed, Jun 21, 2023 at 12:45 PM Anup Sharma <anupnewsmail@gmail.com> wrote:
> > >
> > > The CATEGORIES list and the USER_CATEGORY_INDEX and
> > > KERNEL_CATEGORY_INDEX constants has been introduced.
> > >
> > > The get_or_create_frame function is responsible for retrieving or
> > > creating a frame based on the provided frameString. If the frame
> > > corresponding to the frameString is found in the frameMap, it is
> > > returned. Otherwise, a new frame is created by appending relevant
> > > information to the frameTable's 'data' array and adding the
> > > frameString to the stringTable.
> > >
> > > The index of the newly created frame is added to the frameMap.
> > >
> > > Signed-off-by: Anup Sharma <anupnewsmail@gmail.com>
> > > ---
> > >  .../scripts/python/firefox-gecko-converter.py | 38 +++++++++++++++++++
> > >  1 file changed, 38 insertions(+)
> > >
> > > diff --git a/tools/perf/scripts/python/firefox-gecko-converter.py b/tools/perf/scripts/python/firefox-gecko-converter.py
> > > index 30fc542cfdeb..866751e5d1ce 100644
> > > --- a/tools/perf/scripts/python/firefox-gecko-converter.py
> > > +++ b/tools/perf/scripts/python/firefox-gecko-converter.py
> > > @@ -15,6 +15,13 @@ def isPerfScriptFormat(profile):
> > >      firstLine = profile[:profile.index('\n')]
> > >      return bool(re.match(r'^\S.*?\s+(?:\d+/)?\d+\s+(?:\d+\d+\s+)?[\d.]+:', firstLine))
> > >
> > > +CATEGORIES = [
> > > +{'name': 'User', 'color': 'yellow', 'subcategories': ['Other']},
> > > +{'name': 'Kernel', 'color': 'orange', 'subcategories': ['Other']}
> > > +]
> > > +USER_CATEGORY_INDEX = 0
> > > +KERNEL_CATEGORY_INDEX = 1
> > > +
> > >  def convertPerfScriptProfile(profile):
> > >      def _createtread(name, pid, tid):
> > >          markers = {
> > > @@ -70,6 +77,37 @@ def convertPerfScriptProfile(profile):
> > >                  stackMap[key] = stack
> > >              return stack
> > >
> > > +        frameMap = dict()
> > > +        def get_or_create_frame(frameString):
> > > +            frame = frameMap.get(frameString)
> > > +            if frame is None:
> > > +                frame = len(frameTable['data'])
> > > +                location = len(stringTable)
> > > +                stringTable.append(frameString)
> > > +
> > > +                category = KERNEL_CATEGORY_INDEX if frameString.find('kallsyms') != -1 or frameString.find('/vmlinux') != -1 or frameString.endswith('.ko)') else USER_CATEGORY_INDEX
> >
> > This line is too long, we usually don't allow long lines
> > over 100 characters.
>
> Thanks for your suggestion. I have taken care in latest version.
> Is there any way to add such checks in editor itself ? I used checkpatch.pl
> scripts, however it didnt catch this.

Unfortunately checkpatch.pl doesn't work for python code yet. I think
using mypy types would be useful:
https://github.com/python/mypy
Also having docstring on functions would be useful. Some of the code
has some fairly complex indirection and it'd be nice to understand
why.

Thanks,
Ian

> > Thanks,
> > Namhyung
> >
> >
> > > +                implementation = None
> > > +                optimizations = None
> > > +                line = None
> > > +                relevantForJS = False
> > > +                subcategory = None
> > > +                innerWindowID = 0
> > > +                column = None
> > > +
> > > +                frameTable['data'].append([
> > > +                    location,
> > > +                    relevantForJS,
> > > +                    innerWindowID,
> > > +                    implementation,
> > > +                    optimizations,
> > > +                    line,
> > > +                    column,
> > > +                    category,
> > > +                    subcategory,
> > > +                ])
> > > +                frameMap[frameString] = frame
> > > +            return frame
> > > +
> > >          def addSample(threadName, stackArray, time):
> > >              nonlocal name
> > >              if name != threadName:
> > > --
> > > 2.34.1
> > >