1. Patchew
  2. linux
  3. [PATCH v1 0/6] Mypy and pylint build support
  4. View patch

[PATCH v1 3/6] perf build: Add mypy build tests

Ian Rogers posted 6 patches 1 year, 3 months ago
There is a newer version of this series
[PATCH v1 3/6] perf build: Add mypy build tests
Posted by Ian Rogers 1 year, 3 months ago 
If MYPY=1 is passed to the build then run mypy over python code in
perf. Unlike shellcheck this isn't default on as there are currently
too many errors.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/Build            | 13 +++++++++++++
 tools/perf/Makefile.perf    |  7 ++++++-
 tools/perf/pmu-events/Build | 14 +++++++++++++-
 tools/perf/scripts/Build    | 13 +++++++++++++
 tools/perf/tests/Build      | 13 +++++++++++++
 tools/perf/util/Build       | 13 +++++++++++++
 6 files changed, 71 insertions(+), 2 deletions(-)

diff --git a/tools/perf/Build b/tools/perf/Build
index 5e385f370dd7..312914994c89 100644
--- a/tools/perf/Build
+++ b/tools/perf/Build
@@ -76,3 +76,16 @@ $(OUTPUT)%.shellcheck_log: %
 	$(Q)$(call echo-cmd,test)shellcheck -s bash -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
 
 perf-y += $(SHELL_TEST_LOGS)
+
+ifdef MYPY
+  PY_TESTS := $(shell find python -type f -name '*.py')
+  MYPY_TEST_LOGS := $(PY_TESTS:python/%=python/%.mypy_log)
+else
+  MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+	$(call rule_mkdir)
+	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-y += $(MYPY_TEST_LOGS)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index a9f2a9f6ebf0..333afe29b1e2 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -277,8 +277,13 @@ ifneq ($(SHELLCHECK),)
   endif
 endif
 
+# Runs mypy on perf python files
+ifeq ($(MYPY),1)
+  MYPY := $(shell which mypy 2> /dev/null)
+endif
+
 export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
-export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK
+export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY
 
 include $(srctree)/tools/build/Makefile.include
 
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index d941bc9d16e9..fc1dc810ffb6 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -41,7 +41,19 @@ $(EMPTY_PMU_EVENTS_TEST_LOG): $(EMPTY_PMU_EVENTS_C) $(TEST_EMPTY_PMU_EVENTS_C)
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,test)diff -u $^ 2> $@ || (cat $@ && false)
 
-$(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG) $(EMPTY_PMU_EVENTS_TEST_LOG)
+ifdef MYPY
+  PMU_EVENTS_PY_TESTS := $(wildcard *.py)
+  PMU_EVENTS_MYPY_TEST_LOGS := $(JEVENTS_PY_TESTS:%=%.mypy_log)
+else
+  PMU_EVENTS_MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+	$(call rule_mkdir)
+	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+$(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG) \
+    $(EMPTY_PMU_EVENTS_TEST_LOG) $(PMU_EVENTS_MYPY_TEST_LOGS)
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) pmu-events/arch $@
 endif
diff --git a/tools/perf/scripts/Build b/tools/perf/scripts/Build
index 46f0c6f76dbf..a5350dc6ac50 100644
--- a/tools/perf/scripts/Build
+++ b/tools/perf/scripts/Build
@@ -2,3 +2,16 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y)
   perf-util-$(CONFIG_LIBPERL)   += perl/Perf-Trace-Util/
 endif
 perf-util-$(CONFIG_LIBPYTHON) += python/Perf-Trace-Util/
+
+ifdef MYPY
+  PY_TESTS := $(shell find python -type f -name '*.py')
+  MYPY_TEST_LOGS := $(PY_TESTS:python/%=python/%.mypy_log)
+else
+  MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+	$(call rule_mkdir)
+	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-y += $(MYPY_TEST_LOGS)
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index a51d28c7431c..4d7c38a2bf6c 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -91,3 +91,16 @@ $(OUTPUT)%.shellcheck_log: %
 	$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
 
 perf-test-y += $(SHELL_TEST_LOGS)
+
+ifdef MYPY
+  PY_TESTS := $(shell find tests/shell -type f -name '*.py')
+  MYPY_TEST_LOGS := $(PY_TESTS:tests/shell/%=shell/%.mypy_log)
+else
+  MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+	$(call rule_mkdir)
+	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-test-y += $(MYPY_TEST_LOGS)
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 18cd02ccd3ff..3baa1f41502d 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -409,3 +409,16 @@ $(OUTPUT)%.shellcheck_log: %
 	$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
 
 perf-util-y += $(SHELL_TEST_LOGS)
+
+PY_TESTS := setup.py
+ifdef MYPY
+  MYPY_TEST_LOGS := $(PY_TESTS:%=%.mypy_log)
+else
+  MYPY_TEST_LOGS :=
+endif
+
+$(OUTPUT)%.mypy_log: %
+	$(call rule_mkdir)
+	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-util-y += $(MYPY_TEST_LOGS)
-- 
2.47.0.163.g1226f6d8fa-goog
Re: [PATCH v1 3/6] perf build: Add mypy build tests
Posted by Namhyung Kim 1 year ago 
On Fri, Oct 25, 2024 at 10:23:00AM -0700, Ian Rogers wrote:
> If MYPY=1 is passed to the build then run mypy over python code in
> perf. Unlike shellcheck this isn't default on as there are currently
> too many errors.

Can you please add an example output in the commit log?  Also you need
to add a comment to describe the build option.

Is it ok to pass the option on a system that doesn't have mypy?

Thanks,
Namhyung

> 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/Build            | 13 +++++++++++++
>  tools/perf/Makefile.perf    |  7 ++++++-
>  tools/perf/pmu-events/Build | 14 +++++++++++++-
>  tools/perf/scripts/Build    | 13 +++++++++++++
>  tools/perf/tests/Build      | 13 +++++++++++++
>  tools/perf/util/Build       | 13 +++++++++++++
>  6 files changed, 71 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/perf/Build b/tools/perf/Build
> index 5e385f370dd7..312914994c89 100644
> --- a/tools/perf/Build
> +++ b/tools/perf/Build
> @@ -76,3 +76,16 @@ $(OUTPUT)%.shellcheck_log: %
>  	$(Q)$(call echo-cmd,test)shellcheck -s bash -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
>  
>  perf-y += $(SHELL_TEST_LOGS)
> +
> +ifdef MYPY
> +  PY_TESTS := $(shell find python -type f -name '*.py')
> +  MYPY_TEST_LOGS := $(PY_TESTS:python/%=python/%.mypy_log)
> +else
> +  MYPY_TEST_LOGS :=
> +endif
> +
> +$(OUTPUT)%.mypy_log: %
> +	$(call rule_mkdir)
> +	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
> +
> +perf-y += $(MYPY_TEST_LOGS)
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index a9f2a9f6ebf0..333afe29b1e2 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -277,8 +277,13 @@ ifneq ($(SHELLCHECK),)
>    endif
>  endif
>  
> +# Runs mypy on perf python files
> +ifeq ($(MYPY),1)
> +  MYPY := $(shell which mypy 2> /dev/null)
> +endif
> +
>  export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
> -export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK
> +export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY
>  
>  include $(srctree)/tools/build/Makefile.include
>  
> diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
> index d941bc9d16e9..fc1dc810ffb6 100644
> --- a/tools/perf/pmu-events/Build
> +++ b/tools/perf/pmu-events/Build
> @@ -41,7 +41,19 @@ $(EMPTY_PMU_EVENTS_TEST_LOG): $(EMPTY_PMU_EVENTS_C) $(TEST_EMPTY_PMU_EVENTS_C)
>  	$(call rule_mkdir)
>  	$(Q)$(call echo-cmd,test)diff -u $^ 2> $@ || (cat $@ && false)
>  
> -$(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG) $(EMPTY_PMU_EVENTS_TEST_LOG)
> +ifdef MYPY
> +  PMU_EVENTS_PY_TESTS := $(wildcard *.py)
> +  PMU_EVENTS_MYPY_TEST_LOGS := $(JEVENTS_PY_TESTS:%=%.mypy_log)
> +else
> +  PMU_EVENTS_MYPY_TEST_LOGS :=
> +endif
> +
> +$(OUTPUT)%.mypy_log: %
> +	$(call rule_mkdir)
> +	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
> +
> +$(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG) \
> +    $(EMPTY_PMU_EVENTS_TEST_LOG) $(PMU_EVENTS_MYPY_TEST_LOGS)
>  	$(call rule_mkdir)
>  	$(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) pmu-events/arch $@
>  endif
> diff --git a/tools/perf/scripts/Build b/tools/perf/scripts/Build
> index 46f0c6f76dbf..a5350dc6ac50 100644
> --- a/tools/perf/scripts/Build
> +++ b/tools/perf/scripts/Build
> @@ -2,3 +2,16 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y)
>    perf-util-$(CONFIG_LIBPERL)   += perl/Perf-Trace-Util/
>  endif
>  perf-util-$(CONFIG_LIBPYTHON) += python/Perf-Trace-Util/
> +
> +ifdef MYPY
> +  PY_TESTS := $(shell find python -type f -name '*.py')
> +  MYPY_TEST_LOGS := $(PY_TESTS:python/%=python/%.mypy_log)
> +else
> +  MYPY_TEST_LOGS :=
> +endif
> +
> +$(OUTPUT)%.mypy_log: %
> +	$(call rule_mkdir)
> +	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
> +
> +perf-y += $(MYPY_TEST_LOGS)
> diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
> index a51d28c7431c..4d7c38a2bf6c 100644
> --- a/tools/perf/tests/Build
> +++ b/tools/perf/tests/Build
> @@ -91,3 +91,16 @@ $(OUTPUT)%.shellcheck_log: %
>  	$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
>  
>  perf-test-y += $(SHELL_TEST_LOGS)
> +
> +ifdef MYPY
> +  PY_TESTS := $(shell find tests/shell -type f -name '*.py')
> +  MYPY_TEST_LOGS := $(PY_TESTS:tests/shell/%=shell/%.mypy_log)
> +else
> +  MYPY_TEST_LOGS :=
> +endif
> +
> +$(OUTPUT)%.mypy_log: %
> +	$(call rule_mkdir)
> +	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
> +
> +perf-test-y += $(MYPY_TEST_LOGS)
> diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> index 18cd02ccd3ff..3baa1f41502d 100644
> --- a/tools/perf/util/Build
> +++ b/tools/perf/util/Build
> @@ -409,3 +409,16 @@ $(OUTPUT)%.shellcheck_log: %
>  	$(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
>  
>  perf-util-y += $(SHELL_TEST_LOGS)
> +
> +PY_TESTS := setup.py
> +ifdef MYPY
> +  MYPY_TEST_LOGS := $(PY_TESTS:%=%.mypy_log)
> +else
> +  MYPY_TEST_LOGS :=
> +endif
> +
> +$(OUTPUT)%.mypy_log: %
> +	$(call rule_mkdir)
> +	$(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
> +
> +perf-util-y += $(MYPY_TEST_LOGS)
> -- 
> 2.47.0.163.g1226f6d8fa-goog
>
Re: [PATCH v1 3/6] perf build: Add mypy build tests
Posted by Ian Rogers 1 year ago 
On Fri, Jan 24, 2025 at 3:51 PM Namhyung Kim <namhyung@kernel.org> wrote:
>
> On Fri, Oct 25, 2024 at 10:23:00AM -0700, Ian Rogers wrote:
> > If MYPY=1 is passed to the build then run mypy over python code in
> > perf. Unlike shellcheck this isn't default on as there are currently
> > too many errors.
>
> Can you please add an example output in the commit log?  Also you need
> to add a comment to describe the build option.

It feels more of a developer option, so I'm not sure we should
advertise it. Ideally we'd just default it like the shellcheck case.
I'm not sure of the benefit of snapshotting mypy errors, a quick
internet search will show what mypy does and I'd prefer the patch set
focus on the build infrastructure changes.

> Is it ok to pass the option on a system that doesn't have mypy?

I'd expect the shell/which assignment to MYPY would leave it
undefined, in which case the behavior would match not building with
MYPY=1. It's basically the same logic as with shellchecks. I didn't
test/optimize for people requesting a build option with dependencies
missing.

Thanks,
Ian
Re: [PATCH v1 3/6] perf build: Add mypy build tests
Posted by Namhyung Kim 1 year ago 
On Fri, Jan 24, 2025 at 06:10:09PM -0800, Ian Rogers wrote:
> On Fri, Jan 24, 2025 at 3:51 PM Namhyung Kim <namhyung@kernel.org> wrote:
> >
> > On Fri, Oct 25, 2024 at 10:23:00AM -0700, Ian Rogers wrote:
> > > If MYPY=1 is passed to the build then run mypy over python code in
> > > perf. Unlike shellcheck this isn't default on as there are currently
> > > too many errors.
> >
> > Can you please add an example output in the commit log?  Also you need
> > to add a comment to describe the build option.
> 
> It feels more of a developer option, so I'm not sure we should
> advertise it. Ideally we'd just default it like the shellcheck case.

Yep, but I think we need some (brief) explanation even for a default
developer option.

> I'm not sure of the benefit of snapshotting mypy errors, a quick
> internet search will show what mypy does and I'd prefer the patch set
> focus on the build infrastructure changes.

You are introducing a new thing so you need to explain what it is, how
to use it and how the output look like.

> 
> > Is it ok to pass the option on a system that doesn't have mypy?
> 
> I'd expect the shell/which assignment to MYPY would leave it
> undefined, in which case the behavior would match not building with
> MYPY=1. It's basically the same logic as with shellchecks. I didn't
> test/optimize for people requesting a build option with dependencies
> missing.

Ok, I'm good as long as it handles the case.

Thanks,
Namhyung

Patchew 2.1-devel-b67e4c2

© 2016 - 2026 Red Hat, Inc.