Add skip tests for paranoid level being 3.
Rather than skipping lines starting "Failed", skip lines containing
"failed" - making the behavior consistent with the previous python
version.

Fixes: ec906102e5b7 ("perf test: Fix "perf stat CSV output linter" test on s390")
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/shell/stat+csv_output.sh | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/tools/perf/tests/shell/stat+csv_output.sh b/tools/perf/tests/shell/stat+csv_output.sh
index 38c26f3ef4c1..b165c81fccb4 100755
--- a/tools/perf/tests/shell/stat+csv_output.sh
+++ b/tools/perf/tests/shell/stat+csv_output.sh
@@ -26,9 +26,8 @@ function commachecker()
 
 	while read line
 	do
-		# Check for lines beginning with Failed
-		x=${line:0:6}
-		[ "$x" = "Failed" ] && continue
+		# Skip lines containing failed.
+		(echo "$line" | grep -q "failed") && continue
 
 		# Count the number of commas
 		x=$(echo $line | tr -d -c ',')
@@ -51,6 +50,11 @@ function ParanoidAndNotRoot()
 check_no_args()
 {
 	echo -n "Checking CSV output: no args "
+	if ParanoidAndNotRoot 2
+	then
+		echo "[Skip] paranoid and not root"
+		return
+	fi
 	perf stat -x, true 2>&1 | commachecker --no-args
 	echo "[Success]"
 }
@@ -83,6 +87,11 @@ check_system_wide_no_aggr()
 check_interval()
 {
 	echo -n "Checking CSV output: interval "
+	if ParanoidAndNotRoot 2
+	then
+		echo "[Skip] paranoid and not root"
+		return
+	fi
 	perf stat -x, -I 1000 true 2>&1 | commachecker --interval
 	echo "[Success]"
 }
@@ -91,6 +100,11 @@ check_interval()
 check_event()
 {
 	echo -n "Checking CSV output: event "
+	if ParanoidAndNotRoot 2
+	then
+		echo "[Skip] paranoid and not root"
+		return
+	fi
 	perf stat -x, -e cpu-clock true 2>&1 | commachecker --event
 	echo "[Success]"
 }
-- 
2.37.0.rc0.161.g10f37bed90-goog

On Tue, Jun 28, 2022 at 08:40:07PM -0700, Ian Rogers wrote:
> Add skip tests for paranoid level being 3.
> Rather than skipping lines starting "Failed", skip lines containing
> "failed" - making the behavior consistent with the previous python
> version.

paranoid 3 is an out of tree patch.

On Thu, Jun 30, 2022 at 2:11 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Tue, Jun 28, 2022 at 08:40:07PM -0700, Ian Rogers wrote:
> > Add skip tests for paranoid level being 3.
> > Rather than skipping lines starting "Failed", skip lines containing
> > "failed" - making the behavior consistent with the previous python
> > version.
>
> paranoid 3 is an out of tree patch.

Thanks, what is the right way to resolve this? My desktop appears to
be carrying the patch and I'd like the tests to be as green as
possible.

Thanks,
Ian

On Thu, Jun 30, 2022 at 01:59:54PM -0700, Ian Rogers wrote:
> On Thu, Jun 30, 2022 at 2:11 AM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Tue, Jun 28, 2022 at 08:40:07PM -0700, Ian Rogers wrote:
> > > Add skip tests for paranoid level being 3.
> > > Rather than skipping lines starting "Failed", skip lines containing
> > > "failed" - making the behavior consistent with the previous python
> > > version.
> >
> > paranoid 3 is an out of tree patch.
> 
> Thanks, what is the right way to resolve this? My desktop appears to
> be carrying the patch and I'd like the tests to be as green as
> possible.

Then you desktop is probably running a Debian or derivative distro
kernel. You can run your own kernel, or ask the Debian team to ditch
their hack and use the LSM hooks to further limit perf usage if they
feel this is required.

The big advantage of the LSM hooks is that they can explicitly
white-list the perf binary while dis-allowing random users access to the
syscall. That way perf will still work but the possible exploit
potential is much reduced.

On Fri, Jul 1, 2022 at 1:55 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Thu, Jun 30, 2022 at 01:59:54PM -0700, Ian Rogers wrote:
> > On Thu, Jun 30, 2022 at 2:11 AM Peter Zijlstra <peterz@infradead.org> wrote:
> > >
> > > On Tue, Jun 28, 2022 at 08:40:07PM -0700, Ian Rogers wrote:
> > > > Add skip tests for paranoid level being 3.
> > > > Rather than skipping lines starting "Failed", skip lines containing
> > > > "failed" - making the behavior consistent with the previous python
> > > > version.
> > >
> > > paranoid 3 is an out of tree patch.
> >
> > Thanks, what is the right way to resolve this? My desktop appears to
> > be carrying the patch and I'd like the tests to be as green as
> > possible.
>
> Then you desktop is probably running a Debian or derivative distro
> kernel. You can run your own kernel, or ask the Debian team to ditch
> their hack and use the LSM hooks to further limit perf usage if they
> feel this is required.
>
> The big advantage of the LSM hooks is that they can explicitly
> white-list the perf binary while dis-allowing random users access to the
> syscall. That way perf will still work but the possible exploit
> potential is much reduced.

Thanks, neither rewriting Debian's security to use LSM or running a
custom kernel are going to work in the environment I have. Presumably
it is going to be a matter of policy not to allow this test fix to
land, meaning Debian kernels are going to show failing tests? I can
carry the patch privately but that's a tech-debt, merge-conflict mess.

Thanks,
Ian