From nobody Mon Nov 25 16:48:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0099521B848 for ; Fri, 25 Oct 2024 19:21:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729884102; cv=none; b=k2nG5L3qXA9sKxovmURv9cVknYw7sm0MCJcXUAXXj/7bNa3VJDcpgJnuKE2tr39OQSqGmUh9ITi+nZ9+0AfZV5XP4iN7IgkQa9z0uxFIfqabI6hwBWrC/+jPReUuQyltDWi8+4mrKeL1x4LK3gUTgQ0o9WPIy15OlUVV72zxOvQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729884102; c=relaxed/simple; bh=rNAcF8wM8gvz4H8uQaNqviqYDP2ME6uNIt9wV+wjHHQ=; h=Date:In-Reply-To:Message-Id:Mime-Version:References:Subject:From: To:Content-Type; b=C2J6Jiumx9yb0Ip3wsQ8GWuN+gB3po1RJudZW4Cc3jQ/Er2cQQpxl5uF0N8A+eT0++nIkOkqreFwUYblQYllOAIOSi0VgbG4oUNekIJAWYfZvAoUnrfMa1zujPXCC5vJXi61zCz6uSPI7aVARr93KwogrXLADL2x8RqNFj8h8vw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2HBFvx0v; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2HBFvx0v" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e292dbfd834so4162808276.3 for ; Fri, 25 Oct 2024 12:21:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1729884098; x=1730488898; darn=vger.kernel.org; h=to:from:subject:references:mime-version:message-id:in-reply-to:date :from:to:cc:subject:date:message-id:reply-to; bh=/dxz2W/8vDBHa448LZ9Sd6x6HeOGt9H5wdrppWaYtKk=; b=2HBFvx0v8xJCI17kERjyrZMzHpPctIb37U77ct8HhkDj+0YJ97Qo8Z6sOxHXPZrTJ8 ca834jwrKT0dwAac7k0vj4gEyxRVfc9wor2VFc0lAjXKQuDBfbcSqiyuA0VYSqPUQhQw zmtHSXha5QCh+DETmk3W92DQme6WPoroRQsF/2VZdwGOMfiYXP8rUt/UCkydHAaybL31 BxPP4fOyQhjNpEVFt/zTBvE+mB9eAXne2I86Dt2ycOeCzgZmeqVjxpn33+o2Je5+JvJQ p8o8VMMnSRgRrR2KpXF4DFgL6nO01F/iKBHkr0OCM+Ie2hjP0CYu9clQCbnNo8kPS4WP zo1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729884098; x=1730488898; h=to:from:subject:references:mime-version:message-id:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/dxz2W/8vDBHa448LZ9Sd6x6HeOGt9H5wdrppWaYtKk=; b=PvGJtpzOYrbiwFLWaR5Cekr42B2FZtC9qQ27pwFNdep/CUn1u4sd3cgTeKOcJ/4eVZ lGlDXFYOPrq2iTwXw4Js9SETSQVQN4Xp7TqGj1Q9ANVyyEhb7orHNvHvp5NNdA/zKsx3 XBLaS1k9LTClYFWX1TVFER4o35AI/8k854faF9/VgQ3BhaWCB/Qyx2gWOdDGLPCivFq8 r8z6VcbLr7tmIXCHLFo+l0K9unkJSmAUiuBKh6cZv+iUqrWO3mMu7iGiX+yMcUIkS7sm AwioXEcsaYWnjFoUllLyX0Z/JPc9nSUr6swwZ+T1vLGJyyqBexqotgqnYf4vT0yXN6+0 J2bw== X-Forwarded-Encrypted: i=1; AJvYcCVRgMdwvVjeflP9S6WhLRJQgDER338EiSSciiQdBIo21SUPNpWYbuqoQ1Ua4MbeJigJ/PwnwWhHYUPTUJc=@vger.kernel.org X-Gm-Message-State: AOJu0YxoHqR+4qbmYIjMeaqauSG6YOhko/apHM101eBJx1K5w8koN9No A4z0BrQMuweD68UYzU7PF9VcNFQPpDRW1jFix+LYGi4fy7j5WPbz0q1cUNo9P9pKS45d71n4pbm NjoezQw== X-Google-Smtp-Source: AGHT+IEUKi5wV3YEZPlz2DurrhYIqR/SpWkQ+hDi0Tx37uRYxgfi+MljIzUcCfqShJ1/JNUl8DrAvdnw/RfC X-Received: from irogers.svl.corp.google.com ([2620:15c:2c5:11:5d42:c43e:5e08:3a85]) (user=irogers job=sendgmr) by 2002:a25:c78a:0:b0:e17:8e4f:981a with SMTP id 3f1490d57ef6-e3087c2d8b1mr1832276.11.1729884097267; Fri, 25 Oct 2024 12:21:37 -0700 (PDT) Date: Fri, 25 Oct 2024 12:21:09 -0700 In-Reply-To: <20241025192109.132482-1-irogers@google.com> Message-Id: <20241025192109.132482-11-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241025192109.132482-1-irogers@google.com> X-Mailer: git-send-email 2.47.0.163.g1226f6d8fa-goog Subject: [PATCH v5 10/10] perf test: Sort tests placing exclusive tests last From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , James Clark , Howard Chu , Athira Jajeev , Michael Petlan , Veronika Molnarova , Dapeng Mi , Thomas Richter , Ilya Leoshkevich , Colin Ian King , Weilin Wang , Andi Kleen , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This allows a uniform test numbering even though two passes are used to execute them. Signed-off-by: Ian Rogers --- tools/perf/tests/builtin-test.c | 125 +++++++++++++++++++++----------- 1 file changed, 84 insertions(+), 41 deletions(-) diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-tes= t.c index 2f880f48cdc5..d2cabaa8ad92 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -137,12 +137,6 @@ static struct test_suite *generic_tests[] =3D { NULL, }; =20 -static struct test_suite **tests[] =3D { - generic_tests, - arch_tests, - NULL, /* shell tests created at runtime. */ -}; - static struct test_workload *workloads[] =3D { &workload__noploop, &workload__thloop, @@ -468,10 +462,6 @@ static int start_test(struct test_suite *test, int i, = int subi, struct child_tes return start_command(&(*child)->process); } =20 -#define for_each_test(j, k, t) \ - for (j =3D 0, k =3D 0; j < ARRAY_SIZE(tests); j++, k =3D 0) \ - while ((t =3D tests[j][k++]) !=3D NULL) - /* State outside of __cmd_test for the sake of the signal handler. */ =20 static size_t num_tests; @@ -483,22 +473,21 @@ static void cmd_test_sig_handler(int sig) siglongjmp(cmd_test_jmp_buf, sig); } =20 -static int __cmd_test(int argc, const char *argv[], struct intlist *skipli= st) +static int __cmd_test(struct test_suite **suites, int argc, const char *ar= gv[], + struct intlist *skiplist) { - struct test_suite *t; static int width =3D 0; - unsigned int j, k; int err =3D 0; =20 - for_each_test(j, k, t) { - int len =3D strlen(test_description(t, -1)); + for (struct test_suite **t =3D suites; *t; t++) { + int len =3D strlen(test_description(*t, -1)); =20 if (width < len) width =3D len; =20 - if (has_subtests(t)) { - for (int subi =3D 0, subn =3D num_subtests(t); subi < subn; subi++) { - len =3D strlen(test_description(t, subi)); + if (has_subtests(*t)) { + for (int subi =3D 0, subn =3D num_subtests(*t); subi < subn; subi++) { + len =3D strlen(test_description(*t, subi)); if (width < len) width =3D len; num_tests++; @@ -540,18 +529,18 @@ static int __cmd_test(int argc, const char *argv[], s= truct intlist *skiplist) int child_test_num =3D 0; int i =3D 0; =20 - for_each_test(j, k, t) { + for (struct test_suite **t =3D suites; *t; t++) { int curr =3D i++; =20 - if (!perf_test__matches(test_description(t, -1), curr, argc, argv)) { + if (!perf_test__matches(test_description(*t, -1), curr, argc, argv)) { /* * Test suite shouldn't be run based on * description. See if subtest should. */ bool skip =3D true; =20 - for (int subi =3D 0, subn =3D num_subtests(t); subi < subn; subi++) { - if (perf_test__matches(test_description(t, subi), + for (int subi =3D 0, subn =3D num_subtests(*t); subi < subn; subi++) { + if (perf_test__matches(test_description(*t, subi), curr, argc, argv)) skip =3D false; } @@ -561,24 +550,24 @@ static int __cmd_test(int argc, const char *argv[], s= truct intlist *skiplist) } =20 if (intlist__find(skiplist, i)) { - pr_info("%3d: %-*s:", curr + 1, width, test_description(t, -1)); + pr_info("%3d: %-*s:", curr + 1, width, test_description(*t, -1)); color_fprintf(stderr, PERF_COLOR_YELLOW, " Skip (user override)\n"); continue; } =20 - if (!has_subtests(t)) { - err =3D start_test(t, curr, -1, &child_tests[child_test_num++], + if (!has_subtests(*t)) { + err =3D start_test(*t, curr, -1, &child_tests[child_test_num++], width, pass); if (err) goto err_out; continue; } - for (int subi =3D 0, subn =3D num_subtests(t); subi < subn; subi++) { - if (!perf_test__matches(test_description(t, subi), + for (int subi =3D 0, subn =3D num_subtests(*t); subi < subn; subi++) { + if (!perf_test__matches(test_description(*t, subi), curr, argc, argv)) continue; =20 - err =3D start_test(t, curr, subi, &child_tests[child_test_num++], + err =3D start_test(*t, curr, subi, &child_tests[child_test_num++], width, pass); if (err) goto err_out; @@ -602,27 +591,25 @@ static int __cmd_test(int argc, const char *argv[], s= truct intlist *skiplist) return err; } =20 -static int perf_test__list(int argc, const char **argv) +static int perf_test__list(struct test_suite **suites, int argc, const cha= r **argv) { - unsigned int j, k; - struct test_suite *t; int i =3D 0; =20 - for_each_test(j, k, t) { + for (struct test_suite **t =3D suites; *t; t++) { int curr =3D i++; =20 - if (!perf_test__matches(test_description(t, -1), curr, argc, argv)) + if (!perf_test__matches(test_description(*t, -1), curr, argc, argv)) continue; =20 - pr_info("%3d: %s\n", i, test_description(t, -1)); + pr_info("%3d: %s\n", i, test_description(*t, -1)); =20 - if (has_subtests(t)) { - int subn =3D num_subtests(t); + if (has_subtests(*t)) { + int subn =3D num_subtests(*t); int subi; =20 for (subi =3D 0; subi < subn; subi++) pr_info("%3d:%1d: %s\n", i, subi + 1, - test_description(t, subi)); + test_description(*t, subi)); } } return 0; @@ -661,6 +648,55 @@ static int perf_test__config(const char *var, const ch= ar *value, return 0; } =20 +static struct test_suite **build_suites(void) +{ + /* + * TODO: suites is static to avoid needing to clean up the scripts tests + * for leak sanitizer. + */ + static struct test_suite **suites[] =3D { + generic_tests, + arch_tests, + NULL, + }; + struct test_suite **result; + struct test_suite *t; + size_t n =3D 0, num_suites =3D 0; + + if (suites[2] =3D=3D NULL) + suites[2] =3D create_script_test_suites(); + +#define for_each_test(t) \ + for (size_t i =3D 0, j =3D 0; i < ARRAY_SIZE(suites); i++, j =3D 0) \ + while ((t =3D suites[i][j++]) !=3D NULL) + + for_each_test(t) + num_suites++; + + result =3D calloc(num_suites + 1, sizeof(struct test_suite *)); + + for (int pass =3D 1; pass <=3D 2; pass++) { + for_each_test(t) { + bool exclusive =3D false; + + if (!has_subtests(t)) { + exclusive =3D test_exclusive(t, -1); + } else { + for (int subi =3D 0, subn =3D num_subtests(t); subi < subn; subi++) { + if (test_exclusive(t, subi)) { + exclusive =3D true; + break; + } + } + } + if ((!exclusive && pass =3D=3D 1) || (exclusive && pass =3D=3D 2)) + result[n++] =3D t; + } + } + return result; +#undef for_each_test +} + int cmd_test(int argc, const char **argv) { const char *test_usage[] =3D { @@ -688,6 +724,7 @@ int cmd_test(int argc, const char **argv) const char * const test_subcommands[] =3D { "list", NULL }; struct intlist *skiplist =3D NULL; int ret =3D hists__init(); + struct test_suite **suites; =20 if (ret < 0) return ret; @@ -697,10 +734,13 @@ int cmd_test(int argc, const char **argv) /* Unbuffered output */ setvbuf(stdout, NULL, _IONBF, 0); =20 - tests[2] =3D create_script_test_suites(); argc =3D parse_options_subcommand(argc, argv, test_options, test_subcomma= nds, test_usage, 0); - if (argc >=3D 1 && !strcmp(argv[0], "list")) - return perf_test__list(argc - 1, argv + 1); + if (argc >=3D 1 && !strcmp(argv[0], "list")) { + suites =3D build_suites(); + ret =3D perf_test__list(suites, argc - 1, argv + 1); + free(suites); + return ret; + } =20 if (workload) return run_workload(workload, argc, argv); @@ -728,5 +768,8 @@ int cmd_test(int argc, const char **argv) */ rlimit__bump_memlock(); =20 - return __cmd_test(argc, argv, skiplist); + suites =3D build_suites(); + ret =3D __cmd_test(suites, argc, argv, skiplist); + free(suites); + return ret; } --=20 2.47.0.163.g1226f6d8fa-goog