From nobody Sun Oct 5 18:19:13 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B5322C3276 for ; Thu, 31 Jul 2025 13:26:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753968418; cv=none; b=jyZpZn3WQetUdQnIEfpCmV6EPM0lgB2wqhR1LDQ+5rxKRvI+pu83PO7P8MnqeJQhP+kzhWe2hzP9qiDqWmJ1Ycb3JENP/OF8U7jU7/5m/bXgDWxIVhZpQTTclrr9oZQcdV022eyx8itJ2pF7yXil0t1k/pdyrCxG4u7aHMJjTI0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753968418; c=relaxed/simple; bh=DHxNzNVWUr+O5o8IyHJuN8fdD6GVaiWbG6x9JXShmp0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Content-Type; b=AIxidifS/cqyrEPMlcjGc596Wt1kIHVeqjgV10XbdnnxZOPwB6wmcmynB1BgqA3PHBA4R11AIX5GZZbtnF70sd1CtNeb9XuKhcEedFXigTmeGoBRAfByXMyuG/yBSesON5YMhqnBdsDVgWzIrkS97YEYei2MtQSs5hailjhUtrY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yuzhuo.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EY5BPP2T; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yuzhuo.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EY5BPP2T" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-24011ceafc8so8786105ad.3 for ; Thu, 31 Jul 2025 06:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753968415; x=1754573215; darn=vger.kernel.org; h=to:from:subject:message-id:references:mime-version:in-reply-to:date :from:to:cc:subject:date:message-id:reply-to; bh=bkP7Z4Cqqv/NC5+ok8KneBr4i3bJ/7NR+0dUzs2HMi8=; b=EY5BPP2T7uLDGS8T7JrCdxI6SpxCO3nSoXlqVls2F4J6o45YL5MbeaE7/+yIoTpErl X9JvPt4FPBt0jXyCDmf1Nd5pMf5Boz6Ef/K/qDSphgk9H9NhWhsi2wrgPy3Rdu47YXdV xrsd47wse6ZL5TwDgmphPQGtCburbHN12aM/U14x3SKn7gOa61JSHFdguC18GfsWC87j Sc7ELTWuBWzqmDTMcF0p9+AYKs0X+hjQhxroxwpZFnaIEPgHo3vAWORi06W7HqGlOr3S 9GJm94wEsva/fW7RGkIhqCDayIsiRIUt4rjP4DmXG8dtS5p2f4FgyR1aR3PtScgctQH9 4hpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753968415; x=1754573215; h=to:from:subject:message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bkP7Z4Cqqv/NC5+ok8KneBr4i3bJ/7NR+0dUzs2HMi8=; b=oE4a0g0WaMqPHj/hgPMz7vRcYb2NYuvcxlzfyvsCzth5cowRsWS6Ec+/4gTemBf1oF FifQDkzoDe+1CnqaKNMmHQBIKCvlKvHZ3z8Ejw9Gu+Od2wgWonIBUwdfhY1gHvFZAywt gBBe4w9ngBrgaCf3hIn1E4ONg0qgjjz9WP7O3pNGkWP8y0PxQ+M6Sg+s1T5kz/1WNgEA 7WEsETiR3kSkzfkVhM8UOBcOTKO4zXVd8WrceWTevCRANosRhG1DM8n5lusaaDS9PIbR creHiBW2p7KNe0dqnUC7XE3BVtOyWpy5SZpo3ov9lIT7i9/S9PUzQw4SkdpdRw/+YEC1 Hthw== X-Forwarded-Encrypted: i=1; AJvYcCV31QfXzWR/oNJpyyEdkc2vLyhhTiW66YjvZkf/DLT/HXzpjS/RZJ3KeSN5t/zjmoFJEurRNG1BFHSvNRI=@vger.kernel.org X-Gm-Message-State: AOJu0YxDALOzWBgJOmF5QBOnxlYcPNyzVFmK9YUoVbnVt90bg9L4My9z oeGcIDpiZQUAcqB22bicomMC56AY6PRZidCawzUqWmFIsaaYx++QfDQ8OlxduDCZ70ucRgILPK8 1/PBaog== X-Google-Smtp-Source: AGHT+IEwduAehduBW9EWHJOWUGqQXIKJ02sza7Y6mu7qiMI5Yth6Qmr0VERdlGJ8n+sQiOAH1nTf9mvtt4k= X-Received: from plnf7.prod.google.com ([2002:a17:903:1a67:b0:240:1bbf:686b]) (user=yuzhuo job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:c94b:b0:240:150d:9147 with SMTP id d9443c01a7336-24096a56357mr111905495ad.13.1753968415132; Thu, 31 Jul 2025 06:26:55 -0700 (PDT) Date: Thu, 31 Jul 2025 06:26:11 -0700 In-Reply-To: <20250731132615.938435-1-yuzhuo@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250731132615.938435-1-yuzhuo@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250731132615.938435-2-yuzhuo@google.com> Subject: [PATCH v1 1/5] perf bench: Add RCU benchmark using rcuscale kernel module From: Yuzhuo Jing To: Davidlohr Bueso , "Paul E . McKenney" , Josh Triplett , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Boqun Feng , Uladzislau Rezki , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Liang Kan , Yuzhuo Jing , Yuzhuo Jing , Sebastian Andrzej Siewior , linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-perf-users@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add 'rcu' to the 'perf bench sync' collection. This benchmark depends on the rcuscale kernel module, and also depends on new features in the rcuscale module that exposes control and internal state through debugfs. This patch adds the basic 'once' mode that runs one combination of rcuscale parameters. Command usage is defined as: perf bench sync rcu [options..] [once [..]] * gp_type is one of "sync", "async", "exp" * Valid params can be found from `modinfo rcuscale`. Except that gp_exp, gp_async, and block_start are managed by the benchmark and cannot be set by user. This benchmark parses the modinfo to validate the existence of user provided parameters. It then executes modprobe to load rcuscale. Experiment start/finish is controlled through /sys/kernel/debug/rcuscale/{should_start,test_complete}. After the experiment finishes, it reads /sys/kernel/debug/rcuscale/writer_durations and outputs statistics. The statistic code (print_writer_duration_stats function) is derived from tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuscale.sh. Example output: ./perf bench sync rcu --hist once exp nreaders=3D1 nwriters=3D1 \# Running 'sync/rcu' benchmark: Running experiment with options: gp_exp=3D1 nreaders=3D1 nwriters=3D1 Experiment finished. Histogram bucket size: 0.1 microseconds 8.8 11 8.9 25 9 27 9.1 13 9.2 6 9.3 3 9.4 2 9.5 2 9.6 3 9.7 1 9.8 1 9.9 1 10 2 10.4 1 10.5 1 11.1 1 6025.6 1 Average grace-period duration: 68.734 microseconds Minimum grace-period duration: 8.813 50th percentile grace-period duration: 9.044 90th percentile grace-period duration: 9.625 99th percentile grace-period duration: 10.516 Maximum grace-period duration: 6025.679 Signed-off-by: Yuzhuo Jing --- tools/perf/bench/Build | 1 + tools/perf/bench/bench.h | 1 + tools/perf/bench/sync-rcu.c | 816 ++++++++++++++++++++++++++++++++++++ tools/perf/builtin-bench.c | 1 + 4 files changed, 819 insertions(+) create mode 100644 tools/perf/bench/sync-rcu.c diff --git a/tools/perf/bench/Build b/tools/perf/bench/Build index 13558279fa0e..f694f8715cfc 100644 --- a/tools/perf/bench/Build +++ b/tools/perf/bench/Build @@ -20,6 +20,7 @@ perf-bench-y +=3D breakpoint.o perf-bench-y +=3D pmu-scan.o perf-bench-y +=3D uprobe.o perf-bench-y +=3D sync.o +perf-bench-y +=3D sync-rcu.o perf-bench-y +=3D qspinlock.o =20 perf-bench-$(CONFIG_X86_64) +=3D mem-memcpy-x86-64-asm.o diff --git a/tools/perf/bench/bench.h b/tools/perf/bench/bench.h index 42c0696b05fb..09c5b3af347f 100644 --- a/tools/perf/bench/bench.h +++ b/tools/perf/bench/bench.h @@ -24,6 +24,7 @@ int bench_sched_pipe(int argc, const char **argv); int bench_sched_seccomp_notify(int argc, const char **argv); int bench_sync_qspinlock(int argc, const char **argv); int bench_sync_ticket(int argc, const char **argv); +int bench_sync_rcu(int argc, const char **argv); int bench_syscall_basic(int argc, const char **argv); int bench_syscall_getpgid(int argc, const char **argv); int bench_syscall_fork(int argc, const char **argv); diff --git a/tools/perf/bench/sync-rcu.c b/tools/perf/bench/sync-rcu.c new file mode 100644 index 000000000000..ac85841f0b68 --- /dev/null +++ b/tools/perf/bench/sync-rcu.c @@ -0,0 +1,816 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * RCU scale benchmark using rcuscale kernel module. + * + * 2025 Yuzhuo Jing + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "bench.h" + +#define MAX_OPTS 64 +#define MAX_OPTNAME 64 +#define MAX_OPTTYPE 16 +#define MAX_OPTVALUE 128 + +#define INIT_CAPACITY 1024UL + +/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D Global Options =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D */ + +static bool dryrun; +static unsigned int cooldown =3D 3; +static bool show_hist; +static const char *debugfs =3D "/sys/kernel/debug"; + +static const struct option bench_rcu_options[] =3D { + OPT_BOOLEAN('n', "dryrun", &dryrun, "Dry run mode"), + OPT_UINTEGER('c', "cooldown", &cooldown, + "Sleep time between each run (default: 3 seconds)"), + OPT_BOOLEAN(0, "hist", &show_hist, + "Show histogram of writer durations"), + OPT_STRING(0, "debugfs", &debugfs, "path", + "Debugfs mount point (default: /sys/kernel/debug)"), + OPT_END() +}; + +static const char *const bench_rcu_usage[] =3D { + "RCU benchmark using rcuscale kernel module.", + "", + "perf bench sync rcu [options..]", + "perf bench sync rcu [options..] once [..]", + "", + " : The type of grace period to use: sync, async, exp (expedite= d)", + " This sets the gp_exp or gp_async kernel module parameters.", + " : Any parameter of the rcuscale kernel module, e.g. holdoff= =3D5.", + " Valid options can be found from running `modinfo rcuscale`.= ", + "", + "Notes on param:", + " This benchmark manages gp_exp and gp_async, and sets block_start=3D1.", + " User cannot override those parameters. This benchmark also sets defau= lt", + " values writer_no_print=3D1 and holdoff=3D3, but users may override tho= se.", + " Note that if nwriters=3D0, the rcuscale kernel module will not exit,", + " and the benchmark will sleep indefinitely.", + "", + "Modes:", + " default: Run 'once sync'.", + " once: Run benchmark once, with all parameters passed through to the", + " kernel rcuscale module.", + "", + "Examples:", + " perf bench sync rcu --hist once exp nreaders=3D1 nwriters=3D1 writer_c= pu_offset=3D1", + " perf bench sync rcu once", + " perf bench sync rcu once sync nreaders=3D1 nwriters=3D1 writer_cpu_of= fset=3D1", + "", + "In case perf exited abnormally, user need to unload rcuscale by running:= ", + " modprobe -r rcuscale torture", + "", + "Global options:", // continues to show bench_rcu_options + NULL +}; + +/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D Runtime Options =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D */ + +#define MODPROBE_BASE_COUNT 3 +#define MODPROBE_CMD_MAX (2 * MAX_OPTS + MODPROBE_BASE_COUNT + 1) +/* + * The command line builder for modprobe. The cmd array will be directly + * passed to execvp. + * + * Note: the cmd array does not own the pointers in it. Those argument + * pointers could come from: + * - string literals (e.g. the "modprobe" and "rcuscale" command name) + * - simple_params + */ +struct modprobe_cmd { + const char *cmd[MODPROBE_CMD_MAX]; + size_t count; +}; + +#define MODPROBE_CMD_INIT \ + struct modprobe_cmd modprobe_cmd =3D { \ + { "modprobe", "rcuscale", "block_start=3D1", NULL, }, \ + MODPROBE_BASE_COUNT, \ + } +#define MODPROBE_REMOVE_CMD "modprobe -r rcuscale torture" + +/* + * Generic modprobe parameter definition. This is the storage for an + * instantiated module parameter. This may come from parameters directly + * given by user, or generated. + * + * Format must be "key=3Dvalue". + */ +struct modprobe_param { + char value[MAX_OPTVALUE]; +}; + +/* + * The storage for simple (i.e. non-range) module parameter strings. + */ +static struct modprobe_param simple_params[MAX_OPTS]; +static int simple_params_count; + +static bool in_child; + +struct durations { + u64 *values; + size_t count; + size_t capacity; + u64 sum; +}; + +/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D Override parameters =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D */ + +/* Non-override module parameter. This may be updated by "ratio" command.= */ +struct non_override_param { + char name[MAX_OPTNAME]; +}; +static int non_override_params_count =3D 3; +static struct non_override_param non_override_params[MAX_OPTS] =3D { + { "block_start" }, + { "gp_exp" }, + { "gp_async" }, +}; + +struct overridable_param { + char name[MAX_OPTNAME]; + bool user_overridden; +}; +static struct overridable_param overridable_params[] =3D { + { "writer_no_print", false }, + { "holdoff", false }, +}; +static const int overridable_params_count =3D ARRAY_SIZE(overridable_param= s); + +/* Valid module parameters parsed from modinfo. */ +struct modinfo_parm { + char name[MAX_OPTNAME]; + char type[MAX_OPTTYPE]; +}; +static struct modinfo_parm modinfo_parms[MAX_OPTS]; +static int modinfo_parms_count; + +/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D Cleanup functions =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D */ + +static void unload_module(void) +{ + if (system(MODPROBE_REMOVE_CMD) !=3D 0) + fprintf(stderr, "Failed to unload rcuscale kernel module.\n" + "Please manually remove it using `"MODPROBE_REMOVE_CMD"`.\n"); +} + +static void cleanup(void) +{ + if (in_child) + return; + + unload_module(); +} + +static void signal_handler(int sig) +{ + if (sig) + fprintf(stderr, "perf: Signal %d received\n", sig); + /* cleanup is registered in atexit */ + fprintf(stderr, "Cleaning up...\n"); + exit(1); +} + +static void setup_cleanup(void) +{ + atexit(cleanup); + signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); + signal(SIGSEGV, signal_handler); +} + +/* + * Failure handling. Use this for logical checks, and use "err" for those= with + * external interactions. + */ +#define fail(fmt, ...) \ +do { \ + fprintf(stderr, "perf: "fmt"\n", ##__VA_ARGS__); \ + exit(1); \ +} while (0) + +/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D Modprobe info =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D */ + +/* + * Parse modinfo and store the results in modinfo_parms. Used to determine + * whether an option is valid as a range parameter. + * + * The expected format is: + * nreaders:Number of RCU reader threads (int) + * nwriters:Number of RCU updater threads (int) + */ +static void parse_modinfo(void) +{ + char *line =3D NULL; + size_t len =3D 0; + FILE *fp; + + fp =3D popen("modinfo rcuscale -0 -F parm", "r"); + if (!fp) + err(EXIT_FAILURE, "Failed to run modinfo"); + + while (getdelim(&line, &len, '\0', fp) !=3D -1) { + char *type; + char *remaining =3D NULL; + char *name =3D strtok_r(line, ":", &remaining); + + if (!name) + fail("Failed to parse modinfo parameter name"); + + type =3D strrchr(remaining, '('); + if (!type) + fail("Failed to parse modinfo parameter type"); + remaining =3D NULL; + type =3D strtok_r(type + 1, ")", &remaining); + if (!type) + fail("Failed to parse modinfo parameter type"); + + strlcpy(modinfo_parms[modinfo_parms_count].name, name, MAX_OPTNAME); + strlcpy(modinfo_parms[modinfo_parms_count].type, type, MAX_OPTTYPE); + modinfo_parms_count++; + } + if (!modinfo_parms_count) + fail("Failed to read modinfo"); + + free(line); + pclose(fp); +} + +/* + * Check if the module parameter is an integer. + */ +static bool modparm_is_int(const char *name) +{ + for (int i =3D 0; i < modinfo_parms_count; i++) { + if (strcmp(modinfo_parms[i].name, name) =3D=3D 0) + return strcmp(modinfo_parms[i].type, "int") =3D=3D 0; + } + return false; +} + +/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D Argument parsing =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D */ + +/* + * Reserve memory for a pointer. + * + * If current capacity is 0, the minimum capacity is at least INIT_CAPACIT= Y or + * min_capacity. + * + * The current capacity is doubled if it is less than the minimum capacity. + */ +static void raw_reserve_size(void **ptr, size_t elemsize, + size_t *current_capacity, size_t min_capacity) +{ + size_t new_capacity; + + if (min_capacity =3D=3D 0) + min_capacity =3D INIT_CAPACITY; + + if (*current_capacity >=3D min_capacity) + return; + + new_capacity =3D *current_capacity ?: min_capacity; + while (new_capacity < min_capacity) + new_capacity *=3D 2; + + *ptr =3D realloc(*ptr, new_capacity * elemsize); + if (!*ptr) + fail("Failed to allocate memory"); + *current_capacity =3D new_capacity; +} +#define reserve_size(ptr, current_capacity, min_capacity) \ + raw_reserve_size((void **)(ptr), sizeof(**(ptr)), (current_capacity), (mi= n_capacity)) + +static int parse_int(const char *val) +{ + char *endptr; + long num =3D strtol(val, &endptr, 10); + + if (*endptr !=3D '\0') + fail("Invalid integer format: %s", val); + + return (int)num; +} + +static void simple_params_add(const char *full) +{ + if (simple_params_count >=3D MAX_OPTS) + fail("Too many module parameters"); + strlcpy(simple_params[simple_params_count++].value, full, MAX_OPTVALUE); +} + +static void parse_gp_type(const char *gp_type) +{ + if (strcmp(gp_type, "sync") =3D=3D 0) { + // no new option is added + } else if (strcmp(gp_type, "async") =3D=3D 0) + simple_params_add("gp_async=3D1"); + else if (strcmp(gp_type, "exp") =3D=3D 0) + simple_params_add("gp_exp=3D1"); + else + fail("Invalid grace period type: %s", gp_type); +} + +/* + * Check if the option is already set. + */ +static bool param_has_conflict(const char *key) +{ + for (int i =3D 0; i < non_override_params_count; ++i) { + if (strcmp(key, non_override_params[i].name) =3D=3D 0) + return true; + } + for (int i =3D 0; i < simple_params_count; ++i) { + if (strncmp(key, simple_params[i].value, strlen(key)) =3D=3D 0 + && simple_params[i].value[strlen(key)] =3D=3D '=3D') + return true; + } + /* overridable_params are considered non conflict */ + + return false; +} + +static struct overridable_param *overridable_param_get(const char *key) +{ + for (int i =3D 0; i < overridable_params_count; ++i) + if (strcmp(overridable_params[i].name, key) =3D=3D 0) + return overridable_params + i; + return NULL; +} + +/* + * For overridable_params, if user specifies it, set overridden so that it= will + * not be appended to modprobe cmd. + */ +static inline void param_try_set_user_override(const char *key) +{ + struct overridable_param *param =3D overridable_param_get(key); + + if (param) + param->user_overridden =3D true; +} + +/* + * Validate basics about the parameter name. + * + * Note: This is supposed to only be used during parsing user provided + * arguments. This will also update the "user_overridden" flag for overri= dable + * parameters. + */ +static void check_param_name(const char *name) +{ + if (strlen(name) + 1 > MAX_OPTNAME) + fail("Module parameter name too long: %s", name); + if (param_has_conflict(name)) + fail("Module parameter \"%s\" has conflict", name); + /* Set user overridden if possible */ + param_try_set_user_override(name); + + for (int i =3D 0; i < modinfo_parms_count; ++i) { + if (strcmp(modinfo_parms[i].name, name) =3D=3D 0) + return; + } + fail("Module parameter \"%s\" does not exist in modinfo", name); +} + +/* + * Parse module parameter. Results are stored in params and range_params. + * + * If allow_range is false, all params are stored in params, and checks + * the format cannot be range. + * + * If allow_range is true, params that only has one value will be stored in + * params, and range ones will be stored in range_params. + */ +static void parse_module_params(int argc, const char *argv[]) +{ + while (argc) { + char *saved_ptr =3D NULL; + char *key; + char *value; + char buf[MAX_OPTVALUE] =3D ""; + + if (strnlen(argv[0], MAX_OPTVALUE) >=3D MAX_OPTVALUE - 1) + fail("Module parameter too long: \"%s\"", argv[0]); + strlcpy(buf, argv[0], MAX_OPTVALUE); + + /* Parse keys and values. */ + key =3D strtok_r(buf, "=3D", &saved_ptr); + if (!key) + fail("Failed to parse module option \"%s\"", argv[0]); + check_param_name(key); + + value =3D strtok_r(NULL, "=3D", &saved_ptr); + if (!value || strlen(value) =3D=3D 0) + fail("Cannot find value for module option \"%s\"", key); + if (strlen(value) + 1 > MAX_OPTVALUE) + fail("Module parameter value too long: \"%s\"", value); + + /* Ensure integer type value are integers, but don't need the value. */ + if (modparm_is_int(key)) + parse_int(value); + + simple_params_add(argv[0]); + + argc--; + argv++; + } +} + +/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Expe= riment Result Handling =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D */ + +static void durations_add(struct durations *durations, u64 duration) +{ + reserve_size(&durations->values, &durations->capacity, durations->count += 1); + durations->values[durations->count++] =3D duration; + durations->sum +=3D duration; +} + +/* + * Parse writer durations from debugfs and push them to durations array. + * + * The expected format is writer_id,duration. + * + * Durations are converted to microseconds and stored in durations array. + */ +static struct durations *parse_durations(void) +{ + char durations_path[PATH_MAX]; + FILE *fp =3D NULL; + char *line =3D NULL; + size_t len =3D 0; + ssize_t read; + u64 duration; + struct durations *durations =3D calloc(1, sizeof(*durations)); + + if (!durations) + fail("Failed to allocate memory for durations"); + + snprintf(durations_path, sizeof(durations_path), "%s/rcuscale/writer_dura= tions", debugfs); + + fp =3D fopen(durations_path, "r"); + if (!fp) + err(EXIT_FAILURE, "Failed to open writer_durations"); + + while ((read =3D getline(&line, &len, fp)) !=3D -1) { + if (sscanf(line, "%*d,%lu", &duration) !=3D 1) + fail("Failed to parse writer duration. Line: %s", line); + durations_add(durations, duration); + } + + free(line); + fclose(fp); + + return durations; +} + +static void free_durations(struct durations *durations) +{ + free(durations->values); + free(durations); +} + +/* + * Helper function for sorting. + */ +static int compare_duration(const void *a, const void *b) +{ + u64 aa =3D *(u64 *)a, bb =3D *(u64 *)b; + + return aa < bb ? -1 : !!(aa > bb); +} + +/* + * Print a trimmed zero converted ns to us, without automatic scientific + * notation like %g. + * e.g. 10000 ns -> 10 us + * e.g. 10001 ns -> 10.001 us + * e.g. 10100 ns -> 10.1 us + */ +static char *print_us(char *buf, u64 ns) +{ + int len; + + sprintf(buf, "%"PRIu64, ns / 1000); + if (ns % 1000 =3D=3D 0) + return buf; + + sprintf(buf + strlen(buf), ".%03"PRIu64, ns % 1000); + len =3D strlen(buf); + while (len && buf[len - 1] =3D=3D '0') + buf[--len] =3D '\0'; + + return buf; +} + +/* + * Print statistics of writer durations. + * + * This function is derived from + * tools/testing/selftests/rcutorture/bin/kvm-recheck-rcuscale.sh + * Note that the durations array are in nanoseconds, and are integers. + */ +static void print_writer_duration_stats(const struct durations *d) +{ + size_t pct50, pct90, pct99; + size_t count; + u64 div, last; + char ms_us_buf[30]; + u64 *durations =3D d->values; + size_t durations_count =3D d->count; + + if (durations_count =3D=3D 0) { + printf("No rcuscale records found.\n"); + return; + } + + qsort(durations, durations_count, sizeof(*durations), compare_duration); + + // Calculate percentiles + pct50 =3D max(durations_count * 50 / 100, 1UL); + pct90 =3D max(durations_count * 90 / 100, 1UL); + pct99 =3D max(durations_count * 99 / 100, 1UL); + +#define US_NS 1000 +#define us(ns) print_us(ms_us_buf, (ns)) + + if (show_hist) { + // Calculate histogram bucket size based on 90th percentile + div =3D pow(10, floor(log10((double)durations[pct90 - 1]) + 0.5)) / 100; + if (div <=3D 0) + div =3D 1; + printf("Histogram bucket size: %s microseconds\n", us(div)); + + last =3D durations[0] - 10 * US_NS; + count =3D 0; + for (size_t i =3D 0; i < durations_count; ++i) { + u64 current =3D durations[i] / div * div; + + if (last =3D=3D current) { + count++; + } else { + if (count > 0) + printf("%s %lu\n", us(last), count); + count =3D 1; + last =3D current; + } + } + if (count > 0) + printf("%s %lu\n", us(last), count); + } + + // Print statistics + printf("Average grace-period duration: %s microseconds\n", us(d->sum / du= rations_count)); + printf("Minimum grace-period duration: %s\n", us(durations[0])); + printf("50th percentile grace-period duration: %s\n", us(durations[pct50 = - 1])); + printf("90th percentile grace-period duration: %s\n", us(durations[pct90 = - 1])); + printf("99th percentile grace-period duration: %s\n", us(durations[pct99 = - 1])); + printf("Maximum grace-period duration: %s\n", us(durations[durations_coun= t - 1])); + +#undef US_NS +#undef us +} + +/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D Experiment Functions =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D */ + +/* + * Trigger the experiment by writing 1 to should_start. + */ +static void start_experiment(void) +{ + char path[PATH_MAX]; + FILE *fp; + + snprintf(path, sizeof(path), "%s/rcuscale/should_start", debugfs); + + fp =3D fopen(path, "w"); + if (!fp) + err(EXIT_FAILURE, "Failed to open %s", path); + + if (fprintf(fp, "1\n") < 0) + err(EXIT_FAILURE, "Failed to write %s", path); + + fclose(fp); +} + +/* + * Wait for the experiment to complete by reading test_complete once every + * second, until it is not 0. + */ +static void wait_experiment(void) +{ + char path[PATH_MAX]; + + snprintf(path, sizeof(path), "%s/rcuscale/test_complete", debugfs); + + while (true) { + int finished; + FILE *fp =3D fopen(path, "r"); + + if (!fp) + err(EXIT_FAILURE, "Failed to open %s", path); + + if (fscanf(fp, "%d", &finished) !=3D 1) { + fclose(fp); + err(EXIT_FAILURE, "Failed to read %s", path); + } + + fclose(fp); + + if (finished) + break; + + sleep(1); + } +} + +/* + * Run the constructed modprobe command. + */ +static void run_modprobe(const struct modprobe_cmd *cmd) +{ + int retval; + pid_t pid; + + if (dryrun) + return; + + pid =3D fork(); + if (pid < 0) + err(EXIT_FAILURE, "Failed to fork child process"); + + if (pid =3D=3D 0) { + execvp(cmd->cmd[0], (char *const *)cmd->cmd); + in_child =3D true; + err(EXIT_FAILURE, "Failed to execute modprobe command"); + } + waitpid(pid, &retval, 0); + if (retval) + fail("modprobe failed, exiting."); +} + +/* + * Print modprobe parameters, but skip the base command line, and also skip + * those overridable params not overridden by user. + */ +static void print_params(const struct modprobe_cmd *cmd) +{ + bool printed =3D false; + char keybuf[MAX_OPTNAME]; + struct overridable_param *param; + + printf("Running experiment with options:"); + for (int i =3D MODPROBE_BASE_COUNT; cmd->cmd[i] !=3D NULL; ++i) { + if (sscanf(cmd->cmd[i], "%[^=3D]=3D", keybuf) !=3D 1) + fail("Invalid generated modprobe parameter: %s", cmd->cmd[i]); + param =3D overridable_param_get(keybuf); + if (param =3D=3D NULL || param->user_overridden) { + printed =3D true; + printf(" %s", cmd->cmd[i]); + } + } + if (!printed) + printf(" (default)\n"); + else + printf("\n"); +} + +/* + * Core Experiment function + */ +static void runonce(const struct modprobe_cmd *modprobe_cmd) +{ + struct durations *durations; + + print_params(modprobe_cmd); + run_modprobe(modprobe_cmd); + + if (dryrun) + return; + + /* Start and wait for experiment */ + start_experiment(); + wait_experiment(); + + /* Parse writer durations */ + /* Wait until all kernel threads enter final wait */ + sleep(1); + durations =3D parse_durations(); + unload_module(); + + printf("Experiment finished.\n"); + + /* Print statistics */ + print_writer_duration_stats(durations); + free_durations(durations); +} + +static void modprobe_cmd_add(struct modprobe_cmd *cmd, const char *v) +{ + // 2 for NULL and v + if (cmd->count + 2 >=3D MODPROBE_CMD_MAX) + fail("Too many module parameters"); + cmd->cmd[cmd->count] =3D v; + cmd->cmd[++cmd->count] =3D NULL; +} + +/* + * Append parameters that are overridable by users. + */ +static void modprobe_cmd_add_overridable(struct modprobe_cmd *cmd) +{ + if (!param_has_conflict("writer_no_print")) + modprobe_cmd_add(cmd, "writer_no_print=3D1"); + if (!param_has_conflict("holdoff")) + modprobe_cmd_add(cmd, "holdoff=3D3"); +} + +/* + * Collect simple options into modprobe_cmd. + */ +static void modprobe_collect_simple_options(struct modprobe_cmd *cmd) +{ + for (int i =3D 0; i < simple_params_count; ++i) + modprobe_cmd_add(cmd, simple_params[i].value); + + modprobe_cmd_add_overridable(cmd); +} + +/* + * Test once. Does not allow ranges. + */ +static void test_once(int argc, const char *argv[]) +{ + MODPROBE_CMD_INIT; + + parse_module_params(argc, argv); + + modprobe_collect_simple_options(&modprobe_cmd); + + runonce(&modprobe_cmd); +} + +/* =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D Entry Point =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D */ + +int bench_sync_rcu(int argc, const char **argv) +{ + void (*cmd)(int argc, const char *argv[]); + const char *runmode, *gp_type; + + /* Reset errno to avoid printing irrelavent error string */ + errno =3D 0; + + /* Parse global options first. */ + argc =3D parse_options(argc, argv, bench_rcu_options, bench_rcu_usage, + PARSE_OPT_STOP_AT_NON_OPTION); + + /* The empty case is equivalent to 'once sync'. + * Otherwise, at least two positional options are required: + * once/range/ratio and sync/async/exp + */ + if (argc =3D=3D 0) { + runmode =3D "once"; + gp_type =3D "sync"; + } else if (argc < 2) { + usage_with_options(bench_rcu_usage, bench_rcu_options); + } else { + runmode =3D argv[0]; + gp_type =3D argv[1]; + argc -=3D 2; + argv +=3D 2; + } + + if (strcmp(runmode, "once") =3D=3D 0) + cmd =3D test_once; + else + usage_with_options(bench_rcu_usage, bench_rcu_options); + + parse_gp_type(gp_type); + + parse_modinfo(); + if (system(MODPROBE_REMOVE_CMD) !=3D 0) + err(EXIT_FAILURE, "Unloading existing rcuscale module failed"); + + setup_cleanup(); + + cmd(argc, argv); + + return 0; +} diff --git a/tools/perf/builtin-bench.c b/tools/perf/builtin-bench.c index 8d945b846321..9d2e765c7e16 100644 --- a/tools/perf/builtin-bench.c +++ b/tools/perf/builtin-bench.c @@ -55,6 +55,7 @@ static struct bench sched_benchmarks[] =3D { static struct bench sync_benchmarks[] =3D { { "qspinlock", "Benchmark for queued spinlock", bench_sync_qspinlock }, { "ticket", "Benchmark for ticket spinlock", bench_sync_ticket }, + { "rcu", "Benchmark using rcuscale kernel module", bench_sync_rcu }, { "all", "Run all synchronization benchmarks", NULL }, { NULL, NULL, NULL } }; --=20 2.50.1.565.gc32cd1483b-goog