From nobody Sat Apr 11 02:17:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CB7EC32771 for ; Wed, 17 Aug 2022 15:30:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240456AbiHQPaL (ORCPT ); Wed, 17 Aug 2022 11:30:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41630 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237258AbiHQPaI (ORCPT ); Wed, 17 Aug 2022 11:30:08 -0400 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EF778E0CB for ; Wed, 17 Aug 2022 08:30:07 -0700 (PDT) Received: by mail-pl1-x64a.google.com with SMTP id m5-20020a170902f64500b0016d313f3ce7so8518326plg.23 for ; Wed, 17 Aug 2022 08:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:mime-version:message-id:date:from:to:cc; bh=Qkmccm/z1P8KkL3+STm3aMUEd1Pe32aPYrZdxb0HPrk=; b=Pxvb+ZGChsqiwkr49DHWgMfeYtE2fVHPO36oj7Mj0yUC8eUwQdp7IjUqkfT1BNZ1UB M4sIWk9NkTVEvEOiC0iKLTNnbm3rzh2hMATrXGt7PwnFFbsK8+8ZGq4jj/wi8I0KMbQD sjncZTQ/G6xgxuv+qHsD++IiDXwEVPNHcCLy1VmWSZZHaBKO27XELIirIAl/V1Mq1L8L b0fNNUJ2Jfb7bl6m13muowxadQcJh3BUKLCzYOgRFn93RajEKKsnSpX1t4MXm16EGfIf 2kg6bEX843eHdN+bkC5AXPBYt4l6iBCCtJ7Is/ZvSBgj+CijmVAqDgINAPFNNRSD5Bq3 cASg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:mime-version:message-id:date:x-gm-message-state :from:to:cc; bh=Qkmccm/z1P8KkL3+STm3aMUEd1Pe32aPYrZdxb0HPrk=; b=uKGUlUYbfnLENWYH8VycQ31InGCMzXbKkHld6UHhLJvIt9TphhRTPtCqUMZTUrl9XU XaoMhUx+8qj8DKRxsiEaFvTOTGgPcc13Gf8lO0+833R/kXlXlY6pG1865s4Fcm5HtC9X NWjL65h1k8mTok1QKNGCLQwU0Cs0YrSvnZKm3t8tFr7lr3WvJo8/EUf4MeSOZIkTSFlm J88s+uaweIRNlx8K3ejDSVpi+RnvptwICY8QaDGmN07u/b4ax7yKZU0ZvMK6GhaRVcBo Fwi7CcCvThpWSAhSlsHyOOGE/cRDUJ2KQ0idlKZ+sA7ljPt0afDzicRrCUiodRZ9ZBeD CeHA== X-Gm-Message-State: ACgBeo0eNU7LyXT0t3aKo94/kbr6IrxizMC3M2z88oqaEhmyN5dKnoki SalV00oL9esg7YFVyJzYosyVbVqdVZDy X-Google-Smtp-Source: AA6agR4Vl9pQRf7lXoJiti0d08utPaH3IyHkVgCjs20FK59QowdbfkNojkBF45P/JBKh88yoYDfDR9Q2WAB9 X-Received: from vipin.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:479f]) (user=vipinsh job=sendgmr) by 2002:a17:902:788f:b0:170:8b18:8812 with SMTP id q15-20020a170902788f00b001708b188812mr26741589pll.1.1660750206813; Wed, 17 Aug 2022 08:30:06 -0700 (PDT) Date: Wed, 17 Aug 2022 08:29:56 -0700 Message-Id: <20220817152956.4056410-1-vipinsh@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.37.1.595.g718a3a8f04-goog Subject: [PATCH] KVM: selftests: Run dirty_log_perf_test on specific cpus From: Vipin Sharma To: seanjc@google.com, dmatlack@google.com, pbonzini@redhat.com Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Vipin Sharma Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add command line options to run the vcpus and the main process on the specific cpus on a host machine. This is useful as it provides options to analyze performance based on the vcpus and dirty log worker locations, like on the different numa nodes or on the same numa nodes. Signed-off-by: Vipin Sharma Suggested-by: David Matlack Suggested-by: Sean Christopherson Suggested-by: Paolo Bonzini --- This is based on the discussion at https://lore.kernel.org/lkml/20220801151928.270380-1-vipinsh@google.com/ .../selftests/kvm/access_tracking_perf_test.c | 2 +- .../selftests/kvm/demand_paging_test.c | 2 +- .../selftests/kvm/dirty_log_perf_test.c | 108 +++++++++++++++++- .../selftests/kvm/include/perf_test_util.h | 3 +- .../selftests/kvm/lib/perf_test_util.c | 32 +++++- .../kvm/memslot_modification_stress_test.c | 2 +- 6 files changed, 139 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/kvm/access_tracking_perf_test.c b/tool= s/testing/selftests/kvm/access_tracking_perf_test.c index 1c2749b1481ac..9659462f47478 100644 --- a/tools/testing/selftests/kvm/access_tracking_perf_test.c +++ b/tools/testing/selftests/kvm/access_tracking_perf_test.c @@ -299,7 +299,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) vm =3D perf_test_create_vm(mode, nr_vcpus, params->vcpu_memory_bytes, 1, params->backing_src, !overlap_memory_access); =20 - perf_test_start_vcpu_threads(nr_vcpus, vcpu_thread_main); + perf_test_start_vcpu_threads(nr_vcpus, NULL, vcpu_thread_main); =20 pr_info("\n"); access_memory(vm, nr_vcpus, ACCESS_WRITE, "Populating memory"); diff --git a/tools/testing/selftests/kvm/demand_paging_test.c b/tools/testi= ng/selftests/kvm/demand_paging_test.c index 779ae54f89c40..b9848174d6e7c 100644 --- a/tools/testing/selftests/kvm/demand_paging_test.c +++ b/tools/testing/selftests/kvm/demand_paging_test.c @@ -336,7 +336,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) pr_info("Finished creating vCPUs and starting uffd threads\n"); =20 clock_gettime(CLOCK_MONOTONIC, &start); - perf_test_start_vcpu_threads(nr_vcpus, vcpu_worker); + perf_test_start_vcpu_threads(nr_vcpus, NULL, vcpu_worker); pr_info("Started all vCPUs\n"); =20 perf_test_join_vcpu_threads(nr_vcpus); diff --git a/tools/testing/selftests/kvm/dirty_log_perf_test.c b/tools/test= ing/selftests/kvm/dirty_log_perf_test.c index f99e39a672d3f..68a1d7262f21b 100644 --- a/tools/testing/selftests/kvm/dirty_log_perf_test.c +++ b/tools/testing/selftests/kvm/dirty_log_perf_test.c @@ -8,10 +8,12 @@ * Copyright (C) 2020, Google, Inc. */ =20 +#define _GNU_SOURCE #include #include #include #include +#include #include =20 #include "kvm_util.h" @@ -66,6 +68,8 @@ static u64 dirty_log_manual_caps; static bool host_quit; static int iteration; static int vcpu_last_completed_iteration[KVM_MAX_VCPUS]; +/* Map vcpus to logical cpus on host. */ +static int vcpu_to_lcpu_map[KVM_MAX_VCPUS]; =20 static void vcpu_worker(struct perf_test_vcpu_args *vcpu_args) { @@ -132,6 +136,7 @@ struct test_params { bool partition_vcpu_memory_access; enum vm_mem_backing_src_type backing_src; int slots; + int *vcpu_to_lcpu; }; =20 static void toggle_dirty_logging(struct kvm_vm *vm, int slots, bool enable) @@ -248,7 +253,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) for (i =3D 0; i < nr_vcpus; i++) vcpu_last_completed_iteration[i] =3D -1; =20 - perf_test_start_vcpu_threads(nr_vcpus, vcpu_worker); + perf_test_start_vcpu_threads(nr_vcpus, p->vcpu_to_lcpu, vcpu_worker); =20 /* Allow the vCPUs to populate memory */ pr_debug("Starting iteration %d - Populating\n", iteration); @@ -348,12 +353,74 @@ static void run_test(enum vm_guest_mode mode, void *a= rg) perf_test_destroy_vm(vm); } =20 +static int parse_num(const char *num_str) +{ + int num; + char *end_ptr; + + errno =3D 0; + num =3D (int)strtol(num_str, &end_ptr, 10); + TEST_ASSERT(num_str !=3D end_ptr && *end_ptr =3D=3D '\0', + "Invalid number string.\n"); + TEST_ASSERT(errno =3D=3D 0, "Conversion error: %d\n", errno); + + return num; +} + +static int parse_cpu_list(const char *arg) +{ + char delim[2] =3D ","; + char *cpu, *cpu_list; + int i =3D 0, cpu_num; + + cpu_list =3D strdup(arg); + TEST_ASSERT(cpu_list, "Low memory\n"); + + cpu =3D strtok(cpu_list, delim); + while (cpu) { + cpu_num =3D parse_num(cpu); + TEST_ASSERT(cpu_num >=3D 0, "Invalid cpu number: %d\n", cpu_num); + vcpu_to_lcpu_map[i++] =3D cpu_num; + cpu =3D strtok(NULL, delim); + } + + free(cpu_list); + + return i; +} + +static void assign_dirty_log_perf_test_cpu(const char *arg) +{ + char delim[2] =3D ","; + char *cpu, *cpu_list; + cpu_set_t cpuset; + int cpu_num, err; + + cpu_list =3D strdup(arg); + TEST_ASSERT(cpu_list, "Low memory\n"); + + CPU_ZERO(&cpuset); + cpu =3D strtok(cpu_list, delim); + while (cpu) { + cpu_num =3D parse_num(cpu); + TEST_ASSERT(cpu_num >=3D 0, "Invalid cpu number: %d\n", cpu_num); + CPU_SET(cpu_num, &cpuset); + cpu =3D strtok(NULL, delim); + } + + err =3D sched_setaffinity(0, sizeof(cpu_set_t), &cpuset); + TEST_ASSERT(err =3D=3D 0, "Error in setting dirty log perf test cpu\n"); + + free(cpu_list); +} + static void help(char *name) { puts(""); printf("usage: %s [-h] [-i iterations] [-p offset] [-g] " "[-m mode] [-n] [-b vcpu bytes] [-v vcpus] [-o] [-s mem type]" - "[-x memslots]\n", name); + "[-x memslots] [-c logical cpus for vcpus]" + "[-d cpus to run dirty_log_perf_test]\n", name); puts(""); printf(" -i: specify iteration counts (default: %"PRIu64")\n", TEST_HOST_LOOP_N); @@ -383,6 +450,26 @@ static void help(char *name) backing_src_help("-s"); printf(" -x: Split the memory region into this number of memslots.\n" " (default: 1)\n"); + printf(" -c: Comma separated values of the logical CPUs which will run\n" + " the vCPUs. Number of values should be equal to the number\n" + " of vCPUs.\n\n" + " Example: ./dirty_log_perf_test -v 3 -c 22,43,1\n" + " This means that the vcpu 0 will run on the logical cpu 22,\n" + " vcpu 1 on the logical cpu 43 and vcpu 2 on the logical cpu 1= .\n" + " (default: No cpu mapping)\n\n"); + printf(" -d: Comma separated values of the logical CPUs on which\n" + " dirty_log_perf_test will run. Without -c option, all of\n" + " the vcpus and main process will run on the cpus provided her= e.\n" + " This option also accepts a single cpu. (default: No cpu mapp= ing)\n\n" + " Example 1: ./dirty_log_perf_test -v 3 -c 22,43,1 -d 101\n" + " Main application thread will run on logical cpu 101 and\n" + " vcpus will run on the logical cpus 22, 43 and 1\n\n" + " Example 2: ./dirty_log_perf_test -v 3 -d 101\n" + " Main application thread and vcpus will run on the logical\n" + " cpu 101\n\n" + " Example 3: ./dirty_log_perf_test -v 3 -d 101,23,53\n" + " Main application thread and vcpus will run on logical cpus\n" + " 101, 23 and 53.\n"); puts(""); exit(0); } @@ -396,8 +483,10 @@ int main(int argc, char *argv[]) .partition_vcpu_memory_access =3D true, .backing_src =3D DEFAULT_VM_MEM_SRC, .slots =3D 1, + .vcpu_to_lcpu =3D NULL, }; int opt; + int nr_lcpus =3D -1; =20 dirty_log_manual_caps =3D kvm_check_cap(KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2); @@ -406,8 +495,14 @@ int main(int argc, char *argv[]) =20 guest_modes_append_default(); =20 - while ((opt =3D getopt(argc, argv, "eghi:p:m:nb:f:v:os:x:")) !=3D -1) { + while ((opt =3D getopt(argc, argv, "c:d:eghi:p:m:nb:f:v:os:x:")) !=3D -1)= { switch (opt) { + case 'c': + nr_lcpus =3D parse_cpu_list(optarg); + break; + case 'd': + assign_dirty_log_perf_test_cpu(optarg); + break; case 'e': /* 'e' is for evil. */ run_vcpus_while_disabling_dirty_logging =3D true; @@ -455,6 +550,13 @@ int main(int argc, char *argv[]) } } =20 + if (nr_lcpus !=3D -1) { + TEST_ASSERT(nr_lcpus =3D=3D nr_vcpus, + "Number of vCPUs (%d) are not equal to number of logical cpus provi= ded (%d).", + nr_vcpus, nr_lcpus); + p.vcpu_to_lcpu =3D vcpu_to_lcpu_map; + } + TEST_ASSERT(p.iterations >=3D 2, "The test should have at least two itera= tions"); =20 pr_info("Test iterations: %"PRIu64"\n", p.iterations); diff --git a/tools/testing/selftests/kvm/include/perf_test_util.h b/tools/t= esting/selftests/kvm/include/perf_test_util.h index eaa88df0555a8..bd6c566cfc92e 100644 --- a/tools/testing/selftests/kvm/include/perf_test_util.h +++ b/tools/testing/selftests/kvm/include/perf_test_util.h @@ -53,7 +53,8 @@ void perf_test_destroy_vm(struct kvm_vm *vm); =20 void perf_test_set_wr_fract(struct kvm_vm *vm, int wr_fract); =20 -void perf_test_start_vcpu_threads(int vcpus, void (*vcpu_fn)(struct perf_t= est_vcpu_args *)); +void perf_test_start_vcpu_threads(int vcpus, int *vcpus_to_lcpu, + void (*vcpu_fn)(struct perf_test_vcpu_args *)); void perf_test_join_vcpu_threads(int vcpus); void perf_test_guest_code(uint32_t vcpu_id); =20 diff --git a/tools/testing/selftests/kvm/lib/perf_test_util.c b/tools/testi= ng/selftests/kvm/lib/perf_test_util.c index 9618b37c66f7e..771fbdf3d2c22 100644 --- a/tools/testing/selftests/kvm/lib/perf_test_util.c +++ b/tools/testing/selftests/kvm/lib/perf_test_util.c @@ -2,11 +2,14 @@ /* * Copyright (C) 2020, Google LLC. */ +#define _GNU_SOURCE #include =20 #include "kvm_util.h" #include "perf_test_util.h" #include "processor.h" +#include +#include =20 struct perf_test_args perf_test_args; =20 @@ -260,10 +263,15 @@ static void *vcpu_thread_main(void *data) return NULL; } =20 -void perf_test_start_vcpu_threads(int nr_vcpus, +void perf_test_start_vcpu_threads(int nr_vcpus, int *vcpu_to_lcpu, void (*vcpu_fn)(struct perf_test_vcpu_args *)) { - int i; + int i, err =3D 0; + pthread_attr_t attr; + cpu_set_t cpuset; + + pthread_attr_init(&attr); + CPU_ZERO(&cpuset); =20 vcpu_thread_fn =3D vcpu_fn; WRITE_ONCE(all_vcpu_threads_running, false); @@ -274,7 +282,24 @@ void perf_test_start_vcpu_threads(int nr_vcpus, vcpu->vcpu_idx =3D i; WRITE_ONCE(vcpu->running, false); =20 - pthread_create(&vcpu->thread, NULL, vcpu_thread_main, vcpu); + if (vcpu_to_lcpu) { + CPU_SET(vcpu_to_lcpu[i], &cpuset); + + err =3D pthread_attr_setaffinity_np(&attr, + sizeof(cpu_set_t), + &cpuset); + TEST_ASSERT(err =3D=3D 0, + "vCPU %d could not be mapped to logical cpu %d, error returned: %d= \n", + i, vcpu_to_lcpu[i], err); + + CPU_CLR(vcpu_to_lcpu[i], &cpuset); + } + + err =3D pthread_create(&vcpu->thread, &attr, vcpu_thread_main, + vcpu); + TEST_ASSERT(err =3D=3D 0, + "error in creating vcpu %d thread, error returned: %d\n", + i, err); } =20 for (i =3D 0; i < nr_vcpus; i++) { @@ -283,6 +308,7 @@ void perf_test_start_vcpu_threads(int nr_vcpus, } =20 WRITE_ONCE(all_vcpu_threads_running, true); + pthread_attr_destroy(&attr); } =20 void perf_test_join_vcpu_threads(int nr_vcpus) diff --git a/tools/testing/selftests/kvm/memslot_modification_stress_test.c= b/tools/testing/selftests/kvm/memslot_modification_stress_test.c index 6ee7e1dde4043..246f8cc7bb2b1 100644 --- a/tools/testing/selftests/kvm/memslot_modification_stress_test.c +++ b/tools/testing/selftests/kvm/memslot_modification_stress_test.c @@ -103,7 +103,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) =20 pr_info("Finished creating vCPUs\n"); =20 - perf_test_start_vcpu_threads(nr_vcpus, vcpu_worker); + perf_test_start_vcpu_threads(nr_vcpus, NULL, vcpu_worker); =20 pr_info("Started all vCPUs\n"); =20 --=20 2.37.1.595.g718a3a8f04-goog