From nobody Fri Jun 19 07:16:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98FD0C433EF for ; Wed, 6 Apr 2022 19:33:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231986AbiDFTfi (ORCPT ); Wed, 6 Apr 2022 15:35:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56262 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231968AbiDFTfY (ORCPT ); Wed, 6 Apr 2022 15:35:24 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3D1BA1B60A0; Wed, 6 Apr 2022 10:51:43 -0700 (PDT) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 236FWn9D032195; Wed, 6 Apr 2022 17:51:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=OFAfPY3hhK/weS7YMTSyokrhHm3RHnUzOfo894HLQkY=; b=gfIgmelQvvRMJqziW1LgeCek8Z721WYeh4R8BVJxB8mrOYy6oeJVWfNSsBTqO/741XUT llu4uOGzgdyxWTXsPVSXi5Gxx/xMq9PWTTXm/QFwchd+ve/ByyCz8e8mGwJcrh4gkyls +320GRtFv+cv1LeU32kPjPvOfZiMVSR/EOlkeZGmMMdaZuxp1HtvwRJsY1b1CXKt+7yS rIJlI2edvb97bowG+gNcxNO6+MZQl8zjPj9jpZCFgYNeXcBf71Kae0EHkvLAdEtk1+YS rAeGkcdyEdxZ9tDO/BAp8bA1j6c1RjNseCa8l/wVy+ZPTGHACO776kEkYFBpsnMlO38q sw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 3f977bunbg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:34 +0000 Received: from m0098414.ppops.net (m0098414.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 236HgW4g028102; Wed, 6 Apr 2022 17:51:33 GMT Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0b-001b2d01.pphosted.com with ESMTP id 3f977bunb7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:33 +0000 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 236Hhvcq003113; Wed, 6 Apr 2022 17:51:32 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma01fra.de.ibm.com with ESMTP id 3f6e48xtr9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:31 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 236HpSFu47513908 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 6 Apr 2022 17:51:28 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 300EF4C044; Wed, 6 Apr 2022 17:51:28 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BAB8F4C040; Wed, 6 Apr 2022 17:51:22 +0000 (GMT) Received: from localhost.localdomain (unknown [9.211.90.125]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 6 Apr 2022 17:51:22 +0000 (GMT) From: Athira Rajeev To: acme@kernel.org, jolsa@kernel.org, disgoel@linux.vnet.ibm.com Cc: mpe@ellerman.id.au, linux-perf-users@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.vnet.ibm.com, rnsastry@linux.ibm.com, kjain@linux.ibm.com, linux-kernel@vger.kernel.org, srikar@linux.vnet.ibm.com, irogers@google.com Subject: [PATCH v2 1/4] tools/perf: Fix perf bench futex to correct usage of affinity for machines with #CPUs > 1K Date: Wed, 6 Apr 2022 23:21:10 +0530 Message-Id: <20220406175113.87881-2-atrajeev@linux.vnet.ibm.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220406175113.87881-1-atrajeev@linux.vnet.ibm.com> References: <20220406175113.87881-1-atrajeev@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 92SeOSS3GSzJb9fxzN6HztPdygV3V_ke X-Proofpoint-ORIG-GUID: DV6XDUT-LMXwOtSyY6beI8q_2edOxXKf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.850,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-04-06_09,2022-04-06_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 impostorscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 phishscore=0 suspectscore=0 mlxscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204060087 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" perf bench futex testcase fails on systems with CPU's more than 1K. Testcase: perf bench futex all Failure snippet: <<>>Running futex/hash benchmark... perf: pthread_create: No such file or directory <<>> All the futex benchmarks ( ie hash, lock-api, requeue, wake, wake-parallel ), pthread_create is invoked in respective bench_futex_* function. Though the logs shows direct failure from pthread_create, strace logs showed that actual failure is from "sched_setaffinity" returning EINVAL (invalid argument). This happens because the default mask size in glibc is 1024. To overcome this 1024 CPUs mask size limitation of cpu_set_t, change the mask size using the CPU_*_S macros. Patch addresses this by fixing all the futex benchmarks to use CPU_ALLOC to allocate cpumask, CPU_ALLOC_SIZE for size, and CPU_SET_S to set the mask. Tested-by: Disha Goel Signed-off-by: Athira Rajeev Reported-by: Disha Goel Acked-by: Ian Rogers Reviewed-by: Srikar Dronamraju --- tools/perf/bench/futex-hash.c | 26 +++++++++++++++++++------- tools/perf/bench/futex-lock-pi.c | 21 ++++++++++++++++----- tools/perf/bench/futex-requeue.c | 21 ++++++++++++++++----- tools/perf/bench/futex-wake-parallel.c | 21 ++++++++++++++++----- tools/perf/bench/futex-wake.c | 22 ++++++++++++++++------ 5 files changed, 83 insertions(+), 28 deletions(-) diff --git a/tools/perf/bench/futex-hash.c b/tools/perf/bench/futex-hash.c index 9627b6ab8670..dfce64e551e2 100644 --- a/tools/perf/bench/futex-hash.c +++ b/tools/perf/bench/futex-hash.c @@ -122,12 +122,14 @@ static void print_summary(void) int bench_futex_hash(int argc, const char **argv) { int ret =3D 0; - cpu_set_t cpuset; + cpu_set_t *cpuset; struct sigaction act; unsigned int i; pthread_attr_t thread_attr; struct worker *worker =3D NULL; struct perf_cpu_map *cpu; + int nrcpus; + size_t size; =20 argc =3D parse_options(argc, argv, options, bench_futex_hash_usage, 0); if (argc) { @@ -170,25 +172,35 @@ int bench_futex_hash(int argc, const char **argv) threads_starting =3D params.nthreads; pthread_attr_init(&thread_attr); gettimeofday(&bench__start, NULL); + + nrcpus =3D perf_cpu_map__nr(cpu); + cpuset =3D CPU_ALLOC(nrcpus); + BUG_ON(!cpuset); + size =3D CPU_ALLOC_SIZE(nrcpus); + for (i =3D 0; i < params.nthreads; i++) { worker[i].tid =3D i; worker[i].futex =3D calloc(params.nfutexes, sizeof(*worker[i].futex)); if (!worker[i].futex) goto errmem; =20 - CPU_ZERO(&cpuset); - CPU_SET(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, &cpuset); + CPU_ZERO_S(size, cpuset); =20 - ret =3D pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cp= uset); - if (ret) + CPU_SET_S(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, size, c= puset); + ret =3D pthread_attr_setaffinity_np(&thread_attr, size, cpuset); + if (ret) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); - + } ret =3D pthread_create(&worker[i].thread, &thread_attr, workerfn, (void *)(struct worker *) &worker[i]); - if (ret) + if (ret) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_create"); + } =20 } + CPU_FREE(cpuset); pthread_attr_destroy(&thread_attr); =20 pthread_mutex_lock(&thread_lock); diff --git a/tools/perf/bench/futex-lock-pi.c b/tools/perf/bench/futex-lock= -pi.c index a512a320df74..61c3bb80d4cf 100644 --- a/tools/perf/bench/futex-lock-pi.c +++ b/tools/perf/bench/futex-lock-pi.c @@ -120,11 +120,17 @@ static void *workerfn(void *arg) static void create_threads(struct worker *w, pthread_attr_t thread_attr, struct perf_cpu_map *cpu) { - cpu_set_t cpuset; + cpu_set_t *cpuset; unsigned int i; + int nrcpus =3D perf_cpu_map__nr(cpu); + size_t size; =20 threads_starting =3D params.nthreads; =20 + cpuset =3D CPU_ALLOC(nrcpus); + BUG_ON(!cpuset); + size =3D CPU_ALLOC_SIZE(nrcpus); + for (i =3D 0; i < params.nthreads; i++) { worker[i].tid =3D i; =20 @@ -135,15 +141,20 @@ static void create_threads(struct worker *w, pthread_= attr_t thread_attr, } else worker[i].futex =3D &global_futex; =20 - CPU_ZERO(&cpuset); - CPU_SET(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, &cpuset); + CPU_ZERO_S(size, cpuset); + CPU_SET_S(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, size, c= puset); =20 - if (pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cpuset= )) + if (pthread_attr_setaffinity_np(&thread_attr, size, cpuset)) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); + } =20 - if (pthread_create(&w[i].thread, &thread_attr, workerfn, &worker[i])) + if (pthread_create(&w[i].thread, &thread_attr, workerfn, &worker[i])) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_create"); + } } + CPU_FREE(cpuset); } =20 int bench_futex_lock_pi(int argc, const char **argv) diff --git a/tools/perf/bench/futex-requeue.c b/tools/perf/bench/futex-requ= eue.c index aca47ce8b1e7..2cb013f7ffe5 100644 --- a/tools/perf/bench/futex-requeue.c +++ b/tools/perf/bench/futex-requeue.c @@ -123,22 +123,33 @@ static void *workerfn(void *arg __maybe_unused) static void block_threads(pthread_t *w, pthread_attr_t thread_attr, struct perf_cpu_map *cpu) { - cpu_set_t cpuset; + cpu_set_t *cpuset; unsigned int i; + int nrcpus =3D perf_cpu_map__nr(cpu); + size_t size; =20 threads_starting =3D params.nthreads; =20 + cpuset =3D CPU_ALLOC(nrcpus); + BUG_ON(!cpuset); + size =3D CPU_ALLOC_SIZE(nrcpus); + /* create and block all threads */ for (i =3D 0; i < params.nthreads; i++) { - CPU_ZERO(&cpuset); - CPU_SET(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, &cpuset); + CPU_ZERO_S(size, cpuset); + CPU_SET_S(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, size, c= puset); =20 - if (pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cpuset= )) + if (pthread_attr_setaffinity_np(&thread_attr, size, cpuset)) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); + } =20 - if (pthread_create(&w[i], &thread_attr, workerfn, NULL)) + if (pthread_create(&w[i], &thread_attr, workerfn, NULL)) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_create"); + } } + CPU_FREE(cpuset); } =20 static void toggle_done(int sig __maybe_unused, diff --git a/tools/perf/bench/futex-wake-parallel.c b/tools/perf/bench/fute= x-wake-parallel.c index 888ee6037945..efa5070a5eb3 100644 --- a/tools/perf/bench/futex-wake-parallel.c +++ b/tools/perf/bench/futex-wake-parallel.c @@ -144,22 +144,33 @@ static void *blocked_workerfn(void *arg __maybe_unuse= d) static void block_threads(pthread_t *w, pthread_attr_t thread_attr, struct perf_cpu_map *cpu) { - cpu_set_t cpuset; + cpu_set_t *cpuset; unsigned int i; + int nrcpus =3D perf_cpu_map__nr(cpu); + size_t size; =20 threads_starting =3D params.nthreads; =20 + cpuset =3D CPU_ALLOC(nrcpus); + BUG_ON(!cpuset); + size =3D CPU_ALLOC_SIZE(nrcpus); + /* create and block all threads */ for (i =3D 0; i < params.nthreads; i++) { - CPU_ZERO(&cpuset); - CPU_SET(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, &cpuset); + CPU_ZERO_S(size, cpuset); + CPU_SET_S(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, size, c= puset); =20 - if (pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cpuset= )) + if (pthread_attr_setaffinity_np(&thread_attr, size, cpuset)) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); + } =20 - if (pthread_create(&w[i], &thread_attr, blocked_workerfn, NULL)) + if (pthread_create(&w[i], &thread_attr, blocked_workerfn, NULL)) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_create"); + } } + CPU_FREE(cpuset); } =20 static void print_run(struct thread_data *waking_worker, unsigned int run_= num) diff --git a/tools/perf/bench/futex-wake.c b/tools/perf/bench/futex-wake.c index aa82db51c0ab..3a10f54900c1 100644 --- a/tools/perf/bench/futex-wake.c +++ b/tools/perf/bench/futex-wake.c @@ -97,22 +97,32 @@ static void print_summary(void) static void block_threads(pthread_t *w, pthread_attr_t thread_attr, struct perf_cpu_map *cpu) { - cpu_set_t cpuset; + cpu_set_t *cpuset; unsigned int i; - + size_t size; + int nrcpus =3D perf_cpu_map__nr(cpu); threads_starting =3D params.nthreads; =20 + cpuset =3D CPU_ALLOC(nrcpus); + BUG_ON(!cpuset); + size =3D CPU_ALLOC_SIZE(nrcpus); + /* create and block all threads */ for (i =3D 0; i < params.nthreads; i++) { - CPU_ZERO(&cpuset); - CPU_SET(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, &cpuset); + CPU_ZERO_S(size, cpuset); + CPU_SET_S(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, size, c= puset); =20 - if (pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &cpuset= )) + if (pthread_attr_setaffinity_np(&thread_attr, size, cpuset)) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); + } =20 - if (pthread_create(&w[i], &thread_attr, workerfn, NULL)) + if (pthread_create(&w[i], &thread_attr, workerfn, NULL)) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_create"); + } } + CPU_FREE(cpuset); } =20 static void toggle_done(int sig __maybe_unused, --=20 2.35.1 From nobody Fri Jun 19 07:16:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9E9EC433F5 for ; Wed, 6 Apr 2022 20:13:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234095AbiDFUPn (ORCPT ); Wed, 6 Apr 2022 16:15:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236300AbiDFUNi (ORCPT ); Wed, 6 Apr 2022 16:13:38 -0400 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B18BB60040; Wed, 6 Apr 2022 10:51:48 -0700 (PDT) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 236G8xbb037207; Wed, 6 Apr 2022 17:51:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=8ndA8c5xFaZF+V8VsYWe9PAD5wDhHWHh/EbZ2rHZ214=; b=J1kXuTffGRvpCogdFAz/cqTHO91bY3YQoyGr+A9+PQ8v6/3zKM1PQtzg4VtTOn/0kPW4 LMhSxSnOhOj0J23TVSHMzoMXz/xuKlsCvb4zBUIH/5XEYvFUTT+SsVdWP1QciSfWhjnL 7jwpUjwWbX7toY6ELAY0q5bOJ/tqIzVVMVKaSLjC3ZJlfRN++rqS7zhMhKJOI6xUtjgY rG/fel4dhD1gOrlRWgXreGOGPehh3yLHMGzSMvhs942GWdREXN4Uwiihm9J85/JpOQaM GTKKglXiDfQDeA3gwDfJdBUiPE7xs3nkUy3x6wkAlBFnoRI32ww7MQxAkZfe1pj9j5ty Zw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3f9a9qft54-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:40 +0000 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 236Hgmsd019947; Wed, 6 Apr 2022 17:51:39 GMT Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com with ESMTP id 3f9a9qft4t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:39 +0000 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 236HicXB002932; Wed, 6 Apr 2022 17:51:37 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma06fra.de.ibm.com with ESMTP id 3f6drhpttb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:37 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 236HpYHG41026038 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 6 Apr 2022 17:51:34 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 487404C040; Wed, 6 Apr 2022 17:51:34 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E9C234C046; Wed, 6 Apr 2022 17:51:28 +0000 (GMT) Received: from localhost.localdomain (unknown [9.211.90.125]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 6 Apr 2022 17:51:28 +0000 (GMT) From: Athira Rajeev To: acme@kernel.org, jolsa@kernel.org, disgoel@linux.vnet.ibm.com Cc: mpe@ellerman.id.au, linux-perf-users@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.vnet.ibm.com, rnsastry@linux.ibm.com, kjain@linux.ibm.com, linux-kernel@vger.kernel.org, srikar@linux.vnet.ibm.com, irogers@google.com Subject: [PATCH v2 2/4] tools/perf: Fix perf bench epoll to correct usage of affinity for machines with #CPUs > 1K Date: Wed, 6 Apr 2022 23:21:11 +0530 Message-Id: <20220406175113.87881-3-atrajeev@linux.vnet.ibm.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220406175113.87881-1-atrajeev@linux.vnet.ibm.com> References: <20220406175113.87881-1-atrajeev@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: OVMkAlE1clseF4f9BcFFTUoXeUPF5OVm X-Proofpoint-ORIG-GUID: rvF_PSZj4BgcGV26e9u35LXN93EfB-Vi X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.850,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-04-06_09,2022-04-06_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 malwarescore=0 clxscore=1015 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204060087 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" perf bench epoll testcase fails on systems with CPU's more than 1K. Testcase: perf bench epoll all Result snippet: <<>> Run summary [PID 106497]: 1399 threads monitoring on 64 file-descriptors fo= r 8 secs. perf: pthread_create: No such file or directory <<>> In epoll benchmarks (ctl, wait) pthread_create is invoked in do_threads from respective bench_epoll_* function. Though the logs shows direct failure from pthread_create, the actual failure is from "sched_setaffinity" returning EINVAL (invalid argument). This happens because the default mask size in glibc is 1024. To overcome this 1024 CPUs mask size limitation of cpu_set_t, change the mask size using the CPU_*_S macros. Patch addresses this by fixing all the epoll benchmarks to use CPU_ALLOC to allocate cpumask, CPU_ALLOC_SIZE for size, and CPU_SET_S to set the mask. Tested-by: Disha Goel Signed-off-by: Athira Rajeev Reported-by: Disha Goel Acked-by: Ian Rogers --- tools/perf/bench/epoll-ctl.c | 25 +++++++++++++++++++------ tools/perf/bench/epoll-wait.c | 25 +++++++++++++++++++------ 2 files changed, 38 insertions(+), 12 deletions(-) diff --git a/tools/perf/bench/epoll-ctl.c b/tools/perf/bench/epoll-ctl.c index 1a17ec83d3c4..91c53f6c6d87 100644 --- a/tools/perf/bench/epoll-ctl.c +++ b/tools/perf/bench/epoll-ctl.c @@ -222,13 +222,20 @@ static void init_fdmaps(struct worker *w, int pct) static int do_threads(struct worker *worker, struct perf_cpu_map *cpu) { pthread_attr_t thread_attr, *attrp =3D NULL; - cpu_set_t cpuset; + cpu_set_t *cpuset; unsigned int i, j; int ret =3D 0; + int nrcpus; + size_t size; =20 if (!noaffinity) pthread_attr_init(&thread_attr); =20 + nrcpus =3D perf_cpu_map__nr(cpu); + cpuset =3D CPU_ALLOC(nrcpus); + BUG_ON(!cpuset); + size =3D CPU_ALLOC_SIZE(nrcpus); + for (i =3D 0; i < nthreads; i++) { struct worker *w =3D &worker[i]; =20 @@ -252,22 +259,28 @@ static int do_threads(struct worker *worker, struct p= erf_cpu_map *cpu) init_fdmaps(w, 50); =20 if (!noaffinity) { - CPU_ZERO(&cpuset); - CPU_SET(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, &cpuset); + CPU_ZERO_S(size, cpuset); + CPU_SET_S(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, + size, cpuset); =20 - ret =3D pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &c= puset); - if (ret) + ret =3D pthread_attr_setaffinity_np(&thread_attr, size, cpuset); + if (ret) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); + } =20 attrp =3D &thread_attr; } =20 ret =3D pthread_create(&w->thread, attrp, workerfn, (void *)(struct worker *) w); - if (ret) + if (ret) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_create"); + } } =20 + CPU_FREE(cpuset); if (!noaffinity) pthread_attr_destroy(&thread_attr); =20 diff --git a/tools/perf/bench/epoll-wait.c b/tools/perf/bench/epoll-wait.c index 0d1dd8879197..9469a53ffab9 100644 --- a/tools/perf/bench/epoll-wait.c +++ b/tools/perf/bench/epoll-wait.c @@ -291,9 +291,11 @@ static void print_summary(void) static int do_threads(struct worker *worker, struct perf_cpu_map *cpu) { pthread_attr_t thread_attr, *attrp =3D NULL; - cpu_set_t cpuset; + cpu_set_t *cpuset; unsigned int i, j; int ret =3D 0, events =3D EPOLLIN; + int nrcpus; + size_t size; =20 if (oneshot) events |=3D EPOLLONESHOT; @@ -306,6 +308,11 @@ static int do_threads(struct worker *worker, struct pe= rf_cpu_map *cpu) if (!noaffinity) pthread_attr_init(&thread_attr); =20 + nrcpus =3D perf_cpu_map__nr(cpu); + cpuset =3D CPU_ALLOC(nrcpus); + BUG_ON(!cpuset); + size =3D CPU_ALLOC_SIZE(nrcpus); + for (i =3D 0; i < nthreads; i++) { struct worker *w =3D &worker[i]; =20 @@ -341,22 +348,28 @@ static int do_threads(struct worker *worker, struct p= erf_cpu_map *cpu) } =20 if (!noaffinity) { - CPU_ZERO(&cpuset); - CPU_SET(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, &cpuset); + CPU_ZERO_S(size, cpuset); + CPU_SET_S(perf_cpu_map__cpu(cpu, i % perf_cpu_map__nr(cpu)).cpu, + size, cpuset); =20 - ret =3D pthread_attr_setaffinity_np(&thread_attr, sizeof(cpu_set_t), &c= puset); - if (ret) + ret =3D pthread_attr_setaffinity_np(&thread_attr, size, cpuset); + if (ret) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_attr_setaffinity_np"); + } =20 attrp =3D &thread_attr; } =20 ret =3D pthread_create(&w->thread, attrp, workerfn, (void *)(struct worker *) w); - if (ret) + if (ret) { + CPU_FREE(cpuset); err(EXIT_FAILURE, "pthread_create"); + } } =20 + CPU_FREE(cpuset); if (!noaffinity) pthread_attr_destroy(&thread_attr); =20 --=20 2.35.1 From nobody Fri Jun 19 07:16:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A19FC433EF for ; Wed, 6 Apr 2022 20:07:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233854AbiDFUJe (ORCPT ); Wed, 6 Apr 2022 16:09:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233767AbiDFUIX (ORCPT ); Wed, 6 Apr 2022 16:08:23 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2AA87644; Wed, 6 Apr 2022 10:52:00 -0700 (PDT) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 236Hj4Yi025382; Wed, 6 Apr 2022 17:51:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=BcR3gIKzSiOjVh7Agu1X/eiZNgo2PdhBaIDwVK9ABaA=; b=HVUm6hhtNaDDJh7RPIlOMVAAD/Jtmkx+9xPzhJD/WwvUhlrEdJFL4dTjqeZwbg57SRFt gHmsw+q4OOxvHOfRqTfViH1Z91bvmx0dA/4dp61Mm7YxsqkeBB3zpLWB8k54tM4UEl5h +T7fjGqkh2qfizngWbaRiVqqzDbgaDTFSbsjMQbLO5pWOd5X/A3mKoGQ6r/NKGU8LuW9 lzBPH0we2ihLpGmuRP/v2Y9a/dlf8oA0EsKVkAu9yS2vtvwPbw7MX2Wd6tfF4lKw9vTY GAgruWZNhs2UWclzoIWMX/V1u2p4k4kXLsr9BsKHsICMh00xITz6FCnNi04k6LGQAvd2 9A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3f8twg8usf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:46 +0000 Received: from m0098393.ppops.net (m0098393.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 236HjZKA028010; Wed, 6 Apr 2022 17:51:46 GMT Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 3f8twg8uru-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:46 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 236Hh9TQ018159; Wed, 6 Apr 2022 17:51:43 GMT Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by ppma03ams.nl.ibm.com with ESMTP id 3f6e4908md-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:43 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 236Hpekg39846192 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 6 Apr 2022 17:51:40 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 98ABD4C04A; Wed, 6 Apr 2022 17:51:40 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0C9844C046; Wed, 6 Apr 2022 17:51:35 +0000 (GMT) Received: from localhost.localdomain (unknown [9.211.90.125]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 6 Apr 2022 17:51:34 +0000 (GMT) From: Athira Rajeev To: acme@kernel.org, jolsa@kernel.org, disgoel@linux.vnet.ibm.com Cc: mpe@ellerman.id.au, linux-perf-users@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.vnet.ibm.com, rnsastry@linux.ibm.com, kjain@linux.ibm.com, linux-kernel@vger.kernel.org, srikar@linux.vnet.ibm.com, irogers@google.com Subject: [PATCH v2 3/4] tools/perf: Fix perf numa bench to fix usage of affinity for machines with #CPUs > 1K Date: Wed, 6 Apr 2022 23:21:12 +0530 Message-Id: <20220406175113.87881-4-atrajeev@linux.vnet.ibm.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220406175113.87881-1-atrajeev@linux.vnet.ibm.com> References: <20220406175113.87881-1-atrajeev@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: ex8HKKw-QfxWNZJs90VonxTYaJTMYmbv X-Proofpoint-GUID: J6NBuQE-ikpceLtDq9RlGplaO45gb9dJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.850,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-04-06_09,2022-04-06_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 impostorscore=0 adultscore=0 malwarescore=0 mlxscore=0 bulkscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204060087 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" perf bench numa testcase fails on systems with CPU's more than 1K. Testcase: perf bench numa mem -p 1 -t 3 -P 512 -s 100 -zZ0qcm --thp 1 Snippet of code: <<>> perf: bench/numa.c:302: bind_to_node: Assertion `!(ret)' failed. Aborted (core dumped) <<>> bind_to_node function uses "sched_getaffinity" to save the original cpumask and this call is returning EINVAL ((invalid argument). This happens because the default mask size in glibc is 1024. To overcome this 1024 CPUs mask size limitation of cpu_set_t, change the mask size using the CPU_*_S macros ie, use CPU_ALLOC to allocate cpumask, CPU_ALLOC_SIZE for size. Apart from fixing this for "orig_mask", apply same logic to "mask" as well which is used to setaffinity so that mask size is large enough to represent number of possible CPU's in the system. sched_getaffinity is used in one more place in perf numa bench. It is in "bind_to_cpu" function. Apply the same logic there also. Though currently no failure is reported from there, it is ideal to change getaffinity to work with such system configurations having CPU's more than default mask size supported by glibc. Also fix "sched_setaffinity" to use mask size which is large enough to represent number of possible CPU's in the system. Fixed all places where "bind_cpumask" which is part of "struct thread_data" is used such that bind_cpumask works in all configuration. Tested-by: Disha Goel Signed-off-by: Athira Rajeev Reported-by: Disha Goel Acked-by: Ian Rogers Reviewed-by: Srikar Dronamraju --- tools/perf/bench/numa.c | 97 ++++++++++++++++++++++++++++++----------- 1 file changed, 71 insertions(+), 26 deletions(-) diff --git a/tools/perf/bench/numa.c b/tools/perf/bench/numa.c index f2640179ada9..29e41e32bd88 100644 --- a/tools/perf/bench/numa.c +++ b/tools/perf/bench/numa.c @@ -54,7 +54,7 @@ =20 struct thread_data { int curr_cpu; - cpu_set_t bind_cpumask; + cpu_set_t *bind_cpumask; int bind_node; u8 *process_data; int process_nr; @@ -266,46 +266,71 @@ static bool node_has_cpus(int node) return ret; } =20 -static cpu_set_t bind_to_cpu(int target_cpu) +static cpu_set_t *bind_to_cpu(int target_cpu) { - cpu_set_t orig_mask, mask; + int nrcpus =3D numa_num_possible_cpus(); + cpu_set_t *orig_mask, *mask; + size_t size; int ret; =20 - ret =3D sched_getaffinity(0, sizeof(orig_mask), &orig_mask); - BUG_ON(ret); + orig_mask =3D CPU_ALLOC(nrcpus); + BUG_ON(!orig_mask); + size =3D CPU_ALLOC_SIZE(nrcpus); + CPU_ZERO_S(size, orig_mask); + + ret =3D sched_getaffinity(0, size, orig_mask); + if (ret) { + CPU_FREE(orig_mask); + BUG_ON(ret); + } =20 - CPU_ZERO(&mask); + mask =3D CPU_ALLOC(nrcpus); + BUG_ON(!mask); + CPU_ZERO_S(size, mask); =20 if (target_cpu =3D=3D -1) { int cpu; =20 for (cpu =3D 0; cpu < g->p.nr_cpus; cpu++) - CPU_SET(cpu, &mask); + CPU_SET_S(cpu, size, mask); } else { BUG_ON(target_cpu < 0 || target_cpu >=3D g->p.nr_cpus); - CPU_SET(target_cpu, &mask); + CPU_SET_S(target_cpu, size, mask); } =20 - ret =3D sched_setaffinity(0, sizeof(mask), &mask); + ret =3D sched_setaffinity(0, size, mask); + CPU_FREE(mask); BUG_ON(ret); =20 return orig_mask; } =20 -static cpu_set_t bind_to_node(int target_node) +static cpu_set_t *bind_to_node(int target_node) { - cpu_set_t orig_mask, mask; + int nrcpus =3D numa_num_possible_cpus(); + cpu_set_t *orig_mask, *mask; + size_t size; int cpu; int ret; =20 - ret =3D sched_getaffinity(0, sizeof(orig_mask), &orig_mask); - BUG_ON(ret); + orig_mask =3D CPU_ALLOC(nrcpus); + BUG_ON(!orig_mask); + size =3D CPU_ALLOC_SIZE(nrcpus); + CPU_ZERO_S(size, orig_mask); + + ret =3D sched_getaffinity(0, size, orig_mask); + if (ret) { + CPU_FREE(orig_mask); + BUG_ON(ret); + } =20 - CPU_ZERO(&mask); + mask =3D CPU_ALLOC(nrcpus); + BUG_ON(!mask); + CPU_ZERO_S(size, mask); =20 if (target_node =3D=3D NUMA_NO_NODE) { for (cpu =3D 0; cpu < g->p.nr_cpus; cpu++) - CPU_SET(cpu, &mask); + CPU_SET_S(cpu, size, mask); } else { struct bitmask *cpumask =3D numa_allocate_cpumask(); =20 @@ -313,24 +338,29 @@ static cpu_set_t bind_to_node(int target_node) if (!numa_node_to_cpus(target_node, cpumask)) { for (cpu =3D 0; cpu < (int)cpumask->size; cpu++) { if (numa_bitmask_isbitset(cpumask, cpu)) - CPU_SET(cpu, &mask); + CPU_SET_S(cpu, size, mask); } } numa_free_cpumask(cpumask); } =20 - ret =3D sched_setaffinity(0, sizeof(mask), &mask); + ret =3D sched_setaffinity(0, size, mask); + CPU_FREE(mask); BUG_ON(ret); =20 return orig_mask; } =20 -static void bind_to_cpumask(cpu_set_t mask) +static void bind_to_cpumask(cpu_set_t *mask) { int ret; + size_t size =3D CPU_ALLOC_SIZE(numa_num_possible_cpus()); =20 - ret =3D sched_setaffinity(0, sizeof(mask), &mask); - BUG_ON(ret); + ret =3D sched_setaffinity(0, size, mask); + if (ret) { + CPU_FREE(mask); + BUG_ON(ret); + } } =20 static void mempol_restore(void) @@ -376,7 +406,7 @@ do { \ static u8 *alloc_data(ssize_t bytes0, int map_flags, int init_zero, int init_cpu0, int thp, int init_random) { - cpu_set_t orig_mask; + cpu_set_t *orig_mask; ssize_t bytes; u8 *buf; int ret; @@ -434,6 +464,7 @@ static u8 *alloc_data(ssize_t bytes0, int map_flags, /* Restore affinity: */ if (init_cpu0) { bind_to_cpumask(orig_mask); + CPU_FREE(orig_mask); mempol_restore(); } =20 @@ -589,6 +620,7 @@ static int parse_setup_cpu_list(void) BUG_ON(bind_cpu_0 > bind_cpu_1); =20 for (bind_cpu =3D bind_cpu_0; bind_cpu <=3D bind_cpu_1; bind_cpu +=3D st= ep) { + size_t size =3D CPU_ALLOC_SIZE(g->p.nr_cpus); int i; =20 for (i =3D 0; i < mul; i++) { @@ -608,10 +640,12 @@ static int parse_setup_cpu_list(void) tprintf("%2d", bind_cpu); } =20 - CPU_ZERO(&td->bind_cpumask); + td->bind_cpumask =3D CPU_ALLOC(g->p.nr_cpus); + BUG_ON(!td->bind_cpumask); + CPU_ZERO_S(size, td->bind_cpumask); for (cpu =3D bind_cpu; cpu < bind_cpu+bind_len; cpu++) { BUG_ON(cpu < 0 || cpu >=3D g->p.nr_cpus); - CPU_SET(cpu, &td->bind_cpumask); + CPU_SET_S(cpu, size, td->bind_cpumask); } t++; } @@ -1241,7 +1275,7 @@ static void *worker_thread(void *__tdata) * by migrating to CPU#0: */ if (first_task && g->p.perturb_secs && (int)(stop.tv_sec - last_perturba= nce) >=3D g->p.perturb_secs) { - cpu_set_t orig_mask; + cpu_set_t *orig_mask; int target_cpu; int this_cpu; =20 @@ -1265,6 +1299,7 @@ static void *worker_thread(void *__tdata) printf(" (injecting perturbalance, moved to CPU#%d)\n", target_cpu); =20 bind_to_cpumask(orig_mask); + CPU_FREE(orig_mask); } =20 if (details >=3D 3) { @@ -1398,21 +1433,31 @@ static void init_thread_data(void) =20 for (t =3D 0; t < g->p.nr_tasks; t++) { struct thread_data *td =3D g->threads + t; + size_t cpuset_size =3D CPU_ALLOC_SIZE(g->p.nr_cpus); int cpu; =20 /* Allow all nodes by default: */ td->bind_node =3D NUMA_NO_NODE; =20 /* Allow all CPUs by default: */ - CPU_ZERO(&td->bind_cpumask); + td->bind_cpumask =3D CPU_ALLOC(g->p.nr_cpus); + BUG_ON(!td->bind_cpumask); + CPU_ZERO_S(cpuset_size, td->bind_cpumask); for (cpu =3D 0; cpu < g->p.nr_cpus; cpu++) - CPU_SET(cpu, &td->bind_cpumask); + CPU_SET_S(cpu, cpuset_size, td->bind_cpumask); } } =20 static void deinit_thread_data(void) { ssize_t size =3D sizeof(*g->threads)*g->p.nr_tasks; + int t; + + /* Free the bind_cpumask allocated for thread_data */ + for (t =3D 0; t < g->p.nr_tasks; t++) { + struct thread_data *td =3D g->threads + t; + CPU_FREE(td->bind_cpumask); + } =20 free_data(g->threads, size); } --=20 2.35.1 From nobody Fri Jun 19 07:16:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DF6FC433EF for ; Wed, 6 Apr 2022 20:02:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233362AbiDFUEI (ORCPT ); Wed, 6 Apr 2022 16:04:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235409AbiDFUDu (ORCPT ); Wed, 6 Apr 2022 16:03:50 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5CA4F60DBA; Wed, 6 Apr 2022 10:52:05 -0700 (PDT) Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 236GESm1030105; Wed, 6 Apr 2022 17:51:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=pp1; bh=hWy5I10SyMCKmhD0+kEodm/y+gl5qzddelF/Lh2wtH0=; b=gpCaxE2L4xXcID7HaSzkpTPuYGz8qFLUrwDxZg5+5F0pcGcIWZJaKndpwofexWytg+XR OCfsHHmyh6S+9NRP8dJKBQ8DR/bUU2HAF68vPoWSE95rJNqjq+6KnoWzSKqurm17PiLw VCTZmocNtP0TJn40CdQslYfJSWL1gTzaBDxHj87l514TBUeTWvNtMvBJIoTySaLP0u4o 1Tc+qxMpacYBnIPJHRZkqMXPLYCOkCNoq2l3n5H7eP0nz3GQGNkNoHFuOQavbwxm8/wY TxslO25B3idxtkwMfGy4Qmj211X7M20UC4aghIuv6P6Ra6fi4U+XlYlWT9/wfdxXa5Wv 5A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3f95epwyq6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:53 +0000 Received: from m0098394.ppops.net (m0098394.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 236Hgf9d011564; Wed, 6 Apr 2022 17:51:52 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com with ESMTP id 3f95epwypm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:52 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 236Hhi6M019355; Wed, 6 Apr 2022 17:51:50 GMT Received: from b06cxnps3075.portsmouth.uk.ibm.com (d06relay10.portsmouth.uk.ibm.com [9.149.109.195]) by ppma06ams.nl.ibm.com with ESMTP id 3f6drhr7k4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 Apr 2022 17:51:49 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 236HpllX43057440 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 6 Apr 2022 17:51:47 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DAE814C04A; Wed, 6 Apr 2022 17:51:46 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5C8F54C040; Wed, 6 Apr 2022 17:51:41 +0000 (GMT) Received: from localhost.localdomain (unknown [9.211.90.125]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 6 Apr 2022 17:51:40 +0000 (GMT) From: Athira Rajeev To: acme@kernel.org, jolsa@kernel.org, disgoel@linux.vnet.ibm.com Cc: mpe@ellerman.id.au, linux-perf-users@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.vnet.ibm.com, rnsastry@linux.ibm.com, kjain@linux.ibm.com, linux-kernel@vger.kernel.org, srikar@linux.vnet.ibm.com, irogers@google.com Subject: [PATCH v2 4/4] tools/perf: Fix perf bench numa testcase to check if CPU used to bind task is online Date: Wed, 6 Apr 2022 23:21:13 +0530 Message-Id: <20220406175113.87881-5-atrajeev@linux.vnet.ibm.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220406175113.87881-1-atrajeev@linux.vnet.ibm.com> References: <20220406175113.87881-1-atrajeev@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Z94Fa6rh7GGPCk_QPb0VIRZWRQ5zXAZw X-Proofpoint-ORIG-GUID: 5zuHFViFxtoaVLPnUmFfj3zwISZojtkT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.850,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-04-06_09,2022-04-06_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 mlxscore=0 bulkscore=0 clxscore=1015 priorityscore=1501 phishscore=0 adultscore=0 lowpriorityscore=0 impostorscore=0 malwarescore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204060087 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Perf numa bench test fails with error: Testcase: ./perf bench numa mem -p 2 -t 1 -P 1024 -C 0,8 -M 1,0 -s 20 -zZq --thp 1 --no-data_rand_walk Failure snippet: <<>> Running 'numa/mem' benchmark: # Running main, "perf bench numa numa-mem -p 2 -t 1 -P 1024 -C 0,8 -M 1,0 -s 20 -zZq --thp 1 --no-data_rand_walk" perf: bench/numa.c:333: bind_to_cpumask: Assertion `!(ret)' failed. <<>> The Testcases uses CPU=E2=80=99s 0 and 8. In function "parse_setup_cpu_list= ", There is check to see if cpu number is greater than max cpu=E2=80=99s possi= ble in the system ie via "if (bind_cpu_0 >=3D g->p.nr_cpus || bind_cpu_1 >=3D g->p.nr_cpus) {". But it could happen that system has say 48 CPU=E2=80=99s, but only number of online CPU=E2=80=99s is 0-7. Other= CPU=E2=80=99s are offlined. Since "g->p.nr_cpus" is 48, so function will go ahead and set bit for CPU 8 also in cpumask ( td->bind_cpumask). bind_to_cpumask function is called to set affinity using sched_setaffinity and the cpumask. Since the CPU8 is not present, set affinity will fail here with EINVAL. Fix this issue by adding a check to make sure that, CPU=E2=80=99s provided in the input argument values are online before proceeding further and skip the test. For this, include new helper function "is_cpu_online" in "tools/perf/util/header.c". Since "BIT(x)" definition will get included from header.h, remove that from bench/numa.c Tested-by: Disha Goel Signed-off-by: Athira Rajeev Reported-by: Disha Goel Acked-by: Ian Rogers Reviewed-by: Srikar Dronamraju --- tools/perf/bench/numa.c | 8 ++++++-- tools/perf/util/header.c | 43 ++++++++++++++++++++++++++++++++++++++++ tools/perf/util/header.h | 1 + 3 files changed, 50 insertions(+), 2 deletions(-) diff --git a/tools/perf/bench/numa.c b/tools/perf/bench/numa.c index 29e41e32bd88..7992d79b3e41 100644 --- a/tools/perf/bench/numa.c +++ b/tools/perf/bench/numa.c @@ -34,6 +34,7 @@ #include #include =20 +#include "../util/header.h" #include #include =20 @@ -616,6 +617,11 @@ static int parse_setup_cpu_list(void) return -1; } =20 + if (is_cpu_online(bind_cpu_0) !=3D 1 || is_cpu_online(bind_cpu_1) !=3D 1= ) { + printf("\nTest not applicable, bind_cpu_0 or bind_cpu_1 is offline\n"); + return -1; + } + BUG_ON(bind_cpu_0 < 0 || bind_cpu_1 < 0); BUG_ON(bind_cpu_0 > bind_cpu_1); =20 @@ -786,8 +792,6 @@ static int parse_nodes_opt(const struct option *opt __m= aybe_unused, return parse_node_list(arg); } =20 -#define BIT(x) (1ul << x) - static inline uint32_t lfsr_32(uint32_t lfsr) { const uint32_t taps =3D BIT(1) | BIT(5) | BIT(6) | BIT(31); diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c index 6da12e522edc..3f5fcf5d4b3f 100644 --- a/tools/perf/util/header.c +++ b/tools/perf/util/header.c @@ -983,6 +983,49 @@ static int write_dir_format(struct feat_fd *ff, return do_write(ff, &data->dir.version, sizeof(data->dir.version)); } =20 +#define SYSFS "/sys/devices/system/cpu/" + +/* + * Check whether a CPU is online + * + * Returns: + * 1 -> if CPU is online + * 0 -> if CPU is offline + * -1 -> error case + */ +int is_cpu_online(unsigned int cpu) +{ + char sysfs_cpu[255]; + char buf[255]; + struct stat statbuf; + size_t len; + int fd; + + snprintf(sysfs_cpu, sizeof(sysfs_cpu), SYSFS "cpu%u", cpu); + + if (stat(sysfs_cpu, &statbuf) !=3D 0) + return 0; + + /* + * Check if /sys/devices/system/cpu/cpux/online file + * exists. In kernels without CONFIG_HOTPLUG_CPU, this + * file won't exist. + */ + snprintf(sysfs_cpu, sizeof(sysfs_cpu), SYSFS "cpu%u/online", cpu); + if (stat(sysfs_cpu, &statbuf) !=3D 0) + return 1; + + fd =3D open(sysfs_cpu, O_RDONLY); + if (fd =3D=3D -1) + return -1; + + len =3D read(fd, buf, sizeof(buf) - 1); + buf[len] =3D '\0'; + close(fd); + + return strtoul(buf, NULL, 16); +} + #ifdef HAVE_LIBBPF_SUPPORT static int write_bpf_prog_info(struct feat_fd *ff, struct evlist *evlist __maybe_unused) diff --git a/tools/perf/util/header.h b/tools/perf/util/header.h index c9e3265832d9..0eb4bc29a5a4 100644 --- a/tools/perf/util/header.h +++ b/tools/perf/util/header.h @@ -158,6 +158,7 @@ int do_write(struct feat_fd *fd, const void *buf, size_= t size); int write_padded(struct feat_fd *fd, const void *bf, size_t count, size_t count_aligned); =20 +int is_cpu_online(unsigned int cpu); /* * arch specific callback */ --=20 2.35.1