From nobody Sun Feb 8 12:03:25 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00A48217F27 for ; Mon, 17 Feb 2025 11:23:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739791416; cv=none; b=OjTeRuCNO+C/GgQ8OlCGm3q3oJVDnrjdtijdZfn7GiwR6Ra3TANeOR9kAxseGrS+qGcQACGoqmxaiyn1+jj2Hr87CXxZi6Mi9RiuuGPHH69mTcQM+ZslDeqKcnzRtEt8FRKTJOLREPDmA5FJS6YT1GTJpKlxcNZ12ZmLud09l4A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739791416; c=relaxed/simple; bh=tCn+zXx+bMTWTjCLwfEQuYUE8qVtI5OAQHT0vM17Igs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gp3tAh1JDp3o5rwiYoO5WfQp/zLWQoPSFxTPpCo6d3aFI0rQRElNd7Ld5l7gEs6oVdBvBr4WWuqm3b6arUoKxLr269pCX18dslzlzW+HktGHrIdf6eVe5ZPGffZhGIkU43yHV4MBUXrzAudhpKGSXdvUyooUdtxtMKhuvoByMz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Vqw5pc5w; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Vqw5pc5w" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1739791413; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UmVVq3HHzqRlZLdZZrTZki/75/BrjuDDaU0LBdE44wk=; b=Vqw5pc5wqh9PTT71XgHdCLkRp6B1QzT/K80MetV49uL7A2lL1xDuoWboqvXQ5L6Lx5PNuo qeF3Sr8D5fsSUsHZ4XNge/y/WVW3v9yuFKXHBneRabSM7Ms2Q5zjlrjM1xql/6XLrblSxS ZL9ia0RRJIOL2u7zd1+fMFds/AvWIsE= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-663-yVEZmuJVO1SYSDLbzU-3CA-1; Mon, 17 Feb 2025 06:23:30 -0500 X-MC-Unique: yVEZmuJVO1SYSDLbzU-3CA-1 X-Mimecast-MFC-AGG-ID: yVEZmuJVO1SYSDLbzU-3CA_1739791409 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5F3D51800874; Mon, 17 Feb 2025 11:23:29 +0000 (UTC) Received: from gmonaco-thinkpadt14gen3.rmtit.com (unknown [10.44.32.190]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BAE811955BD4; Mon, 17 Feb 2025 11:23:25 +0000 (UTC) From: Gabriele Monaco To: linux-kernel@vger.kernel.org, Andrew Morton , Ingo Molnar , Peter Zijlstra , Mathieu Desnoyers , linux-mm@kvack.org Cc: Gabriele Monaco , Ingo Molnar , "Paul E. McKenney" Subject: [PATCH 1/2] sched: Compact RSEQ concurrency IDs in batches Date: Mon, 17 Feb 2025 12:23:16 +0100 Message-ID: <20250217112317.258716-2-gmonaco@redhat.com> In-Reply-To: <20250217112317.258716-1-gmonaco@redhat.com> References: <20250217112317.258716-1-gmonaco@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Currently, the task_mm_cid_work function is called in a task work triggered by a scheduler tick to frequently compact the mm_cids of each process for each core. This can delay the execution of the corresponding thread for the entire duration of the function, negatively affecting the response in case of real time tasks. In practice, we observe task_mm_cid_work increasing the latency of 30-35us on a 128 cores system, this order of magnitude is meaningful under PREEMPT_RT. Run the task_mm_cid_work in batches of up to CONFIG_RSEQ_CID_SCAN_BATCH cpus, this contains the duration of the delay for each scan. Also improve the duration by iterating for all present cpus and not for all possible. The task_mm_cid_work already contains a mechanism to avoid running more frequently than every 100ms, considering the function runs at every tick, assuming ticks every 1ms (HZ=3D1000 is common on distros) and assuming an unfavorable scenario of 1/10 ticks during task T runtime, we can compact the CIDs for task T in about 130ms by setting CONFIG_RSEQ_CID_SCAN_BATCH to 10 on a 128 cores machine. This value also drastically reduces the task work duration and is a more acceptable latency for the aforementioned machine. Fixes: 223baf9d17f2 ("sched: Fix performance regression introduced by mm_ci= d") Signed-off-by: Gabriele Monaco --- include/linux/mm_types.h | 8 ++++++++ init/Kconfig | 12 ++++++++++++ kernel/sched/core.c | 27 ++++++++++++++++++++++++--- 3 files changed, 44 insertions(+), 3 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 0234f14f2aa6b..1e0e491d2c5c2 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -867,6 +867,13 @@ struct mm_struct { * When the next mm_cid scan is due (in jiffies). */ unsigned long mm_cid_next_scan; + /* + * @mm_cid_scan_cpu: Which cpu to start from in the next scan + * + * Scan in batches of CONFIG_RSEQ_CID_SCAN_BATCH after each scan + * save the next cpu index here (or 0 if we are done) + */ + unsigned int mm_cid_scan_cpu; /** * @nr_cpus_allowed: Number of CPUs allowed for mm. * @@ -1249,6 +1256,7 @@ static inline void mm_init_cid(struct mm_struct *mm, = struct task_struct *p) raw_spin_lock_init(&mm->cpus_allowed_lock); cpumask_copy(mm_cpus_allowed(mm), &p->cpus_mask); cpumask_clear(mm_cidmask(mm)); + mm->mm_cid_scan_cpu =3D 0; } =20 static inline int mm_alloc_cid_noprof(struct mm_struct *mm, struct task_st= ruct *p) diff --git a/init/Kconfig b/init/Kconfig index d0d021b3fa3b3..39f1d4c7980c0 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1813,6 +1813,18 @@ config DEBUG_RSEQ =20 If unsure, say N. =20 +config RSEQ_CID_SCAN_BATCH + int "Number of CPUs to scan every time we attempt mm_cid compaction" + range 1 NR_CPUS + default 10 + depends on SCHED_MM_CID + help + CPUs are scanned pseudo-periodically to compact the CID of each task, + this operation can take a longer amount of time on systems with many + CPUs, resulting in higher scheduling latency for the current task. + A higher value means the CID is compacted faster, but results in + higher scheduling latency. + config CACHESTAT_SYSCALL bool "Enable cachestat() system call" if EXPERT default y diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 9aecd914ac691..8d1cce4ed62c6 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -10536,7 +10536,7 @@ static void task_mm_cid_work(struct callback_head *= work) struct task_struct *t =3D current; struct cpumask *cidmask; struct mm_struct *mm; - int weight, cpu; + int weight, cpu, from_cpu, to_cpu; =20 SCHED_WARN_ON(t !=3D container_of(work, struct task_struct, cid_work)); =20 @@ -10546,6 +10546,15 @@ static void task_mm_cid_work(struct callback_head = *work) mm =3D t->mm; if (!mm) return; + cpu =3D from_cpu =3D READ_ONCE(mm->mm_cid_scan_cpu); + to_cpu =3D from_cpu + CONFIG_RSEQ_CID_SCAN_BATCH; + if (from_cpu > cpumask_last(cpu_present_mask)) { + from_cpu =3D 0; + to_cpu =3D CONFIG_RSEQ_CID_SCAN_BATCH; + } + if (from_cpu !=3D 0) + /* Delay scan only if we are done with all cpus. */ + goto cid_compact; old_scan =3D READ_ONCE(mm->mm_cid_next_scan); next_scan =3D now + msecs_to_jiffies(MM_CID_SCAN_DELAY); if (!old_scan) { @@ -10561,17 +10570,29 @@ static void task_mm_cid_work(struct callback_head= *work) return; if (!try_cmpxchg(&mm->mm_cid_next_scan, &old_scan, next_scan)) return; + +cid_compact: + if (!try_cmpxchg(&mm->mm_cid_scan_cpu, &cpu, to_cpu)) + return; cidmask =3D mm_cidmask(mm); /* Clear cids that were not recently used. */ - for_each_possible_cpu(cpu) + cpu =3D from_cpu; + for_each_cpu_from(cpu, cpu_present_mask) { + if (cpu =3D=3D to_cpu) + break; sched_mm_cid_remote_clear_old(mm, cpu); + } weight =3D cpumask_weight(cidmask); /* * Clear cids that are greater or equal to the cidmask weight to * recompact it. */ - for_each_possible_cpu(cpu) + cpu =3D from_cpu; + for_each_cpu_from(cpu, cpu_present_mask) { + if (cpu =3D=3D to_cpu) + break; sched_mm_cid_remote_clear_weight(mm, cpu, weight); + } } =20 void init_sched_mm_cid(struct task_struct *t) --=20 2.48.1 From nobody Sun Feb 8 12:03:25 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45F2F2135C7 for ; Mon, 17 Feb 2025 11:23:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739791423; cv=none; b=V7lqiikAx9jaRLOfXddxAdlPo8DKNTbUK246DABxeIW1LY8lROHy0Q42KJgAa68FAkRPcjPX39sTxaIaO+5nU3X9W9JOBX0sSUWgcxWq6svGqexyaWt4xbiaHNcGDnb/NMNb5tMaESw6AnlMQFn/AA4L/tXR0kRzUxrB298gv4o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739791423; c=relaxed/simple; bh=sIfXqq6LgX08DnF2ZKJioqs25c99aohveTdkwtTDipY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F4nApDbdDks+hSsecCH0hm0GKf+Jvvot1e5lyte/ywUGq9ejgSSNM3S6yLd8MebGBSBNuMwApbTiLNZD1CJ9O2c0pQtnHsvPxHS2YrPHNCJg8dqLX+HLRaZCHOuducHw+M6zvTe1H+4QBqiTddn0kQHvWzLoLuU3X/rPTzPhftU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KS8Dus4r; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KS8Dus4r" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1739791420; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e7SDIYW07YQZs6j+00IUK84Snt1dAMAtfJPlH6mtonk=; b=KS8Dus4rX+ZD1wsSoRxnan8TTKGm4oLi1vv+M+u0Zv4SJHoX6dIjSESN1A9S/8uR5Xkk+W rkUE0aQ3EZZs+WXNbFluGw1aDacPcaP7cBZp/htHFt7iDqc0BslqxI1sE1unWotsc/NsMz uC0fUfwGWnL3/0uahz9eYfkO1WJzGTo= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-679-xVzBnSylOfy1UFIzXwO6tQ-1; Mon, 17 Feb 2025 06:23:34 -0500 X-MC-Unique: xVzBnSylOfy1UFIzXwO6tQ-1 X-Mimecast-MFC-AGG-ID: xVzBnSylOfy1UFIzXwO6tQ_1739791413 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 48E96180087C; Mon, 17 Feb 2025 11:23:33 +0000 (UTC) Received: from gmonaco-thinkpadt14gen3.rmtit.com (unknown [10.44.32.190]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 377481955BCB; Mon, 17 Feb 2025 11:23:29 +0000 (UTC) From: Gabriele Monaco To: linux-kernel@vger.kernel.org, Mathieu Desnoyers , Peter Zijlstra , "Paul E. McKenney" , Shuah Khan , linux-kselftest@vger.kernel.org Cc: Gabriele Monaco , Ingo Molnar Subject: [PATCH 2/2] rseq/selftests: Add test for mm_cid compaction Date: Mon, 17 Feb 2025 12:23:17 +0100 Message-ID: <20250217112317.258716-3-gmonaco@redhat.com> In-Reply-To: <20250217112317.258716-1-gmonaco@redhat.com> References: <20250217112317.258716-1-gmonaco@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" A task in the kernel (task_mm_cid_work) runs somewhat periodically to compact the mm_cid for each process. Add a test to validate that it runs correctly and timely. The test spawns 1 thread pinned to each CPU, then each thread, including the main one, runs in short bursts for some time. During this period, the mm_cids should be spanning all numbers between 0 and nproc. At the end of this phase, a thread with high enough mm_cid (>=3D nproc/2) is selected to be the new leader, all other threads terminate. After some time, the only running thread should see 0 as mm_cid, if that doesn't happen, the compaction mechanism didn't work and the test fails. Since mm_cid compaction is less likely for tasks running in short bursts, we increase the likelihood by just running a busy loop at every iteration. This compaction is a best effort work and this behaviour is currently acceptable. The test never fails if only 1 core is available, in which case, we cannot test anything as the only available mm_cid is 0. Reviewed-by: Mathieu Desnoyers Signed-off-by: Gabriele Monaco --- tools/testing/selftests/rseq/.gitignore | 1 + tools/testing/selftests/rseq/Makefile | 2 +- .../selftests/rseq/mm_cid_compaction_test.c | 208 ++++++++++++++++++ 3 files changed, 210 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/rseq/mm_cid_compaction_test.c diff --git a/tools/testing/selftests/rseq/.gitignore b/tools/testing/selfte= sts/rseq/.gitignore index 16496de5f6ce4..2c89f97e4f737 100644 --- a/tools/testing/selftests/rseq/.gitignore +++ b/tools/testing/selftests/rseq/.gitignore @@ -3,6 +3,7 @@ basic_percpu_ops_test basic_percpu_ops_mm_cid_test basic_test basic_rseq_op_test +mm_cid_compaction_test param_test param_test_benchmark param_test_compare_twice diff --git a/tools/testing/selftests/rseq/Makefile b/tools/testing/selftest= s/rseq/Makefile index 5a3432fceb586..ce1b38f46a355 100644 --- a/tools/testing/selftests/rseq/Makefile +++ b/tools/testing/selftests/rseq/Makefile @@ -16,7 +16,7 @@ OVERRIDE_TARGETS =3D 1 =20 TEST_GEN_PROGS =3D basic_test basic_percpu_ops_test basic_percpu_ops_mm_ci= d_test param_test \ param_test_benchmark param_test_compare_twice param_test_mm_cid \ - param_test_mm_cid_benchmark param_test_mm_cid_compare_twice + param_test_mm_cid_benchmark param_test_mm_cid_compare_twice mm_cid_compa= ction_test =20 TEST_GEN_PROGS_EXTENDED =3D librseq.so =20 diff --git a/tools/testing/selftests/rseq/mm_cid_compaction_test.c b/tools/= testing/selftests/rseq/mm_cid_compaction_test.c new file mode 100644 index 0000000000000..8808500466d02 --- /dev/null +++ b/tools/testing/selftests/rseq/mm_cid_compaction_test.c @@ -0,0 +1,208 @@ +// SPDX-License-Identifier: LGPL-2.1 +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../kselftest.h" +#include "rseq.h" + +#define VERBOSE 0 +#define printf_verbose(fmt, ...) \ + do { \ + if (VERBOSE) \ + printf(fmt, ##__VA_ARGS__); \ + } while (0) + +/* 0.5 s */ +#define RUNNER_PERIOD 500000 +/* Number of runs before we terminate or get the token */ +#define THREAD_RUNS 5 + +/* + * Number of times we check that the mm_cid were compacted. + * Checks are repeated every RUNNER_PERIOD. + */ +#define MM_CID_COMPACT_TIMEOUT 10 + +struct thread_args { + int cpu; + int num_cpus; + pthread_mutex_t *token; + pthread_barrier_t *barrier; + pthread_t *tinfo; + struct thread_args *args_head; +}; + +static void __noreturn *thread_runner(void *arg) +{ + struct thread_args *args =3D arg; + int i, ret, curr_mm_cid; + cpu_set_t cpumask; + + CPU_ZERO(&cpumask); + CPU_SET(args->cpu, &cpumask); + ret =3D pthread_setaffinity_np(pthread_self(), sizeof(cpumask), &cpumask); + if (ret) { + errno =3D ret; + perror("Error: failed to set affinity"); + abort(); + } + pthread_barrier_wait(args->barrier); + + for (i =3D 0; i < THREAD_RUNS; i++) + usleep(RUNNER_PERIOD); + curr_mm_cid =3D rseq_current_mm_cid(); + /* + * We select one thread with high enough mm_cid to be the new leader. + * All other threads (including the main thread) will terminate. + * After some time, the mm_cid of the only remaining thread should + * converge to 0, if not, the test fails. + */ + if (curr_mm_cid >=3D args->num_cpus / 2 && + !pthread_mutex_trylock(args->token)) { + printf_verbose( + "cpu%d has mm_cid=3D%d and will be the new leader.\n", + sched_getcpu(), curr_mm_cid); + for (i =3D 0; i < args->num_cpus; i++) { + if (args->tinfo[i] =3D=3D pthread_self()) + continue; + ret =3D pthread_join(args->tinfo[i], NULL); + if (ret) { + errno =3D ret; + perror("Error: failed to join thread"); + abort(); + } + } + pthread_barrier_destroy(args->barrier); + free(args->tinfo); + free(args->token); + free(args->barrier); + free(args->args_head); + + for (i =3D 0; i < MM_CID_COMPACT_TIMEOUT; i++) { + curr_mm_cid =3D rseq_current_mm_cid(); + printf_verbose("run %d: mm_cid=3D%d on cpu%d.\n", i, + curr_mm_cid, sched_getcpu()); + if (curr_mm_cid =3D=3D 0) + exit(EXIT_SUCCESS); + /* + * Currently mm_cid compaction is less likely for tasks + * running in short bursts: increase likelihood by just + * running for some time doing nothing. + */ + for (int j =3D 0; j < 0xffff; j++) + for (int k =3D 0; k < 0xffff; k++) + asm(""); + usleep(RUNNER_PERIOD); + } + exit(EXIT_FAILURE); + } + printf_verbose("cpu%d has mm_cid=3D%d and is going to terminate.\n", + sched_getcpu(), curr_mm_cid); + pthread_exit(NULL); +} + +int test_mm_cid_compaction(void) +{ + cpu_set_t affinity; + int i, j, ret =3D 0, num_threads; + pthread_t *tinfo; + pthread_mutex_t *token; + pthread_barrier_t *barrier; + struct thread_args *args; + + sched_getaffinity(0, sizeof(affinity), &affinity); + num_threads =3D CPU_COUNT(&affinity); + tinfo =3D calloc(num_threads, sizeof(*tinfo)); + if (!tinfo) { + perror("Error: failed to allocate tinfo"); + return -1; + } + args =3D calloc(num_threads, sizeof(*args)); + if (!args) { + perror("Error: failed to allocate args"); + ret =3D -1; + goto out_free_tinfo; + } + token =3D malloc(sizeof(*token)); + if (!token) { + perror("Error: failed to allocate token"); + ret =3D -1; + goto out_free_args; + } + barrier =3D malloc(sizeof(*barrier)); + if (!barrier) { + perror("Error: failed to allocate barrier"); + ret =3D -1; + goto out_free_token; + } + if (num_threads =3D=3D 1) { + fprintf(stderr, "Cannot test on a single cpu. " + "Skipping mm_cid_compaction test.\n"); + /* only skipping the test, this is not a failure */ + goto out_free_barrier; + } + pthread_mutex_init(token, NULL); + ret =3D pthread_barrier_init(barrier, NULL, num_threads); + if (ret) { + errno =3D ret; + perror("Error: failed to initialise barrier"); + goto out_free_barrier; + } + for (i =3D 0, j =3D 0; i < CPU_SETSIZE && j < num_threads; i++) { + if (!CPU_ISSET(i, &affinity)) + continue; + args[j].num_cpus =3D num_threads; + args[j].tinfo =3D tinfo; + args[j].token =3D token; + args[j].barrier =3D barrier; + args[j].cpu =3D i; + args[j].args_head =3D args; + if (!j) { + /* The first thread is the main one */ + tinfo[0] =3D pthread_self(); + ++j; + continue; + } + ret =3D pthread_create(&tinfo[j], NULL, thread_runner, &args[j]); + if (ret) { + errno =3D ret; + perror("Error: failed to create thread"); + abort(); + } + ++j; + } + printf_verbose("Started %d threads.\n", num_threads); + + /* Also main thread will terminate if it is not selected as leader */ + thread_runner(&args[0]); + + /* only reached in case of errors */ +out_free_barrier: + free(barrier); +out_free_token: + free(token); +out_free_args: + free(args); +out_free_tinfo: + free(tinfo); + + return ret; +} + +int main(int argc, char **argv) +{ + if (!rseq_mm_cid_available()) { + fprintf(stderr, "Error: rseq_mm_cid unavailable\n"); + return -1; + } + if (test_mm_cid_compaction()) + return -1; + return 0; +} --=20 2.48.1