From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95CAE175A8D; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161094; cv=none; b=gypLBpmLCRhFKK+j8hOKVFtpJWWhaUgN67r3R+9jK6iRZh0UxdC/8MQIz/XlnZxV7iJNy68nRdcCsbmHXHmqShiDSfQx+yox06jpOlnK2Y02XrXjNdDEAvhltz44sMH5k9RBmeTK1niBZsJJqu4v4A8PDXRKRTztyFxwM0nnQdg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161094; c=relaxed/simple; bh=QyBWHTeCx5uqp65H5AmxJz/GR+kdlpZn3SAwBleh3O4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oW0uxNdPY5uNLFjYMjm7W3d4JNy9BkGDRwlx/PjbtvNcLrEN7K7KbFNJS+85qDdFUBHslc+s2AgZ3WrsDHfbt7RVW2wGg3j236F2nDrSCgfh7LuCh6w99gqU1/moXDzrb1U5JCFdGoQ2t8bvl9VvjPixUuxFCc4eprLYEXk7I68= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=edCu6KiN; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="edCu6KiN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 356121F000E9; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=n+ZolLsRH55PYQDrNZ8XplwgNVinZ6I37JOCU9BiGYg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=edCu6KiNENohZoWXHE9KcYahldzB6Sz+Qjoy+Rw/NrGp4PQpk1GwEEydQMznYWoAn nSAc4ws5/gK6MhuqoXgytVry9UYzcDZZ6OjdwreiQnI7l3/yQWJn2QcL2aPARlZWnG vp1QkBOf2HoZ8WrMc05Ka18tH0vzvf5/Vv96WSjsSdXJY/UsYMPwt5FjiJSRVwQalO D1HAj+HAtl3OcMHAKlVYcRiuTlcUNF43dEsd/3yp1hRmiya0U4G3smW+TJRPeUHAG5 E/aOjuWixsPIQAVfjQ50oUua2qvW3p+S0nXbzpu/MvJ07Hj4IwphyV+K3iSNtr9YK4 YlODpiykbXLpg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 2503ACE0569; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Mathieu Desnoyers , Nicholas Piggin , Michael Ellerman , Greg Kroah-Hartman , Sebastian Andrzej Siewior , "Paul E. McKenney" , Will Deacon , Peter Zijlstra , Boqun Feng , Alan Stern , John Stultz , Linus Torvalds , Andrew Morton , Frederic Weisbecker , Joel Fernandes , Josh Triplett , Uladzislau Rezki , Lai Jiangshan , Zqiang , Ingo Molnar , Waiman Long , Mark Rutland , Thomas Gleixner , Vlastimil Babka , maged.michael@gmail.com, Mateusz Guzik , Jonas Oberhauser , linux-mm@kvack.org, lkmm@lists.linux.dev Subject: [PATCH RFC v2 01/24] hazptr: Implement Hazard Pointers Date: Wed, 15 Jul 2026 17:17:46 -0700 Message-Id: <20260716001809.11084-1-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mathieu Desnoyers This API provides existence guarantees of objects through Hazard Pointers [1] (hazptr). Its main benefit over RCU is that it allows fast reclaim of HP-protected pointers without needing to wait for a grace period. This implementation has 4 statically allocated hazard pointer slots per cpu for the fast path, and relies on a on-stack backup slot allocated by the hazard pointer user as fallback in case no per-cpu slot is available. It integrates with the scheduler to migrate per-CPU slots to the backup slot on context switch. This ensures that the per-CPU slots won't be used by blocked or preempted tasks holding on hazard pointers for a long time. References: [1]: M. M. Michael, "Hazard pointers: safe memory reclamation for lock-free objects," in IEEE Transactions on Parallel and Distributed Systems, vol. 15, no. 6, pp. 491-504, June 2004 Link: https://lpc.events/event/19/contributions/2082/ Link: https://lore.kernel.org/lkml/j3scdl5iymjlxavomgc6u5ndg3svhab6ga23dr36= o4f5mt333w@7xslvq6b6hmv/ Link: https://lpc.events/event/18/contributions/1731/ Signed-off-by: Mathieu Desnoyers Cc: Nicholas Piggin Cc: Michael Ellerman Cc: Greg Kroah-Hartman Cc: Sebastian Andrzej Siewior Cc: "Paul E. McKenney" Cc: Will Deacon Cc: Peter Zijlstra Cc: Boqun Feng Cc: Alan Stern Cc: John Stultz Cc: Linus Torvalds Cc: Andrew Morton Cc: Frederic Weisbecker Cc: Joel Fernandes Cc: Josh Triplett Cc: Uladzislau Rezki Cc: Steven Rostedt Cc: Lai Jiangshan Cc: Zqiang Cc: Ingo Molnar Cc: Waiman Long Cc: Mark Rutland Cc: Thomas Gleixner Cc: Vlastimil Babka Cc: maged.michael@gmail.com Cc: Mateusz Guzik Cc: Jonas Oberhauser Cc: rcu@vger.kernel.org Cc: linux-mm@kvack.org Cc: lkmm@lists.linux.dev Signed-off-by: Paul E. McKenney --- include/linux/hazptr.h | 197 +++++++++++++++++++++++++++++++++ init/main.c | 2 + kernel/Makefile | 2 +- kernel/hazptr.c | 242 +++++++++++++++++++++++++++++++++++++++++ kernel/sched/core.c | 2 + 5 files changed, 444 insertions(+), 1 deletion(-) create mode 100644 include/linux/hazptr.h create mode 100644 kernel/hazptr.c diff --git a/include/linux/hazptr.h b/include/linux/hazptr.h new file mode 100644 index 00000000000000..461f481a480b9c --- /dev/null +++ b/include/linux/hazptr.h @@ -0,0 +1,197 @@ +// SPDX-FileCopyrightText: 2024 Mathieu Desnoyers +// +// SPDX-License-Identifier: LGPL-2.1-or-later + +#ifndef _LINUX_HAZPTR_H +#define _LINUX_HAZPTR_H + +/* + * hazptr: Hazard Pointers + * + * This API provides existence guarantees of objects through hazard + * pointers. + * + * Its main benefit over RCU is that it allows fast reclaim of + * HP-protected pointers without needing to wait for a grace period. + * + * References: + * + * [1]: M. M. Michael, "Hazard pointers: safe memory reclamation for + * lock-free objects," in IEEE Transactions on Parallel and + * Distributed Systems, vol. 15, no. 6, pp. 491-504, June 2004 + */ + +#include +#include +#include +#include + +/* 4 slots (each sizeof(hazptr_slot_item)) fit in a single 64-byte cache l= ine. */ +#define NR_HAZPTR_PERCPU_SLOTS 4 +#define HAZPTR_WILDCARD ((void *) 0x1UL) + +/* + * Hazard pointer slot. + */ +struct hazptr_slot { + void *addr; +}; + +struct hazptr_overflow_list; + +struct hazptr_backup_slot { + struct hlist_node overflow_node; + struct hazptr_slot slot; + /* Overflow list where the backup slot is added. */ + struct hazptr_overflow_list *overflow_list; +}; + +struct hazptr_ctx { + struct hazptr_slot *slot; + /* Backup slot in case all per-CPU slots are used. */ + struct hazptr_backup_slot backup_slot; + struct hlist_node preempt_node; +}; + +struct hazptr_slot_ctx { + struct hazptr_ctx *ctx; +}; + +struct hazptr_slot_item { + struct hazptr_slot slot; + struct hazptr_slot_ctx ctx; +}; + +struct hazptr_percpu_slots { + struct hazptr_slot_item items[NR_HAZPTR_PERCPU_SLOTS]; +} ____cacheline_aligned; + +DECLARE_PER_CPU(struct hazptr_percpu_slots, hazptr_percpu_slots); + +void *__hazptr_acquire(struct hazptr_ctx *ctx, void * const * addr_p); + +/* + * hazptr_synchronize: Wait until @addr is released from all slots. + * + * Wait to observe that each slot contains a value that differs from + * @addr before returning. + * Should be called from preemptible context. + */ +void hazptr_synchronize(void *addr); + +/* + * hazptr_chain_backup_slot: Chain backup slot into overflow list. + * + * Set backup slot address to @addr, and chain it into the overflow + * list. + */ +struct hazptr_slot *hazptr_chain_backup_slot(struct hazptr_ctx *ctx); + +/* + * hazptr_unchain_backup_slot: Unchain backup slot from overflow list. + */ +void hazptr_unchain_backup_slot(struct hazptr_ctx *ctx); + +static inline +bool hazptr_slot_is_backup(struct hazptr_ctx *ctx, struct hazptr_slot *slo= t) +{ + return slot =3D=3D &ctx->backup_slot.slot; +} + +static inline +void hazptr_note_context_switch(void) +{ + struct hazptr_percpu_slots *percpu_slots =3D this_cpu_ptr(&hazptr_percpu_= slots); + unsigned int idx; + + for (idx =3D 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) { + struct hazptr_slot_item *item =3D &percpu_slots->items[idx]; + struct hazptr_slot *slot =3D &item->slot, *backup_slot; + struct hazptr_ctx *ctx; + + if (!slot->addr) + continue; + ctx =3D item->ctx.ctx; + backup_slot =3D hazptr_chain_backup_slot(ctx); + /* + * Move hazard pointer from the per-CPU slot to the + * backup slot. This requires hazard pointer + * synchronize to iterate on per-CPU slots with + * load-acquire before iterating on the overflow list. + */ + WRITE_ONCE(backup_slot->addr, slot->addr); + /* + * store-release orders store to backup slot addr before + * store to per-CPU slot addr. + */ + smp_store_release(&slot->addr, NULL); + /* Use the backup slot for context. */ + ctx->slot =3D backup_slot; + } +} + +/* + * hazptr_acquire: Load pointer at address and protect with hazard pointer. + * + * Load @addr_p, and protect the loaded pointer with hazard pointer. + * When using hazptr_acquire from interrupt handlers, the acquired slots + * need to be released before returning from the interrupt handler. + * + * Returns a non-NULL protected address if the loaded pointer is non-NULL. + * Returns NULL if the loaded pointer is NULL. + * + * On success the protected hazptr slot is stored in @ctx->slot. + */ +static inline +void *hazptr_acquire(struct hazptr_ctx *ctx, void * const *addr_p) +{ + struct hazptr_percpu_slots *percpu_slots; + struct hazptr_slot_item *slot_item; + struct hazptr_slot *slot; + void *addr; + + guard(preempt)(); + percpu_slots =3D this_cpu_ptr(&hazptr_percpu_slots); + slot_item =3D &percpu_slots->items[0]; + slot =3D &slot_item->slot; + if (unlikely(slot->addr)) + return __hazptr_acquire(ctx, addr_p); + WRITE_ONCE(slot->addr, HAZPTR_WILDCARD); /* Store B */ + + /* Memory ordering: Store B before Load A. */ + smp_mb(); + + /* + * Load @addr_p after storing wildcard to the hazard pointer slot. + */ + addr =3D READ_ONCE(*addr_p); /* Load A */ + + /* + * We don't care about ordering of Store C. It will simply + * replace the wildcard by a more specific address. If addr is + * NULL, we simply store NULL into the slot. + */ + WRITE_ONCE(slot->addr, addr); /* Store C */ + slot_item->ctx.ctx =3D ctx; + ctx->slot =3D slot; + return addr; +} + +/* Release the protected hazard pointer from @slot. */ +static inline +void hazptr_release(struct hazptr_ctx *ctx, void *addr) +{ + struct hazptr_slot *slot; + + if (!addr) + return; + guard(preempt)(); + slot =3D ctx->slot; + smp_store_release(&slot->addr, NULL); + if (unlikely(hazptr_slot_is_backup(ctx, slot))) + hazptr_unchain_backup_slot(ctx); +} + +void hazptr_init(void); + +#endif /* _LINUX_HAZPTR_H */ diff --git a/init/main.c b/init/main.c index e363232b428b47..e41f3ae436d496 100644 --- a/init/main.c +++ b/init/main.c @@ -107,6 +107,7 @@ #include #include #include +#include #include =20 #include @@ -1065,6 +1066,7 @@ void start_kernel(void) workqueue_init_early(); =20 rcu_init(); + hazptr_init(); kvfree_rcu_init(); =20 /* Trace events are available after this */ diff --git a/kernel/Makefile b/kernel/Makefile index 1e1a31673577d6..8961c8660d0dce 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -7,7 +7,7 @@ obj-y =3D fork.o exec_domain.o exec_state.o panic.o \ cpu.o exit.o softirq.o resource.o \ sysctl.o capability.o ptrace.o user.o \ signal.o sys.o umh.o workqueue.o pid.o task_work.o \ - extable.o params.o \ + extable.o params.o hazptr.o \ kthread.o sys_ni.o nsproxy.o nstree.o nscommon.o \ notifier.o ksysfs.o cred.o reboot.o \ async.o range.o smpboot.o ucount.o regset.o ksyms_common.o diff --git a/kernel/hazptr.c b/kernel/hazptr.c new file mode 100644 index 00000000000000..a63ac681cb85df --- /dev/null +++ b/kernel/hazptr.c @@ -0,0 +1,242 @@ +// SPDX-FileCopyrightText: 2024 Mathieu Desnoyers +// +// SPDX-License-Identifier: LGPL-2.1-or-later + +/* + * hazptr: Hazard Pointers + */ + +#include +#include +#include +#include +#include +#include + +struct hazptr_overflow_list { + raw_spinlock_t lock; /* Lock protecting overflow list and list generatio= n. */ + struct hlist_head head; /* Overflow list head. */ + uint64_t gen; /* Overflow list generation. */ +}; + +/* + * Flip between two lists to guarantee list scan forward progress even + * with frequent generation counter increments. The list additions are + * always done on a different list than the one used for scan. The scan + * successively iterates on both lists. Therefore, only list removals + * can cause the iteration to retry, and the number of removals is + * limited to the number of list elements. + */ +struct hazptr_overflow_list_flip { + struct mutex lock; /* Mutex protecting add_idx from concurrent updates. = */ + unsigned int add_idx; /* Index of current flip-list to add to. */ + struct hazptr_overflow_list array[2]; +}; + +static DEFINE_PER_CPU(struct hazptr_overflow_list_flip, percpu_overflow_li= st_flip); + +DEFINE_PER_CPU(struct hazptr_percpu_slots, hazptr_percpu_slots); +EXPORT_PER_CPU_SYMBOL_GPL(hazptr_percpu_slots); + +static +struct hazptr_slot *hazptr_get_free_percpu_slot(struct hazptr_ctx *ctx) +{ + struct hazptr_percpu_slots *percpu_slots =3D this_cpu_ptr(&hazptr_percpu_= slots); + unsigned int idx; + + for (idx =3D 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) { + struct hazptr_slot_item *item =3D &percpu_slots->items[idx]; + struct hazptr_slot *slot =3D &item->slot; + + if (!slot->addr) { + item->ctx.ctx =3D ctx; + return slot; + } + } + /* All slots are in use. */ + return NULL; +} + +/* + * Hazard pointer acquire slow path. + * Called with preemption disabled. + */ +void *__hazptr_acquire(struct hazptr_ctx *ctx, void * const *addr_p) +{ + struct hazptr_slot *slot =3D hazptr_get_free_percpu_slot(ctx); + void *addr; + + /* + * If all the per-CPU slots are already in use, fallback + * to the backup slot. + */ + if (unlikely(!slot)) + slot =3D hazptr_chain_backup_slot(ctx); + WRITE_ONCE(slot->addr, HAZPTR_WILDCARD); /* Store B */ + + /* Memory ordering: Store B before Load A. */ + smp_mb(); + + /* + * Load @addr_p after storing wildcard to the hazard pointer slot. + */ + addr =3D READ_ONCE(*addr_p); /* Load A */ + + /* + * We don't care about ordering of Store C. It will simply + * replace the wildcard by a more specific address. If addr is + * NULL, we simply store NULL into the slot. + */ + WRITE_ONCE(slot->addr, addr); /* Store C */ + ctx->slot =3D slot; + if (!addr && hazptr_slot_is_backup(ctx, slot)) + hazptr_unchain_backup_slot(ctx); + return addr; +} +EXPORT_SYMBOL_GPL(__hazptr_acquire); + +/* + * Perform piecewise iteration on overflow list waiting until "addr" is + * not present. Raw spinlock is released and taken between each list + * item and busy loop iteration. The overflow list generation is checked + * each time the lock is taken to validate that the list has not changed + * before resuming iteration or busy wait. If the generation has + * changed, retry the entire list traversal. + */ +static +void hazptr_synchronize_overflow_list(struct hazptr_overflow_list *overflo= w_list, void *addr) +{ + struct hazptr_backup_slot *backup_slot; + uint64_t snapshot_gen; + unsigned long flags; + + raw_spin_lock_irqsave(&overflow_list->lock, flags); +retry: + snapshot_gen =3D overflow_list->gen; + hlist_for_each_entry(backup_slot, &overflow_list->head, overflow_node) { + /* Busy-wait if node is found. */ + for (;;) { + void *load_addr =3D smp_load_acquire(&backup_slot->slot.addr); /* Load = B */ + + if (load_addr !=3D addr && load_addr !=3D HAZPTR_WILDCARD) + break; + raw_spin_unlock_irqrestore(&overflow_list->lock, flags); + cpu_relax(); + raw_spin_lock_irqsave(&overflow_list->lock, flags); + if (overflow_list->gen !=3D snapshot_gen) + goto retry; + } + raw_spin_unlock_irqrestore(&overflow_list->lock, flags); + /* + * Release raw spinlock, validate generation after + * re-acquiring the lock. + */ + raw_spin_lock_irqsave(&overflow_list->lock, flags); + if (overflow_list->gen !=3D snapshot_gen) + goto retry; + } + raw_spin_unlock_irqrestore(&overflow_list->lock, flags); +} + +static +void hazptr_synchronize_cpu_slots(int cpu, void *addr) +{ + struct hazptr_percpu_slots *percpu_slots =3D per_cpu_ptr(&hazptr_percpu_s= lots, cpu); + unsigned int idx; + + for (idx =3D 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) { + struct hazptr_slot_item *item =3D &percpu_slots->items[idx]; + + /* Busy-wait if node is found. */ + smp_cond_load_acquire(&item->slot.addr, VAL !=3D addr && VAL !=3D HAZPTR= _WILDCARD); /* Load B */ + } +} + +/* + * hazptr_synchronize: Wait until @addr is released from all slots. + * + * Wait to observe that each slot contains a value that differs from + * @addr before returning. + * Should be called from preemptible context. + */ +void hazptr_synchronize(void *addr) +{ + int cpu; + + /* + * Busy-wait should only be done from preemptible context. + */ + lockdep_assert_preemption_enabled(); + + /* + * Store A precedes hazptr_scan(): it unpublishes addr (sets it to + * NULL or to a different value), and thus hides it from hazard + * pointer readers. + */ + if (!addr) + return; + /* Memory ordering: Store A before Load B. */ + smp_mb(); + /* Scan all CPUs slots. */ + for_each_possible_cpu(cpu) { + struct hazptr_overflow_list_flip *overflow_list_flip =3D per_cpu_ptr(&pe= rcpu_overflow_list_flip, cpu); + unsigned int scan_idx; + + /* Scan CPU slots. */ + hazptr_synchronize_cpu_slots(cpu, addr); + + /* + * Scan backup slots in percpu overflow lists. + * Forward progress is guaranteed by scanning one list + * while new elements are added into the other list. + */ + guard(mutex)(&overflow_list_flip->lock); + scan_idx =3D overflow_list_flip->add_idx ^ 1; + hazptr_synchronize_overflow_list(&overflow_list_flip->array[scan_idx], a= ddr); + /* Flip current list. */ + WRITE_ONCE(overflow_list_flip->add_idx, scan_idx); + hazptr_synchronize_overflow_list(&overflow_list_flip->array[scan_idx ^ 1= ], addr); + } +} +EXPORT_SYMBOL_GPL(hazptr_synchronize); + +struct hazptr_slot *hazptr_chain_backup_slot(struct hazptr_ctx *ctx) +{ + struct hazptr_overflow_list_flip *overflow_list_flip =3D this_cpu_ptr(&pe= rcpu_overflow_list_flip); + unsigned int list_idx =3D READ_ONCE(overflow_list_flip->add_idx); + struct hazptr_overflow_list *overflow_list =3D &overflow_list_flip->array= [list_idx]; + struct hazptr_slot *slot =3D &ctx->backup_slot.slot; + + slot->addr =3D NULL; + guard(raw_spinlock_irqsave)(&overflow_list->lock); + overflow_list->gen++; + hlist_add_head(&ctx->backup_slot.overflow_node, &overflow_list->head); + ctx->backup_slot.overflow_list =3D overflow_list; + return slot; +} +EXPORT_SYMBOL_GPL(hazptr_chain_backup_slot); + +void hazptr_unchain_backup_slot(struct hazptr_ctx *ctx) +{ + struct hazptr_overflow_list *overflow_list =3D ctx->backup_slot.overflow_= list; + + guard(raw_spinlock_irqsave)(&overflow_list->lock); + overflow_list->gen++; + hlist_del(&ctx->backup_slot.overflow_node); +} +EXPORT_SYMBOL_GPL(hazptr_unchain_backup_slot); + +void __init hazptr_init(void) +{ + int cpu; + + for_each_possible_cpu(cpu) { + struct hazptr_overflow_list_flip *overflow_list_flip =3D per_cpu_ptr(&pe= rcpu_overflow_list_flip, cpu); + + mutex_init(&overflow_list_flip->lock); + for (int i =3D 0; i < 2; i++) { + raw_spin_lock_init(&overflow_list_flip->array[i].lock); + INIT_HLIST_HEAD(&overflow_list_flip->array[i].head); + } + } +} diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 96226707c2f613..e3fb70c3843689 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -60,6 +60,7 @@ #include #include #include +#include #include #include #include @@ -7087,6 +7088,7 @@ static void __sched notrace __schedule(int sched_mode) local_irq_disable(); rcu_note_context_switch(preempt); migrate_disable_switch(rq, prev); + hazptr_note_context_switch(); =20 /* * Make sure that signal_pending_state()->signal_pending() below --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95D6819C556; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161092; cv=none; b=qrj7ZpVkUDWPog+keXrTiORhj9+DztAXAeuSUJLGdT3AxH7bLyDFsi7E+nyJWp/h9qF09VmnAQwfCKhn4jCoY2pWZfdic6AOsDd9z3hCZUM2KPcyLDR5QDK1PHpBdScaZI6NPLshokIdfwR27MSy7+BJAYDtN9iruZnPRcHWsLU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161092; c=relaxed/simple; bh=5CQy2SixWdszyZBFvsnR1ERCtjWgrC0RnKvY/mF+7wE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OZmPDypFYsufwhmR52l0rfDS+BD0wwJOtlomVyZKI3rRiHAgBOutkQ7nzKQw/0UMqXYSohmWAJqlBlrttXOqmxV0RZAFekdudfgXBdYCtglqaqBSFwllc2OxrDNR/WeT6uuVz0+0xSfNvA0/Gt3EU55RDfKXzH7GXktBAX1oAL4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kQVQALlM; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kQVQALlM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4DB6A1F00A3A; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=hKV57X1DTBu0Cfp4aIMAmc3dhW9PqHag1BVOJPYPt6w=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=kQVQALlMKzkVNZDp0tUiKlTFJlzW7PNfLfZZa71OTwrJktCxeNXT6I2yXIJKtZV9/ XcK0VhLCMqa4lveym9swW5gzozxiDw/mnHJ2cNEFq9EM5/ADcmtKE0QYrrUGVCKBHh 3JHf6zQ77TKGIz5Ge53hgAsaero4GCA8vofFQrbXmKtbppQq7Y01KS+Jp50//WA3YR blbugshSUZm80HIrUDDqVlUX82qaaXWWsNFGT0yvwTZn/ZyuYY4rYA4P6dmzzmljUO yms6J2vLS5dOlgqSaWW1IWjnNyUn9SpZhxl6ajKm91AZCp+v/dXe/rR4rrRm3vfY9K 2/oxUtbsSZjYg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 280A8CE0B87; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Mathieu Desnoyers , Boqun Feng , "Paul E . McKenney" Subject: [PATCH RFC v2 02/24] hazptr: Add refscale test Date: Wed, 15 Jul 2026 17:17:47 -0700 Message-Id: <20260716001809.11084-2-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mathieu Desnoyers Add the refscale test for hazptr to measure the reader side performance. Signed-off-by: Mathieu Desnoyers Co-developed-by: Boqun Feng Signed-off-by: Paul E. McKenney --- kernel/rcu/refscale.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/kernel/rcu/refscale.c b/kernel/rcu/refscale.c index a2d9d75d88a10d..2320b3fedec736 100644 --- a/kernel/rcu/refscale.c +++ b/kernel/rcu/refscale.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -1200,6 +1201,47 @@ static const struct ref_scale_ops typesafe_seqlock_o= ps =3D { .name =3D "typesafe_seqlock" }; =20 +static void ref_hazptr_read_section(const int nloops) +{ + static void *ref_hazptr_read_section_ptr =3D ref_hazptr_read_section; + int i; + + for (i =3D nloops; i >=3D 0; i--) { + struct hazptr_ctx ctx; + void *addr; + + addr =3D hazptr_acquire(&ctx, &ref_hazptr_read_section_ptr); + hazptr_release(&ctx, addr); + } +} + +static void ref_hazptr_delay_section(const int nloops, const int udl, cons= t int ndl) +{ + static void *ref_hazptr_delay_section_ptr =3D ref_hazptr_delay_section; + int i; + + for (i =3D nloops; i >=3D 0; i--) { + struct hazptr_ctx ctx; + void *addr; + + addr =3D hazptr_acquire(&ctx, &ref_hazptr_delay_section_ptr); + un_delay(udl, ndl); + hazptr_release(&ctx, addr); + } +} + +static bool ref_hazptr_init(void) +{ + return true; +} + +static const struct ref_scale_ops hazptr_ops =3D { + .init =3D ref_hazptr_init, + .readsection =3D ref_hazptr_read_section, + .delaysection =3D ref_hazptr_delay_section, + .name =3D "hazptr" +}; + static void rcu_scale_one_reader(void) { if (readdelay <=3D 0) @@ -1504,6 +1546,7 @@ ref_scale_init(void) &sched_clock_ops, &clock_ops, &jiffies_ops, &preempt_ops, &bh_ops, &irq_ops, &irqsave_ops, &typesafe_ref_ops, &typesafe_lock_ops, &typesafe_seqlock_ops, + &hazptr_ops, }; =20 if (!torture_init_begin(scale_type, verbose)) --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AF031F936; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161093; cv=none; b=DbBf7zgilhyK8fpjQxbqCp/0N9ltYW8WCnn4lGjwHOUszJFypVfLOVhSrs0xrResMzKHgBnpTcN4PIySToOcGQDfs8ApFBbuCL8h7uDr2/Ybudj0lrT7ghFSpfc3pbhDEyBhGGiq2pp6O0t+iawZ2hXWICG6MdLIFCqWDB0suCQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161093; c=relaxed/simple; bh=cEFy5Z4SVq9QRHL6OBD0YY8yyqsTPR9ahljwhud2yBw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QXnB1+90cdCtW2CGCehZn12Qvc5Oa8+9o/eYMNO/vz4zt40S9yXpg7g9nQE2nHdrGvWN7mEyspEtG8Jg3ayMcMbQ+rZ7K5TQVPTheZR9CCCdXXsizbydPleIQv4fEDxaQa3zQRSIBv2OZpdedZhgqU5h7loLId0X9ybcTS0L91Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ry6QhdVo; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ry6QhdVo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 550021F00AC4; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=LFpG011Bg1Fu+D9zv5ydI3cr1OWU5y5JPbigcrJXvOk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Ry6QhdVokhza5OyGhjig+E5vD2xepDZZULXK8/bqde+ZqAvgXOMp9Mg5AI4DagZK7 9R2qvi7LQoEj8EBGEP+ORfyyiOkpWaXN/SBPDHAIfJ/Ws/FCaPRcuGuf+sDU9sKM54 dJu8tdICDIWz5pYEdeYCAUb92bNshX623HHdtl0N2MArWEMn6weIYBJcyIO+NblBnq q8qSMAOLd4gTUszSN0MZVoQDxehhCRmqZ4ZvpLcQVRMQc0lat1TD3pLplImtldv3nC Odsl4Gloae5ykXic6pIlDMBG5ub7XpSd5Vbp8U80p7alV0tglWyW/Vk5uMyj/5P+od aFKMdTMoJa3UQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 2AF40CE0B8B; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 03/24] torture: Add a hazptrtorture.c torture test Date: Wed, 15 Jul 2026 17:17:48 -0700 Message-Id: <20260716001809.11084-3-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds a torture test for hazard pointers. The initial version simply acquires and releases the hazard pointers without nesting, each from within the context of a single task. [ paulmck: Apply kernel test robot feedback. ] Signed-off-by: Paul E. McKenney --- include/linux/torture.h | 2 +- kernel/rcu/Kconfig.debug | 12 + kernel/rcu/Makefile | 1 + kernel/rcu/hazptrtorture.c | 682 ++++++++++++++++++ kernel/rcu/update.c | 3 +- tools/testing/selftests/rcutorture/bin/kvm.sh | 6 +- .../rcutorture/configs/hazptr/CFLIST | 2 + .../rcutorture/configs/hazptr/CFcommon | 2 + .../rcutorture/configs/hazptr/NOPREEMPT | 17 + .../rcutorture/configs/hazptr/PREEMPT | 14 + .../configs/hazptr/ver_functions.sh | 40 + 11 files changed, 776 insertions(+), 5 deletions(-) create mode 100644 kernel/rcu/hazptrtorture.c create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/CFLIST create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/CFcom= mon create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/NOPRE= EMPT create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/PREEM= PT create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/ver_f= unctions.sh diff --git a/include/linux/torture.h b/include/linux/torture.h index c9b47d13830295..66d2d444428aef 100644 --- a/include/linux/torture.h +++ b/include/linux/torture.h @@ -131,7 +131,7 @@ void _torture_stop_kthread(char *m, struct task_struct = **tp); #endif void torture_sched_set_normal(struct task_struct *t, int nice); =20 -#if IS_ENABLED(CONFIG_RCU_TORTURE_TEST) || IS_MODULE(CONFIG_RCU_TORTURE_TE= ST) || IS_ENABLED(CONFIG_LOCK_TORTURE_TEST) || IS_MODULE(CONFIG_LOCK_TORTUR= E_TEST) +#if IS_ENABLED(CONFIG_RCU_TORTURE_TEST) || IS_ENABLED(CONFIG_LOCK_TORTURE_= TEST) || IS_ENABLED(CONFIG_HAZPTR_TORTURE_TEST) long torture_sched_setaffinity(pid_t pid, const struct cpumask *in_mask, b= ool dowarn); #endif =20 diff --git a/kernel/rcu/Kconfig.debug b/kernel/rcu/Kconfig.debug index e078e988773d6a..fe64356f008841 100644 --- a/kernel/rcu/Kconfig.debug +++ b/kernel/rcu/Kconfig.debug @@ -113,6 +113,18 @@ config RCU_REF_SCALE_TEST Say M if you want to build it as a module instead. Say N if you are unsure. =20 +config HAZPTR_TORTURE_TEST + tristate "Torture tests for hazard pointers" + depends on DEBUG_KERNEL + select TORTURE_TEST + default n + help + This option provides in-kernel hazard-pointer stress tests. + + Say Y here if you want hazard-pointer testing built into the kernel. + Say M if you want to build them as a module instead. + Say N if you are unsure. + config RCU_CPU_STALL_TIMEOUT int "RCU CPU stall timeout in seconds" depends on RCU_STALL_COMMON diff --git a/kernel/rcu/Makefile b/kernel/rcu/Makefile index 0cfb009a99b9f7..478ef2237fee6e 100644 --- a/kernel/rcu/Makefile +++ b/kernel/rcu/Makefile @@ -10,6 +10,7 @@ endif obj-y +=3D update.o sync.o obj-$(CONFIG_TREE_SRCU) +=3D srcutree.o obj-$(CONFIG_TINY_SRCU) +=3D srcutiny.o +obj-$(CONFIG_HAZPTR_TORTURE_TEST) +=3D hazptrtorture.o obj-$(CONFIG_RCU_TORTURE_TEST) +=3D rcutorture.o obj-$(CONFIG_RCU_SCALE_TEST) +=3D rcuscale.o obj-$(CONFIG_RCU_REF_SCALE_TEST) +=3D refscale.o diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c new file mode 100644 index 00000000000000..70f12d2a149069 --- /dev/null +++ b/kernel/rcu/hazptrtorture.c @@ -0,0 +1,682 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Hazard-pointer module-based torture test facility + * + * Copyright (c) 2026 Meta Platforms, Inc. and affiliates. + * + * Author: Paul E. McKenney + */ + +#define pr_fmt(fmt) fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rcu.h" + +MODULE_DESCRIPTION("Hazard-pointer module-based torture test facility"); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Paul E. McKenney "); + +torture_param(int, irqreader, 1, "Allow hazard-pointer readers from irq ha= ndlers"); +torture_param(int, nreaders, -1, "Number of hazard-pointer reader threads"= ); +torture_param(int, onoff_holdoff, 0, "Time after boot before CPU hotplugs = (s)"); +torture_param(int, onoff_interval, 0, "Time between CPU hotplugs (jiffies)= , 0=3Ddisable"); +torture_param(int, preempt_duration, 0, "Preemption duration (ms), zero to= disable"); +torture_param(int, preempt_interval, MSEC_PER_SEC, "Interval between preem= ptions (ms)"); +torture_param(int, shuffle_interval, 3, "Number of seconds between shuffle= s"); +torture_param(int, shutdown_secs, 0, "Shutdown time (s), <=3D zero to disa= ble."); +torture_param(int, stat_interval, 60, "Number of seconds between stats pri= ntk()s"); +torture_param(int, stutter, 5, "Number of seconds to run/halt test"); +torture_param(int, verbose, 1, "Enable verbose debugging printk()s"); + +static char *torture_type =3D "hazptr"; +module_param(torture_type, charp, 0444); +MODULE_PARM_DESC(torture_type, "Type of hazard pointers to torture (hazptr= , ...)"); + +static int nrealreaders; +static struct task_struct *writer_task; +static struct task_struct *preempt_task; +static struct task_struct **reader_tasks; +static struct task_struct *stats_task; + +#define HAZPTR_TORTURE_PIPE_LEN 10 + +// Update-side data structure used to check RCU readers. +struct hazptr_torture { + void *obj_hazptr; + int htort_pipe_count; + struct list_head htort_free; +}; + +static LIST_HEAD(hazptr_torture_freelist); +static struct hazptr_torture *hazptr_torture_current; +static unsigned long hazptr_torture_current_version; +static struct hazptr_torture hazptr_tortures[10 * HAZPTR_TORTURE_PIPE_LEN]; +static DEFINE_SPINLOCK(hazptr_torture_lock); +static DEFINE_PER_CPU(long [HAZPTR_TORTURE_PIPE_LEN + 1], hazptr_torture_c= ount); +static atomic_t hazptr_torture_wcount[HAZPTR_TORTURE_PIPE_LEN + 1]; +static atomic_t n_hazptr_torture_alloc; +static atomic_t n_hazptr_torture_alloc_fail; +static atomic_t n_hazptr_torture_free; +static atomic_t n_hazptr_torture_error; +static struct list_head hazptr_torture_removed; + +static int hazptr_torture_writer_state; +#define HTWS_FIXED_DELAY 0 +#define HTWS_DELAY 1 +#define HTWS_REPLACE 2 +#define HTWS_SYNC 3 +#define HTWS_STUTTER 4 +#define HTWS_STOPPING 5 +static const char * const hazptr_torture_writer_state_names[] =3D { + "HTWS_FIXED_DELAY", + "HTWS_DELAY", + "HTWS_REPLACE", + "HTWS_SYNC", + "HTWS_STUTTER", + "HTWS_STOPPING", +}; + +static const char *hazptr_torture_writer_state_getname(void) +{ + unsigned int i =3D READ_ONCE(hazptr_torture_writer_state); + + if (i >=3D ARRAY_SIZE(hazptr_torture_writer_state_names)) + return "???"; + return hazptr_torture_writer_state_names[i]; +} + +/* + * Allocate an element from the hazptr_tortures pool. + */ +static struct hazptr_torture *hazptr_torture_alloc(void) +{ + struct list_head *p; + + spin_lock_bh(&hazptr_torture_lock); + if (list_empty(&hazptr_torture_freelist)) { + atomic_inc(&n_hazptr_torture_alloc_fail); + spin_unlock_bh(&hazptr_torture_lock); + return NULL; + } + atomic_inc(&n_hazptr_torture_alloc); + p =3D hazptr_torture_freelist.next; + list_del_init(p); + spin_unlock_bh(&hazptr_torture_lock); + return container_of(p, struct hazptr_torture, htort_free); +} + +/* + * Free an element to the hazptr_tortures pool. + */ +static void +hazptr_torture_free(struct hazptr_torture *p) +{ + atomic_inc(&n_hazptr_torture_free); + spin_lock_bh(&hazptr_torture_lock); + list_add_tail(&p->htort_free, &hazptr_torture_freelist); + spin_unlock_bh(&hazptr_torture_lock); +} + +/* + * Update object in the pipe. This should be invoked after a suitable tim= e. + */ +static bool +hazptr_torture_pipe_update_one(struct hazptr_torture *rp) +{ + int i; + + i =3D rp->htort_pipe_count; + if (i > HAZPTR_TORTURE_PIPE_LEN) + i =3D HAZPTR_TORTURE_PIPE_LEN; + atomic_inc(&hazptr_torture_wcount[i]); + WRITE_ONCE(rp->htort_pipe_count, i + 1); + ASSERT_EXCLUSIVE_WRITER(rp->htort_pipe_count); + if (i + 1 >=3D HAZPTR_TORTURE_PIPE_LEN) + return true; + return false; +} + +/* + * Update all callbacks in the pipe each time period. + */ +static void +hazptr_torture_pipe_update(struct hazptr_torture *old_rp) +{ + struct hazptr_torture *rp; + struct hazptr_torture *rp1; + + if (old_rp) + list_add(&old_rp->htort_free, &hazptr_torture_removed); + list_for_each_entry_safe(rp, rp1, &hazptr_torture_removed, htort_free) { + if (hazptr_torture_pipe_update_one(rp)) { + list_del(&rp->htort_free); + hazptr_torture_free(rp); + } + } +} + +/* + * Operations vector for selecting different types of tests. + */ + +struct hazptr_torture_ops { + void (*init)(void); + void (*cleanup)(void); + struct hazptr_torture *((*readlock)(struct hazptr_ctx **hcpp)); + void (*read_delay)(struct torture_random_state *rrsp); + void (*readunlock)(struct hazptr_ctx *hcp, struct hazptr_torture *htp); + void (*sync)(void *htp); + int irq_capable; + int onstack_ctx; + const char *name; +}; + +static struct hazptr_torture_ops *cur_ops; + +/* + * Definitions for hazard-pointer torture testing. + */ + +static struct hazptr_torture *hazptr_torture_read_lock(struct hazptr_ctx *= *hcpp) +{ + struct hazptr_ctx *hcp =3D kmalloc(sizeof(*hcp), GFP_KERNEL); + + *hcpp =3D hcp; + if (!hcp) + return NULL; + return (struct hazptr_torture *)hazptr_acquire(hcp, (void *)&hazptr_tortu= re_current); +} + +static void hazptr_read_delay(struct torture_random_state *rrsp) +{ + const unsigned long shortdelay_us =3D 200; + unsigned long longdelay_ms =3D 300; + + /* We want a short delay sometimes to make a reader delay the grace + * period, and we want a long delay occasionally to trigger + * force_quiescent_state. */ + + if (!(torture_random(rrsp) % (nrealreaders * 2000 * longdelay_ms))) { + if ((preempt_count() & HARDIRQ_MASK) || softirq_count()) + longdelay_ms =3D 5; /* Avoid triggering BH limits. */ + mdelay(longdelay_ms); + } + if (!(torture_random(rrsp) % (nrealreaders * 2 * shortdelay_us))) + udelay(shortdelay_us); + if (!preempt_count() && !(torture_random(rrsp) % (nrealreaders * 500))) + torture_preempt_schedule(); /* QS only if preemptible. */ +} + +static void hazptr_torture_read_unlock(struct hazptr_ctx *hcp, struct hazp= tr_torture *htp) +{ + if (hcp) { + hazptr_release(hcp, htp); + if (cur_ops->onstack_ctx) + kfree(hcp); + } +} + +static void hazptr_sync_torture_init(void) +{ + INIT_LIST_HEAD(&hazptr_torture_removed); +} + +static struct hazptr_torture_ops hazptr_ops =3D { + .init =3D hazptr_sync_torture_init, + .readlock =3D hazptr_torture_read_lock, + .read_delay =3D hazptr_read_delay, + .readunlock =3D hazptr_torture_read_unlock, + .sync =3D hazptr_synchronize, + .irq_capable =3D 1, + .onstack_ctx =3D 1, + .name =3D "hazptr" +}; + +/* + * Hazard-pointer torture writer kthread. Repeatedly substitutes a new + * structure for that pointed to by hazptr_torture_current, freeing the + * old structure after a series of timeouts (the "pipeline"). + */ +static int +hazptr_torture_writer(void *arg) +{ + bool booting_still =3D false; + int i; + unsigned long j; + int oldnice =3D task_nice(current); + struct hazptr_torture *rp; + struct hazptr_torture *old_rp; + static DEFINE_TORTURE_RANDOM(rand); + bool stutter_waited; + + VERBOSE_TOROUT_STRING("hazptr_torture_writer task started"); + // If the system is still booting, let it finish. + j =3D jiffies; + while (!torture_must_stop() && !rcu_inkernel_boot_has_ended()) { + booting_still =3D true; + schedule_timeout_interruptible(HZ); + } + if (booting_still) + pr_alert("%s" TORTURE_FLAG " Waited %lu jiffies for boot to complete.\n", + torture_type, jiffies - j); + + do { + hazptr_torture_writer_state =3D HTWS_FIXED_DELAY; + torture_hrtimeout_us(500, 1000, &rand); + rp =3D hazptr_torture_alloc(); + if (rp =3D=3D NULL) + continue; + rp->htort_pipe_count =3D 0; + ASSERT_EXCLUSIVE_WRITER(rp->htort_pipe_count); + hazptr_torture_writer_state =3D HTWS_DELAY; + udelay(torture_random(&rand) & 0x3ff); + hazptr_torture_writer_state =3D HTWS_REPLACE; + old_rp =3D READ_ONCE(hazptr_torture_current); + smp_store_release(&hazptr_torture_current, rp); + smp_wmb(); /* Mods to old_rp must follow smp_store_release() */ + if (old_rp) { + i =3D old_rp->htort_pipe_count; + if (i > HAZPTR_TORTURE_PIPE_LEN) + i =3D HAZPTR_TORTURE_PIPE_LEN; + atomic_inc(&hazptr_torture_wcount[i]); + WRITE_ONCE(old_rp->htort_pipe_count, + old_rp->htort_pipe_count + 1); + ASSERT_EXCLUSIVE_WRITER(old_rp->htort_pipe_count); + + hazptr_torture_writer_state =3D HTWS_SYNC; + cur_ops->sync((void *)old_rp); + hazptr_torture_pipe_update(old_rp); + } + + WRITE_ONCE(hazptr_torture_current_version, hazptr_torture_current_versio= n + 1); + hazptr_torture_writer_state =3D HTWS_STUTTER; + stutter_waited =3D stutter_wait("hazptr_torture_writer"); + if (stutter_waited && !torture_must_stop()) + for (i =3D 0; i < ARRAY_SIZE(hazptr_tortures); i++) + if (list_empty(&hazptr_tortures[i].htort_free) && + READ_ONCE(hazptr_torture_current) !=3D &hazptr_tortures[i]) { + tracing_off(); + WARN(1, "%s: htort_pipe_count: %d\n", __func__, hazptr_tortures[i].ht= ort_pipe_count); + rcu_ftrace_dump(DUMP_ALL); + break; + } + if (stutter_waited) + sched_set_normal(current, oldnice); + } while (!torture_must_stop()); + hazptr_torture_current =3D NULL; // Let stats task know that we are done. + hazptr_torture_writer_state =3D HTWS_STOPPING; + torture_kthread_stopping("hazptr_torture_writer"); + return 0; +} + +/* + * Hazard-pointer torture reader kthread. Repeatedly dereferences + * hazptr_torture_current, incrementing the corresponding element of the + * pipeline array. The counter in the element should never be greater + * than 1, otherwise, the hazard-pointer implementation is broken. + */ +static int hazptr_torture_reader(void *arg) +{ + struct hazptr_ctx *hcp; + struct hazptr_torture *htp; + unsigned long lastsleep =3D jiffies; + long myid =3D (long)arg; + int mynumonline =3D myid; + int pipe_count; + DEFINE_TORTURE_RANDOM(rand); + + VERBOSE_TOROUT_STRING("hazptr_torture_reader task started"); + set_user_nice(current, MAX_NICE); + do { + htp =3D cur_ops->readlock(&hcp); + if (!htp) { + schedule_timeout_interruptible(HZ / 10); + continue; + } + if (time_after(jiffies, lastsleep) && !torture_must_stop()) { + torture_hrtimeout_us(500, 1000, &rand); + lastsleep =3D jiffies + 10; + } + cur_ops->read_delay(&rand); + preempt_disable(); + pipe_count =3D READ_ONCE(htp->htort_pipe_count); + if (pipe_count > HAZPTR_TORTURE_PIPE_LEN) { + // Should not happen in a correct RCU implementation, + // happens quite often for torture_type=3Dbusted. + pipe_count =3D HAZPTR_TORTURE_PIPE_LEN; + } + if (pipe_count > 1) + rcu_ftrace_dump(DUMP_ALL); + __this_cpu_inc(hazptr_torture_count[pipe_count]); + preempt_enable(); + cur_ops->readunlock(hcp, htp); + while (!torture_must_stop() && + (torture_num_online_cpus() < mynumonline || !rcu_inkernel_boot_ha= s_ended())) + schedule_timeout_interruptible(HZ / 5); + stutter_wait("hazptr_torture_reader"); + } while (!torture_must_stop()); + torture_kthread_stopping("hazptr_torture_reader"); + return 0; +} + +/* + * Print torture statistics. Caller must ensure that there is only one + * call to this function at a given time!!! This is normally accomplished + * by relying on the module system to only have one copy of the module + * loaded, and then by giving the hazptr_torture_stats kthread full control + * (or the init/cleanup functions when hazptr_torture_stats thread is + * not running). + */ +static void +hazptr_torture_stats_print(void) +{ + const char *cp =3D hazptr_torture_writer_state_getname();; + int cpu; + int i; + long pipesummary[HAZPTR_TORTURE_PIPE_LEN + 1] =3D { 0 }; + long batchsummary[HAZPTR_TORTURE_PIPE_LEN + 1] =3D { 0 }; + struct hazptr_torture *rtcp; + static unsigned long rtcv_snap =3D ULONG_MAX; + static bool splatted; + struct task_struct *wtp; + + for_each_possible_cpu(cpu) + for (i =3D 0; i < HAZPTR_TORTURE_PIPE_LEN + 1; i++) + pipesummary[i] +=3D READ_ONCE(per_cpu(hazptr_torture_count, cpu)[i]); + for (i =3D HAZPTR_TORTURE_PIPE_LEN; i >=3D 0; i--) { + if (pipesummary[i] !=3D 0) + break; + } // The value of variable "i" is used later, so don't clobber it! + + pr_alert("%s%s ", torture_type, TORTURE_FLAG); + rtcp =3D READ_ONCE(hazptr_torture_current); + pr_cont("rtc: %p %s: %lu %s tfle: %d rta: %d rtaf: %d rtf: %d ", + rtcp, + rtcp && !rcu_stall_is_suppressed_at_boot() ? "ver" : "VER", + hazptr_torture_current_version, + cp, + list_empty(&hazptr_torture_freelist), + atomic_read(&n_hazptr_torture_alloc), + atomic_read(&n_hazptr_torture_alloc_fail), + atomic_read(&n_hazptr_torture_free)); + torture_onoff_stats(); + + pr_alert("%s%s ", torture_type, TORTURE_FLAG); + if (i > 1) { + pr_cont("%s", "!!! "); + atomic_inc(&n_hazptr_torture_error); + WARN_ON_ONCE(i > 1); // Too-short grace period + } + pr_cont("Reader Pipe: "); + for (i =3D 0; i < HAZPTR_TORTURE_PIPE_LEN + 1; i++) + pr_cont(" %ld", pipesummary[i]); + pr_cont("\n"); + + pr_alert("%s%s ", torture_type, TORTURE_FLAG); + pr_cont("Reader Batch: "); + for (i =3D 0; i < HAZPTR_TORTURE_PIPE_LEN + 1; i++) + pr_cont(" %ld", batchsummary[i]); + pr_cont("\n"); + + pr_alert("%s%s ", torture_type, TORTURE_FLAG); + pr_cont("Free-Block Circulation: "); + for (i =3D 0; i < HAZPTR_TORTURE_PIPE_LEN + 1; i++) { + pr_cont(" %d", atomic_read(&hazptr_torture_wcount[i])); + } + pr_cont("\n"); + + if (rtcv_snap =3D=3D hazptr_torture_current_version && + READ_ONCE(hazptr_torture_current) && + rcu_inkernel_boot_has_ended()) { + int __maybe_unused flags =3D 0; + unsigned long __maybe_unused gp_seq =3D 0; + + wtp =3D READ_ONCE(writer_task); + pr_alert("??? Writer stall state %s(%d) g%lu f%#x ->state %#x cpu %d\n", + hazptr_torture_writer_state_getname(), + hazptr_torture_writer_state, gp_seq, flags, + wtp =3D=3D NULL ? ~0U : wtp->__state, + wtp =3D=3D NULL ? -1 : (int)task_cpu(wtp)); + if (!splatted && wtp) { + sched_show_task(wtp); + splatted =3D true; + } + rcu_ftrace_dump(DUMP_ALL); + } + rtcv_snap =3D hazptr_torture_current_version; +} + +/* + * Periodically prints torture statistics, if periodic statistics printing + * was specified via the stat_interval module parameter. + */ +static int +hazptr_torture_stats(void *arg) +{ + VERBOSE_TOROUT_STRING("hazptr_torture_stats task started"); + do { + schedule_timeout_interruptible(stat_interval * HZ); + hazptr_torture_stats_print(); + torture_shutdown_absorb("hazptr_torture_stats"); + } while (!torture_must_stop()); + torture_kthread_stopping("hazptr_torture_stats"); + return 0; +} + +static void +hazptr_torture_print_module_parms(struct hazptr_torture_ops *cur_ops, cons= t char *tag) +{ + pr_alert("%s" TORTURE_FLAG + "--- %s: nreaders=3D%d " + "stat_interval=3D%d verbose=3D%d " + "shuffle_interval=3D%d stutter=3D%d irqreader=3D%d " + "onoff_interval=3D%d onoff_holdoff=3D%d\n", + torture_type, tag, nrealreaders, + stat_interval, verbose, + shuffle_interval, stutter, irqreader, + onoff_interval, onoff_holdoff); +} + +// Randomly preempt online CPUs. +static int hazptr_torture_preempt(void *unused) +{ + int cpu =3D -1; + DEFINE_TORTURE_RANDOM(rand); + + schedule_timeout_idle(onoff_holdoff * HZ); + do { + // Wait for preempt_interval ms with up to 100us fuzz. + torture_hrtimeout_ms(preempt_interval, 100, &rand); + // Select online CPU. + cpu =3D cpumask_next(cpu, cpu_online_mask); + if (cpu >=3D nr_cpu_ids) + cpu =3D cpumask_next(-1, cpu_online_mask); + WARN_ON_ONCE(cpu >=3D nr_cpu_ids); + // Move to that CPU, if can't do so, retry later. + if (torture_sched_setaffinity(current->pid, cpumask_of(cpu), false)) + continue; + // Preempt at high-ish priority, then reset to normal. + sched_set_fifo(current); + torture_sched_setaffinity(current->pid, cpu_present_mask, true); + mdelay(preempt_duration); + sched_set_normal(current, 0); + stutter_wait("hazptr_torture_preempt"); + } while (!torture_must_stop()); + torture_kthread_stopping("hazptr_torture_preempt"); + return 0; +} + +static void +hazptr_torture_cleanup(void) +{ + int i; + + if (torture_cleanup_begin()) + return; + if (!cur_ops) { + torture_cleanup_end(); + return; + } + + torture_stop_kthread(hazptr_torture_preempt, preempt_task); + torture_stop_kthread(hazptr_torture_writer, writer_task); + + if (reader_tasks) { + for (i =3D 0; i < nrealreaders; i++) + torture_stop_kthread(hazptr_torture_reader, + reader_tasks[i]); + kfree(reader_tasks); + reader_tasks =3D NULL; + } + + torture_stop_kthread(hazptr_torture_stats, stats_task); + + /* Do torture-type-specific cleanup operations. */ + if (cur_ops->cleanup !=3D NULL) + cur_ops->cleanup(); + + hazptr_torture_stats_print(); /* -After- the stats thread is stopped! */ + if (atomic_read(&n_hazptr_torture_error)) + hazptr_torture_print_module_parms(cur_ops, "End of test: FAILURE"); + else if (torture_onoff_failures()) + hazptr_torture_print_module_parms(cur_ops, "End of test: HAZPTR_HOTPLUG"= ); + else + hazptr_torture_print_module_parms(cur_ops, "End of test: SUCCESS"); + torture_cleanup_end(); +} + +static int __init hazptr_torture_init(void) +{ + long i; + int cpu; + int firsterr =3D 0; + static struct hazptr_torture_ops *torture_ops[] =3D { &hazptr_ops, }; + + if (!torture_init_begin(torture_type, verbose)) + return -EBUSY; + + /* Process args and tell the world that the torturer is on the job. */ + for (i =3D 0; i < ARRAY_SIZE(torture_ops); i++) { + cur_ops =3D torture_ops[i]; + if (strcmp(torture_type, cur_ops->name) =3D=3D 0) + break; + } + if (i =3D=3D ARRAY_SIZE(torture_ops)) { + pr_alert("hazptr-torture: invalid torture type: \"%s\"\n", torture_type); + pr_alert("hazptr-torture types:"); + for (i =3D 0; i < ARRAY_SIZE(torture_ops); i++) + pr_cont(" %s", torture_ops[i]->name); + pr_cont("\n"); + firsterr =3D -EINVAL; + cur_ops =3D NULL; + goto unwind; + } + + if (cur_ops->init) + cur_ops->init(); + + if (nreaders >=3D 0) { + nrealreaders =3D nreaders; + } else { + nrealreaders =3D num_online_cpus() - 2 - nreaders; + if (nrealreaders <=3D 0) + nrealreaders =3D 1; + } + hazptr_torture_print_module_parms(cur_ops, "Start of test"); + + /* Set up the freelist. */ + INIT_LIST_HEAD(&hazptr_torture_freelist); + for (i =3D 0; i < ARRAY_SIZE(hazptr_tortures); i++) + list_add_tail(&hazptr_tortures[i].htort_free, &hazptr_torture_freelist); + + /* Initialize the statistics so that each run gets its own numbers. */ + + hazptr_torture_current =3D NULL; + hazptr_torture_current_version =3D 0; + atomic_set(&n_hazptr_torture_alloc, 0); + atomic_set(&n_hazptr_torture_alloc_fail, 0); + atomic_set(&n_hazptr_torture_free, 0); + atomic_set(&n_hazptr_torture_error, 0); + for (i =3D 0; i < HAZPTR_TORTURE_PIPE_LEN + 1; i++) + atomic_set(&hazptr_torture_wcount[i], 0); + for_each_possible_cpu(cpu) { + for (i =3D 0; i < HAZPTR_TORTURE_PIPE_LEN + 1; i++) + per_cpu(hazptr_torture_count, cpu)[i] =3D 0; + } + + /* Start up the kthreads. */ + + reader_tasks =3D kzalloc_objs(reader_tasks[0], nrealreaders); + for (i =3D 0; i < nrealreaders; i++) { + firsterr =3D torture_create_kthread(hazptr_torture_reader, (void *)i, + reader_tasks[i]); + if (torture_init_error(firsterr)) + goto unwind; + } + + firsterr =3D torture_create_kthread(hazptr_torture_writer, NULL, writer_t= ask); + if (torture_init_error(firsterr)) + goto unwind; + + firsterr =3D torture_onoff_init(onoff_holdoff * HZ, onoff_interval, NULL); + if (torture_init_error(firsterr)) + goto unwind; + + if (stat_interval > 0) { + firsterr =3D torture_create_kthread(hazptr_torture_stats, NULL, stats_ta= sk); + if (torture_init_error(firsterr)) + goto unwind; + } + if (shuffle_interval > 0) { + firsterr =3D torture_shuffle_init(shuffle_interval * HZ); + if (torture_init_error(firsterr)) + goto unwind; + } + if (stutter < 0) + stutter =3D 0; + if (stutter) { + int t; + + t =3D stutter * HZ; + firsterr =3D torture_stutter_init(stutter * HZ, t); + if (torture_init_error(firsterr)) + goto unwind; + } + firsterr =3D torture_shutdown_init(shutdown_secs, hazptr_torture_cleanup); + if (torture_init_error(firsterr)) + goto unwind; + if (preempt_duration > 0) { + firsterr =3D torture_create_kthread(hazptr_torture_preempt, NULL, preemp= t_task); + if (torture_init_error(firsterr)) + goto unwind; + } + + torture_init_end(); + return 0; + +unwind: + torture_init_end(); + hazptr_torture_cleanup(); + if (shutdown_secs) { + WARN_ON(!IS_MODULE(CONFIG_HAZPTR_TORTURE_TEST)); + kernel_power_off(); + } + return firsterr; +} + +module_init(hazptr_torture_init); +module_exit(hazptr_torture_cleanup); diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c index b62735a6788423..2a778b8ab4ad78 100644 --- a/kernel/rcu/update.c +++ b/kernel/rcu/update.c @@ -44,6 +44,7 @@ #include #include #include +#include =20 #define CREATE_TRACE_POINTS =20 @@ -525,7 +526,7 @@ EXPORT_SYMBOL_GPL(do_trace_rcu_torture_read); do { } while (0) #endif =20 -#if IS_ENABLED(CONFIG_RCU_TORTURE_TEST) || IS_MODULE(CONFIG_RCU_TORTURE_TE= ST) || IS_ENABLED(CONFIG_LOCK_TORTURE_TEST) || IS_MODULE(CONFIG_LOCK_TORTUR= E_TEST) +#if IS_ENABLED(CONFIG_RCU_TORTURE_TEST) || IS_ENABLED(CONFIG_LOCK_TORTURE_= TEST) || IS_ENABLED(CONFIG_HAZPTR_TORTURE_TEST) /* Get rcutorture access to sched_setaffinity(). */ long torture_sched_setaffinity(pid_t pid, const struct cpumask *in_mask, b= ool dowarn) { diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/= selftests/rcutorture/bin/kvm.sh index 65b04b8327330a..32c3199976388d 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm.sh @@ -91,7 +91,7 @@ usage () { echo " --remote" echo " --results absolute-pathname" echo " --shutdown-grace seconds" - echo " --torture lock|rcu|rcuscale|refscale|scf|X*" + echo " --torture hazptr|lock|rcu|rcuscale|refscale|scf|X*" echo " --trust-make" exit 1 } @@ -256,9 +256,9 @@ do shift ;; --torture) - checkarg --torture "(suite name)" "$#" "$2" '^\(lock\|rcu\|rcuscale\|ref= scale\|scf\|X.*\)$' '^--' + checkarg --torture "(suite name)" "$#" "$2" '^\(hazptr\|lock\|rcu\|rcusc= ale\|refscale\|scf\|X.*\)$' '^--' TORTURE_SUITE=3D$2 - TORTURE_MOD=3D"`echo $TORTURE_SUITE | sed -e 's/^\(lock\|rcu\|scf\)$/\1t= orture/'`" + TORTURE_MOD=3D"`echo $TORTURE_SUITE | sed -e 's/^\(hazptr\|lock\|rcu\|sc= f\)$/\1torture/'`" shift if test "$TORTURE_SUITE" =3D rcuscale || test "$TORTURE_SUITE" =3D refsc= ale then diff --git a/tools/testing/selftests/rcutorture/configs/hazptr/CFLIST b/too= ls/testing/selftests/rcutorture/configs/hazptr/CFLIST new file mode 100644 index 00000000000000..4d62eb4a39f999 --- /dev/null +++ b/tools/testing/selftests/rcutorture/configs/hazptr/CFLIST @@ -0,0 +1,2 @@ +NOPREEMPT +PREEMPT diff --git a/tools/testing/selftests/rcutorture/configs/hazptr/CFcommon b/t= ools/testing/selftests/rcutorture/configs/hazptr/CFcommon new file mode 100644 index 00000000000000..c440d227007dce --- /dev/null +++ b/tools/testing/selftests/rcutorture/configs/hazptr/CFcommon @@ -0,0 +1,2 @@ +CONFIG_HAZPTR_TORTURE_TEST=3Dy +CONFIG_PRINTK_TIME=3Dy diff --git a/tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT b/= tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT new file mode 100644 index 00000000000000..e2da430abe4d70 --- /dev/null +++ b/tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT @@ -0,0 +1,17 @@ +CONFIG_SMP=3Dy +CONFIG_NR_CPUS=3D16 +CONFIG_PREEMPT_LAZY=3Dy +CONFIG_PREEMPT_NONE=3Dn +CONFIG_PREEMPT_VOLUNTARY=3Dn +CONFIG_PREEMPT=3Dn +CONFIG_PREEMPT_DYNAMIC=3Dn +CONFIG_HZ_PERIODIC=3Dn +CONFIG_NO_HZ_IDLE=3Dy +CONFIG_NO_HZ_FULL=3Dn +CONFIG_HOTPLUG_CPU=3Dy +CONFIG_SUSPEND=3Dn +CONFIG_HIBERNATION=3Dn +CONFIG_DEBUG_LOCK_ALLOC=3Dn +CONFIG_PROVE_LOCKING=3Dn +CONFIG_KPROBES=3Dn +CONFIG_FTRACE=3Dn diff --git a/tools/testing/selftests/rcutorture/configs/hazptr/PREEMPT b/to= ols/testing/selftests/rcutorture/configs/hazptr/PREEMPT new file mode 100644 index 00000000000000..b8ea4364b20b7b --- /dev/null +++ b/tools/testing/selftests/rcutorture/configs/hazptr/PREEMPT @@ -0,0 +1,14 @@ +CONFIG_SMP=3Dy +CONFIG_NR_CPUS=3D16 +CONFIG_PREEMPT_NONE=3Dn +CONFIG_PREEMPT_VOLUNTARY=3Dn +CONFIG_PREEMPT=3Dy +CONFIG_HZ_PERIODIC=3Dn +CONFIG_NO_HZ_IDLE=3Dy +CONFIG_NO_HZ_FULL=3Dn +CONFIG_HOTPLUG_CPU=3Dy +CONFIG_SUSPEND=3Dn +CONFIG_HIBERNATION=3Dn +CONFIG_DEBUG_LOCK_ALLOC=3Dn +CONFIG_PROVE_LOCKING=3Dn +CONFIG_DEBUG_OBJECTS_RCU_HEAD=3Dn diff --git a/tools/testing/selftests/rcutorture/configs/hazptr/ver_function= s.sh b/tools/testing/selftests/rcutorture/configs/hazptr/ver_functions.sh new file mode 100644 index 00000000000000..a28ea2f292e453 --- /dev/null +++ b/tools/testing/selftests/rcutorture/configs/hazptr/ver_functions.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0+ +# +# Kernel-version-dependent shell functions for the rest of the scripts. +# +# Claude created this file, and I quote: +# +# "I created [this file] modeled on the lock torture +# version. It defines per_version_boot_params to pass +# hazptrtorture.shutdown_secs=3D$3, hazptrtorture.stat_interval=3D15, +# hazptrtorture.verbose=3D1, and optional CPU-hotplug parameters to +# the kernel command line." +# +# I therefore kept locktorture's ver_functions.sh copyright notice: +# +# Copyright (C) Meta Platforms, Inc. and affiliates. +# +# Authors: Paul E. McKenney + +# hazptrtorture_param_onoff bootparam-string config-file +# +# Adds onoff hazptrtorture module parameters to kernels having it. +hazptrtorture_param_onoff () { + if ! bootparam_hotplug_cpu "$1" && configfrag_hotplug_cpu "$2" + then + echo CPU-hotplug kernel, adding hazptrtorture onoff. 1>&2 + echo hazptrtorture.onoff_interval=3D3 hazptrtorture.onoff_holdoff=3D30 + fi +} + +# per_version_boot_params bootparam-string config-file seconds +# +# Adds per-version torture-module parameters to kernels supporting them. +per_version_boot_params () { + echo `hazptrtorture_param_onoff "$1" "$2"` \ + hazptrtorture.stat_interval=3D15 \ + hazptrtorture.shutdown_secs=3D$3 \ + hazptrtorture.verbose=3D1 \ + $1 +} --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95D0617B418; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161092; cv=none; b=b++blSWnOlu17lq+b9K+CGmmH6aBJSDYBVPNANa4thVwk0jMp9wMyOPLoVsK8T7EiRQtNS9ZNsk1bILlQPAMFfxfVgXoUBSq9oQxvJWP4PUZ9ZY7rQr2R8Rph1ndtBwD03bF6Fi+O1540MQ4uGEHy81Ca9UqK3LZF3YFB2J+n7s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161092; c=relaxed/simple; bh=k5jwZOVr3f1E5CeV2/EAlnYNwKzSLw3bW83Q6O+tBfY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kDqJ3fpmhxweVbsLwpNe8n1ldQzuDlDtx4LcadfHd6F7xgBpMi5M0ZhaAMQiWXyb/WwoiV3SqWfkOcneOi6BlA8iqExbtpdQk5N1UiyMvjV7UPKoYrNxPHoJf0DbOLuaZ4+P/RAaJSdG36TWtp2SHcZAWwGc4t3MjBtVGC5DICM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FBffTixB; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FBffTixB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5BC4F1F00ACF; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=Z+drtNt6IhXy3FJaowefTkmFOsaKLhYUt8b8U2czzzg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=FBffTixBC7xBecvpxOGKlo+EKHlVzdot5lEYOvIlx8w7pGGZdGNLeNP4AfHZPZq9p IkF8r6/HFU6j8q1dtAirpTELUKhQqVcHr4uhbkjyDA97IgwEVSwOsO7Q66Co8VPsox +OGWQ0cmZwGA1V36LQBeqHntcKLeOSebr/Fb8rke5yHhHLS8hhdmAdT2BdM00vSP7r Ik8OgCDdr9n4EJQ+LGZX7HIIcD8jyyug6flDYIwJctlFPyH+c9toLKUnNrkwLUtb1D +5bZRDDIzrq2+NFhejmtfVVs6xFHKHKb7gqcrMnyUjYINmmqw9jid0DmvyQgZ9NNft CJwyxkdXTbPNw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 2D838CE0C44; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 04/24] hazptrtorture: Add testing of on-stack hazptr_ctx structures Date: Wed, 15 Jul 2026 17:17:49 -0700 Message-Id: <20260716001809.11084-4-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds a test using on-stack hazptr_ctx structures, in contrast with the per-CPU structures used by the initial test. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 48 ++++++++++++++----- .../rcutorture/configs/hazptr/NOPREEMPT.boot | 1 + 2 files changed, 38 insertions(+), 11 deletions(-) create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/NOPRE= EMPT.boot diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 70f12d2a149069..0f25637c82876b 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -30,7 +30,6 @@ MODULE_DESCRIPTION("Hazard-pointer module-based torture t= est facility"); MODULE_LICENSE("GPL"); MODULE_AUTHOR("Paul E. McKenney "); =20 -torture_param(int, irqreader, 1, "Allow hazard-pointer readers from irq ha= ndlers"); torture_param(int, nreaders, -1, "Number of hazard-pointer reader threads"= ); torture_param(int, onoff_holdoff, 0, "Time after boot before CPU hotplugs = (s)"); torture_param(int, onoff_interval, 0, "Time between CPU hotplugs (jiffies)= , 0=3Ddisable"); @@ -188,7 +187,8 @@ struct hazptr_torture_ops { static struct hazptr_torture_ops *cur_ops; =20 /* - * Definitions for hazard-pointer torture testing. + * Definitions for hazard-pointer torture testing using per-CPU hazptr_ctx + * structures. */ =20 static struct hazptr_torture *hazptr_torture_read_lock(struct hazptr_ctx *= *hcpp) @@ -242,10 +242,33 @@ static struct hazptr_torture_ops hazptr_ops =3D { .readunlock =3D hazptr_torture_read_unlock, .sync =3D hazptr_synchronize, .irq_capable =3D 1, - .onstack_ctx =3D 1, .name =3D "hazptr" }; =20 +/* + * Definitions for hazard-pointer torture testing using on-stack + * hazptr_ctx structures. + */ + +static struct hazptr_torture *hazptr_torture_read_lock_stack(struct hazptr= _ctx **hcpp) +{ + struct hazptr_torture *htp; + + htp =3D (struct hazptr_torture *)hazptr_acquire(*hcpp, (void *)&hazptr_to= rture_current); + return htp; +} + +static struct hazptr_torture_ops hazptr_stack_ops =3D { + .init =3D hazptr_sync_torture_init, + .readlock =3D hazptr_torture_read_lock_stack, + .read_delay =3D hazptr_read_delay, + .readunlock =3D hazptr_torture_read_unlock, + .sync =3D hazptr_synchronize, + .irq_capable =3D 1, + .onstack_ctx =3D 1, + .name =3D "hazptr-stack" +}; + /* * Hazard-pointer torture writer kthread. Repeatedly substitutes a new * structure for that pointed to by hazptr_torture_current, freeing the @@ -331,7 +354,8 @@ hazptr_torture_writer(void *arg) */ static int hazptr_torture_reader(void *arg) { - struct hazptr_ctx *hcp; + struct hazptr_ctx hc; + struct hazptr_ctx *hcp =3D &hc; struct hazptr_torture *htp; unsigned long lastsleep =3D jiffies; long myid =3D (long)arg; @@ -482,13 +506,15 @@ hazptr_torture_print_module_parms(struct hazptr_tortu= re_ops *cur_ops, const char { pr_alert("%s" TORTURE_FLAG "--- %s: nreaders=3D%d " - "stat_interval=3D%d verbose=3D%d " - "shuffle_interval=3D%d stutter=3D%d irqreader=3D%d " - "onoff_interval=3D%d onoff_holdoff=3D%d\n", + "onoff_interval=3D%d onoff_holdoff=3D%d " + "preempt_duration=3D%d preempt_interval=3D%d " + "shuffle_interval=3D%d shutdown_secs=3D%d stat_interval=3D%d stutter=3D= %d " + "verbose=3D%d\n", torture_type, tag, nrealreaders, - stat_interval, verbose, - shuffle_interval, stutter, irqreader, - onoff_interval, onoff_holdoff); + onoff_interval, onoff_holdoff, + preempt_duration, preempt_interval, + shuffle_interval, shutdown_secs, stat_interval, stutter, + verbose); } =20 // Randomly preempt online CPUs. @@ -564,7 +590,7 @@ static int __init hazptr_torture_init(void) long i; int cpu; int firsterr =3D 0; - static struct hazptr_torture_ops *torture_ops[] =3D { &hazptr_ops, }; + static struct hazptr_torture_ops *torture_ops[] =3D { &hazptr_ops, &hazpt= r_stack_ops, }; =20 if (!torture_init_begin(torture_type, verbose)) return -EBUSY; diff --git a/tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT.bo= ot b/tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT.boot new file mode 100644 index 00000000000000..1d09a6446080a1 --- /dev/null +++ b/tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT.boot @@ -0,0 +1 @@ +hazptrtorture.torture_type=3Dhazptr-stack --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95C42145355; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161092; cv=none; b=QZX7vXCf1oGmL/5ZnesVRY7tnKxxjpJvijTGnDyDj4YpuZho1Gr3o0ECB32LdmVF9T2t2Zyaah20Gyah6LRikXrLhKkSWqwJMYKSa4xSsZoB16bInm+rBVo5hbeYR8QiByPrZ4lh5UYPUGQ1SQWWeVE2vXz5NCY9JGw+RydHIIA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161092; c=relaxed/simple; bh=//FELBnZMe/1P4IlYUwD9U/a8/x4j15TVXGrWC50gRU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NFwOyFQ3aicW4XJtSpcSw9pE/ZIx2P0LGjKu8aUWQXVDysbBwYM87mV2iRL5J/ViXCrc65cvzd2hE5uSvOy2dHK9MCairLDhXh7Y3aE20gIQpCx526n7FaKvQxJJbxeBQsaI3gF4lUcSBLeI39zSHDiY5FmXRi3vEwf65QFa7TE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jUbIUpxQ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jUbIUpxQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6F46E1F00ADF; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=D9KA50uigaEU66rKjO2UKiByjh8eGkvgbDo4WqhIl9M=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=jUbIUpxQPM2dGNs6AvsAj/kEIc9rEJ46EmqYMTXm/tdjnGQGaJ+0BGsDyEV0KnvCc WGbTz+CLDECmxC7AuKWk4MEQu3t6GFyRf1uKdor3x/1JkHbFzTMucE0kN2jhJsVoLR x8fWkThVYB0UzFXi49cO49HFB3R2mi1sYr+YXonLlebzeVea5wC6t1g9n3RdrecP99 zEkiTok76DFwxxHo7X+yomGjMV0C6vQNORuJPypBmDCKATq7LuCcSiPM9ka06ElDRy 4jNugcDu04F6fflMN1/Yph3tPzL6gYS1ZsvUQhTUKFds4iFrnhgZlUuunJrG1aic/V bFFFMsItzbmGA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 2FD92CE0C57; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 05/24] hazptrtorture: Add microsecond-scale sleep in readers Date: Wed, 15 Jul 2026 17:17:50 -0700 Message-Id: <20260716001809.11084-5-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds a default-disabled reader_sleep_us module parameter that causes the hazard-pointer reader to unconditionally sleep for the specified number of microseconds. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 0f25637c82876b..4dd31104779fe8 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -35,6 +35,7 @@ torture_param(int, onoff_holdoff, 0, "Time after boot bef= ore CPU hotplugs (s)"); torture_param(int, onoff_interval, 0, "Time between CPU hotplugs (jiffies)= , 0=3Ddisable"); torture_param(int, preempt_duration, 0, "Preemption duration (ms), zero to= disable"); torture_param(int, preempt_interval, MSEC_PER_SEC, "Interval between preem= ptions (ms)"); +torture_param(int, reader_sleep_us, 0, "Reader sleep duration (us)"); torture_param(int, shuffle_interval, 3, "Number of seconds between shuffle= s"); torture_param(int, shutdown_secs, 0, "Shutdown time (s), <=3D zero to disa= ble."); torture_param(int, stat_interval, 60, "Number of seconds between stats pri= ntk()s"); @@ -219,6 +220,8 @@ static void hazptr_read_delay(struct torture_random_sta= te *rrsp) udelay(shortdelay_us); if (!preempt_count() && !(torture_random(rrsp) % (nrealreaders * 500))) torture_preempt_schedule(); /* QS only if preemptible. */ + if (reader_sleep_us > 0) + torture_hrtimeout_us(reader_sleep_us, 0, NULL); } =20 static void hazptr_torture_read_unlock(struct hazptr_ctx *hcp, struct hazp= tr_torture *htp) @@ -508,11 +511,13 @@ hazptr_torture_print_module_parms(struct hazptr_tortu= re_ops *cur_ops, const char "--- %s: nreaders=3D%d " "onoff_interval=3D%d onoff_holdoff=3D%d " "preempt_duration=3D%d preempt_interval=3D%d " + "reader_sleep_us=3D%d " "shuffle_interval=3D%d shutdown_secs=3D%d stat_interval=3D%d stutter=3D= %d " "verbose=3D%d\n", torture_type, tag, nrealreaders, onoff_interval, onoff_holdoff, preempt_duration, preempt_interval, + reader_sleep_us, shuffle_interval, shutdown_secs, stat_interval, stutter, verbose); } --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE8061E5B9A; Thu, 16 Jul 2026 00:18:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161095; cv=none; b=skefnvULof4Z11rvU6tvnIzyf3qZ4dhCujnib9y8bCey+bjpWm/POipeLo+WeNOjhnUdfwhQAy5iaI0xcPlc7pI1Ov9E57tm9L2qftguaDJFhOHeYy2bYY2rfGyY4XjLCz50DJwQjnjp4YYGcs4Wt85CTTgRBeD5UGs+WPAc4bU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161095; c=relaxed/simple; bh=woD7Az2kWlV9PTw7pihphtuKL4FPWPYj4HN0zS+xMec=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BsKSVuiwWKQ2Y4sdwybKyEXYMRX4BD95ZRMnOuG9je6Ooz0Bu+mMpCg9+L+3aww0c4kVDH+ntMKY0UEMhzWF7J+uu7OkWx1qOTXZfoK7QM6YxTwSsumfcdcLmjha+2JWmnL7NO/bMsVgUUN45E1VWuD0FjpGm2X9s/gchB33Q58= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Jl67WNAg; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Jl67WNAg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7638C1F01559; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=SFxIxkKLFyMdSx2whKbiwqYFPjCjFGvqgtrJbBlbYeY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Jl67WNAgUA1KE0EMqag8knVPJejl16SD3w8fWjdf00B3aowHc2AfvchHFsdn6RnLF D5m5xTOxrBoUjJjIhYsihI4EhFv9eR90aKHTjyWwv8MecY8Tr5GJmY8TqT2Ujo2op2 y3fVtNloCw0Oms7NQsCHXIHZvfo0GDQHL8CRfCN/vSLhDMXwDNBsDfUm5fuvIDJ3C/ aqBm6/z13k5nzsCggzccwPN5AKRRfg8zCvLyl59cweJEtyPXDGna/iVUoFAiWNvGme nEImEeOhy+f2ygLcLW5dI1AK0PTQywdoTH+ue1T2ol3RSDzuUzSipHEcJdw2CYU37E RHy0bVt95iuhA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 32302CE0D3B; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 06/24] hazptrtorture: Enable system-independent CPU overcommit Date: Wed, 15 Jul 2026 17:17:51 -0700 Message-Id: <20260716001809.11084-6-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit interprets negative values of the nreaders module parameter as a number of readers per CPU, so that hazptrtorture.nreaders=3D-5 would spawn five hazard-pointer reader kthreads per CPU. As CPUs go offline, a number of readers determined by the overcommit are idled, so that again when hazptrtorture.nreaders=3D-5, five readers would be idled for each offline CPU. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 4dd31104779fe8..21a72667554fe0 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -362,7 +362,7 @@ static int hazptr_torture_reader(void *arg) struct hazptr_torture *htp; unsigned long lastsleep =3D jiffies; long myid =3D (long)arg; - int mynumonline =3D myid; + int mynumonline =3D myid % nr_cpu_ids; int pipe_count; DEFINE_TORTURE_RANDOM(rand); =20 @@ -623,7 +623,7 @@ static int __init hazptr_torture_init(void) if (nreaders >=3D 0) { nrealreaders =3D nreaders; } else { - nrealreaders =3D num_online_cpus() - 2 - nreaders; + nrealreaders =3D num_online_cpus() * -nreaders; if (nrealreaders <=3D 0) nrealreaders =3D 1; } --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE70E1DE8AD; Thu, 16 Jul 2026 00:18:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161094; cv=none; b=IpAC5fzjBNeQc/N8ZZliePyc+M4f331mzgmdxDG9YMqmivAnQjPaOA98xjnT/x4XwBReIoNps+Y3F2R0Tg1+ccNy49WPUbi0Wox7QR0jgkdBFP0jqQMsUZDCRYHjAqgELboaqqsmIh+3tSFjgZO4Pi29GFi0RNk8hADXN0ag028= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161094; c=relaxed/simple; bh=MVYVSEwgX9ErXlL3oNlXG1ram+EvlEnkLLeYLffDHVI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PgWENY1ciUcTg2Ul4pULG6HHn5Rgh4UOri74F3UKsRWEOWrgjzv7dzk3w8tsHBgXuP5ieW2PldEOL7FqWM8r9daTGJKY43/v5TnaCUfdas6idLo7cGbyriZQso0nEUA+pm5q5gWDDDsguJPoTKds/xwyHUkMRjx850G02hyimxI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WChgXTfi; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WChgXTfi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 860A41F00ADB; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=WhHqRMaeXfu70Of3FC59od/tkpMMpTIm2Z1mPB13wwc=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=WChgXTfi4Ccs3qdg+1AaiZ7xocuQZR1CYDvm1IuQ34zLbJNLxH25CRdQMs822+Ib9 AdKcFQORm2ADBXBb2gyE78zjDL8EAmDaeLL6NXxrMBbEMjMT2zXCMlWdpiSZQryCDk XtlObGSLQ5+7AmkxG9yPqgHtbcN9oERk5Wc5Z8DvQgdEHPWREeoJM5piAQlZ+RCjrl dEBuR3TUrAgT4uMIWPd2LS7PPwWq1f3CJcv7mAN+PJgR6blNoRKVCifcTvGhslAUSG 1M52862a/2fnDtEV+jPGAJZZGWQ2h4+PYG54k3xRkh2Spetysz5VEnahKNoOK+8Ydh D/COGE8/FC92g== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 34AA6CE0E0B; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 07/24] torture: Add a stutter_will_wait() function Date: Wed, 15 Jul 2026 17:17:52 -0700 Message-Id: <20260716001809.11084-7-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds a stutter_will_wait() function that returns true if a call to stutter_wait() at that same time would have waited. Of course, the passage of time means that the return value might become immediately stale, so this should be periodically polled on the one hand, or used only for heuristic purposes on the other. The initial use case for this function is to clean up references to objects that might otherwise be held across the stutter interval, which could result in false-positive failures. Signed-off-by: Paul E. McKenney --- include/linux/torture.h | 1 + kernel/torture.c | 23 +++++++++++++++++++++-- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/include/linux/torture.h b/include/linux/torture.h index 66d2d444428aef..b8e5d0f3c2d8f9 100644 --- a/include/linux/torture.h +++ b/include/linux/torture.h @@ -98,6 +98,7 @@ void torture_shutdown_absorb(const char *title); int torture_shutdown_init(int ssecs, void (*cleanup)(void)); =20 /* Task stuttering, which forces load/no-load transitions. */ +bool stutter_will_wait(void); bool stutter_wait(const char *title); int torture_stutter_init(int s, int sgap); =20 diff --git a/kernel/torture.c b/kernel/torture.c index 77cb3589b19f9c..bcd2e9c8a26356 100644 --- a/kernel/torture.c +++ b/kernel/torture.c @@ -727,6 +727,26 @@ static ktime_t stutter_till_abs_time; static int stutter; static int stutter_gap; =20 +static bool _stutter_will_wait(ktime_t *till_ns) +{ + *till_ns =3D READ_ONCE(stutter_till_abs_time); + if (*till_ns && ktime_before(ktime_get(), *till_ns)) + return true; + return false; +} + +/* + * Will stutter_wait() actually stutter? The returned result is of + * course immediately stale due to the passage of time. + */ +bool stutter_will_wait(void) +{ + ktime_t till_ns; + + return _stutter_will_wait(&till_ns); +} +EXPORT_SYMBOL_GPL(stutter_will_wait); + /* * Block until the stutter interval ends. This must be called periodically * by all running kthreads that need to be subject to stuttering. @@ -737,8 +757,7 @@ bool stutter_wait(const char *title) ktime_t till_ns; =20 cond_resched_tasks_rcu_qs(); - till_ns =3D READ_ONCE(stutter_till_abs_time); - if (till_ns && ktime_before(ktime_get(), till_ns)) { + if (_stutter_will_wait(&till_ns)) { torture_hrtimeout_ns(till_ns, 0, HRTIMER_MODE_ABS, NULL); ret =3D true; } --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE7631DF980; Thu, 16 Jul 2026 00:18:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161095; cv=none; b=Ik1WsGtU2RqUDuFIKsHt8UR/kUufBlVwjB0a91gztyI+W+7eh259GAz347p4dKCJDIGyiZ5hS2VVoLniNSFV99gmfKghCmX3CGjzUG2KUzWsKNqt53GnEQfltqF2lf8obr8/4nemoWvJWVSup9OruCUxgr3L3MrvJd89o54NMMI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161095; c=relaxed/simple; bh=KCTqDcoSNBl3bpRi6VKE0ye3sMYgJQD/SmQAZWZocO4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jV5UIJmrXnCvDC+9bRzSMP4+DfCpH2sjBnrv9GniL1CzzYLlTwC+gRKdNADOtmDpQxNc4rWaGudLhWswuQiVIVBMy1+fPBSwgU2/VQQYEb6oSYN9viBLkkpvVgBLYiQM10JKQapW54UBPJ/nSbSTXFAg3S4nJ4XsoO/2jTkx9zs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TkR4nF2J; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TkR4nF2J" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8D0F11F0155B; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=OrmEDktF9ULNSe1gracoYBb5VOuwxPcRL2J82R2YnPk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=TkR4nF2JBXmEn4tOoSVvnT0SA7ey+cZevvJyRSZdvrJzaEHCbNDwiNCUvKrqU6T1y El70jAiQK9/33ElHZk9jX3ZIJzUrQ1BNYh1WC/zKjx58E7sZ2P4g1yhLw+X8LiS2i2 JT5hG3ht2noLiRrscSC3opBFkmJjBvGTaqiCrAURcB5quoY0/tV5/v/KCw6kuQC9Da y7lzncT70jH9N5GX7uAlUUBA3vgg+6+QAhMyO+Z3jPYKoN7JXrNY4wdvCi6+0OFOC1 4WiPHUO5Hqxnx9mrqzHN8ZuiPmOKXtCgof9Hf38rcxP5fTOmRxiL7JlMIETw0MxYA4 TKmxbZHGRy2Sg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 36F0CCE0E2A; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 08/24] hazptrtorture: Use mnemonic local variables for context information Date: Wed, 15 Jul 2026 17:17:53 -0700 Message-Id: <20260716001809.11084-8-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit introduces can_sleep and short_spin local variables to hazptr_read_delay(). The can_sleep variable records whether unconstrained sleeping is feasible, and the short_spin variable checks for the NMI handlers, IRQ handlers, or disabled interrupts/softirqs suggesting short spin times. While in the area, it disables the reader_sleep_us module parameter when invoked where sleeping is not permitted. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 21a72667554fe0..adaf250ecfd585 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -204,23 +204,25 @@ static struct hazptr_torture *hazptr_torture_read_loc= k(struct hazptr_ctx **hcpp) =20 static void hazptr_read_delay(struct torture_random_state *rrsp) { + const bool can_sleep =3D !preempt_count() && !irqs_disabled(); const unsigned long shortdelay_us =3D 200; unsigned long longdelay_ms =3D 300; + const bool short_spin =3D irqs_disabled() || irq_count(); =20 /* We want a short delay sometimes to make a reader delay the grace * period, and we want a long delay occasionally to trigger * force_quiescent_state. */ =20 if (!(torture_random(rrsp) % (nrealreaders * 2000 * longdelay_ms))) { - if ((preempt_count() & HARDIRQ_MASK) || softirq_count()) + if (short_spin) longdelay_ms =3D 5; /* Avoid triggering BH limits. */ mdelay(longdelay_ms); } if (!(torture_random(rrsp) % (nrealreaders * 2 * shortdelay_us))) udelay(shortdelay_us); - if (!preempt_count() && !(torture_random(rrsp) % (nrealreaders * 500))) + if (can_sleep && !(torture_random(rrsp) % (nrealreaders * 500))) torture_preempt_schedule(); /* QS only if preemptible. */ - if (reader_sleep_us > 0) + if (can_sleep && reader_sleep_us > 0) torture_hrtimeout_us(reader_sleep_us, 0, NULL); } =20 --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDD991DD877; Thu, 16 Jul 2026 00:18:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161094; cv=none; b=QZui2QIGFMkyzeeZYO7npkbLHMg2HIoD8qTIadcucI5FGOQUVYpLaOHXW28Asr7wwRh1ffBJAzkkzu2psR9ugenHblj5S3HMf850uVJMCY9/ZCqrvxXKDAqeUrtF1vPWC0HiclE/By2c7fVtSDUuXP0fMVwI17nkFxN5zOZdQRE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161094; c=relaxed/simple; bh=ZYDLakhWjRDXD/lE4lRucRuog7QGo2bBbwJt2eMu1wk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mYqSGlVnCI7w2d3xLQPISW8W15uz07QvioI/Mxq3vcTB0CTswmY4LQe2KWD0WQ0+AfSGeb12FMn4mT5T1ouMiHrM16xMyTyrtiQTagARPAx4NCEwUxPRlUx3/Yv4NaoHa6BtzmFrXhY0nR22K2zjQQxyaJY51ehszIdxsM0lx4I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SzVfq33t; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SzVfq33t" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 926FB1F01558; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=Zg19oyFNV4LWAa8FUqpybXusDjwvYpSzum/Y8JLe4ZY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=SzVfq33tzTcQf5SoZ1uGpGcfr5rCqbrMGAFlR2RCXGNUiiCg1el4oOyjTUw0vEe0I RxhEU0dDPNTeQbY33BHpINkD8/RtJ19LPeOhMT10PVBLdHUvw/Xc+88PxPARRmaQAk JC1AAEv4KtNuOSnHbbZkWG5yJygqsOaAUbJn9w94caqDHMXMfJv/kx3yldp4mSdCQN Fks/FPjNF+pNJPpQ8Ku0s9ESuCQDsP4wjgrfaCj18Mk5f4uTIVzV5DIL0bceiHJpjd QFJYNxgHavQgq1w+BYmXS6vzuPrY3SBhGzOhqkYECkG7dyVUROAm8OZtVd3zcaIsTX tMr9esSXl29bg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 39748CE0E5D; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 09/24] hazptrtorture: Split hazptr_torture_reader_tail() from hazptr_torture_reader() Date: Wed, 15 Jul 2026 17:17:54 -0700 Message-Id: <20260716001809.11084-9-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit splits a new hazptr_torture_reader_tail() function out of hazptr_torture_reader(). This will allow hazptr_torture_reader() to pass hazard pointers off to other tasks and to various types of handlers, and those hazard pointers can in turn be passed to this new hazptr_torture_reader_tail() function to complete processing. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 42 +++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 14 deletions(-) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index adaf250ecfd585..3d559cfb85816d 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -351,6 +351,31 @@ hazptr_torture_writer(void *arg) return 0; } =20 +/* + * Do the delay, the accounting, and the release. This in intended to + * be invoked from hazptr_torture_reader, but also for hazard pointers + * sent off to interrupt handlers and the like. + */ +static void hazptr_torture_reader_tail(struct hazptr_ctx *hcp, struct hazp= tr_torture *htp, + struct torture_random_state *trsp) +{ + int pipe_count; + + cur_ops->read_delay(trsp); + preempt_disable(); + pipe_count =3D READ_ONCE(htp->htort_pipe_count); + if (pipe_count > HAZPTR_TORTURE_PIPE_LEN) { + // Should not happen in a correct hazptr implementation, + // happens quite often for TBD torture_type=3Dbusted. + pipe_count =3D HAZPTR_TORTURE_PIPE_LEN; + } + if (pipe_count > 1) + rcu_ftrace_dump(DUMP_ALL); + __this_cpu_inc(hazptr_torture_count[pipe_count]); + preempt_enable(); + cur_ops->readunlock(hcp, htp); +} + /* * Hazard-pointer torture reader kthread. Repeatedly dereferences * hazptr_torture_current, incrementing the corresponding element of the @@ -365,7 +390,6 @@ static int hazptr_torture_reader(void *arg) unsigned long lastsleep =3D jiffies; long myid =3D (long)arg; int mynumonline =3D myid % nr_cpu_ids; - int pipe_count; DEFINE_TORTURE_RANDOM(rand); =20 VERBOSE_TOROUT_STRING("hazptr_torture_reader task started"); @@ -373,6 +397,8 @@ static int hazptr_torture_reader(void *arg) do { htp =3D cur_ops->readlock(&hcp); if (!htp) { + // Still starting up or allocation failure, + // so get out of the way. schedule_timeout_interruptible(HZ / 10); continue; } @@ -380,19 +406,7 @@ static int hazptr_torture_reader(void *arg) torture_hrtimeout_us(500, 1000, &rand); lastsleep =3D jiffies + 10; } - cur_ops->read_delay(&rand); - preempt_disable(); - pipe_count =3D READ_ONCE(htp->htort_pipe_count); - if (pipe_count > HAZPTR_TORTURE_PIPE_LEN) { - // Should not happen in a correct RCU implementation, - // happens quite often for torture_type=3Dbusted. - pipe_count =3D HAZPTR_TORTURE_PIPE_LEN; - } - if (pipe_count > 1) - rcu_ftrace_dump(DUMP_ALL); - __this_cpu_inc(hazptr_torture_count[pipe_count]); - preempt_enable(); - cur_ops->readunlock(hcp, htp); + hazptr_torture_reader_tail(hcp, htp, &rand); while (!torture_must_stop() && (torture_num_online_cpus() < mynumonline || !rcu_inkernel_boot_ha= s_ended())) schedule_timeout_interruptible(HZ / 5); --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 434661F872D; Thu, 16 Jul 2026 00:18:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; cv=none; b=ZOMQsgfSBjT2HP513OgzE8ZAb/zxy42aBxPO/dpegl0mYzKRtsQEENxIuC7fChGF2eg+JYG+O+QLLJB8L62NSWBoPxnCvON3DcU/GuVykxlLqcgVvktudcAonsohOHZX6vpsAAvWtevcj6OESXbiIdZ+JVLoiuQSRqHfYDCEqj4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; c=relaxed/simple; bh=ShaFJLZTzZkuuouNZqg1D0xWB0Hi657+jMGET9iDMjE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Rv9LdWY8PddbtQHj2iTBLq9nWwdRF7I22/kP5R4tJMOwpKwM169MHh3XA9foTXcZ3+9d9O7WbPbRrA4rK9Uwg13uw5d6fXFA+Tadq+ANb2l3X29WVZ0ntb1DdvI8/1egs57vu1GBUiBWrromSsVpqsVyawJ2PwryiDFfdh7COqE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VUcPbs+2; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VUcPbs+2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9B8541F0155D; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=xjkbRlOLknSaMSCbByQmhZguL3vIQRJh7+dw3pDUs6Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=VUcPbs+2cSLctJFMpGIsZhkWGQcAGiat7bNMyGasv8r0gONKNN8ewauovAtvYoAir OTBm2tOdoXwSlAhK8utI6TI0UIE5PWBvOxrCNLq+Ftuz4eSY18DJYzj8YhOa4ejL7b WnSOKHVO3anYu4bgv40sAoHdUMUpwgJwEA9NRDrT4ofannaqel/jgxeXYOzG201RqT edLtNkBc9YQ4IRlqcOttXTJDU6b+iKZZVMO1ZzBUPYBuqcZ/ccP57vyVx14nTERbW9 5Qao6ZOREyDgbuvpJSRfuHPqE/VPscjtzUcCvWCpO1R6ndptJjpmCo/9LaaU5PqxKo /vrobmckrU5XA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 3BD04CE0E86; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 10/24] hazptrtorture: Add kthread to release deferred hazard pointers Date: Wed, 15 Jul 2026 17:17:55 -0700 Message-Id: <20260716001809.11084-10-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds a kthread to release deferred hazard pointers, which will be used to test acquiring a hazard pointer in one task and releasing it in another. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 90 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 3d559cfb85816d..de2b4e45d9599a 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -30,6 +30,8 @@ MODULE_DESCRIPTION("Hazard-pointer module-based torture t= est facility"); MODULE_LICENSE("GPL"); MODULE_AUTHOR("Paul E. McKenney "); =20 +torture_param(int, kthread_do_pending_ms, -1, + "Delay between cleanups for deferred hazard pointers (ms), zero to = disable"); torture_param(int, nreaders, -1, "Number of hazard-pointer reader threads"= ); torture_param(int, onoff_holdoff, 0, "Time after boot before CPU hotplugs = (s)"); torture_param(int, onoff_interval, 0, "Time between CPU hotplugs (jiffies)= , 0=3Ddisable"); @@ -50,6 +52,7 @@ static int nrealreaders; static struct task_struct *writer_task; static struct task_struct *preempt_task; static struct task_struct **reader_tasks; +static struct task_struct *do_pending_task; static struct task_struct *stats_task; =20 #define HAZPTR_TORTURE_PIPE_LEN 10 @@ -74,6 +77,14 @@ static atomic_t n_hazptr_torture_free; static atomic_t n_hazptr_torture_error; static struct list_head hazptr_torture_removed; =20 +// State for a deferred (AKA pending) hazard pointer +struct hazptr_pending { + struct llist_node hpp_node; + struct hazptr_ctx hpp_hc; + struct hazptr_torture *hpp_htp; +}; +static DEFINE_PER_CPU(struct llist_head, hazptr_pending); + static int hazptr_torture_writer_state; #define HTWS_FIXED_DELAY 0 #define HTWS_DELAY 1 @@ -416,6 +427,76 @@ static int hazptr_torture_reader(void *arg) return 0; } =20 +/* + * Release the specified CPU's set of deferred/pending hazard pointers. + */ +static void hazptr_torture_do_one_pending(int cpu, struct torture_random_s= tate *trsp) +{ + struct hazptr_pending *hppp; + struct hazptr_pending *hppp1; + struct llist_head *llhp; + struct llist_node *llnp; + + llhp =3D per_cpu_ptr(&hazptr_pending, cpu); + llnp =3D llist_del_all(llhp); + if (!llnp) + return; + llist_for_each_entry_safe(hppp, hppp1, llnp, hpp_node) { + hazptr_torture_reader_tail(&hppp->hpp_hc, hppp->hpp_htp, trsp); + kfree(hppp); + } +} + +/* + * Hazard-pointer release of deferred/pending hazard pointers. + */ +static int hazptr_torture_do_pending(void *arg) +{ + int cpu =3D 0; + DEFINE_TORTURE_RANDOM(rand); + + VERBOSE_TOROUT_STRING("hazptr_torture_do_pending task started"); + do { + if (stutter_will_wait()) { + for_each_possible_cpu(cpu) + hazptr_torture_do_one_pending(cpu, &rand); + } else { + cpu =3D cpumask_next_wrap(cpu, cpu_possible_mask); + hazptr_torture_do_one_pending(cpu, &rand); + } + if (torture_must_stop()) + torture_hrtimeout_ms(kthread_do_pending_ms, USEC_PER_MSEC, &rand); + // Omit stutter_wait() because this function needs to do cleanup. + } while (!torture_must_stop()); + torture_kthread_stopping("hazptr_torture_do_pending"); + return 0; +} + +/* + * Spawn hazptr_torture_do_pending() if there is something for it to do. + */ +static int hazptr_torture_do_pending_init(void) +{ + if (kthread_do_pending_ms =3D=3D -1) + kthread_do_pending_ms =3D cur_ops->onstack_ctx ? 0 : 3; + if (kthread_do_pending_ms < 0) { + pr_alert("Cannot have negative kthread_do_pending_ms, disabling deferral= .\n"); + goto err_out; + } + if (cur_ops->onstack_ctx && kthread_do_pending_ms) { + pr_alert("Cannot defer onstack hazptr_ctx, disabling deferral.\n"); + goto err_out; + } + if (!kthread_do_pending_ms) + return 0; + return torture_create_kthread(hazptr_torture_do_pending, NULL, do_pending= _task); + +err_out: + WARN_ON(IS_BUILTIN(CONFIG_HAZPTR_TORTURE_TEST)); + kthread_do_pending_ms =3D 0; + return 0; +} + /* * Print torture statistics. Caller must ensure that there is only one * call to this function at a given time!!! This is normally accomplished @@ -525,12 +606,14 @@ hazptr_torture_print_module_parms(struct hazptr_tortu= re_ops *cur_ops, const char { pr_alert("%s" TORTURE_FLAG "--- %s: nreaders=3D%d " + "kthread_do_pending_ms=3D%d " "onoff_interval=3D%d onoff_holdoff=3D%d " "preempt_duration=3D%d preempt_interval=3D%d " "reader_sleep_us=3D%d " "shuffle_interval=3D%d shutdown_secs=3D%d stat_interval=3D%d stutter=3D= %d " "verbose=3D%d\n", torture_type, tag, nrealreaders, + kthread_do_pending_ms, onoff_interval, onoff_holdoff, preempt_duration, preempt_interval, reader_sleep_us, @@ -579,6 +662,7 @@ hazptr_torture_cleanup(void) return; } =20 + torture_stop_kthread(hazptr_torture_do_pending, do_pending_task); torture_stop_kthread(hazptr_torture_preempt, preempt_task); torture_stop_kthread(hazptr_torture_writer, writer_task); =20 @@ -667,6 +751,12 @@ static int __init hazptr_torture_init(void) =20 /* Start up the kthreads. */ =20 + // This must be before the readers in order to set up the module + // parameters used by the readers. + firsterr =3D hazptr_torture_do_pending_init(); + if (torture_init_error(firsterr)) + goto unwind; + reader_tasks =3D kzalloc_objs(reader_tasks[0], nrealreaders); for (i =3D 0; i < nrealreaders; i++) { firsterr =3D torture_create_kthread(hazptr_torture_reader, (void *)i, --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 434C31FF1B5; Thu, 16 Jul 2026 00:18:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161094; cv=none; b=fQQjTmKnYUqUvDUlanA6eJeQ18YjPJ4xVSom8T06aLX0xBySLTo7kWF9tGEhDzgjp+SK5OnLoyy8Hmqj7FlmChaxS25EdmL+McVjdP03eOP4lp2s0Q9pHjZAtfzBUoBXT5jBD6ld2JXRCExAbm7tOCzpOd+5+fQVTMUPpJpcl3M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161094; c=relaxed/simple; bh=Q6bgJqNPOT9m4p4ZMYGtiBC53GoA2H3n/hREj9jILi8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OXzsPFev3vlzBR3iXMEjReOpcvIsi97Any4fARIO9xH3V9oGx13fqQ3oIlu9PnPYXm7wCZOJsUSGJRMYSh0O/cwXyvQpf57CCEGQlqhXBo5XuCywbBFQBitwAsOAsd9FAGiQUIf+Nv6sAoWetNolzIgMbaEzYJqVsJZFKPVkmBc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PbB3zntv; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PbB3zntv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A366C1F0155C; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=urMuuqdtRK9CjaXQHSEO2Bp6QaroTLrsoc3/R9EDtnY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=PbB3zntvn2YYAJaA0jtXQ5+5gnbf712eJZaiAkzTKCGl/ynwiveALMT9musImqaZO RSLh9r8pn9EijXM0iJJ/AJyXYqxgeJjM1umVmVqX7bWIIntMpt0hVOWLZ/keW/WVmJ HBBIpr7IWJ1f6sEMrrlDenKpwKInm5uwy9qwAvelL9nd7MVeN9mNafyUkpL05YYQ1I JIq5f2L3cfnSOvx6C+Ip8HOd6Pz2KidhW43+ZuGeVQtukvYcwitJX4C7odeSCRpRn2 0pxi1J4CclppkLAP6sakmNxd9kFb8GVGtamCb2c1JH6x4FHOI26VvWQwe7garjl7cL Iwf0eN+Lk5oAA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 3E395CE0E8B; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 11/24] hazptrtorture: Defer release of hazard pointers Date: Wed, 15 Jul 2026 17:17:56 -0700 Message-Id: <20260716001809.11084-11-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit creates the defer_modulus module parameter, so that the releases of one out of defer_modulus hazard-pointer acquisitions will be deferred. This parameter defaults to -1, which results in a value of 1000*nr_cpu_ids to be used. This parameter must be zero for hazptr_torture runs that set cur_ops->onstack_ctx, because otherwise we would get on-stack data races. It must also be zero when the kthread_do_pending_ms module parameter is zero. Setting the defer_modulus module parameter to a value less than -1 will result in disabling hazard-pointer deferral. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 87 +++++++++++++++++++++++++------------- 1 file changed, 58 insertions(+), 29 deletions(-) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index de2b4e45d9599a..8b1315e2d1d1a6 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -30,6 +30,7 @@ MODULE_DESCRIPTION("Hazard-pointer module-based torture t= est facility"); MODULE_LICENSE("GPL"); MODULE_AUTHOR("Paul E. McKenney "); =20 +torture_param(int, defer_modulus, -1, "Defer once per specified # of hazpt= r ops, zero to disable"); torture_param(int, kthread_do_pending_ms, -1, "Delay between cleanups for deferred hazard pointers (ms), zero to = disable"); torture_param(int, nreaders, -1, "Number of hazard-pointer reader threads"= ); @@ -187,7 +188,7 @@ hazptr_torture_pipe_update(struct hazptr_torture *old_r= p) struct hazptr_torture_ops { void (*init)(void); void (*cleanup)(void); - struct hazptr_torture *((*readlock)(struct hazptr_ctx **hcpp)); + struct hazptr_torture *((*readlock)(struct hazptr_ctx *hcpp)); void (*read_delay)(struct torture_random_state *rrsp); void (*readunlock)(struct hazptr_ctx *hcp, struct hazptr_torture *htp); void (*sync)(void *htp); @@ -203,14 +204,12 @@ static struct hazptr_torture_ops *cur_ops; * structures. */ =20 -static struct hazptr_torture *hazptr_torture_read_lock(struct hazptr_ctx *= *hcpp) +static struct hazptr_torture *hazptr_torture_read_lock(struct hazptr_ctx *= hcpp) { - struct hazptr_ctx *hcp =3D kmalloc(sizeof(*hcp), GFP_KERNEL); + struct hazptr_torture *htp; =20 - *hcpp =3D hcp; - if (!hcp) - return NULL; - return (struct hazptr_torture *)hazptr_acquire(hcp, (void *)&hazptr_tortu= re_current); + htp =3D (struct hazptr_torture *)hazptr_acquire(hcpp, (void *)&hazptr_tor= ture_current); + return htp; } =20 static void hazptr_read_delay(struct torture_random_state *rrsp) @@ -241,8 +240,6 @@ static void hazptr_torture_read_unlock(struct hazptr_ct= x *hcp, struct hazptr_tor { if (hcp) { hazptr_release(hcp, htp); - if (cur_ops->onstack_ctx) - kfree(hcp); } } =20 @@ -266,17 +263,9 @@ static struct hazptr_torture_ops hazptr_ops =3D { * hazptr_ctx structures. */ =20 -static struct hazptr_torture *hazptr_torture_read_lock_stack(struct hazptr= _ctx **hcpp) -{ - struct hazptr_torture *htp; - - htp =3D (struct hazptr_torture *)hazptr_acquire(*hcpp, (void *)&hazptr_to= rture_current); - return htp; -} - static struct hazptr_torture_ops hazptr_stack_ops =3D { .init =3D hazptr_sync_torture_init, - .readlock =3D hazptr_torture_read_lock_stack, + .readlock =3D hazptr_torture_read_lock, .read_delay =3D hazptr_read_delay, .readunlock =3D hazptr_torture_read_unlock, .sync =3D hazptr_synchronize, @@ -387,6 +376,20 @@ static void hazptr_torture_reader_tail(struct hazptr_c= tx *hcp, struct hazptr_tor cur_ops->readunlock(hcp, htp); } =20 +/* + * Defer the specified hazard pointer to some other context. + */ +static void hazptr_torture_defer(struct hazptr_pending *hppp, struct tortu= re_random_state *trsp) +{ + int cpu =3D torture_random(trsp) % nr_cpu_ids; + struct llist_head *llhp; + + guard(preempt)(); + cpu =3D cpumask_next_wrap(cpu, cpu_online_mask); + llhp =3D per_cpu_ptr(&hazptr_pending, cpu); + llist_add(&hppp->hpp_node, llhp); +} + /* * Hazard-pointer torture reader kthread. Repeatedly dereferences * hazptr_torture_current, incrementing the corresponding element of the @@ -395,9 +398,9 @@ static void hazptr_torture_reader_tail(struct hazptr_ct= x *hcp, struct hazptr_tor */ static int hazptr_torture_reader(void *arg) { - struct hazptr_ctx hc; - struct hazptr_ctx *hcp =3D &hc; - struct hazptr_torture *htp; + bool can_defer =3D !cur_ops->onstack_ctx && kthread_do_pending_ms && defe= r_modulus; + struct hazptr_pending hpp; + struct hazptr_pending *hppp =3D cur_ops->onstack_ctx ? &hpp : NULL; unsigned long lastsleep =3D jiffies; long myid =3D (long)arg; int mynumonline =3D myid % nr_cpu_ids; @@ -406,10 +409,17 @@ static int hazptr_torture_reader(void *arg) VERBOSE_TOROUT_STRING("hazptr_torture_reader task started"); set_user_nice(current, MAX_NICE); do { - htp =3D cur_ops->readlock(&hcp); - if (!htp) { - // Still starting up or allocation failure, - // so get out of the way. + if (!hppp) { + hppp =3D kmalloc_obj(*hppp, GFP_KERNEL); + if (!hppp) { + // Allocation failure, so get out of the way. + schedule_timeout_interruptible(HZ / 10); + continue; + } + } + hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); + if (!hppp->hpp_htp) { + // Still starting up, so get out of the way. schedule_timeout_interruptible(HZ / 10); continue; } @@ -417,7 +427,12 @@ static int hazptr_torture_reader(void *arg) torture_hrtimeout_us(500, 1000, &rand); lastsleep =3D jiffies + 10; } - hazptr_torture_reader_tail(hcp, htp, &rand); + if (can_defer && !(torture_random(&rand) % defer_modulus)) { + hazptr_torture_defer(hppp, &rand); + hppp =3D NULL; + } else { + hazptr_torture_reader_tail(&hppp->hpp_hc, hppp->hpp_htp, &rand); + } while (!torture_must_stop() && (torture_num_online_cpus() < mynumonline || !rcu_inkernel_boot_ha= s_ended())) schedule_timeout_interruptible(HZ / 5); @@ -478,7 +493,10 @@ static int hazptr_torture_do_pending(void *arg) static int hazptr_torture_do_pending_init(void) { if (kthread_do_pending_ms =3D=3D -1) - kthread_do_pending_ms =3D cur_ops->onstack_ctx ? 0 : 3; + kthread_do_pending_ms =3D (cur_ops->onstack_ctx || defer_modulus =3D=3D = 0) ? 0 : 3; + if (defer_modulus =3D=3D -1) + defer_modulus =3D (cur_ops->onstack_ctx || + kthread_do_pending_ms =3D=3D 0) ? 0 : 1000 * nr_cpu_ids; if (kthread_do_pending_ms < 0) { pr_alert("Cannot have negative kthread_do_pending_ms, disabling deferral= .\n"); goto err_out; @@ -487,6 +505,16 @@ static int hazptr_torture_do_pending_init(void) pr_alert("Cannot defer onstack hazptr_ctx, disabling deferral.\n"); goto err_out; } + if (defer_modulus < 0) { + pr_alert("Cannot have negative defer_modulus (%d), disabling deferral.\n= ", + defer_modulus); + goto err_out; + } + if (!kthread_do_pending_ms !=3D !defer_modulus) { + pr_alert("Pending kthread (%d) & deferral (%d) don't match, disabling de= ferral.\n", + kthread_do_pending_ms, defer_modulus); + goto err_out; + } if (!kthread_do_pending_ms) return 0; return torture_create_kthread(hazptr_torture_do_pending, NULL, do_pending= _task); @@ -494,6 +522,7 @@ static int hazptr_torture_do_pending_init(void) err_out: WARN_ON(IS_BUILTIN(CONFIG_HAZPTR_TORTURE_TEST)); kthread_do_pending_ms =3D 0; + defer_modulus =3D 0; return 0; } =20 @@ -606,14 +635,14 @@ hazptr_torture_print_module_parms(struct hazptr_tortu= re_ops *cur_ops, const char { pr_alert("%s" TORTURE_FLAG "--- %s: nreaders=3D%d " - "kthread_do_pending_ms=3D%d " + "defer_modulus=3D%d kthread_do_pending_ms=3D%d " "onoff_interval=3D%d onoff_holdoff=3D%d " "preempt_duration=3D%d preempt_interval=3D%d " "reader_sleep_us=3D%d " "shuffle_interval=3D%d shutdown_secs=3D%d stat_interval=3D%d stutter=3D= %d " "verbose=3D%d\n", torture_type, tag, nrealreaders, - kthread_do_pending_ms, + defer_modulus, kthread_do_pending_ms, onoff_interval, onoff_holdoff, preempt_duration, preempt_interval, reader_sleep_us, --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6E44221FCD; Thu, 16 Jul 2026 00:18:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161095; cv=none; b=lj+TofzPKUGyor6t0Q+0ytkOS74DT8yaHKqaGKGlObv96CzfbsophPxUJY/HVAD2rCOvWibKi8I/lC3jdvIswoKWeKD11Wp5YY/LLB5LJZJRF6uhHRSE53JinZwchcfjYT0N4TPfAeJLaZGfGqmVIRJg2nZ7pQwSAsMOW7DoudY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161095; c=relaxed/simple; bh=M6BVgZFSjILX5HaRUmyWETnbG02BYPqnpcsDUfsXXbA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qKe8l7ZhYWp44E6Ot8GSSnKffzljVXPq6i3ZAJX6WdKwzezL0a4y1sCRcOa/rcPexjijCsYV9/O3ZGaWau0Ak4O6EMQeLgnCl05NE4EHRWMHQBD/mDjAhOyQ7U0xv+cbsOhyBJ/9q7uXjWyS8tK0i0ciC3Oa+2W1FHY2sA32kEs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ddsru8iT; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ddsru8iT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A75F11F01560; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=o2QKJb1SmUARMMLEAEctoR5D5DIckgM5bthyhdFJHLg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ddsru8iTWOAIVD8vlpGVCpkR7Vm+Abkp36rkU4MrtLBVenOxLqj2uGVENQM/bjhWO uBbGB8hoi42udhDib2CP7I8VRlQddoyhwezzffXNC8qsK1GqJuDHo7UmMXLWQJ/Dh5 b9bapr1KaVhR+zdB30lbiYUMBnUXzN8x1d+8frNjIiJS19MkdcoylPvfXQMLGX/o/t NxPs62NBCTOVUeGg9Egvo3e0vZ8Nv3SqQFIKj9fHsYp1J8n2EPaLsU9EdfMtQtDOJE cgwg+MuqqonfXp8Ty2PVOdouORzzZpsLIJBN/l7SHs5R7Pabh1eEHzxRKAu01ofvDf xRILLdytXXt2Q== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 408A3CE0E8C; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 12/24] hazptrtorture: Add irq_acquire to acquire hazptr from irq Date: Wed, 15 Jul 2026 17:17:57 -0700 Message-Id: <20260716001809.11084-12-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds the irq_acquire module parameter, which specifies how often hazard pointers will be acquired from an smp_call_function_single() handler. For example, a value of 10 would result in one of of ten hazard-pointer acquisitions taking place in a handler, and probably also death by excessive numbers of handlers. A value of zero disables, and a value of -1 makes the frequency decrease as a function of the number of CPUs. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 8b1315e2d1d1a6..3304eb450d7e32 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -31,6 +31,8 @@ MODULE_LICENSE("GPL"); MODULE_AUTHOR("Paul E. McKenney "); =20 torture_param(int, defer_modulus, -1, "Defer once per specified # of hazpt= r ops, zero to disable"); +torture_param(int, irq_acquire, -1, + "Acquire hazard pointers from irq handlers once per specified #, ze= ro to disable"); torture_param(int, kthread_do_pending_ms, -1, "Delay between cleanups for deferred hazard pointers (ms), zero to = disable"); torture_param(int, nreaders, -1, "Number of hazard-pointer reader threads"= ); @@ -351,6 +353,16 @@ hazptr_torture_writer(void *arg) return 0; } =20 +/* + * Acquire a hazard pointer from an smp_call_function handler. + */ +static void hazptr_torture_acquire(void *hppp_in) +{ + struct hazptr_pending *hppp =3D hppp_in; + + hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); +} + /* * Do the delay, the accounting, and the release. This in intended to * be invoked from hazptr_torture_reader, but also for hazard pointers @@ -399,6 +411,7 @@ static void hazptr_torture_defer(struct hazptr_pending = *hppp, struct torture_ran static int hazptr_torture_reader(void *arg) { bool can_defer =3D !cur_ops->onstack_ctx && kthread_do_pending_ms && defe= r_modulus; + int cpu =3D 0; struct hazptr_pending hpp; struct hazptr_pending *hppp =3D cur_ops->onstack_ctx ? &hpp : NULL; unsigned long lastsleep =3D jiffies; @@ -417,7 +430,16 @@ static int hazptr_torture_reader(void *arg) continue; } } - hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); + if (irq_acquire && !(torture_random(&rand) % irq_acquire)) { + guard(preempt)(); + cpu =3D cpumask_next_wrap(cpu, cpu_online_mask); + if (cpu !=3D smp_processor_id()) + smp_call_function_single(cpu, hazptr_torture_acquire, hppp, 1); + else + hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); + } else { + hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); + } if (!hppp->hpp_htp) { // Still starting up, so get out of the way. schedule_timeout_interruptible(HZ / 10); @@ -635,14 +657,14 @@ hazptr_torture_print_module_parms(struct hazptr_tortu= re_ops *cur_ops, const char { pr_alert("%s" TORTURE_FLAG "--- %s: nreaders=3D%d " - "defer_modulus=3D%d kthread_do_pending_ms=3D%d " + "defer_modulus=3D%d irq_acquire=3D%d kthread_do_pending_ms=3D%d " "onoff_interval=3D%d onoff_holdoff=3D%d " "preempt_duration=3D%d preempt_interval=3D%d " "reader_sleep_us=3D%d " "shuffle_interval=3D%d shutdown_secs=3D%d stat_interval=3D%d stutter=3D= %d " "verbose=3D%d\n", torture_type, tag, nrealreaders, - defer_modulus, kthread_do_pending_ms, + defer_modulus, irq_acquire, kthread_do_pending_ms, onoff_interval, onoff_holdoff, preempt_duration, preempt_interval, reader_sleep_us, @@ -786,6 +808,13 @@ static int __init hazptr_torture_init(void) if (torture_init_error(firsterr)) goto unwind; =20 + if (irq_acquire =3D=3D -1) { + irq_acquire =3D 1000 * nr_cpu_ids; + } else if (irq_acquire < 0) { + pr_alert("Cannot have irq_acquire (%d) < -1, disabling.\n", irq_acquire); + WARN_ON(IS_BUILTIN(CONFIG_HAZPTR_TORTURE_TEST)); + irq_acquire =3D 0; + } reader_tasks =3D kzalloc_objs(reader_tasks[0], nrealreaders); for (i =3D 0; i < nrealreaders; i++) { firsterr =3D torture_create_kthread(hazptr_torture_reader, (void *)i, --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11458223DD4; Thu, 16 Jul 2026 00:18:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161095; cv=none; b=s4X4DgKfnbd7qcNxckU1ssu754XgogQGi6pBqOSB3Z7alW9FhRyLy9S/vXVlSWW8TzRSughCBTbm0mnA/P9t8iVl87kmRWSEtu4ycUhoYYQYX62VjWX+Ljr6G2x8F0V3UcWYWFuv0fPKTw7UJ9QxBIXaJ/4Accf7wTK+xNmfYis= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161095; c=relaxed/simple; bh=q5x7h/cKxZhYRIDtmDdIaqS5ptNKMn+4az5MYemtxRU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NBzWILA/JddUGiUXtgYUVneoEL13d6zDqsVKrOaVON4sYkdkx/90/VhL/UACsAFP/UE1BxWBN8Ud4AmCV2rZIU4avYrJR17MGdQfjA8aJI+kQa2q2DHAT+zy7nHtrrf4R1/rRmHU2PCBmY0Q5bjQDvzWWhb2TtsI9erOI2wYjG4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=csf1T6D8; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="csf1T6D8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AD6371F00A3E; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=3CvywNr0WbXOGx0JTNUPTDTUNzLnOXqHzFE6wFFrR5s=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=csf1T6D8JqrGZsEcDKWkDiSyH+M/AKUgFMKfL9u48n+1HNOAqcnKohhOCBkJ1/GCK 3VBvqbbvV0UHUBAOSYkYc8Vf+qEyNNcOIZiKrOI7oB7mL3U3jmeRCAk/jQ9kMIAlfa h97apb+Yt9VE8NnwCLLGwfGDXcbMWoFBurdwRmDKIEdo9XP8CSI7QBq4aMjj4revZ5 7GX2khwfKhw1w7OiYOMEFGSEBCUqvBUpvnLxcpBEbTA2z0/XM3T/uhhOGDlDimN0gu JJp9EEldZyjF/bWwl848Qa+vRBDO1GejsCnreePyhb2Gi2gxqHPk/FO6HaWln22bHs pU3KtKbW1PZlw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 430DFCE0F14; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Kunwu Chan , Zqiang , Wang Lian , "Paul E . McKenney" Subject: [PATCH RFC v2 13/24] hazptrtorture: Use task_state_to_char() for task-state reporting Date: Wed, 15 Jul 2026 17:17:58 -0700 Message-Id: <20260716001809.11084-13-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kunwu Chan Use the kernel's standard symbolic task-state representation instead of printing raw hexadecimal task-state values. Suggested-by: Zqiang Co-developed-by: Wang Lian Signed-off-by: Wang Lian Signed-off-by: Kunwu Chan Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 3304eb450d7e32..266fecb00105f9 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -621,10 +621,10 @@ hazptr_torture_stats_print(void) unsigned long __maybe_unused gp_seq =3D 0; =20 wtp =3D READ_ONCE(writer_task); - pr_alert("??? Writer stall state %s(%d) g%lu f%#x ->state %#x cpu %d\n", + pr_alert("??? Writer stall state %s(%d) g%lu f%#x ->state %c cpu %d\n", hazptr_torture_writer_state_getname(), hazptr_torture_writer_state, gp_seq, flags, - wtp =3D=3D NULL ? ~0U : wtp->__state, + wtp =3D=3D NULL ? '?' : task_state_to_char(wtp), wtp =3D=3D NULL ? -1 : (int)task_cpu(wtp)); if (!splatted && wtp) { sched_show_task(wtp); --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75D2F231827; Thu, 16 Jul 2026 00:18:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161096; cv=none; b=JFrwoBw7AALi+jhvFzXIFJaDNZeC505t7pnsi0C14Or7LdgD9T4P55JH/8Wp+V1UjbfyO/JWD7wvSkryDutYoyfZoqloWFV3T/y6+0oxEPZ6poMiYup+hVnD7QwlXKYnKCvc5YCBpYbXPkKiajj79Xwx7nBsb5pa/SCOnnkDhXo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161096; c=relaxed/simple; bh=nJ1ctnOBPalzNp6ZnLD2UE5OKoMg8XFO+FWhVnoARMY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ksE1Nh+NZJsx8g78VxtAnQ9nYqE/VUAL6hMqtnTNLNH4MFX/41OKify2WicR0jBYbMvlX/k3BHjDBlvCIviiSBdmeaV2xdn61EYPCMo61lic6SPrgqtCbFl56+LO3DA2uasxt7qpvdvIAGa2rpoyhRW3BeKHfO9g94PXmrLQzaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kFqqpjyA; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kFqqpjyA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BBD131F00A3F; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=vcVacDQqDFrvp1JfQX66udVDl3nvOv16wymKh+hFy3k=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=kFqqpjyABQQknFAjJXX+878VqVnehCq0zqPLDkmozkjbqu+RQtNg4nTHgMm88gNmw MxVHLIYmDS2tHLO91ddeZ9h/UyxQkdXzjv8uSJpn+HTBc8uH/U4ZLdTRPl3UxUf+kY JDpBAki+YOvKB6nFNo3arNd5zJ8wWPTSkdZ1PBrVaDRNi5u64jsSPG/slId1AFQ6m2 191FxvoXdwCL9OZR7E/j+rZ4hJcpxEgTxSUcm6E7dSJQYQULmH0w8xTx3AfOp8nDxm Nfd1hvcH/XqxiHmfN6Uysu95gf4ut/LnrPPBkb0jWnl9tG9E8cXsemhYixQBIyw8fP AfDtsCEMY6+1A== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 45596CE0F42; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 14/24] hazptrtorture: Pass hazptr_pending to hazptr_torture_reader_tail() Date: Wed, 15 Jul 2026 17:17:59 -0700 Message-Id: <20260716001809.11084-14-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit passes a hazptr_pending structure instead of a pair of pointers to the hazptr_torture_reader_tail() function in order to make it easier to test the releasing of hazard pointers from interrupt handlers. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 266fecb00105f9..625ed90912a2c6 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -368,9 +368,11 @@ static void hazptr_torture_acquire(void *hppp_in) * be invoked from hazptr_torture_reader, but also for hazard pointers * sent off to interrupt handlers and the like. */ -static void hazptr_torture_reader_tail(struct hazptr_ctx *hcp, struct hazp= tr_torture *htp, - struct torture_random_state *trsp) +static void +hazptr_torture_reader_tail(struct hazptr_pending *hppp, struct torture_ran= dom_state *trsp) { + struct hazptr_ctx *hcp =3D &hppp->hpp_hc; + struct hazptr_torture *htp =3D hppp->hpp_htp; int pipe_count; =20 cur_ops->read_delay(trsp); @@ -453,7 +455,7 @@ static int hazptr_torture_reader(void *arg) hazptr_torture_defer(hppp, &rand); hppp =3D NULL; } else { - hazptr_torture_reader_tail(&hppp->hpp_hc, hppp->hpp_htp, &rand); + hazptr_torture_reader_tail(hppp, &rand); } while (!torture_must_stop() && (torture_num_online_cpus() < mynumonline || !rcu_inkernel_boot_ha= s_ended())) @@ -479,7 +481,7 @@ static void hazptr_torture_do_one_pending(int cpu, stru= ct torture_random_state * if (!llnp) return; llist_for_each_entry_safe(hppp, hppp1, llnp, hpp_node) { - hazptr_torture_reader_tail(&hppp->hpp_hc, hppp->hpp_htp, trsp); + hazptr_torture_reader_tail(hppp, trsp); kfree(hppp); } } --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 460232264B0; Thu, 16 Jul 2026 00:18:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; cv=none; b=vAUWJUXaFTa+2Qec4SnBoGX+BPEChQN+eTjo3ZCvgLwTPMVgqOvxe85/OPUbvlHAJrQxTHeDwbDvagoY8IZIgBMR1Om641L18HmadNgqkStPk3uKkLepcirWZev39crRYIo5q5W+2ksLS2gNAc/izEuMuMoVHLr0XkLyutE5Flo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; c=relaxed/simple; bh=7Pyke/o79sTDfkUqS45VlLOxDMP8MIL1i7PXIg6sUu0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KEHYsn/3UpOeZWU2/vCWFN83a/B4t6+ULhRfOBpdxcYlf1G/vlygfX8WUMtIs8Uzw5lStUY3JohfzsGT9Jm82TE4dj1E7FamVuMjI+5a7Yu9A/RsZCwPuGToKjXdAPlJXd+IrSHhJqaqi0u9wR/gSj+UM4qbhrvcnjS9XSVSGk4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SvP8yra/; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SvP8yra/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B6D6B1F00A3D; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=AAbLUnNrcnqFC0CF5ESAMKua77koOg9Ol3O6aMT/qRc=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=SvP8yra/uNjYWoI5kgfJAusi/owJ0qAIS//DRKTW2g8y8v8B3wCv/MsRSFJ+cVO6K kbPyB+h8DV2HA1gBLY/lwHo1TSZAyUCCaBPSlWzy16a+60rYDelYzg0wPbgAjEqqja UvsPGW/WaQHUMfWPYcOIkH70FQvOzJ5w8yHYj3ThPTbm3nBs8DeRD9/JgND6hl4iC2 lpv07/a8kWb3lEjYSpFei92+uykNKmgx0bYEEoczF2xuQTsIhIIO+zchiaiBB0yMX3 pMzU1+QDb71jyBV/kMIi2aT7F8WfkESpWyCAtMbFdICM0gSKw0HV+fJoVR4HdjEFpM 2ksItvwfz96Xw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 47D1ECE0F4C; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 15/24] hazptrtorture: Add the ability to disable the writer kthread Date: Wed, 15 Jul 2026 17:18:00 -0700 Message-Id: <20260716001809.11084-15-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds a hazptrtorture.nwriters module parameter that, when set to zero, disables hazard-pointer update-side activity. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 625ed90912a2c6..0752cedb2f5570 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -36,6 +36,7 @@ torture_param(int, irq_acquire, -1, torture_param(int, kthread_do_pending_ms, -1, "Delay between cleanups for deferred hazard pointers (ms), zero to = disable"); torture_param(int, nreaders, -1, "Number of hazard-pointer reader threads"= ); +torture_param(int, nwriters, 1, "Number of hazard-pointer writer threads, = 0 or 1"); torture_param(int, onoff_holdoff, 0, "Time after boot before CPU hotplugs = (s)"); torture_param(int, onoff_interval, 0, "Time between CPU hotplugs (jiffies)= , 0=3Ddisable"); torture_param(int, preempt_duration, 0, "Preemption duration (ms), zero to= disable"); @@ -658,14 +659,14 @@ static void hazptr_torture_print_module_parms(struct hazptr_torture_ops *cur_ops, cons= t char *tag) { pr_alert("%s" TORTURE_FLAG - "--- %s: nreaders=3D%d " + "--- %s: nreaders=3D%d nwriters=3D%d " "defer_modulus=3D%d irq_acquire=3D%d kthread_do_pending_ms=3D%d " "onoff_interval=3D%d onoff_holdoff=3D%d " "preempt_duration=3D%d preempt_interval=3D%d " "reader_sleep_us=3D%d " "shuffle_interval=3D%d shutdown_secs=3D%d stat_interval=3D%d stutter=3D= %d " "verbose=3D%d\n", - torture_type, tag, nrealreaders, + torture_type, tag, nrealreaders, nwriters, defer_modulus, irq_acquire, kthread_do_pending_ms, onoff_interval, onoff_holdoff, preempt_duration, preempt_interval, @@ -825,9 +826,12 @@ static int __init hazptr_torture_init(void) goto unwind; } =20 - firsterr =3D torture_create_kthread(hazptr_torture_writer, NULL, writer_t= ask); - if (torture_init_error(firsterr)) - goto unwind; + if (nwriters) { + WARN_ON(IS_BUILTIN(CONFIG_HAZPTR_TORTURE_TEST) && nwriters !=3D 1); + firsterr =3D torture_create_kthread(hazptr_torture_writer, NULL, writer_= task); + if (torture_init_error(firsterr)) + goto unwind; + } =20 firsterr =3D torture_onoff_init(onoff_holdoff * HZ, onoff_interval, NULL); if (torture_init_error(firsterr)) --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9278F23183F; Thu, 16 Jul 2026 00:18:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161096; cv=none; b=ASqKR9W1SQQOhiMkP+9OixUdYIqH1s1FboXIKgej0n0C6P2LQJ9R8Vd2eimi30UBM9e022xQnXjymRV4jvM9QqfVWOt8+10vA7UnThJCB3ji/LXIM8WA5NQA6rLtGGf3bA4y0Jl/00ilDxDDsfv09QW3Tgbv04jTrq3rMl9EaIg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161096; c=relaxed/simple; bh=zvWR6bgmBcz8v6mYV8M7NJqQ2X/y1muO0aIYKPzGGDA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GHeBdl9gJfG2vQN8g3N4VFOnYbgpNOjbujOjZiYRKwaPTFJtaHXDMaQ5mwPdfvkyJbGiHMGhrjGDTx8rZcMN0jXVGvDG98ZG7KTpzk6cWNFHzzgfl9tdt82bI8Cs4a/DurJ/QnFswc1Y2VH4IgK11kPSU0CVt95W80pQPVgn+WE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jMO3Gv6W; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jMO3Gv6W" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC4331F01561; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=oO2sm1j7Nq9Prg/AKFwcQRXORcrI5Vi3nU5SVYQ9V+o=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=jMO3Gv6WanyGQuOXaw2lzBFd5/u6pZ+MdVjYZNK8JeYzxDc1qD+OWTOLyaZaInf8V 8M9Ha7Znzw8mPqCFiPTtk8MzfmFOQKvMZp76wi+UASYp9Jv2JzgktZGr/hlu96JLkb SXzQfGowEZdjpZdAggdvOtX/6LUezTsGOHw+TXdvTTucbeL2XivcgJ4oa0zezwcWZi ll6wE7ZKj4LBVsSlUa7SmzgcX8PAqybMMuiJOCNB3W9eR0jVMbqjWIaOIrZx6z1z1L l0JxAfdT2am61meJe1yUzoxJP2lC0mni99serrlI+I4jUegnTeZBTbi2cRPnrNLykV WgfkrBhzESvdg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 4A6E5CE0F64; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 16/24] hazptrtorture: Add irq_release to release hazptr from irq Date: Wed, 15 Jul 2026 17:18:01 -0700 Message-Id: <20260716001809.11084-16-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit adds the irq_release module parameter, which specifies how often hazard pointers will be released from an smp_call_function_single() handler. For example, a value of 10 would result in one of of ten hazard-pointer acquisitions taking place in a handler, and probably also death by excessive numbers of handlers. A value of zero disables, and a value of -1 makes the frequency decrease as a function of the number of CPUs. Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 0752cedb2f5570..fb25f8351659a4 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -33,6 +33,8 @@ MODULE_AUTHOR("Paul E. McKenney "); torture_param(int, defer_modulus, -1, "Defer once per specified # of hazpt= r ops, zero to disable"); torture_param(int, irq_acquire, -1, "Acquire hazard pointers from irq handlers once per specified #, ze= ro to disable"); +torture_param(int, irq_release, -1, + "Release hazard pointers from irq handlers once per specified #, ze= ro to disable"); torture_param(int, kthread_do_pending_ms, -1, "Delay between cleanups for deferred hazard pointers (ms), zero to = disable"); torture_param(int, nreaders, -1, "Number of hazard-pointer reader threads"= ); @@ -364,6 +366,16 @@ static void hazptr_torture_acquire(void *hppp_in) hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); } =20 +/* + * Release a hazard pointer from an smp_call_function handler. + */ +static void hazptr_torture_release(void *hppp_in) +{ + struct hazptr_pending *hppp =3D hppp_in; + + cur_ops->readunlock(&hppp->hpp_hc, hppp->hpp_htp); +} + /* * Do the delay, the accounting, and the release. This in intended to * be invoked from hazptr_torture_reader, but also for hazard pointers @@ -372,6 +384,7 @@ static void hazptr_torture_acquire(void *hppp_in) static void hazptr_torture_reader_tail(struct hazptr_pending *hppp, struct torture_ran= dom_state *trsp) { + int cpu; struct hazptr_ctx *hcp =3D &hppp->hpp_hc; struct hazptr_torture *htp =3D hppp->hpp_htp; int pipe_count; @@ -388,7 +401,13 @@ hazptr_torture_reader_tail(struct hazptr_pending *hppp= , struct torture_random_st rcu_ftrace_dump(DUMP_ALL); __this_cpu_inc(hazptr_torture_count[pipe_count]); preempt_enable(); - cur_ops->readunlock(hcp, htp); + if (irq_release && !(torture_random(trsp) % irq_release)) { + guard(preempt)(); + cpu =3D cpumask_next_wrap(smp_processor_id(), cpu_online_mask); + smp_call_function_single(cpu, hazptr_torture_release, hppp, 1); + } else { + cur_ops->readunlock(hcp, htp); + } } =20 /* @@ -660,14 +679,14 @@ hazptr_torture_print_module_parms(struct hazptr_tortu= re_ops *cur_ops, const char { pr_alert("%s" TORTURE_FLAG "--- %s: nreaders=3D%d nwriters=3D%d " - "defer_modulus=3D%d irq_acquire=3D%d kthread_do_pending_ms=3D%d " + "defer_modulus=3D%d irq_acquire=3D%d irq_release=3D%d kthread_do_pendin= g_ms=3D%d " "onoff_interval=3D%d onoff_holdoff=3D%d " "preempt_duration=3D%d preempt_interval=3D%d " "reader_sleep_us=3D%d " "shuffle_interval=3D%d shutdown_secs=3D%d stat_interval=3D%d stutter=3D= %d " "verbose=3D%d\n", torture_type, tag, nrealreaders, nwriters, - defer_modulus, irq_acquire, kthread_do_pending_ms, + defer_modulus, irq_acquire, irq_release, kthread_do_pending_ms, onoff_interval, onoff_holdoff, preempt_duration, preempt_interval, reader_sleep_us, @@ -818,6 +837,13 @@ static int __init hazptr_torture_init(void) WARN_ON(IS_BUILTIN(CONFIG_HAZPTR_TORTURE_TEST)); irq_acquire =3D 0; } + if (irq_release =3D=3D -1) { + irq_release =3D 1000 * nr_cpu_ids; + } else if (irq_release < 0) { + pr_alert("Cannot have irq_release (%d) < -1, disabling.\n", irq_release); + WARN_ON(IS_BUILTIN(CONFIG_HAZPTR_TORTURE_TEST)); + irq_release =3D 0; + } reader_tasks =3D kzalloc_objs(reader_tasks[0], nrealreaders); for (i =3D 0; i < nrealreaders; i++) { firsterr =3D torture_create_kthread(hazptr_torture_reader, (void *)i, --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C896923BF9F; Thu, 16 Jul 2026 00:18:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161096; cv=none; b=lkR59OC4SMIqIagc3NZ2jLtH/yYLQ9PJNzYF58AWonXk7buxNheko6Sk4QPqFT5mduY6LB14EwH4vf3JDsp7bp+USuIn4GSKNQFy5Ztnxat7R03lw2zoVwE0/JYBl5ujZ8lTaVpldMTm2F49s5IUILCwnHpkpuXDiNoXyz0oDoI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161096; c=relaxed/simple; bh=/zonpul43VppgukDMsBQNGiTbbiYXq6Xs4Unnezskg0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XOeBYG3kHhrS6P9KV+vDWSetoY5u5kASjDuuMl3Zcw+oh0QBFAzgmnUXiIKbjKAWSaG7DedlZIt5RX3kk5/CKZrv3agM+ehnL+seD2XCi33vPobCqFTOFrdxQKRG2tFSV/NdxHZO9TJ21lOpcYJm/u+F6OhpaCgnKBRu9pypELc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=W47wNrJt; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="W47wNrJt" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C34F71F0155E; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=h5ky3EvNHhzgz0w9Yhq3ozQuB2Sf01a5zXkBYIpuLOI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=W47wNrJtuYoqllGQIYj/5oYX+Utp4AtghTpwYbr/KTcJeI3AKaR6/N5GMvfGrgYJd BggDDEsmsA1SsDX6JPMhAXV5jkBlJ556mQFo9rAMyXJEj0Jm6rmfB8HI5nORatp7AD 2ITiMfYWuXmQ0vbWob7rtcspkKqLPbEoQEH5u5600hElHCOLGHeFPmCXMcOg15hadZ m9crNzGrnF/Z7hH3A8dfwGBpzW8hMhQhFIORD/TuxHThx6q6cwdyiMj9BmjoMPnFzZ j5B0mLzu2vl277A4N7usvanHpyfIRj3Jeowqs3vfqCBjDH18lkV7aC+FJz19NtChCr DBCu/+BccO+pQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 4CBDDCE0F67; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 17/24] hazptrtorture: Accumulate operation statistics Date: Wed, 15 Jul 2026 17:18:02 -0700 Message-Id: <20260716001809.11084-17-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit accumulates and prints counts of the number and types of hazard-pointer operations that the test performed. Signed-off-by: Paul E. McKenney --- include/linux/torture.h | 3 +++ kernel/rcu/hazptrtorture.c | 28 ++++++++++++++++++++++++++-- kernel/torture.c | 15 +++++++++++++++ 3 files changed, 44 insertions(+), 2 deletions(-) diff --git a/include/linux/torture.h b/include/linux/torture.h index b8e5d0f3c2d8f9..1b6d5be641f8d5 100644 --- a/include/linux/torture.h +++ b/include/linux/torture.h @@ -136,4 +136,7 @@ void torture_sched_set_normal(struct task_struct *t, in= t nice); long torture_sched_setaffinity(pid_t pid, const struct cpumask *in_mask, b= ool dowarn); #endif =20 +/* Atomic per-CPU counters. */ +s64 torture_sum_pcpu_atomic_long(atomic_long_t __percpu *pcp); + #endif /* __LINUX_TORTURE_H */ diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index fb25f8351659a4..311b387d6ae755 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -81,6 +81,12 @@ static atomic_t n_hazptr_torture_alloc; static atomic_t n_hazptr_torture_alloc_fail; static atomic_t n_hazptr_torture_free; static atomic_t n_hazptr_torture_error; +static DEFINE_PER_CPU(atomic_long_t, hazptr_torture_acquires); +static DEFINE_PER_CPU(atomic_long_t, hazptr_torture_releases); +static DEFINE_PER_CPU(atomic_long_t, hazptr_torture_acquires_irq); +static DEFINE_PER_CPU(atomic_long_t, hazptr_torture_releases_irq); +static DEFINE_PER_CPU(atomic_long_t, hazptr_torture_releases_defer); +static DEFINE_PER_CPU(atomic_long_t, hazptr_torture_releases_undefer); static struct list_head hazptr_torture_removed; =20 // State for a deferred (AKA pending) hazard pointer @@ -364,6 +370,7 @@ static void hazptr_torture_acquire(void *hppp_in) struct hazptr_pending *hppp =3D hppp_in; =20 hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); + atomic_long_inc(per_cpu_ptr(&hazptr_torture_acquires_irq, raw_smp_process= or_id())); } =20 /* @@ -374,6 +381,7 @@ static void hazptr_torture_release(void *hppp_in) struct hazptr_pending *hppp =3D hppp_in; =20 cur_ops->readunlock(&hppp->hpp_hc, hppp->hpp_htp); + atomic_long_inc(per_cpu_ptr(&hazptr_torture_releases_irq, raw_smp_process= or_id())); } =20 /* @@ -407,6 +415,7 @@ hazptr_torture_reader_tail(struct hazptr_pending *hppp,= struct torture_random_st smp_call_function_single(cpu, hazptr_torture_release, hppp, 1); } else { cur_ops->readunlock(hcp, htp); + atomic_long_inc(per_cpu_ptr(&hazptr_torture_releases, raw_smp_processor_= id())); } } =20 @@ -422,6 +431,7 @@ static void hazptr_torture_defer(struct hazptr_pending = *hppp, struct torture_ran cpu =3D cpumask_next_wrap(cpu, cpu_online_mask); llhp =3D per_cpu_ptr(&hazptr_pending, cpu); llist_add(&hppp->hpp_node, llhp); + atomic_long_inc(per_cpu_ptr(&hazptr_torture_releases_defer, raw_smp_proce= ssor_id())); } =20 /* @@ -455,12 +465,17 @@ static int hazptr_torture_reader(void *arg) if (irq_acquire && !(torture_random(&rand) % irq_acquire)) { guard(preempt)(); cpu =3D cpumask_next_wrap(cpu, cpu_online_mask); - if (cpu !=3D smp_processor_id()) + if (cpu !=3D smp_processor_id()) { smp_call_function_single(cpu, hazptr_torture_acquire, hppp, 1); - else + } else { hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); + atomic_long_inc(per_cpu_ptr(&hazptr_torture_acquires, + raw_smp_processor_id())); + } } else { hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); + atomic_long_inc(per_cpu_ptr(&hazptr_torture_acquires, + raw_smp_processor_id())); } if (!hppp->hpp_htp) { // Still starting up, so get out of the way. @@ -502,6 +517,8 @@ static void hazptr_torture_do_one_pending(int cpu, stru= ct torture_random_state * return; llist_for_each_entry_safe(hppp, hppp1, llnp, hpp_node) { hazptr_torture_reader_tail(hppp, trsp); + atomic_long_inc(per_cpu_ptr(&hazptr_torture_releases_undefer, + raw_smp_processor_id())); kfree(hppp); } } @@ -611,6 +628,13 @@ hazptr_torture_stats_print(void) atomic_read(&n_hazptr_torture_alloc_fail), atomic_read(&n_hazptr_torture_free)); torture_onoff_stats(); + pr_cont("acq: %lld rel: %lld acqirq: %lld relirq: %lld reldefer: %lld rel= undefer %lld\n", + torture_sum_pcpu_atomic_long(&hazptr_torture_acquires), + torture_sum_pcpu_atomic_long(&hazptr_torture_releases), + torture_sum_pcpu_atomic_long(&hazptr_torture_acquires_irq), + torture_sum_pcpu_atomic_long(&hazptr_torture_releases_irq), + torture_sum_pcpu_atomic_long(&hazptr_torture_releases_defer), + torture_sum_pcpu_atomic_long(&hazptr_torture_releases_undefer)); =20 pr_alert("%s%s ", torture_type, TORTURE_FLAG); if (i > 1) { diff --git a/kernel/torture.c b/kernel/torture.c index bcd2e9c8a26356..bb9e1349418a19 100644 --- a/kernel/torture.c +++ b/kernel/torture.c @@ -1007,3 +1007,18 @@ void torture_sched_set_normal(struct task_struct *t,= int nice) sched_set_normal(t, realnice); } EXPORT_SYMBOL_GPL(torture_sched_set_normal); + +/* + * Sum the specified per-CPU atomic_long_t variable. + */ +s64 torture_sum_pcpu_atomic_long(atomic_long_t __percpu *pcp) +{ + int cpu; + s64 sum =3D 0; + + for_each_possible_cpu(cpu) { + sum +=3D atomic_long_read(per_cpu_ptr(pcp, cpu)); + } + return sum; +} +EXPORT_SYMBOL_GPL(torture_sum_pcpu_atomic_long); --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5EAC23D283; Thu, 16 Jul 2026 00:18:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161096; cv=none; b=m+Igw28ipJDHbxgg6ngILIXj4nA+DbgqPpSe2suBrMcVjBU29bLs7lPpo8OflnnIjjjwlF9adQnCJcW/BCWBoKil/1bJDbetQzecZWHVtm5hp57/uh+xDxvG7o7Kt07gKJfuSa+ngujZxidRf7UDkBF+seMPSKKMa8kFU2HXLgM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161096; c=relaxed/simple; bh=gpQurLnFPrnkDNe+MWnj+a7OVfK1lfa352oLOHf8xQw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PXXQReZiDIfWJvm8s2swGK7rgec+Zla8T0ERQ0QzA3P+6vnUQkzSuzcsvrVmbj0gqsmkpDA3RIQW5X/8h93T2iAnhh7xUvzSUmwm5SGWuyw32FifA8Ou1sA9PueaH1apkoApydtreFHmBtqW1vOS2TLVZUR2SCIFYSe3kla8iw0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DLPtPVIj; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DLPtPVIj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C9A4B1F00ACA; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=Tk/d40pMOZOxOEG/n4oOtTc7jUK/Df01qF5T3EGToyY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=DLPtPVIj2ZbrlkywyeGSZfEQGXe3GTMlgQZm71TqX2bclYUyPsyYX9c1uVxEWseOf Q819E0GrO3jc+6DRFgXSYGDG/w8VndAQb13qLlsyTVgGtVhokX8cgwC6Njzeoos3ue PQ8PFtqa/K1X68PZcENkTgRHIIyk1FzOUoR0lrnT9Mft+O4RfRZ82TPw87D4fnw1A6 skI7qDC05kdO3oR3ATOrZokJUO8c0tTmgNb4SwKb+FqL6Ef4AX4twbjIeJFoBz8bjq QeIW8GXpOFBylUn4oV9XnCxZ8U+Z93PEEE1eSupDnWi6eNDA9UJfwwDjCaYeO2+wGt cYX96dM5k7D1g== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 4F2E6CE0F71; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 18/24] doc: Add hazptrtorture module parameters Date: Wed, 15 Jul 2026 17:18:03 -0700 Message-Id: <20260716001809.11084-18-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add the hazard-pointer-torture module parameters to kernel-parameters.txt. Signed-off-by: Paul E. McKenney --- .../admin-guide/kernel-parameters.txt | 96 +++++++++++++++++++ 1 file changed, 96 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index b5493a7f8f2281..7977ce806a2647 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1951,6 +1951,102 @@ Kernel parameters for 64-bit NUMA, off otherwise. Format: 0 | 1 (for off | on) =20 + hazptrtorture.defer_modulus=3D [KNL] + Defer release of the hazard pointer on average + one out of the specified number of times. Zero + disables. Negative one defaults to 1,000 times + nr_cpu_ids, unless hazptr.kthread_do_pending_ms is + equal to zero, in which case it instead defaults + to zero (disabled). + + hazptrtorture.irq_acquire=3D [KNL] + Acquire hazard pointers from an irq handler on average + one out of the specified number of times. Zero + disables. Negative one defaults to 1,000 times + nr_cpu_ids. + + hazptrtorture.irq_release=3D [KNL] + Release hazard pointers from an irq handler on average + one out of the specified number of times. Zero + disables. Negative one defaults to 1,000 times + nr_cpu_ids. + + hazptrtorture.kthread_do_pending_ms=3D [KNL] + Interval between cleanup of pending (deferred) + hazard-pointer releases in milliseconds. + Zero disables. Negative one defaults to three + milliseconds, unless hazptr.kthread_do_pending_ms + is equal to zero, in which case it instead + defaults to zero (disabled). + + hazptrtorture.nreaders=3D [KNL] + Number of hazard-pointer reader kthreads, each + of which repeatedly acquires and releases hazard + pointers. If the value is zero, one reader + is spawned. If the value is less than zero, + the absolute value is multiplied by the number + of online CPUs at initialization time. + + hazptrtorture.nwriters=3D [KNL] + Controls whether or not there is a writer. + There is at most one writer, so this value must + be zero or one. + + hazptrtorture.onoff_holdoff=3D [KNL] + Set time (s) after boot for CPU-hotplug testing. + + hazptrtorture.onoff_interval=3D [KNL] + Set time (jiffies) between CPU-hotplug operations, + or zero to disable CPU-hotplug testing. + + hazptrtorture.preempt_duration=3D [KNL] + Set duration (in milliseconds) of preemptions + by a high-priority FIFO real-time task. Set to + zero (the default) to disable. The CPUs to + preempt are selected randomly from the set that + are online at a given point in time. Races with + CPUs going offline are ignored, with that attempt + at preemption skipped. + + hazptrtorture.preempt_interval=3D [KNL] + Set interval (in milliseconds, defaulting to one + second) between preemptions by a high-priority + FIFO real-time task. This delay is mediated + by an hrtimer and is further fuzzed to avoid + inadvertent synchronizations. + + hazptrtorture.reader_sleep_us=3D [KNL] + Set occassional read-side sleep time in + microseconds, defaulting to zero for disabled. + + hazptrtorture.shuffle_interval=3D [KNL] + Set task-shuffle interval (seconds). Shuffling + tasks allows some CPUs to go into dyntick-idle + mode during the hazptrtorture test. + + hazptrtorture.shutdown_secs=3D [KNL] + Set time (s) after boot system shutdown. This + is useful for hands-off automated testing. + + hazptrtorture.stat_interval=3D [KNL] + Time (s) between statistics printk()s. + + hazptrtorture.stutter=3D [KNL] + Time (s) to stutter testing, for example, + specifying three seconds (the default) causes + the test to run for three seconds, wait + for five seconds, and so on. This tests the + hazard-pointers primitives' ability to transition + abruptly to and from idle. Setting this to zero + disables stuttering. + + hazptrtorture.torture_type=3D [KNL] + Specify the hazard-pointers implementation + to test. + + hazptrtorture.verbose=3D [KNL] + Enable additional printk() statements. + hd=3D [EIDE] (E)IDE hard drive subsystem geometry Format: ,, =20 --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B0782472A2; Thu, 16 Jul 2026 00:18:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; cv=none; b=Br8ZWjKzdq7LGdZ8E2IPBXaQwQsTSQL1EvsyHDHmEIv/BXsBbbu/Kg3bKF4WFC9LtBbLLg10aWL+0QLfGJNeIuwUPDz25nHQr6e5QaR0ODLURrc/FvUrzM3ltyJE/JQfaEAD2Vm17mEjcXlv7Tj4o0G8mlfFGWcj3D65BqFHPJI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; c=relaxed/simple; bh=g33j4FMeJZmbihNcQPD8E3FIRztKnqE/UjXk5EguQa4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sXVKZ0X/tVxqLcusJ22xtuT5ToUiflrufdkDUK0LVdn+TVNrHmD1drS6aNWRBtibowNtwXAgJVIhGebEzub317cysCoSoyaVfcnWlb6XeiNsxVRw4iqQb5WST69en+35zgFkvi0MX6j1NTgxMmjk2aLRx39FBBQ0kSRT2W3DsyM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UWDMh877; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UWDMh877" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CBEEF1F01562; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=EKacui6kllvNGfuSReTdOlVtHpqrAdOu+NWLYtwgPkk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=UWDMh877XUQepw0OsbHKLovZp/FjITWiGZvqowJXxyVIrFXzqlj9XYHTVDCUosWes zaNy4ycAzxm7zN8BuIMz3H+/eKGIrUhlhfgCbbrrlJFy/kWhu0lJ3QD8vNYHvy8u71 aPxVEH5e/MAZV87AD7F5MQsLg1WYJ1kscm/BHU2/JOxZ+t3PdOQdZPb84ARfuiBUsY EBDA7RAZ9hZZBspwnEB93SOaGiG+pbkLC171m0CmmpfolfvjF0jrnsShQ5ts8CeQ1v xjnURL73cLAGaw7zDEGGZ/Nq/39WKJx0kjEZJiwxlSBQK5GUBoyUtQ8+85i/NaZ04W ZhcwAAeC9jgZA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 5184CCE0F72; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Mathieu Desnoyers , "Paul E . McKenney" Subject: [PATCH RFC v2 19/24] hazptr: Permit detaching hazard pointers from contexts Date: Wed, 15 Jul 2026 17:18:04 -0700 Message-Id: <20260716001809.11084-19-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mathieu Desnoyers Provide a new hazptr_detach() function that detaches a given hazard pointer from its acquisition context. This context might be a task or an interrupt handler. [ paulmck: s/hazptr_detach_from_task/hazptr_detach/ per Mathieu. ] Signed-off-by: Mathieu Desnoyers Signed-off-by: Paul E. McKenney --- include/linux/hazptr.h | 55 ++++++++++++++++++++++++++++-------------- 1 file changed, 37 insertions(+), 18 deletions(-) diff --git a/include/linux/hazptr.h b/include/linux/hazptr.h index 461f481a480b9c..fd19579d1260e7 100644 --- a/include/linux/hazptr.h +++ b/include/linux/hazptr.h @@ -98,6 +98,41 @@ bool hazptr_slot_is_backup(struct hazptr_ctx *ctx, struc= t hazptr_slot *slot) return slot =3D=3D &ctx->backup_slot.slot; } =20 +/* Internal helper. */ +static inline +void hazptr_promote_to_backup_slot(struct hazptr_ctx *ctx, struct hazptr_s= lot *slot) +{ + struct hazptr_slot *backup_slot; + + backup_slot =3D hazptr_chain_backup_slot(ctx); + /* + * Move hazard pointer from the per-CPU slot to the + * backup slot. This requires hazard pointer + * synchronize to iterate on per-CPU slots with + * load-acquire before iterating on the overflow list. + */ + WRITE_ONCE(backup_slot->addr, slot->addr); + /* + * store-release orders store to backup slot addr before + * store to per-CPU slot addr. + */ + smp_store_release(&slot->addr, NULL); + /* Use the backup slot for context. */ + ctx->slot =3D backup_slot; +} + +static inline +void hazptr_detach(struct hazptr_ctx *ctx) +{ + struct hazptr_slot *slot; + + guard(preempt)(); + slot =3D ctx->slot; + if (unlikely(hazptr_slot_is_backup(ctx, slot))) + return; + hazptr_promote_to_backup_slot(ctx, slot); +} + static inline void hazptr_note_context_switch(void) { @@ -106,27 +141,11 @@ void hazptr_note_context_switch(void) =20 for (idx =3D 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) { struct hazptr_slot_item *item =3D &percpu_slots->items[idx]; - struct hazptr_slot *slot =3D &item->slot, *backup_slot; - struct hazptr_ctx *ctx; + struct hazptr_slot *slot =3D &item->slot; =20 if (!slot->addr) continue; - ctx =3D item->ctx.ctx; - backup_slot =3D hazptr_chain_backup_slot(ctx); - /* - * Move hazard pointer from the per-CPU slot to the - * backup slot. This requires hazard pointer - * synchronize to iterate on per-CPU slots with - * load-acquire before iterating on the overflow list. - */ - WRITE_ONCE(backup_slot->addr, slot->addr); - /* - * store-release orders store to backup slot addr before - * store to per-CPU slot addr. - */ - smp_store_release(&slot->addr, NULL); - /* Use the backup slot for context. */ - ctx->slot =3D backup_slot; + hazptr_promote_to_backup_slot(item->ctx.ctx, slot); } } =20 --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19E4524501D; Thu, 16 Jul 2026 00:18:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; cv=none; b=A23abbjKIwNQIuux8DTiPVhkrEdsfbeeEuC3F6UDRXtA1CrIS2aqzL+OtvRsVZA4QATXl+YfZkg0fJtmd7/qmYtLfop8+MR7w6mWO7R6bqfx15dxDTiyQE0w1XcMynprTHT3TWp5tDx+IDWUSxG1cYFUH9oS9/hpgmzwhis9wIA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; c=relaxed/simple; bh=/B+fi8ukPwEgpj/yLUn0uXe7/WHkFf7Da4S66mrAOyA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lgWN095CUBUW535SQqcdF8px4cZar8I7fYtsaSBVm2ZPIFrbbMZSSvhIeo4ZRw9/qDA/YB7UDf7dV3/2P+o8Vq6qo4F4/yRryRkZl++RxSP3RrWQHgXCM6WqLu+OHXQk5sXo1gmcceKHIwKfuCyLjLmdPuFdvRA6P+M0Yae5Ktc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LKHffkn7; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LKHffkn7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D52F21F01563; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=rk8Zt8AGXLeYFlr5hpQ5Pkkm2AOavSzFFbJr+gqcPSU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=LKHffkn7/QRXKJ3vO62b4DAiSRv9QaoG2KL+ozWC8MS7u4xHSJcGS/4o7rQcoL0B3 iMCS8mXC95W9XVQdbjsc19lQENm+PTGR3ByuYKP003idp2hnpqXwkGUBaBU+4SdI23 d7rHBxbtfPHs4n20U+FxTHH2zVeEldibOJ3k1Z6FXAKrr8A6JFwMGHjCPE5hjzKUQA TCW3dPl6vamVpEtInSN1icffsKx5AdZbXkf+Kw5rTfNp4GgKVNVeaEVARVSHdvX675 Hn2DxyoTLYiEzHG5zC6Zh/Fg0gkx1T4Wh1Ak562uIQ+GNhP2vamb8A7mpUZDeS1vuO /dI+z0eIjVSrQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 53CB7CE0F7B; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Mathieu Desnoyers , "Paul E . McKenney" Subject: [PATCH RFC v2 20/24] hazptrtorture: Detach deferred and IPIed hazard pointers Date: Wed, 15 Jul 2026 17:18:05 -0700 Message-Id: <20260716001809.11084-20-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mathieu Desnoyers Pass to hazptr_detach() those hazard pointers that are to be released in some other task or within the context of an IPI handler. [ paulmck: s/hazptr_detach_from_task/hazptr_detach/ per Mathieu. ] Signed-off-by: Mathieu Desnoyers Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index 311b387d6ae755..ac0cb5b9f4f0f6 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -411,6 +411,7 @@ hazptr_torture_reader_tail(struct hazptr_pending *hppp,= struct torture_random_st preempt_enable(); if (irq_release && !(torture_random(trsp) % irq_release)) { guard(preempt)(); + hazptr_detach(&hppp->hpp_hc); cpu =3D cpumask_next_wrap(smp_processor_id(), cpu_online_mask); smp_call_function_single(cpu, hazptr_torture_release, hppp, 1); } else { @@ -428,6 +429,7 @@ static void hazptr_torture_defer(struct hazptr_pending = *hppp, struct torture_ran struct llist_head *llhp; =20 guard(preempt)(); + hazptr_detach(&hppp->hpp_hc); cpu =3D cpumask_next_wrap(cpu, cpu_online_mask); llhp =3D per_cpu_ptr(&hazptr_pending, cpu); llist_add(&hppp->hpp_node, llhp); --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86777231836; Thu, 16 Jul 2026 00:18:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; cv=none; b=fCWPI4k1duzhv46HvLyS4miytamUL43rMJH820/UW65+gmcoeSRvWf/0sVVrO3y9DkJYQc9wUTPY/Uxe8DSFAYVoS6p9rh9J1pXCRvt2uURnm1LOa9wY4Jy/oqHbfXe7I3Lo0a0FvT2V1aT7NT/mV+W7CCUezXJt1QlevJq63m0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; c=relaxed/simple; bh=4eN7brpsI5g8Lv0mxzQWj+z0Yn6cj2Lx5n6r7omFm20=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DhCR8jqlfUhd75GNvNDf99bi3ZlCgVUyk6z7kehdHb5kRkyu+4GZjtbJAUkswQlN8CSgIPyBkG0NhAijoF9tfQxb8q6WY6TCInwwce/8DShBZ2i8G/7HTTSaLy/HhWkD2+EWLrqRdnRH9ofmrjn8+hv7hoAmBz7w9KroK7+AjYg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nj/GOS9X; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nj/GOS9X" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7DC91F00ADE; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=ihQGCuCLa8CwmWPpe0a9bR7kxFnuDDG5DtCRq0y+o7Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=nj/GOS9Xx8ImfIs7cQO5fIjNRIT+FFnvCFXT3MaIapOCxcGBCo+IWsQQAVoe4cX/S yMvxZRDX7d4KPIeE5xYXSeAZelCVkxXe3CID0wwI4Uz0cpYzTf1zz0a27oixrsW/pE 7RIbUfsFEgPF8Kmqvu+gFoA+XlDXTlFnsxbsONS78c8kUlA9vWXJcvbJeMb01Vbl8t WYH8PMnqxltvA/JQAMsXPkJnDXeXCWHcbr2hSYzEr6bZ2JGk13WRcTTeVs7IUGtc6h TJ+5H6qM3AUwkb9H+fX3fD92uEq6dsUMaph/9FAkf59IgzjdVFPaWHpzvQyajHz7kC DAZ5ux/QolIJQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 5792ACE0F7E; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Mathieu Desnoyers , "Paul E . McKenney" Subject: [PATCH RFC v2 21/24] hazptr: Introduce CONFIG_HAZPTR_DEBUG misuse detection Date: Wed, 15 Jul 2026 17:18:06 -0700 Message-Id: <20260716001809.11084-21-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mathieu Desnoyers Introduce Hazard Pointers debug assert, which detects misuse of hazard pointers, namely failure to detach the hazard pointer from its owner thread before releasing it from a different thread. Prints the following to the console when a failure is detected: Hazard Pointer (addr=3D000000006885a05f) released on remote task without be= ing detached from task. Acquire: caller=3Dhazptr_torture_read_lock+0x43/0xa= 0 [hazptrtorture], pid=3D3727, cpu=3D1. Release: pid=3D3725, cpu=3D139. WARNING: ./include/linux/hazptr.h:225 at hazptr_torture_read_unlock+0x68/0x= f0 [hazptrtorture], CPU#139: hazptr_torture_/3725 Modules linked in: hazptrtorture torture nft_masq nft_chain_nat nf_nat nf_c= onntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables nfnetlink CPU: 139 UID: 0 PID: 3725 Comm: hazptr_torture_ Not tainted 7.1.0-rc4+ #9 P= REEMPT(full) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16= .3-2 04/01/2014 RIP: 0010:hazptr_torture_read_unlock+0x74/0xf0 [hazptrtorture] Code: 74 31 8b 4f 40 4c 8b 43 48 49 c7 c2 22 c9 56 c0 48 c7 c7 3b c9 56 c0 = 4c 8d 1d b8 32 f1 ff 52 48 89 fa 4c 89 df 50 51 4c 89 d1 <67> 48 0f b9 3a 4= 8 83 c4 18 48 8b 03 48 8d 53 18 48 c7 00 00 00 00 RSP: 0018:ff621a2a479b3de0 EFLAGS: 00010293 RAX: 0000000000000e8d RBX: ff12ea30d3913488 RCX: ffffffffc056c922 RDX: ffffffffc056c93b RSI: ffffffffc0562280 RDI: ffffffffc0562030 RBP: ff621a2a479b3e50 R08: ffffffffc064ee53 R09: 0000000000000e8f R10: ffffffffc056c922 R11: ffffffffc0562030 R12: 0000000000000000 R13: ffffffffc0562280 R14: ff12ea30d3913488 R15: ff621a2a479b3e50 FS: 0000000000000000(0000) GS:ff12ea5011a84000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f39b6e4b010 CR3: 0000000114c6e005 CR4: 0000000000771ef0 PKRU: 55555554 Call Trace: hazptr_torture_reader_tail+0x8e/0x210 [hazptrtorture] hazptr_torture_reader+0x145/0xb30 [hazptrtorture] ? srso_alias_return_thunk+0x5/0xfbef5 ? set_cpus_allowed_ptr+0x36/0x60 ? srso_alias_return_thunk+0x5/0xfbef5 ? srso_alias_return_thunk+0x5/0xfbef5 ? __pfx_hazptr_torture_reader+0x10/0x10 [hazptrtorture] kthread+0xdf/0x120 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x216/0x2d0 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 ---[ end trace 0000000000000000 ]--- Signed-off-by: Mathieu Desnoyers Signed-off-by: Paul E. McKenney --- include/linux/hazptr.h | 38 ++++++++++++++++++++++++++++++++++++++ kernel/rcu/Kconfig.debug | 9 +++++++++ 2 files changed, 47 insertions(+) diff --git a/include/linux/hazptr.h b/include/linux/hazptr.h index fd19579d1260e7..70db3ca7b95be0 100644 --- a/include/linux/hazptr.h +++ b/include/linux/hazptr.h @@ -51,6 +51,11 @@ struct hazptr_ctx { /* Backup slot in case all per-CPU slots are used. */ struct hazptr_backup_slot backup_slot; struct hlist_node preempt_node; +#ifdef CONFIG_HAZPTR_DEBUG + bool detach_task, detach_cpu; /* Whether the ctx has been detached from t= ask/cpu. */ + int acquire_pid, acquire_cpu; /* Note the task and cpu number at acquire.= */ + unsigned long acquire_caller; /* Acquire instruction pointer. */ +#endif }; =20 struct hazptr_slot_ctx { @@ -127,6 +132,9 @@ void hazptr_detach(struct hazptr_ctx *ctx) struct hazptr_slot *slot; =20 guard(preempt)(); +#ifdef CONFIG_HAZPTR_DEBUG + ctx->detach_task =3D ctx->detach_cpu =3D true; +#endif slot =3D ctx->slot; if (unlikely(hazptr_slot_is_backup(ctx, slot))) return; @@ -145,6 +153,9 @@ void hazptr_note_context_switch(void) =20 if (!slot->addr) continue; +#ifdef CONFIG_HAZPTR_DEBUG + item->ctx.ctx->detach_cpu =3D true; +#endif hazptr_promote_to_backup_slot(item->ctx.ctx, slot); } } @@ -173,6 +184,12 @@ void *hazptr_acquire(struct hazptr_ctx *ctx, void * co= nst *addr_p) percpu_slots =3D this_cpu_ptr(&hazptr_percpu_slots); slot_item =3D &percpu_slots->items[0]; slot =3D &slot_item->slot; +#ifdef CONFIG_HAZPTR_DEBUG + ctx->detach_cpu =3D ctx->detach_task =3D false; + ctx->acquire_pid =3D current->pid; + ctx->acquire_cpu =3D smp_processor_id(); + ctx->acquire_caller =3D _THIS_IP_; +#endif if (unlikely(slot->addr)) return __hazptr_acquire(ctx, addr_p); WRITE_ONCE(slot->addr, HAZPTR_WILDCARD); /* Store B */ @@ -196,6 +213,26 @@ void *hazptr_acquire(struct hazptr_ctx *ctx, void * co= nst *addr_p) return addr; } =20 +#ifdef CONFIG_HAZPTR_DEBUG +/* Called with preemption disabled. */ +static inline +void hazptr_release_debug(struct hazptr_ctx *ctx, void *addr) +{ + int pid =3D current->pid, cpu =3D smp_processor_id(); + bool warn_remote_cpu =3D !ctx->detach_cpu && ctx->acquire_cpu !=3D cpu, + warn_remote_task =3D !ctx->detach_task && ctx->acquire_pid !=3D pid; + + WARN_ONCE(warn_remote_cpu || warn_remote_task, + "Hazard Pointer (addr=3D%p) released on remote %s without %s. Acquire: c= aller=3D%pS, pid=3D%d, cpu=3D%d. Release: pid=3D%d, cpu=3D%d.", + addr, + warn_remote_task ? "task" : "cpu", + warn_remote_task ? "being detached from task" : "context switch", + (void *) ctx->acquire_caller, ctx->acquire_pid, ctx->acquire_cpu, pid, c= pu); +} +#else +static inline void hazptr_release_debug(struct hazptr_ctx *ctx, void *addr= ) { } +#endif + /* Release the protected hazard pointer from @slot. */ static inline void hazptr_release(struct hazptr_ctx *ctx, void *addr) @@ -205,6 +242,7 @@ void hazptr_release(struct hazptr_ctx *ctx, void *addr) if (!addr) return; guard(preempt)(); + hazptr_release_debug(ctx, addr); slot =3D ctx->slot; smp_store_release(&slot->addr, NULL); if (unlikely(hazptr_slot_is_backup(ctx, slot))) diff --git a/kernel/rcu/Kconfig.debug b/kernel/rcu/Kconfig.debug index fe64356f008841..9c3f72474cb501 100644 --- a/kernel/rcu/Kconfig.debug +++ b/kernel/rcu/Kconfig.debug @@ -251,4 +251,13 @@ config TRIVIAL_PREEMPT_RCU =20 This has no value for production and is only for testing. =20 +config HAZPTR_DEBUG + bool "Provide debugging asserts for Hazard Pointers" + depends on DEBUG_KERNEL + default n + help + This option provides consistency checks for Hazard pointers. + + Say Y here if you want to enable those assert, N otherwise. + endmenu # "RCU Debugging" --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 861981F3B85; Thu, 16 Jul 2026 00:18:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; cv=none; b=AeK5m2XOW0tUNjS4t8OgVnCmKLAnjTg3ER+0bmkFhNtQFhHTRZMJP+jU+5/oFwRJ93VllAQhFjoCEzuIx2N21DwTY7/QnKX/g9pyps3cxFvN6+QOjJcYgEhCTDc0a3hZIQVXb15tR9ziz1xaYfh0cj3p2/SIgddnjoEcGwXoFYw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; c=relaxed/simple; bh=6oVzcSBj6kE8wjYY1W4nlz+iO87tEMu2odbs3kQBbUA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=t+k/bOUUA4kacZkOSaQeHyMYzCIn8osd97NwYSRvUIW6Q+UvWt+9GlOZ1tej3Man4kDBSOm3o6Pdf1SnMVGtOLn53ENMrs30yE7nIMjPNkx5/QUQeATQ8gTi6b2f0T0rFXfClNzb7LvY375x7+RznIcV6ZQGFl/Ijewb9MvDsdc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CU2b58TX; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CU2b58TX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D92D91F01566; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=BR7mqeQqi1nfzRNjsh6eZW1e50dQgGiaIf2ARC6PPoQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=CU2b58TXF3wKbn72zP+YgK/WYxyJeWjA8P/XrK+diKJk1Vv/0wWvBTS2glUKxStp8 ATeIpX/Kgd/LFm5WwHdBKXe1wh/vO9ecmldlAobiaISD3Yf5sydug30ZrwxDMND9Q/ eiZEBBfEetCFyw2e/qZU6nuus9P8ktsoKX8TozUzJjLjbSgurl50y8pZB/dwBZEim2 JxjODgnYVIykbyUq4ZWRJTq+/JVLERyVxGbqW9vPdGxEa+6b4nLFznc2bIl/cfHKLi 5pLOURWMVsDUsxbmxprd+IuG03Rgn99q2UI5TiT4x9+ub4QVaexmV7T3Ek6+McA4Tc txjRrqavZKqzw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 59F4BCE0F85; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Mathieu Desnoyers , "Paul E . McKenney" Subject: [PATCH RFC v2 22/24] hazptrtorture: Fix hazptr ownership issue Date: Wed, 15 Jul 2026 17:18:07 -0700 Message-Id: <20260716001809.11084-22-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mathieu Desnoyers hazptr_torture_acquire is invoked from IPI on remote CPUs. It needs to immediately detach the hazptr from its local task so it can be released from other tasks. Without this fix, the hazptrtorture test fails loudly with a OOPS. It passes with the fix applied. [ This applies on top of linux-rcu dev branch at commit e5ee2ea2d00ae5a0ca53ccf9c9100f2357e1bd08 ] [ paulmck: s/hazptr_detach_from_task/hazptr_detach/ per Mathieu. ] Signed-off-by: Mathieu Desnoyers Signed-off-by: Paul E. McKenney --- kernel/rcu/hazptrtorture.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/kernel/rcu/hazptrtorture.c b/kernel/rcu/hazptrtorture.c index ac0cb5b9f4f0f6..b2999f577af8b1 100644 --- a/kernel/rcu/hazptrtorture.c +++ b/kernel/rcu/hazptrtorture.c @@ -370,6 +370,11 @@ static void hazptr_torture_acquire(void *hppp_in) struct hazptr_pending *hppp =3D hppp_in; =20 hppp->hpp_htp =3D cur_ops->readlock(&hppp->hpp_hc); + /* + * Acquiring a hazard pointer from a remote CPU. + * Detach hazptr from its task so it can be released by another task. + */ + hazptr_detach(&hppp->hpp_hc); atomic_long_inc(per_cpu_ptr(&hazptr_torture_acquires_irq, raw_smp_process= or_id())); } =20 @@ -411,6 +416,10 @@ hazptr_torture_reader_tail(struct hazptr_pending *hppp= , struct torture_random_st preempt_enable(); if (irq_release && !(torture_random(trsp) % irq_release)) { guard(preempt)(); + /* + * Handling release from a remote CPU. + * Detach hazptr from its task so it can be released by another task. + */ hazptr_detach(&hppp->hpp_hc); cpu =3D cpumask_next_wrap(smp_processor_id(), cpu_online_mask); smp_call_function_single(cpu, hazptr_torture_release, hppp, 1); @@ -429,6 +438,10 @@ static void hazptr_torture_defer(struct hazptr_pending= *hppp, struct torture_ran struct llist_head *llhp; =20 guard(preempt)(); + /* + * Handling release from a remote CPU. + * Detach hazptr from its task so it can be released by another task. + */ hazptr_detach(&hppp->hpp_hc); cpu =3D cpumask_next_wrap(cpu, cpu_online_mask); llhp =3D per_cpu_ptr(&hazptr_pending, cpu); --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEB9725B085; Thu, 16 Jul 2026 00:18:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161098; cv=none; b=bYunt+fibudWej43IVUl5rH3WdXigeEiD9e8Fx6gi4Bjrtn+hB4kxS3sGNOYgsu/+qWb63LdxkYCfImDE9VRXosYZVT4wyCOWxIru6cEi1pI8pz/FVdfd22sarjlvpAG97KXjAPrUfE0tVTDxN9poqRcQLBJ2CsSgLISRzd+5xA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161098; c=relaxed/simple; bh=BjFUTrvtwU5D3BOnMP1eF/iFS9josaPdGig40fY7wWE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iuuVtDEjiYtKBjk61rUlDSL/LIBnFY5ZgcpFkm6cDRiy+N4WIX1rSQUIa9FoIq22mm6ZUvpQOfM/HuYadc2l8NuearoF5VQDF+FbuO5b8gwUYQw+8HCZ6BVpS9LZVvfswcOgVwVjOZ+KfiIr0mFEkOn5fuxshG2UX5MGCEBGJ3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MaEHg55+; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MaEHg55+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DC5861F0155A; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=S2IRIOjVtcezmaUWCwimKB7xhL3Hl7qs/F5PWQvhzSA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=MaEHg55+NsF8rdZHexUuV1EHp7jkAFXGTIxw7b/Eu8kq/2bOnXH613theuYivcdPg W6sjCXuEhL8t6HuTtnP0esPXxbbmB9lzNy97V9UPfLFz5Gp/32dAgpPKSLuOniTT7V ov0Xuv00F+kVtqeWznc/4SQmb7N/NPefHiVX+/BEn58zZaFB8AqysBcCp9/dhOKp1l P98MgV7dChiqw5gW37kjqGZWq22U0IVg2Pr6NOw7BquUs1kB9mLca2YL/D2y69AiTH MPRhu6jkZgG7nnjAvR4edVVfwSBluZexsnIY4+rgcMMXY64NJdvyR1QaqBK7FgmhNo 0SIq/Y9DL9HXA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 5C74BCE0F9F; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH RFC v2 23/24] hazptrtorture: Enable CONFIG_HAZPTR_DEBUG Date: Wed, 15 Jul 2026 17:18:08 -0700 Message-Id: <20260716001809.11084-23-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This commit enables the CONFIG_HAZPTR_DEBUG Kconfig option in order to more easily debug hazard-pointer handoff issues. Signed-off-by: Paul E. McKenney --- tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT | 2 ++ tools/testing/selftests/rcutorture/configs/hazptr/PREEMPT | 2 ++ 2 files changed, 4 insertions(+) diff --git a/tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT b/= tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT index e2da430abe4d70..405941f3b50f3e 100644 --- a/tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT +++ b/tools/testing/selftests/rcutorture/configs/hazptr/NOPREEMPT @@ -15,3 +15,5 @@ CONFIG_DEBUG_LOCK_ALLOC=3Dn CONFIG_PROVE_LOCKING=3Dn CONFIG_KPROBES=3Dn CONFIG_FTRACE=3Dn +CONFIG_DEBUG_KERNEL=3Dy +CONFIG_HAZPTR_DEBUG=3Dy diff --git a/tools/testing/selftests/rcutorture/configs/hazptr/PREEMPT b/to= ols/testing/selftests/rcutorture/configs/hazptr/PREEMPT index b8ea4364b20b7b..c5b12f3bef7429 100644 --- a/tools/testing/selftests/rcutorture/configs/hazptr/PREEMPT +++ b/tools/testing/selftests/rcutorture/configs/hazptr/PREEMPT @@ -12,3 +12,5 @@ CONFIG_HIBERNATION=3Dn CONFIG_DEBUG_LOCK_ALLOC=3Dn CONFIG_PROVE_LOCKING=3Dn CONFIG_DEBUG_OBJECTS_RCU_HEAD=3Dn +CONFIG_DEBUG_KERNEL=3Dy +CONFIG_HAZPTR_DEBUG=3Dy --=20 2.40.1 From nobody Sat Jul 25 16:48:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB39725F7A5; Thu, 16 Jul 2026 00:18:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; cv=none; b=jBfssdfqFPbBJCS4q8qbccNyxzix0LJ6TNa6id+jIRJ1+mY0A1Lz7kW/71pSknUSOLumG+2MWx+9XzpFRXjJacG6Q2fxxliDO2FR9zAr7md5vQRzVEXF7GDkq+i8A6Zz7REZR+v77wIhGCQnaa2ODqbLSXmHOKeSGKJ8BJiEqFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784161097; c=relaxed/simple; bh=mxJOPif/jV8Wcsk1RJDevaQLZEfK2ZjRh6SiVmLZHKg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nzmXQ7WTWpN7m4PALmnF42QsRRD6Q1pmeDkj/OoMVRpf16BSePAMETb/s0OgFEYSo4lV4mK2C8Xooll/e9IZPMrbgzRo/gvqUf1cr1jVe4SBmtQ1fZ6wYIcIHbkp+h70EzLarhDzMX5/yA+gsoYPhrrVL+/hWbnzci3JRgclCmk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=n1ZTZZmV; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="n1ZTZZmV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E362E1F01564; Thu, 16 Jul 2026 00:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784161091; bh=4Fw3aNAw+royQiOfx6tNUiBUQW4ffB4TkgOEDFkEVR4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=n1ZTZZmVSF87WeIftSeLjIJYJEqcERtPpknSWZf/tKCKl1cSZ6iu9KURxSTYk/Gd5 wT7gZmLgxowVwj00TRj2W39Y1goTgL/cbwef+/TlEKvlpxcZwXPlMLPK26VmLFHNS1 B7fhnQQ21htG6zijiTnw8As6385ojiSbWvBefhkVYx5A+ScnTQcnAfUwXQe+ly/Cvv uyKxZepFSUXbZEyV3sDmNVH3UCnQ35fDM3R+XfKskmP9XEQRLm5xtQvcFplqRd4FpI r4zp0hO9zxuLgktxQHbDi8Uu63vsGIv5KjYsDhmzLpzn0pG7Tye6g4Mf8jDZfBX5ts 1njenqJ5RkaAA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 5EE22CE0569; Wed, 15 Jul 2026 17:18:11 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Mathieu Desnoyers , Boqun Feng Subject: [PATCH RFC v2 24/24] hazptr: Upgrade kernel-doc headers Date: Wed, 15 Jul 2026 17:18:09 -0700 Message-Id: <20260716001809.11084-24-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> References: <23e34c2e-67fd-45da-b130-e70a131a59ea@paulmck-laptop> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Upgrade the kernel-doc headers for hazptr_acquire(), hazptr_release(), and hazptr_detach_from_task(). [ paulmck: s/hazptr_detach_from_task/hazptr_detach/ per Mathieu. ] Signed-off-by: Paul E. McKenney Cc: Mathieu Desnoyers Cc: Boqun Feng --- include/linux/hazptr.h | 75 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 65 insertions(+), 10 deletions(-) diff --git a/include/linux/hazptr.h b/include/linux/hazptr.h index 70db3ca7b95be0..80232eb6a592d5 100644 --- a/include/linux/hazptr.h +++ b/include/linux/hazptr.h @@ -75,12 +75,16 @@ DECLARE_PER_CPU(struct hazptr_percpu_slots, hazptr_perc= pu_slots); =20 void *__hazptr_acquire(struct hazptr_ctx *ctx, void * const * addr_p); =20 -/* - * hazptr_synchronize: Wait until @addr is released from all slots. +/** + * hazptr_synchronize: Wait for release from hazard-pointer protection + * + * @addr: The address to be released from hazard-pointer protection * - * Wait to observe that each slot contains a value that differs from - * @addr before returning. - * Should be called from preemptible context. + * Wait for the specified @addr to be released from protection from all + * hazard pointers. The caller should make @addr inaccessible to all + * hazard-pointer readers before invoking this function. + * + * Must be called from preemptible context. */ void hazptr_synchronize(void *addr); =20 @@ -126,6 +130,28 @@ void hazptr_promote_to_backup_slot(struct hazptr_ctx *= ctx, struct hazptr_slot *s ctx->slot =3D backup_slot; } =20 +/** + * hazptr_detach - Allow a hazard pointer to be released in some other con= text + * + * @ctx: The hazard-pointer context to be detached. + * + * By default, a given hazptr_acquire() and the corresponding + * hazptr_release() must run in a single execution context, for example, + * the context of a single task or a single interrupt handler. When you + * have acquired a hazard pointer in one context and need to release it + * in another, you must invoke hazptr_detach() on that hazard pointer's + * context. It is permissible to invoke hazptr_detach() multiple times + * on the same @ctx while it is protecting the same pointer, however, + * the first invocation absolutely must be in the same context that did + * the hazptr_acquire(), and must take place after the return from that + * hazptr_acquire(). + * + * For example, if a hazard pointer is acquired by a task and released + * by a timer handler, that task would need to pass the hazard pointer's + * context to hazptr_detach() after return from the hazptr_acquire() and + * before arming the timer (or at least before the handler had a chance + * to access that hazard-pointer context). + */ static inline void hazptr_detach(struct hazptr_ctx *ctx) { @@ -160,12 +186,25 @@ void hazptr_note_context_switch(void) } } =20 -/* - * hazptr_acquire: Load pointer at address and protect with hazard pointer. +/** + * hazptr_acquire - Load pointer at address and protect with hazard pointe= r. + * + * @ctx: The hazard-pointer context to be passed to hazptr_release(). + * @addr_p: Pointer to the pointer that is to be hazard-pointer protected. * * Load @addr_p, and protect the loaded pointer with hazard pointer. - * When using hazptr_acquire from interrupt handlers, the acquired slots - * need to be released before returning from the interrupt handler. + * This protection is roughly similar to (but way faster than) that of a + * reference counter, and ends with a later call to hazptr_release(). + * + * By default, the call to hazptr_release() must be running in the same + * execution context as the corresponding hazptr_acquire(), for example, + * within the same task or interrupt handler. When it is necessary to + * instead call hazptr_release() from some other context, pass @ctx to + * hazptr_detach() in the original context after invoking hazptr_acquire() + * but before making the hazard pointer available to that other context. + * + * It is not permissible to invoke hazptr_acquire() twice on the same @ctx + * without an intervening hazptr_release(). * * Returns a non-NULL protected address if the loaded pointer is non-NULL. * Returns NULL if the loaded pointer is NULL. @@ -233,7 +272,23 @@ void hazptr_release_debug(struct hazptr_ctx *ctx, void= *addr) static inline void hazptr_release_debug(struct hazptr_ctx *ctx, void *addr= ) { } #endif =20 -/* Release the protected hazard pointer from @slot. */ +/** + * hazptr_release - Release the specified hazard pointer + * + * @ctx: The hazard-pointer context that was passed to hazptr_acquire(). + * @addr_p: The pointer that is to be hazard-pointer unprotected. + * + * Release the protected hazard pointer recorded in @ctx. + * + * By default, hazptr_release() must execute in the same execution context + * that invoked the corresponding hazptr_acquire(), for example, within the + * same task or the same interrupt handler. However, if this restriction + * is problematic for your use case, please see hazptr_detach(). + * + * It is permissible (though unwise from a maintainability viewpoint) + * to invoke hazptr_release() twice on the same @ctx without an intervening + * hazptr_acquire(). + */ static inline void hazptr_release(struct hazptr_ctx *ctx, void *addr) { --=20 2.40.1