From nobody Thu Sep 24 17:48:55 2026 Received: from mail-pj2-f13.google.com (mail-pj2-f13.google.com [74.125.227.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC1F14F93D1 for ; Tue, 22 Sep 2026 07:10:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.141 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790061020; cv=none; b=aQ6TvaS6vWqRIUeUnckRSVg0edE6gzTIZglA+3kVFEytxHw/Mzh58+VQ57FWrpQDGmIx7e7XbY3oxPyknOS0q8rGImd0anXv8Pgv+ZO8zre2dmozTInTuO5XWIFwV8BNVF9B5bMRxBiJsCc1WCDCAM8rAToRxm/HCQE/gfLDEx8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790061020; c=relaxed/simple; bh=72LNOQJjrtL3qWz6jvLkQ4Fallgw3xB29OlIJrXyRUo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TklUrIVQnu49GM/nxP77joSVlwwM4tgVzotL1vc0BVGXh5HYa1NKfzU1eFOBNI0M9tYPa24ONc+fwXSphZ0d5y4/rVZsRCblAwxGeZjPTVvn05WK8q2sqtRQGpN6U4I6T4OW3JkRxpSUbTapKD6ZDMTHrbSOa8Hc0BTM56QwrOU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=rfbmh/8b; arc=none smtp.client-ip=74.125.227.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rfbmh/8b" Received: by mail-pj2-f13.google.com with SMTP id 98e67ed59e1d1-398beb616f5so1607118a91.1 for ; Tue, 22 Sep 2026 00:10:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1790061017; x=1790665817; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=zci1WHdZ5lBidKxN/C/mCTiN5fZd/8DPxW3bn9MBBYQ=; b=rfbmh/8brNWO8si/SpVurmORLwjgrRwxYSeeShImvTFn2mP8VTCSfZbttojnSDTjR6 sMAGLDJ1Gu6jRJsD/aef2nUOZFdhn2e7OzV1TVg0RDUJGWz/jT/5H4ZbowSOqaZjeQC0 PFzAut1vTOdjScf8xdbpn1894TyQZvXGTKjaaWqd6OkpDz3fN891Q1C7i14AuAajcWAz WsoLaDB6P3qbmmLpqXQbFafIOFWVRjP5iiQ4YjBmku628VOJe5G477dX4QTA783n0SdO HwWqr/0vYQV4WvZuJ6kPEM2V3XF0RLrRZoVUm9Y3Ow4lKlhpWn9MMPaAhLN95wr/ZHoz noOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790061017; x=1790665817; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=zci1WHdZ5lBidKxN/C/mCTiN5fZd/8DPxW3bn9MBBYQ=; b=H1YolVyKA6r9fewCMg2i4R1SPDVDFFQnRp6To532sAdRz4kWoNyoLwBwqhC35zGSc6 zg/QIKDzItCKypY4TAICKdr5p2ilt3Ta39pT9XKFqNMlUrl6cWZaJnz/1fi+eQDJtQZt jD5AtaiXxHYZhzXz6jaHMg1t+esebEcRPHeS5HaL8wLIVrkOy/4wUKCGphp55usSgsFO +8INa+L1UQOmV6VtnNF8UKczRUml8WPkyARQ4hdV43CMzZ58P7LDIPMsg9CDyluSYlmr 1pbI8+GM/dVcd0ipJc6xxMwFI8EPMZbEGQd2VTWAHSjDfEUTHJHDni7eFjsjdUZdBiCh dNfg== X-Forwarded-Encrypted: i=1; AKwUvByoAWPQ+BjMRvRNSvC/X4en8rzvWGv4HxjJboK1o1r2yTp/BJG+uh7s1A5NbqmmpHzfJZIX8x6cIOvidgE=@vger.kernel.org X-Gm-Message-State: AFuF++nLOEuDa4ZxNJSHVbegKHE+4HmHiSTftRlHfG99NH4fbdFtIT8F 3aRbQN6q852OFiNQrqoiq4ywaVTTInEOmLED5T98by6WO2oIoNhNmCMZ X-Gm-Gg: AYBFou2uHDKD+Va+EEsVdZL+NL4P3M+KbZTLX+efE82rMfxmAq1/8V2JS1QjvBgiwBX bDxWC0UCPAPHtC/qAiIJFAcc7IUukNHEsYaTLOySOU1fnrO0UfstqtmJNmyRWXh62MJH0iThuGo Esx0j4MV38YZYCp1Bz9XWeJ3Zqc2ZMr4oyA6BXRJxOHgvpfQFT8tCDwDH85jnSEddb/Cytuh+36 3Rpayuxx5cCWDpxOlRFhuPls9YYdL+tH80zA5Ip60L2MrErfpl5yqm4DQVWkXqr8RLpLTj+XVMx GlxldklhZaEK4QZgZVefivRLNNSWPcdJYMbQurCCULfplDiZcHi+8mWTi2lDCEvJgrW0HqbWpAY J9s0/kBZliy9M1E2TGF/4Gsh19O0fQ/b6OlAgndrKijtsVvdcqqWdivJZjiYl8a8JuDjNg+XXMC vt57HacgqnG8m5udx4IpR+8nMBiJASW0N5NqdBiFuXak/xvMHQANTd0LZ1sAU7Se5LPmcWFx0Wa cvUKZC/8JsekUd+pQ== X-Received: by 2002:a17:90b:1d46:b0:3a0:2900:f55e with SMTP id 98e67ed59e1d1-3a0736a923dmr127880a91.29.1790061016560; Tue, 22 Sep 2026 00:10:16 -0700 (PDT) Received: from kernel.tail6741c6.ts.net ([185.220.238.42]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3a067409854sm3079414a91.7.2026.09.22.00.10.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 00:10:16 -0700 (PDT) From: Kunwu Chan To: stern@rowland.harvard.edu, parri.andrea@gmail.com, will@kernel.org, peterz@infradead.org, boqun@kernel.org, npiggin@gmail.com, dhowells@redhat.com, j.alglave@ucl.ac.uk, luc.maranget@inria.fr, paulmck@kernel.org, corbet@lwn.net, mingo@redhat.com, dave@stgolabs.net, josh@joshtriplett.org, frederic@kernel.org, neeraj.upadhyay@kernel.org, urezki@gmail.com Cc: akiyks@gmail.com, dlustig@nvidia.com, joelagnelf@nvidia.com, skhan@linuxfoundation.org, rdunlap@infradead.org, longman@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, kunwu.chan@gmail.com, include@grrlz.net, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, lkmm@lists.linux.dev, linux-doc@vger.kernel.org, rcu@vger.kernel.org, lianux.mm@gmail.com Subject: [RFC/WIP PATCH 1/4] hazptr: add shared-scan kthread Date: Tue, 22 Sep 2026 15:09:47 +0800 Message-ID: <20260922070950.4173245-2-kunwu.chan@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260922070950.4173245-1-kunwu.chan@gmail.com> References: <20260922070950.4173245-1-kunwu.chan@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Batch concurrent hazptr_synchronize() callers into a shared scan cycle, avoiding redundant scans of the per-CPU slots. Queue waiters to a kthread and let each scan cycle make one pass over all CPUs. Each waiter tracks per-CPU progress for both wildcard generations, allowing multiple waiters to share the same scan. Flip the wildcard before scanning. New acquires then use the new generation, so the old-generation mask makes forward progress even under a steady stream of readers. Waiters that remain blocked are retried after a short delay. Fall back to the existing direct two-phase scan if the scan kthread is unavailable or waiter state cannot be allocated. Signed-off-by: Kunwu Chan --- kernel/hazptr.c | 274 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 274 insertions(+) diff --git a/kernel/hazptr.c b/kernel/hazptr.c index d3d1050d92cf..ce553a61b119 100644 --- a/kernel/hazptr.c +++ b/kernel/hazptr.c @@ -12,6 +12,10 @@ #include #include #include +#include +#include +#include +#include =20 /* * The current hazard pointer wildcard. Flips between 1UL and 2UL to guara= ntee @@ -209,12 +213,251 @@ void hazptr_scan_period(void *addr, void *scan_wildc= ard) } } =20 +/* + * Batch hazptr_synchronize() callers through a shared scan kthread. + */ + +struct hazptr_waiter { + struct list_head node; + void *addr; + struct completion done; + /* + * Per-wildcard-generation progress masks. A CPU bit is + * cleared when the scan observes neither @addr nor that + * generation's wildcard on the CPU. + */ + unsigned long *cpu_mask; /* 2 * BITS_TO_LONGS(nr_cpu_ids) */ +}; + +/* Return waiter @w's progress mask for wildcard generation @gen. */ +static unsigned long *hazptr_waiter_mask(struct hazptr_waiter *w, int gen) +{ + return w->cpu_mask + gen * BITS_TO_LONGS(nr_cpu_ids); +} + +struct hazptr_scan_state { + struct task_struct *kthread; + struct swait_queue_head wq; + bool wakeup; + struct mutex lock; + struct list_head pending; + struct list_head scanning; /* kthread only */ +}; +static struct hazptr_scan_state hazptr_scan; + +/* + * Check a CPU's overflow lists. A backup slot can hold a wildcard + * because __hazptr_acquire() writes the wildcard to any slot, + * including backup slots from hazptr_chain_backup_slot(). + * + * @addr: address the waiter is waiting on + * @old_wc: wildcard value of the pre-flip generation + * @new_wc: wildcard value of the post-flip generation + * @has_old: set if any overflow slot holds @old_wc + * @has_new: set if any overflow slot holds @new_wc + * + * Returns true if @addr is present. + */ +static bool hazptr_ovf_list_blocked(int cpu, void *addr, + void *old_wc, void *new_wc, + bool *has_old, bool *has_new) +{ + struct hazptr_overflow_list_flip *ovf =3D per_cpu_ptr(&percpu_overflow_li= st_flip, cpu); + bool found_addr =3D false; + int i; + + for (i =3D 0; i < 2; i++) { + struct hazptr_overflow_list *list =3D &ovf->array[i]; + struct hazptr_backup_slot *b; + unsigned long flags; + + raw_spin_lock_irqsave(&list->lock, flags); + hlist_for_each_entry(b, &list->head, overflow_node) { + /* Pairs with smp_store_release in hazptr_release(). */ + void *val =3D smp_load_acquire(&b->slot.addr); + + if (val =3D=3D addr) + found_addr =3D true; + else if (val =3D=3D old_wc) + *has_old =3D true; + else if (val =3D=3D new_wc) + *has_new =3D true; + } + raw_spin_unlock_irqrestore(&list->lock, flags); + } + return found_addr; +} + +/* + * Move pending waiters to ->scanning, flip the wildcard, then make + * one pass over all CPUs. Clear per-waiter bits for CPUs that no + * longer hold the waiter address or the corresponding wildcard. + * + * After the flip, new acquires use the new wildcard. The old + * generation therefore makes forward progress and is fully cleared + * after enough scan cycles. + */ +static void hazptr_scan_do_cycle(void) +{ + void *old_wc, *new_wc; + unsigned int old_idx, new_idx; + int cpu; + struct hazptr_waiter *w, *n; + LIST_HEAD(done); + + mutex_lock(&hazptr_wildcard_lock); + + mutex_lock(&hazptr_scan.lock); + list_splice_tail_init(&hazptr_scan.pending, &hazptr_scan.scanning); + mutex_unlock(&hazptr_scan.lock); + + if (list_empty(&hazptr_scan.scanning)) { + mutex_unlock(&hazptr_wildcard_lock); + return; + } + + old_wc =3D READ_ONCE(hazptr_wildcard); + new_wc =3D flip_wildcard(old_wc); + WRITE_ONCE(hazptr_wildcard, new_wc); + old_idx =3D (unsigned long)old_wc - 1; + new_idx =3D 1 - old_idx; + + /* + * One pass over all CPUs for the per-CPU slots, checking + * overflow lists for the remaining waiters. + */ + for_each_possible_cpu(cpu) { + struct hazptr_percpu_slots *slots =3D per_cpu_ptr(&hazptr_percpu_slots, = cpu); + void *vals[NR_HAZPTR_PERCPU_SLOTS]; + bool has_old =3D false, has_new =3D false; + unsigned int idx; + + for (idx =3D 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) { + /* Pairs with smp_store_release in hazptr_release(). */ + vals[idx] =3D smp_load_acquire(&slots->items[idx].slot.addr); + if (vals[idx] =3D=3D old_wc) + has_old =3D true; + else if (vals[idx] =3D=3D new_wc) + has_new =3D true; + } + + list_for_each_entry(w, &hazptr_scan.scanning, node) { + bool has_addr =3D false; + + if (!test_bit(cpu, hazptr_waiter_mask(w, old_idx)) && + !test_bit(cpu, hazptr_waiter_mask(w, new_idx))) + continue; /* Both bits already clear. */ + for (idx =3D 0; idx < NR_HAZPTR_PERCPU_SLOTS; idx++) { + if (vals[idx] =3D=3D w->addr) { + has_addr =3D true; + break; + } + } + if (!has_addr) + has_addr =3D hazptr_ovf_list_blocked(cpu, w->addr, + old_wc, new_wc, &has_old, &has_new); + if (has_addr) + continue; + if (!has_old) + __clear_bit(cpu, hazptr_waiter_mask(w, old_idx)); + if (!has_new) + __clear_bit(cpu, hazptr_waiter_mask(w, new_idx)); + } + } + + mutex_unlock(&hazptr_wildcard_lock); + + /* Complete waiters whose masks are both empty. */ + list_for_each_entry_safe(w, n, &hazptr_scan.scanning, node) { + if (bitmap_empty(hazptr_waiter_mask(w, 0), nr_cpu_ids) && + bitmap_empty(hazptr_waiter_mask(w, 1), nr_cpu_ids)) + list_move(&w->node, &done); + } + + list_for_each_entry_safe(w, n, &done, node) { + list_del_init(&w->node); + complete(&w->done); + } +} + +/* + * Shared scan kthread for hazptr_synchronize() waiters. + */ +static int hazptr_scan_kthread(void *unused) +{ + for (;;) { + bool idle; + + swait_event_idle_exclusive(hazptr_scan.wq, + READ_ONCE(hazptr_scan.wakeup)); + + hazptr_scan_do_cycle(); + + mutex_lock(&hazptr_scan.lock); + idle =3D list_empty(&hazptr_scan.pending) && + list_empty(&hazptr_scan.scanning); + if (idle) + WRITE_ONCE(hazptr_scan.wakeup, false); + mutex_unlock(&hazptr_scan.lock); + + if (idle) + continue; + /* Waiters still blocked: retry after a polling delay. */ + schedule_timeout_idle(1); + } + return 0; +} + +/* + * Queue @addr for scan-thread processing, then sleep until the scan + * thread observes that @addr is no longer held by any hazard pointer. + * Returns false if the waiter masks cannot be allocated, in which + * case the caller falls back to the direct scan. + */ +static bool hazptr_synchronize_queued(void *addr) +{ + struct hazptr_waiter waiter =3D { + .addr =3D addr, + }; + unsigned long *masks; + unsigned int mask_longs =3D BITS_TO_LONGS(nr_cpu_ids); + + masks =3D kcalloc(2, mask_longs * sizeof(unsigned long), GFP_KERNEL); + if (!masks) + return false; + bitmap_fill(masks, nr_cpu_ids); + bitmap_fill(masks + mask_longs, nr_cpu_ids); + waiter.cpu_mask =3D masks; + + init_completion(&waiter.done); + INIT_LIST_HEAD(&waiter.node); + + /* Enqueue and wake the scan kthread. */ + mutex_lock(&hazptr_scan.lock); + list_add_tail(&waiter.node, &hazptr_scan.pending); + if (!READ_ONCE(hazptr_scan.wakeup)) { + WRITE_ONCE(hazptr_scan.wakeup, true); + swake_up_one(&hazptr_scan.wq); + } + mutex_unlock(&hazptr_scan.lock); + + /* Sleep until the scan thread completes this waiter. */ + wait_for_completion(&waiter.done); + kfree(masks); + return true; +} + /* * hazptr_synchronize: Wait until @addr is released from all slots. * * Wait to observe that each slot contains a value that differs from * @addr before returning. * Should be called from preemptible context. + * + * If the scan kthread is running, the caller is queued and the scan + * thread performs the work, allowing multiple concurrent callers to + * share a single scan cycle. Otherwise, the existing direct + * two-phase scan is used as a fallback. */ void hazptr_synchronize(void *addr) { @@ -235,6 +478,13 @@ void hazptr_synchronize(void *addr) /* Memory ordering: Store A before Load B. */ smp_mb(); =20 + /* Use the scan thread if available. */ + /* Pairs with smp_store_release in hazptr_scan_init(). */ + if (smp_load_acquire(&hazptr_scan.kthread) && + hazptr_synchronize_queued(addr)) + return; + + /* Fallback: direct two-phase wildcard scan. */ guard(mutex)(&hazptr_wildcard_lock); scan_wildcard =3D flip_wildcard(hazptr_wildcard); hazptr_scan_period(addr, scan_wildcard); @@ -282,3 +532,27 @@ void __init hazptr_init(void) } } } + +/* + * Initialize the scan kthread. On failure falls back to the direct + * scan (busy-wait) path at synchronize time. + * core_initcall ensures the scheduler is ready before kthread_run. + */ +static int __init hazptr_scan_init(void) +{ + struct task_struct *t; + + init_swait_queue_head(&hazptr_scan.wq); + mutex_init(&hazptr_scan.lock); + INIT_LIST_HEAD(&hazptr_scan.pending); + INIT_LIST_HEAD(&hazptr_scan.scanning); + + t =3D kthread_run(hazptr_scan_kthread, NULL, "hazptr_scan"); + if (!IS_ERR(t)) + /* Pairs with smp_load_acquire in hazptr_synchronize(). */ + smp_store_release(&hazptr_scan.kthread, t); + else + pr_warn("hazptr: scan thread failed, using direct scan\n"); + return 0; +} +core_initcall(hazptr_scan_init); --=20 2.43.0 From nobody Thu Sep 24 17:48:55 2026 Received: from mail-pj2-f12.google.com (mail-pj2-f12.google.com [74.125.227.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33D3A4E36F7 for ; Tue, 22 Sep 2026 07:10:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790061028; cv=none; b=t7qb7Cz8rt359PCqRsEuBSvbJu1N4sKSGY5IGucjU/f2UupL3iQzsb6MsZZyZUvBg+9QBgUF6AxePnLrYQkZxQyxjDzng7gQlrE5XdTSHTSyb9GPnW0NDeLUWrhfuShlPlY9xd08PstYhOw4eB1k2EeDN4j1XTDEFsYjoVVR/zY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790061028; c=relaxed/simple; bh=Z5Gkdx2wsDeiiFNXcQT9qA1i2avBskmfCcp53LcmQc4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nAjzTGCXcanXfWz+0cyzod4656DrLbRXSUokG7OfPsa64rMUrE/3yfZiGXMRAWEZLe0v/NSQIQrrnGRo3YufSRSRhTNz1Gt1qxHCUDENexOfB4D9HK2DrhLk1osfdSrr3JOdJAeR1xr5xi86J0A++Fy4LMWpYTFlqn+CqMfbYA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NzzOmESi; arc=none smtp.client-ip=74.125.227.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NzzOmESi" Received: by mail-pj2-f12.google.com with SMTP id 98e67ed59e1d1-396ccafb751so3214618a91.2 for ; Tue, 22 Sep 2026 00:10:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1790061025; x=1790665825; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=AVuEObhXGSApyrXTtBablhdwNNS63ZjQVW0XfLpFgyY=; b=NzzOmESibYOv9tLC6NXFE5gdg3UrV7JUTRLp3KLr8c8SwcQgV134oeeMAxgxvbD+hy 0B+aiuba3bFt28hAZN22Z+B17Jp5rUyNv1ISpw/OnhLWZPEQKedMgHMvOM2Nvi8bGexX cO2mDzs42bkpFX95Ymvnw3zAhlDGTL03ac9xAbVwNt+ZjkaC+Wm9C0bxeAFdye0grSp6 SZdTz26ox6rZw+SJK2uB8kItsTKMCYy1GOT/4lw1sGyOdLw+WzytPAjmpUxsSTbvvrmO giC2/QzZDnv50XuR9riLxvyOpDVwQo5s2laaLFEVL7valJZHRZofs/SLUq5q2fZkCzCD PHsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790061025; x=1790665825; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=AVuEObhXGSApyrXTtBablhdwNNS63ZjQVW0XfLpFgyY=; b=deHov/RRPBve3R+/JHfjJrVh/bpwBHofjw/riNJ8SuF66Rwfxr8VlawEi0fhrOkn12 fLjz6/JjWxO87ysTxB4Bx6W5giotMFZiRhmIrzlplhfdSE3zMVMDxblMS5o6ln3o5ckb /Z6yswG4RPPcYNXsU3zCJ7ypVolpIi5wUQAm87javPZdZD68Wgzrx0TbzX5wGYqwPbEP 2BlHK7ddjSfQwKUY69ZNxUT+07hiz4s62ugOI8zi8B19OoUKx+VP6WLYzsTmllRF1wJk VILcRxgfmdc0EVJUn/bscRFITE2i5zJL/F8SKw2BU29Hry6MU4Dj8bqeN9x8igf6iO5E PsuA== X-Forwarded-Encrypted: i=1; AKwUvByJKy/UuD8o0JPCNBv6Ea3+QJ+duswOq5XnwP1LS/7LcJyZndeapUnaH8jAqsbIO3d5I6dNB9zqZgmFTKI=@vger.kernel.org X-Gm-Message-State: AFuF++mNFZ+CXo1IWIbcn2JnXE7FVDVxtEkwbz7+7C9Jl9Swpmh6+38D 0FZIWJXRRiB96vxQp3B8asNuOmqaIlbXep33D1siypobWzR6d831vPse X-Gm-Gg: AYBFou1ByuS5Cm/Mtg4Kz6T2cA8UPK2yPp4TOxan9xrihTu2EllYWG4CDCd/3EvlyhB IN7E4YWPIybX6/SHNODKJfeGP9FtgYp60wNgtmGbdzunYsBRRKGIzJpLAokAscu6MCBOGp8g+JB CyvYBQllvy7ahR9C7f9GJBqk2ReqsQTvgAqQbo0P3f9e4rijjMwuMnQ7Pcgv4JbRwIbXFVrrnn0 37hCoWy2xKR1Vq2lUTEntKbXH8syoiiV1ADwMbaVqMaZ7Vbi6/kc8AHsL80/NpKmD97PAwSx6sD ElEatULQ/b0lNh1mS8f0J8f2OY2r67fhtjeDRBSRTUNpLG53sUbn3WPejy1mWprR5JEwiR2JTzP lpNscpC2qcXHlJLNC4j4pMQ5sxuQyN2J+j2m/12x8i71CB+pc2rGjkq3KB54gdTLcqmaXfoQvrf aVSUCZejFDsjhdxtRV7OL5sXuDIAQt9rhb3N91Ulq6Yzg3drmklwXnqrjN9yvNeCTIw5kXB1y/W uf8QXRqr67Nga7mSw== X-Received: by 2002:a17:90b:4d8f:b0:39e:6a7f:eef1 with SMTP id 98e67ed59e1d1-3a0731ef87bmr231688a91.22.1790061025401; Tue, 22 Sep 2026 00:10:25 -0700 (PDT) Received: from kernel.tail6741c6.ts.net ([185.220.238.42]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3a067409854sm3079414a91.7.2026.09.22.00.10.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 00:10:24 -0700 (PDT) From: Kunwu Chan To: stern@rowland.harvard.edu, parri.andrea@gmail.com, will@kernel.org, peterz@infradead.org, boqun@kernel.org, npiggin@gmail.com, dhowells@redhat.com, j.alglave@ucl.ac.uk, luc.maranget@inria.fr, paulmck@kernel.org, corbet@lwn.net, mingo@redhat.com, dave@stgolabs.net, josh@joshtriplett.org, frederic@kernel.org, neeraj.upadhyay@kernel.org, urezki@gmail.com Cc: akiyks@gmail.com, dlustig@nvidia.com, joelagnelf@nvidia.com, skhan@linuxfoundation.org, rdunlap@infradead.org, longman@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, kunwu.chan@gmail.com, include@grrlz.net, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, lkmm@lists.linux.dev, linux-doc@vger.kernel.org, rcu@vger.kernel.org, lianux.mm@gmail.com Subject: [RFC/WIP PATCH 2/4] locking/lockdep: use hazptr to wait for dynamic key lookups Date: Tue, 22 Sep 2026 15:09:48 +0800 Message-ID: <20260922070950.4173245-3-kunwu.chan@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260922070950.4173245-1-kunwu.chan@gmail.com> References: <20260922070950.4173245-1-kunwu.chan@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" lockdep_unregister_key() waits for is_dynamic_key() callers with synchronize_rcu_expedited(), which sends IPIs to every online CPU. Have is_dynamic_key() mark the hash bucket with a hazard pointer and use hazptr_synchronize() to wait specifically for those traversals. The hash bucket address from keyhashentry() is stable, making it a suitable hazptr synchronize target. The rest of the key hashlist lifetime (hlist_del_rcu/call_rcu) remains RCU-based. This adapts the lockdep use case from Boqun Feng's hazard-pointer series to the current hazptr API. Signed-off-by: Kunwu Chan --- kernel/locking/lockdep.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index c56a7f91d72e..f67d847f9abf 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -58,6 +58,7 @@ #include #include #include +#include =20 #include =20 @@ -1280,14 +1281,24 @@ static bool is_dynamic_key(const struct lock_class_= key *key) =20 hash_head =3D keyhashentry(key); =20 - rcu_read_lock(); - hlist_for_each_entry_rcu(k, hash_head, hash_entry) { - if (k =3D=3D key) { - found =3D true; - break; + /* + * The traversal is protected by a hazard pointer rather + * than an RCU read-side critical section. + */ + { + struct hazptr_ctx ctx; + void *bucket =3D hash_head; + void *addr; + + addr =3D hazptr_acquire(&ctx, &bucket); + hlist_for_each_entry_rcu(k, hash_head, hash_entry, 1) { + if (k =3D=3D key) { + found =3D true; + break; + } } + hazptr_release(&ctx, addr); } - rcu_read_unlock(); =20 return found; } @@ -6683,11 +6694,10 @@ void lockdep_unregister_key(struct lock_class_key *= key) * * Some operations like __qdisc_destroy() will call this in a debug * kernel, and the network traffic is disabled while waiting, hence - * the delay of the wait matters in debugging cases. Currently use a - * synchronize_rcu_expedited() to speed up the wait at the cost of - * system IPIs. TODO: Replace RCU with hazptr for this. + * the delay of the wait matters in debugging cases. Replace the + * expedited RCU wait with hazptr_synchronize(). */ - synchronize_rcu_expedited(); + hazptr_synchronize(keyhashentry(key)); } EXPORT_SYMBOL_GPL(lockdep_unregister_key); =20 --=20 2.43.0 From nobody Thu Sep 24 17:48:55 2026 Received: from mail-pz2-f42.google.com (mail-pz2-f42.google.com [74.125.228.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DA2650129B for ; Tue, 22 Sep 2026 07:10:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790061037; cv=none; b=uqRQgaWFo/ZJfQ5gvHH8fYyivQeJ2S3kXTfaa1QKr2JBeotwc2zNcHU61pIYIXatRF5TTBET/aTO7kR73Flk73nSRz7zxiPe1IH0f9N8+h+7/W3yZqdGGDrKOMgQ73VC/d7zG1FJDO+9sAE/Xo27Gp48eLVsmjdqXSYRHZWDDf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790061037; c=relaxed/simple; bh=m4OHMwMbzXlh2XQ6c0Igw8Fqh29mNUw6HniKWr6Zo5A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=S+eDCXcdXbmC+iTJ6x8z8sjrzwylFUZWMFNKJm7IWyGp2OAZZ9KF9OJUSLGTSRmPkVOwsEXNDQDpp+toqwXa2z8tOSATnEvUBIMIQqRjtlq2vxFpbujdRhToljgnEY0PG7D6tepa8Nz/QHe6zSeXLI4jFQtZeMsLffxEcqhum4E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RyHmGLyN; arc=none smtp.client-ip=74.125.228.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RyHmGLyN" Received: by mail-pz2-f42.google.com with SMTP id 41be03b00d2f7-cc1cea4c79fso1632449a12.0 for ; Tue, 22 Sep 2026 00:10:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1790061035; x=1790665835; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=y92+ar2Zyy71HW4Z81q+yVXNw3aTL5K091H9SB9aDmE=; b=RyHmGLyN8QNLpEWO+QQfYWhuCr2Qu9gBT3fJLDRzl9vgw/+nwS6Eh931FXXl9WiFX2 C9OWnKIabkV8FlgrFoofY1OrcMr273G+PzaHpT8WxheKgvLPo7AqY9YnNrowYprQmFK2 UGd6/m8fUbzSYiWCfxaORTKEcfsXnOX5w9DSeUS+dbJVpYQZl/dxatY+xZHY+u16SnnT Ovv+bEwZeGPv/B3HShGKwCUeYqOV+p892oMWxRcGCZApaUe6f/QGLp7cDx3NRmTwfBQY FOswVRT2HNxPoXJpCeIaJ77TBXh3RJkkqC4WuTDaMLIqG7LdeLytpm8cBxB0zryyuXst ESng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790061035; x=1790665835; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=y92+ar2Zyy71HW4Z81q+yVXNw3aTL5K091H9SB9aDmE=; b=uHsVAlYEAc2XQqx90KndKUIZVzjtvd4SP9ziUrIuO+qU0LKEedXtfvSKTDjfmCv3Ic E/IGqf0xUl6+5nqLkM72BslofXg8i+AcceKCrttqRNbXnAUDwKAZ0aVOnoXVNfhE73mf ioongwC83Pl+FBRS+FSAiHm4JVbjeiqelwsyUMF4RESHuOwsaAnVC7OTrPiTpn+J7m8J 9pRk62P3/J7qjDD308ynuH/9j2N4tkVShr9g+ZNSL6tMihQRYOnDygKxKMtGUuAwJkuW ffwTWeSsHZpNWwgdM8TTqi3Rhhh4hXwVl3u3EiD6t8jNhhcGJY/iEO2bxYOA20YHTV8x botg== X-Forwarded-Encrypted: i=1; AKwUvBzDH4tybn2lFRhi+OdzYX6CLCXp/KXU5Wxj8h2p/b17yGYQhAvBidAw87x2COZWXuPoYMKPh/T5skgupao=@vger.kernel.org X-Gm-Message-State: AFuF++nsNquJLPT62qV/pixTZKJE9rTKftDoM/oGR6az+Cz+rROD6Ii5 t2VBiVEv4j1JE0NfNhgpIdvnT38LybxWQAbQFi9sACqI0kxN3h5ZAyER X-Gm-Gg: AYBFou3+/VBwdv/7fxn39Det9dxKlk6+FedH8geqtFJp03N0jX2EInCHc6FqIw5IViH gCAcKSe8oraRYA/uVKBzVj/AfWTyW/Z6Oa9P/+KsvTW63vW8dntYQT2CymTp3tEy76dUhdQ+1iW 9zsAT4gUPnBzjNbT7krWnYQqe3TBP9NPKmwf10/u2PBR05akN4JUjfbPC3szjWdafyS6sBb2cQe 25vaUwNm38FCsk0Xw0ozJBrJKHzr7Y5Ca1arKbHecgV3JsRtibMwGgHZo07qwY6pTC3Xjp2j/4x I84cSxDuGdQ5UdOO8GrEuH1nqKUStOzLmj1YaPGdZ/RFFZ4z6A2qX7KvlbfsOGyCbAXwPiTpPfF M0aUTYYAZaKPTFqH45gycFu/c1o99F6Ox7qsQB1c+iTTnWy/bdvAt2xwnsQ200jbD5sxoDvrVsO w6ej4EU0rxr3K8KtkSnIWQysHp6hjprhFewy2ngK6xsdvd8kSSTFTGiuRKqks7dJ71OxHf19EBv qMKM9IO6oXnzS4TMQ== X-Received: by 2002:a17:90b:3806:b0:39e:6c6a:2091 with SMTP id 98e67ed59e1d1-3a073235b91mr260254a91.50.1790061035059; Tue, 22 Sep 2026 00:10:35 -0700 (PDT) Received: from kernel.tail6741c6.ts.net ([185.220.238.42]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3a067409854sm3079414a91.7.2026.09.22.00.10.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 00:10:34 -0700 (PDT) From: Kunwu Chan To: stern@rowland.harvard.edu, parri.andrea@gmail.com, will@kernel.org, peterz@infradead.org, boqun@kernel.org, npiggin@gmail.com, dhowells@redhat.com, j.alglave@ucl.ac.uk, luc.maranget@inria.fr, paulmck@kernel.org, corbet@lwn.net, mingo@redhat.com, dave@stgolabs.net, josh@joshtriplett.org, frederic@kernel.org, neeraj.upadhyay@kernel.org, urezki@gmail.com Cc: akiyks@gmail.com, dlustig@nvidia.com, joelagnelf@nvidia.com, skhan@linuxfoundation.org, rdunlap@infradead.org, longman@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, kunwu.chan@gmail.com, include@grrlz.net, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, lkmm@lists.linux.dev, linux-doc@vger.kernel.org, rcu@vger.kernel.org, lianux.mm@gmail.com Subject: [RFC/WIP PATCH 3/4] rcuscale: add hazptr scale type Date: Tue, 22 Sep 2026 15:09:49 +0800 Message-ID: <20260922070950.4173245-4-kunwu.chan@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260922070950.4173245-1-kunwu.chan@gmail.com> References: <20260922070950.4173245-1-kunwu.chan@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add hazptr reader and synchronize operations so that synchronize latency can be measured alongside RCU and SRCU. The read side acquires the hazard pointer in readlock() and holds it until readunlock(), matching the RCU/SRCU reader model. The address of a static object serves as the synchronize target, which is stable and never reclaimed. Both normal and expedited sync map to hazptr_synchronize(), since hazptr has no expedited concept. Signed-off-by: Kunwu Chan --- kernel/rcu/rcuscale.c | 65 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 64 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/rcuscale.c b/kernel/rcu/rcuscale.c index 1097ec15879c..072ddf9526c3 100644 --- a/kernel/rcu/rcuscale.c +++ b/kernel/rcu/rcuscale.c @@ -39,6 +39,7 @@ #include #include #include +#include #include =20 #include "rcu.h" @@ -418,6 +419,66 @@ static struct rcu_scale_ops tasks_tracing_ops =3D { =20 #endif // #else // #ifdef CONFIG_TASKS_TRACE_RCU =20 +#if IS_ENABLED(CONFIG_HAZPTR_TORTURE_TEST) + +static int hazptr_scale_obj; /* Stable, non-NULL, never reclaimed. */ +static void *hazptr_scale_ptr =3D &hazptr_scale_obj; + +struct hazptr_scale_state { + struct hazptr_ctx ctx; + void *addr; +}; +static DEFINE_PER_CPU(struct hazptr_scale_state, hazptr_scale_state); + +static int hazptr_scale_read_lock(void) +{ + struct hazptr_scale_state *state =3D this_cpu_ptr(&hazptr_scale_state); + + preempt_disable(); + state->addr =3D hazptr_acquire(&state->ctx, &hazptr_scale_ptr); + return 0; +} + +static void hazptr_scale_read_unlock(int idx) +{ + struct hazptr_scale_state *state =3D this_cpu_ptr(&hazptr_scale_state); + + udelay(10); + hazptr_release(&state->ctx, state->addr); + preempt_enable(); +} + +static unsigned long hazptr_scale_completed(void) +{ + return 0; +} + +static void hazptr_scale_sync(void) +{ + hazptr_synchronize(hazptr_scale_ptr); +} + +static void hazptr_scale_sync_exp(void) +{ + hazptr_synchronize(hazptr_scale_ptr); +} + +static struct rcu_scale_ops hazptr_scale_ops =3D { + .ptype =3D 0, + .readlock =3D hazptr_scale_read_lock, + .readunlock =3D hazptr_scale_read_unlock, + .get_gp_seq =3D hazptr_scale_completed, + .gp_diff =3D NULL, + .sync =3D hazptr_scale_sync, + .exp_sync =3D hazptr_scale_sync_exp, + .name =3D "hazptr", +}; + +#define HAZPTR_SCALE_OPS &hazptr_scale_ops, +#else +#define HAZPTR_SCALE_OPS +#endif + static unsigned long rcuscale_seq_diff(unsigned long new, unsigned long ol= d) { if (!cur_ops->gp_diff) @@ -1110,7 +1171,9 @@ rcu_scale_init(void) long i; long j; static struct rcu_scale_ops *scale_ops[] =3D { - &rcu_ops, &srcu_ops, &srcud_ops, TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_= OPS + &rcu_ops, &srcu_ops, &srcud_ops, + TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_OPS + HAZPTR_SCALE_OPS }; =20 if (!torture_init_begin(scale_type, verbose)) --=20 2.43.0 From nobody Thu Sep 24 17:48:55 2026 Received: from mail-pz2-f12.google.com (mail-pz2-f12.google.com [74.125.228.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3F4E4FD7BA for ; Tue, 22 Sep 2026 07:10:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790061048; cv=none; b=jvec9QTjJWjkjAQjOtr5BtFC6nkT/CGrIWdt08AXXOFDJrJB31W0+ch1uAnMUgsAtp29Pczd9J4OAj9ZjXbdsd9O86kRRi8lE4cYwx7zSzzZtLq4Q1bSJBbBqRVirbfgUUk5EoMvmgyfW0wK90e0E+D8UadDlicL31f0SGDEd0s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790061048; c=relaxed/simple; bh=JeXul2Nkf2zeS2OWaaoOg5njiRzZPTKUOP2PjjDjokQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=N/YLwmyvjhJymnzZDFLfw3JIPRm2HAmf6kVgdciT8OcxYMEebu1ypcHa2BwTMsm5iUtBKx+wfuhHSU0HdydJBKeICqQ83SKcnnsfz2tnzkQjDXzE1MQKlGxqs1YYpgWb9afn8ntRwbU2Dvxj2Vb84BI57x0LbnyJQZHODUTalRU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=sdTPOEdm; arc=none smtp.client-ip=74.125.228.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sdTPOEdm" Received: by mail-pz2-f12.google.com with SMTP id 41be03b00d2f7-cc1cea4c79fso1632513a12.0 for ; Tue, 22 Sep 2026 00:10:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1790061046; x=1790665846; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=fM25rpREpN0YJ4vGmgefvM6EGacbAogSR9saanUz1wI=; b=sdTPOEdmJXwieL/77+UQ5ZopnJVY8eQ0VbwKBd42xXAxXLTxn2xTYQVnWxnlv+6hqT IA+Mo5NUhqX95i1TSjolIymQh0ErsvyY524YmOXfToegh7Vi6zoGjeAWDN+1AqdikQ9n 3dWFm8++MfBftbgcwdOYspdJcpWwPNw9f5GSSMH5hFqxyuGcWjdeaRQkVjAvhgkaqpMb 8JubUW3fyaVu+fndevBZuwVwyg6xLgaQKP1JeNtkqJnKyuU04XcP5gzcymUafxfgtrPG rkaOt7IzFSiHRsEplM6gvhDMcierdgdoGWStAh4KyXZSv09Yn7SGr2tua18MqTC9/D68 mf1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790061046; x=1790665846; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=fM25rpREpN0YJ4vGmgefvM6EGacbAogSR9saanUz1wI=; b=0ch9WqMu0lbKPeJEuA0rd1ZDG4xrAKLvCF2slRXEYxZcQ/KU+msJpf6r4jCxrJFqH2 nQpVC1lZfI1QmPM00DPv2BS1rYeAagKFms943dHp08OUQClK2Si9IbabWZo7mu8q6VII qpoZOCZHLFVFAkrcJLfc1HpFCw20NZnY/x7DSX+KiQHt+2FcbzAFkPqKLvs4s5La8gdO j8HHfeIhaPpX8k9GAQuMQ1x/fMAIm2sGWEUnurLlYtdAb8o3eH/myS1mkDMH8MRtkj5k cVbFFlt6Z1uZHY6Kmh7w77vPjcyozVSvluTubdTARYwcZoQEj6RQWiW/PAkxDBJ3k2q6 GirA== X-Forwarded-Encrypted: i=1; AKwUvBy92Ifoy3y+a5YI3ealx0euzPIMHSWTvbM3mBY240SL8jJKO/vZY+WiLJsbT95UYt6oM6TIcU5OiQgExjw=@vger.kernel.org X-Gm-Message-State: AFuF++nlsLX8VgGPkAK2LniVzLwV7HnaO5mF7P8g9V1EuKmRdVHXpQ38 EJ81UwGcbFSZI/RO/ATychcID+ngm3eK9K3NkEyQ/8DA2T8ZbIOOTYWl X-Gm-Gg: AYBFou3JQCmoLqHB4LAU4vPwUbkzI0G2U/nyE6d6/BzkERN5iZX69UgsMalMIEkY4Uv UHbxsEtVxVpdG9BhIngWhRAHQK36UKwmVS0eY9FDJMcR9j3a0EI/Ia/BX+nmfKM/pUraVAPyPcA qK794xv/rHoAGJ4qmwe+3nXgr+vMKjBrvTN4rvnF9sKhl8Q/pQgCcWWrG2/rlL50e2MwQp5YDKb R0K79DyVZwvEAkwkkOFcwNDEnVUBQYvL89gRzY/Oan8h2UWhefF0Jo9X46rwrutlB1ZVvB0lpYx w+oKAgLhG1sXDiIRxQew70HvU4AeToAhjjNwrOwrWnKhr82SmZEID1bCH/YzeNazjr7xjKQuo2G Znt8xZDgqd1S/NhcIrlBMBQMKD8RTBAlV00mMTPc7ChG5l4bW4iwuOHjRluCGP1mZgdfzXKpQPi sZzlMZ5zjsZpUE7/0LVIHFg5xFI59s7cBJ7frl5tIDhQ4hux3IvCqiMTUzA2MSccoLWtdzUmXWz JhhVI5h89Yw++uzTQ== X-Received: by 2002:a17:90b:58d0:b0:3a0:4668:4cb0 with SMTP id 98e67ed59e1d1-3a07322a5bfmr239358a91.47.1790061045855; Tue, 22 Sep 2026 00:10:45 -0700 (PDT) Received: from kernel.tail6741c6.ts.net ([185.220.238.42]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3a067409854sm3079414a91.7.2026.09.22.00.10.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 00:10:45 -0700 (PDT) From: Kunwu Chan To: stern@rowland.harvard.edu, parri.andrea@gmail.com, will@kernel.org, peterz@infradead.org, boqun@kernel.org, npiggin@gmail.com, dhowells@redhat.com, j.alglave@ucl.ac.uk, luc.maranget@inria.fr, paulmck@kernel.org, corbet@lwn.net, mingo@redhat.com, dave@stgolabs.net, josh@joshtriplett.org, frederic@kernel.org, neeraj.upadhyay@kernel.org, urezki@gmail.com Cc: akiyks@gmail.com, dlustig@nvidia.com, joelagnelf@nvidia.com, skhan@linuxfoundation.org, rdunlap@infradead.org, longman@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, qiang.zhang@linux.dev, kunwu.chan@gmail.com, include@grrlz.net, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, lkmm@lists.linux.dev, linux-doc@vger.kernel.org, rcu@vger.kernel.org, lianux.mm@gmail.com Subject: [RFC/WIP PATCH 4/4] Documentation/litmus-tests: add hazptr acquire-before-scan test Date: Tue, 22 Sep 2026 15:09:50 +0800 Message-ID: <20260922070950.4173245-5-kunwu.chan@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260922070950.4173245-1-kunwu.chan@gmail.com> References: <20260922070950.4173245-1-kunwu.chan@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an LKMM test for the hazard-pointer publication protocol used by the lockdep conversion. The test checks that the reclaimer cannot miss a reader hazard pointer publication while the reader still observes the pre-unpublish pointer. The smp_mb() pair provides the required ordering. Models the resolved-publication case; the in-flight wildcard window is covered by the two-phase wildcard scan of the v3 implementation. Verified with herd7: Never 0 6. Signed-off-by: Kunwu Chan --- .../hazptr/hazptr-acquire-before-scan.litmus | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 Documentation/litmus-tests/hazptr/hazptr-acquire-before= -scan.litmus diff --git a/Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.l= itmus b/Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus new file mode 100644 index 000000000000..19df97f1c3b2 --- /dev/null +++ b/Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus @@ -0,0 +1,49 @@ +C hazptr-acquire-before-scan + +(* + * Result: Never + * + * The reclaimer unpublishes the pointer, executes smp_mb(), then + * scans the hazard-pointer slot. The reader publishes the + * protected address, executes smp_mb(), then loads the pointer. + * + * The smp_mb() pair forbids the reclaimer from missing the + * publication while the reader still observes the pre-unpublish + * pointer. + * + * This is the publication protocol used by the lockdep + * is_dynamic_key()/lockdep_unregister_key() conversion. + * + * Models the resolved-publication case; the in-flight wildcard + * window is covered by the two-phase wildcard scan of the v3 + * implementation. + *) + +{ +int ptr =3D 1; (* 1: points to the object, 0: unpublished. *) +int hp =3D 0; (* Hazard-pointer slot: 0: empty, 1: holds the addr. *) +int data =3D 1; (* Object payload; 0: reclaimed. *) +} + +P0(int *ptr, int *hp, int *data) +{ + int r0; + + WRITE_ONCE(*ptr, 0); /* Store A: unpublish. */ + smp_mb(); + r0 =3D READ_ONCE(*hp); /* Load B: scan. */ + WRITE_ONCE(*data, 0); /* Reclaim. */ +} + +P1(int *ptr, int *hp, int *data) +{ + int r0; + int r1; + + WRITE_ONCE(*hp, 1); /* Store B: publish. */ + smp_mb(); + r0 =3D READ_ONCE(*ptr); /* Load A: load pointer. */ + r1 =3D READ_ONCE(*data); /* Access object. */ +} + +exists (0:r0=3D0 /\ 1:r0=3D1) --=20 2.43.0