From nobody Tue Apr 7 12:37:03 2026 Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC6B11B4244 for ; Fri, 3 Apr 2026 14:24:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226251; cv=none; b=btOxEwhu4m/MgugKK8HQu9kjIuhBfjA3deNf/CdwwpeYtCPyoA6m6HYZFPdJCGwQNLyqmxoTmtpeF0s66k06SThJMFJD4bd5W0xexqRmhjhMDFi1yHwQcBRca8uFVFm6TjEDc+p1V3h9QWnIfPM/v9LAAKyTy4LeYpW/lJlIHy0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226251; c=relaxed/simple; bh=JEbOGTbCGqklP/cu1D/wFazp39cERX3Y6A2nBWEIH1w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZpW0Ef1kolgNaeCt93EDKTxj3iQbiGfj3an45fDxh+5Fp/vKcAnFV3X/xB4MfSkd0Lr3WO/byTYJYprINzDchXS4E0nX9pouFgJvbPvzY/EAOeCkuclrZsHb3JdDAtzay5QOZSlIA5OrlEatPLNfKhqPaL8BbHPMp4evzzyt76A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Q378Xafm; arc=none smtp.client-ip=209.85.210.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q378Xafm" Received: by mail-ot1-f49.google.com with SMTP id 46e09a7af769-7d750eeaec3so878685a34.0 for ; Fri, 03 Apr 2026 07:24:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775226249; x=1775831049; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MSvAMBdnCWX1J3zGwiw7xvxdZPUTObrMMVmHYUBfq3Q=; b=Q378XafmGn7OWO0vn2QVZgBFVnHmcXHoWRQFt86EobZyIWP0X5ES8LhmKz3Cy6Zfo2 9iQuMXbtBjC41XuYJ+Z/1q39Ttz+3Uw1zPvpXxKfoZfRHxMPKHS0UggPwieWgR0UOrMQ 0T49W5nuDNAQ4XYI8GTrUJ1rLAYBLaE17kzrvOlQJ05OWTBM51+lEtzQw6LqYBvbKFBT h/yRvipuF1j1nVgySkFUICnOFWab+rrG9N5AGvba+aLC7eGhiSXVAihUCY5oyIvWU4Fv zufywE0z6xCuqbfg3YykucRCmFF/ZmBVTZjzIkL10W9rhg6YoHawyo4wovt0AbH5s7W5 AGiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775226249; x=1775831049; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=MSvAMBdnCWX1J3zGwiw7xvxdZPUTObrMMVmHYUBfq3Q=; b=psg2j5XvVZQjAoJskjw3LuHn5LgMA2EF6KVMONCkmssDDJcc2QjjiC+xqAa534sG+9 A8LF7pW0B+vR4D2K/s+6HGa8GRCAvbtu/ZOsD8y0Pb/ZFmyHXmS916c+XCDJzqO1w3Gi adsyLxVmsDs+/vB+MOY7Zs59HWdB9DyOQQJdZvLSr/FKQEgYvKFmbMg2JqURa0U3HOMV QFGQlWZuL9CRrHRn+P6bLQhhvQ/W1PJJaC+9TmJO7Tsfk2HP1wCvJd5ITscKgGxpv8RA k0VxnjVYqsUgIOOrrXriC4KZdh9Vf9rpT80eBFGHyyarJPfhY4+kP8jURXGhh67g8NYt uLkQ== X-Forwarded-Encrypted: i=1; AJvYcCWBS/kPBXKx2EFeMkj9rdpSRWIaMeEUl3r6puRCSdLKVKZffk5Omub5Aa6gmPvCQlNJ+eomYBkQIr6DLZc=@vger.kernel.org X-Gm-Message-State: AOJu0Yw0lydgVdPdZ2qTDStZyDMjKtUW2t3Ubry5pWtXqjYg+UXuJcEJ V0ztye6TtHZCdkmbsGb2z6jCXQ6qMAtdW0/zqJg8V+Kud0Vh6O6uPjy0 X-Gm-Gg: ATEYQzznYUmyxC7uU3P6pOALOScp7ojl5kC87aPIGDhQFi0BE14IQUuDYmhNdCeIfbB cz8RNQGVkaWLYV5ySnANQpeixwL9ufyB+VTtCcbq4sbVStuCO+mffFwgD5paTztZA6GWDaiRqmp nGU8r3eXMaK0l4L4Vqpg2u4oDyWzBfXc1uV0B348815HD/mOwAis21H+COfBSTNDB7GsEk4uqQV KIQiF8vArGQMPEv/Lp8HuCtjhfEWgZVnPfQHQ4WxtBlxa3PG8kSAQnH3mYnL9G7KyUXAGE4kOGX xwhltz8nGJflNHZLAkUGW+IuKRnYgFRlooPYISjnB6vYDgB1/TSWUavDBAe5PN/JtHxsOsGP9bd HMZvnRMnom4f9BrmKtNCu7bEmoZuBhbg2ytnTfwG0LI+V5F/1jol9NfUWv8s1l4fjRTeeG6uKJ6 AGlZoQLib1F8nwj/kcqp06AmroGCtge4Jp3NLl1QFcSNNyNC6uhGehUvu5H1HpEW0EKsQHK7Rv4 ZWvgagnDvaFlg== X-Received: by 2002:a05:6830:6084:b0:7d7:cb13:3fe9 with SMTP id 46e09a7af769-7dbb7112eaamr2165259a34.19.1775226248589; Fri, 03 Apr 2026 07:24:08 -0700 (PDT) Received: from frodo.raven-morpho.ts.net (c-98-38-17-99.hsd1.co.comcast.net. [98.38.17.99]) by smtp.googlemail.com with ESMTPSA id 46e09a7af769-7dbb7c3a0c8sm1806671a34.12.2026.04.03.07.24.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Apr 2026 07:24:07 -0700 (PDT) From: Jim Cromie To: peterz@infradead.org, gregkh@linuxfoundation.org Cc: jpoimboe@kernel.org, jbaron@akamai.com, aliceryhl@google.com, rostedt@goodmis.org, ardb@kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, Jim Cromie Subject: [PATCH 1/5] jump_label: add queueing API for batched static key updates Date: Fri, 3 Apr 2026 08:23:57 -0600 Message-ID: <20260403142401.1387033-2-jim.cromie@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260403142401.1387033-1-jim.cromie@gmail.com> References: <20260403142401.1387033-1-jim.cromie@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce a new queueing API for static keys to allow batching of instruction updates. This is provided for subsystems like dynamic-debug, which may toggle thousands of static keys in a single operation. Currently, this suffers from (N)$ overhead due to repeated machine-wide `stop_machine` or `text_poke_sync` synchronizations. The new API includes: - static_key_enable_queued(key) - static_key_disable_queued(key) - static_key_apply_queued() (the global synchronization barrier) - Corresponding static_branch_*_queued(x) macros. The _queued functionality is achieved by adding a 'bool apply' parameter to 3 existing midlayer static functions, and using it to decide whether or not to call arch_jump_label_transform_apply(). By not explicitly calling it, we allow the transform queue to fill 1st. This amortizes the cost of the IPI sync to 1/N (256 on x86). 1. __jump_label_update(struct static_key *key, bool apply) this implements the if (apply) arch_jump_label_transform_apply(); 2. __jump_label_mod_update(struct static_key *key, bool apply) extends _queued feature to modules' static-keys 3. rename jump_label_update(struct static_key *key) to __jump_label_update_key(struct static_key *key, bool apply) and wrap it with 1-liners: jump_label_update() & jump_label_update_queued() On architectures supporting HAVE_JUMP_LABEL_BATCH, the _queued variants lazily fill the architecture's patch queue. It is safe to mix queued and immediate updates; any non-queued static_key_enable/disable call will effectively "flush" any previously queued changes as a side effect. Or just call static_key_apply_queued(). On architectures that do not need a patch queue (e.g., those with atomic patching) or that do not support batching, the API transparently falls back to immediate patching. Signed-off-by: Jim Cromie --- include/linux/jump_label.h | 18 +++++++ kernel/jump_label.c | 104 +++++++++++++++++++++++++++++-------- 2 files changed, 100 insertions(+), 22 deletions(-) diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index fdb79dd1ebd8..283ee9360026 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -234,6 +234,9 @@ extern void static_key_slow_dec_cpuslocked(struct stati= c_key *key); extern int static_key_count(struct static_key *key); extern void static_key_enable(struct static_key *key); extern void static_key_disable(struct static_key *key); +extern void static_key_enable_queued(struct static_key *key); +extern void static_key_disable_queued(struct static_key *key); +extern void static_key_apply_queued(void); extern void static_key_enable_cpuslocked(struct static_key *key); extern void static_key_disable_cpuslocked(struct static_key *key); extern enum jump_label_type jump_label_init_type(struct jump_entry *entry); @@ -340,6 +343,18 @@ static inline void static_key_disable(struct static_ke= y *key) atomic_set(&key->enabled, 0); } =20 +static inline void static_key_enable_queued(struct static_key *key) +{ + static_key_enable(key); +} + +static inline void static_key_disable_queued(struct static_key *key) +{ + static_key_disable(key); +} + +static inline void static_key_apply_queued(void) {} + #define static_key_enable_cpuslocked(k) static_key_enable((k)) #define static_key_disable_cpuslocked(k) static_key_disable((k)) =20 @@ -535,6 +550,9 @@ extern bool ____wrong_branch_error(void); =20 #define static_branch_enable(x) static_key_enable(&(x)->key) #define static_branch_disable(x) static_key_disable(&(x)->key) +#define static_branch_enable_queued(x) static_key_enable_queued(&(x)->key) +#define static_branch_disable_queued(x) static_key_disable_queued(&(x)->k= ey) +#define static_branch_apply_queued() static_key_apply_queued() #define static_branch_enable_cpuslocked(x) static_key_enable_cpuslocked(&(= x)->key) #define static_branch_disable_cpuslocked(x) static_key_disable_cpuslocked(= &(x)->key) =20 diff --git a/kernel/jump_label.c b/kernel/jump_label.c index 7cb19e601426..9826aefd77e5 100644 --- a/kernel/jump_label.c +++ b/kernel/jump_label.c @@ -91,6 +91,7 @@ jump_label_sort_entries(struct jump_entry *start, struct = jump_entry *stop) } =20 static void jump_label_update(struct static_key *key); +static void jump_label_update_queued(struct static_key *key); =20 /* * There are similar definitions for the !CONFIG_JUMP_LABEL case in jump_l= abel.h. @@ -250,10 +251,57 @@ void static_key_disable(struct static_key *key) } EXPORT_SYMBOL_GPL(static_key_disable); =20 +void static_key_enable_queued(struct static_key *key) +{ + STATIC_KEY_CHECK_USE(key); + + if (atomic_read(&key->enabled) > 0) { + /* + * already enabled, don't act. + * warn if static-key-inc was used. + */ + WARN_ON_ONCE(atomic_read(&key->enabled) !=3D 1); + return; + } + + scoped_guard(jump_label_lock) { + if (atomic_read(&key->enabled) =3D=3D 0) { + /* + * set transitional value, update, + * then set stable enabled state + */ + atomic_set(&key->enabled, -1); + jump_label_update_queued(key); + atomic_set_release(&key->enabled, 1); + } + } +} + +void static_key_disable_queued(struct static_key *key) +{ + STATIC_KEY_CHECK_USE(key); + + if (atomic_read(&key->enabled) !=3D 1) { + /* + * not simply enabled; is disabled, inc-enabled, or + * in transition. don't act. warn if inc-enabled. + */ + WARN_ON_ONCE(atomic_read(&key->enabled) !=3D 0); + return; + } + + scoped_guard(jump_label_lock) { + /* + * update if simply enabled. dont act if in transition. + */ + if (atomic_cmpxchg(&key->enabled, 1, 0) =3D=3D 1) + jump_label_update_queued(key); + } +} + static bool static_key_dec_not_one(struct static_key *key) { int v; - /* * Go into the slow path if key::enabled is less than or equal than * one. One is valid to shut down the key, anything less than one @@ -488,28 +536,16 @@ static bool jump_label_can_update(struct jump_entry *= entry, bool init) return true; } =20 -#ifndef HAVE_JUMP_LABEL_BATCH static void __jump_label_update(struct static_key *key, struct jump_entry *entry, struct jump_entry *stop, - bool init) + bool init, bool apply) { for (; (entry < stop) && (jump_entry_key(entry) =3D=3D key); entry++) { - if (jump_label_can_update(entry, init)) - arch_jump_label_transform(entry, jump_label_type(entry)); - } -} -#else -static void __jump_label_update(struct static_key *key, - struct jump_entry *entry, - struct jump_entry *stop, - bool init) -{ - for (; (entry < stop) && (jump_entry_key(entry) =3D=3D key); entry++) { - if (!jump_label_can_update(entry, init)) continue; =20 +#ifdef HAVE_JUMP_LABEL_BATCH if (!arch_jump_label_transform_queue(entry, jump_label_type(entry))) { /* * Queue is full: Apply the current queue and try again. @@ -517,10 +553,24 @@ static void __jump_label_update(struct static_key *ke= y, arch_jump_label_transform_apply(); BUG_ON(!arch_jump_label_transform_queue(entry, jump_label_type(entry))); } +#else + arch_jump_label_transform(entry, jump_label_type(entry)); +#endif } - arch_jump_label_transform_apply(); +#ifdef HAVE_JUMP_LABEL_BATCH + if (apply) + arch_jump_label_transform_apply(); +#endif } + +void static_key_apply_queued(void) +{ +#ifdef HAVE_JUMP_LABEL_BATCH + jump_label_lock(); + arch_jump_label_transform_apply(); + jump_label_unlock(); #endif +} =20 void __init jump_label_init(void) { @@ -671,7 +721,7 @@ static int __jump_label_mod_text_reserved(void *start, = void *end) return ret; } =20 -static void __jump_label_mod_update(struct static_key *key) +static void __jump_label_mod_update(struct static_key *key, bool apply) { struct static_key_mod *mod; =20 @@ -692,7 +742,7 @@ static void __jump_label_mod_update(struct static_key *= key) else stop =3D m->jump_entries + m->num_jump_entries; __jump_label_update(key, mod->entries, stop, - m && m->state =3D=3D MODULE_STATE_COMING); + m && m->state =3D=3D MODULE_STATE_COMING, apply); } } =20 @@ -762,7 +812,7 @@ static int jump_label_add_module(struct module *mod) /* Only update if we've changed from our initial state */ do_poke: if (jump_label_type(iter) !=3D jump_label_init_type(iter)) - __jump_label_update(key, iter, iter_stop, true); + __jump_label_update(key, iter, iter_stop, true, true); } =20 return 0; @@ -892,7 +942,7 @@ int jump_label_text_reserved(void *start, void *end) return ret; } =20 -static void jump_label_update(struct static_key *key) +static void jump_label_update_key(struct static_key *key, bool apply) { struct jump_entry *stop =3D __stop___jump_table; bool init =3D system_state < SYSTEM_RUNNING; @@ -901,7 +951,7 @@ static void jump_label_update(struct static_key *key) struct module *mod; =20 if (static_key_linked(key)) { - __jump_label_mod_update(key); + __jump_label_mod_update(key, apply); return; } =20 @@ -916,7 +966,17 @@ static void jump_label_update(struct static_key *key) entry =3D static_key_entries(key); /* if there are no users, entry can be NULL */ if (entry) - __jump_label_update(key, entry, stop, init); + __jump_label_update(key, entry, stop, init, apply); +} + +static void jump_label_update(struct static_key *key) +{ + jump_label_update_key(key, true); +} + +static void jump_label_update_queued(struct static_key *key) +{ + jump_label_update_key(key, false); } =20 #ifdef CONFIG_STATIC_KEYS_SELFTEST --=20 2.53.0 From nobody Tue Apr 7 12:37:03 2026 Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D181E31AF31 for ; Fri, 3 Apr 2026 14:24:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226253; cv=none; b=uFgcbNNxroe2renZLsln0CqBAkYfHBJgvwlR8V75UMLS0XzeVUsDcaAfAhfDKQ/x88Q/dEa4/Sn7+F2zpD91f4Iz80jokpr8BHKNqWRS7wSggG9QBx13sD1AzJlfbL5z1JldM7jrLqNEf7gi7V1zeWTl5MmBP0L1Fa07r/k2GpA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226253; c=relaxed/simple; bh=FOLvAW/2MDBWDNIX2ItmusLKTznPVoyS8kB8SIYc0YY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ncxo/Zt5kpdCjwuHE7MFphCqm126dF+YdRwxrDx4WlEbVXNv20w+LmmENZYOb61qIJEwAJ6jsyhPYe3haaltbM6Wke4t3GPdRvMd8O+Q6aGjT+qLWe4e+87SQSswry0AaW60N1B8rsB4G184T3zoKy7MQxvAl1ADz2hZo2/QYIA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iSlAk2Qv; arc=none smtp.client-ip=209.85.210.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iSlAk2Qv" Received: by mail-ot1-f41.google.com with SMTP id 46e09a7af769-7d55b97f358so1304439a34.3 for ; Fri, 03 Apr 2026 07:24:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775226250; x=1775831050; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=97ZO5vwgea/5I8mCoklLcrJeYS/yYQVoVQGULeyvPBY=; b=iSlAk2Qv4WxNC46wXfszeXKLVu4cfW/Omz6tgEVNHauIi3RekcuuwdOLk1coCTZbhf T2zLGqM/nRev12JgvG0Oa3hU50Xt/s+ykuTaQgH2JziGwaTgYXYpA/9CS09pkxjEHJd/ ipOKq5JBj65OLFgEMwVM645F35img57ql5FbGV8KDKouJbbgWbZ+GAimgPrxQeelRhgr bYzyLAiteYNYbM5qxwwqnfizy8VyKDPwbA8SVxq0Sp1JxVx8HgbH8OSdKTwfsNA52TTV 6ZXeuw/hDQwDyLbecIobLxy8T2+4T9BduEQDBPcz8tchQo1j73K6/V8S63jDlEIZ+1FP 4k7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775226250; x=1775831050; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=97ZO5vwgea/5I8mCoklLcrJeYS/yYQVoVQGULeyvPBY=; b=WK3iO3HXw+9stAAVx7kIr48OpDI+qPZi3necDSHNOiOoUjVE7wd3ehRWuJsEKq+n7T HssKa5Xc2AQiiq3EN0naR+PfK0oYRPPJtkS2tgN8kAk28O6Iap0XFZWjoRzNZF5PpkbI rSnW4RaaGUNlf/c2O7eFNjXEyBH49zLE5nLl5wgiMybAGWoRlh7PA1llDjORkho9ipUf yu9mp+isdyHU2YDyQtaVabR0kNmYSlNxzgeCLa7YE8oJ8Hw0KhNIixlL6WnF0G8IlvDa igsi4sPBMe5RoCopJvqcUUxwWLhYtCLp0oazNRrfN00Z4PEH8s2c2KBLlqmrtM0d0E3l e7bA== X-Forwarded-Encrypted: i=1; AJvYcCWpi7Sn0HZxWclkPxkhdFJp2JkTPLca9ZgnEWeJRSseRmlKZBjS0lcTJ+HFBrBFdzBcl0N/lLxgazJUgr8=@vger.kernel.org X-Gm-Message-State: AOJu0Yw0hgxc4twd/8Fpg24ee6E4z91FShITwX9o+IfUfH5XgEj9yTL4 laCx3M0885puPv8OiP1zok2002Sr68hiQJOtAggXgoCjRWLj7lMiUIqu X-Gm-Gg: ATEYQzx1CplrTCQSDs915B7dD16t69yGxUvl9Uj2pPfB3w2vuP65lN6EwpgqCX+HitN wkUvWzSr03sduAfw/yD+8mmtWC/rkD+kZnUZNgi26ywhYX3oUgfsxTXk+xttPJ+wUsS1hsrCosG olV6Naby2D2Nsy/4L69+1+MgKbbfDFr8Ljvnl6+phXYP1fGgpqRi6AXbWO5hypse57VIkLsbdY7 nm+VRVxEy49zxpn8M3Mzy2IBlRoTx5tBZ1k+15dmZQRUd/Jznw+maiwjhrMvRW7uqMFYWOn6RNq qte75axJJoEln657LGvqpRw4SyXyF982Nzl77NHz+c8Eh3hSfVuA5Tx/778qaqd+CH8qVil0MyI jgkS4IOUfmpDw2ImeJZZ0wzHnaSR7eyoTS/ceeM2kegJEsq1kktwEXXZiTJllS4PKNtdXL9Tfww 9hGjBBydLatjQwy5TX9yE7CBQhBzQI5FiNtcr5loNxDzy8vbeGy0iKx2HW+lixDr2AWUWBKKLis g37nhBF5kzYDg== X-Received: by 2002:a05:6830:6285:b0:7d7:f15b:e399 with SMTP id 46e09a7af769-7dbb712a397mr2332148a34.16.1775226249693; Fri, 03 Apr 2026 07:24:09 -0700 (PDT) Received: from frodo.raven-morpho.ts.net (c-98-38-17-99.hsd1.co.comcast.net. [98.38.17.99]) by smtp.googlemail.com with ESMTPSA id 46e09a7af769-7dbb7c3a0c8sm1806671a34.12.2026.04.03.07.24.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Apr 2026 07:24:09 -0700 (PDT) From: Jim Cromie To: peterz@infradead.org, gregkh@linuxfoundation.org Cc: jpoimboe@kernel.org, jbaron@akamai.com, aliceryhl@google.com, rostedt@goodmis.org, ardb@kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, Jim Cromie Subject: [PATCH 2/5] x86/alternative.c: sort text-pokes before flushing the queue Date: Fri, 3 Apr 2026 08:23:58 -0600 Message-ID: <20260403142401.1387033-3-jim.cromie@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260403142401.1387033-1-jim.cromie@gmail.com> References: <20260403142401.1387033-1-jim.cromie@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Until now, x86 can queue jump_label text-pokes as long as the poke-addr is monotonically increasing, but flushes the queue when a new poke-addr is less than the previous. Dynamic-debug now uses the queued static-key API, but the advantage is limited; we see a ~2x reduction in IPIs. Although the pr_debug descriptors are ordered, the patch-address in them are not; about 1/2 violate the ordering constraint. So this patch drops that requirement, and sorts the text-pokes by their address before applying them. Doing so lets us fill the queue before sorting then flushing, giving a dramatic ~125x reduction in IPIs over the traditional single IPI per pr_debug. Other arches don't need a queue, and so have nothing to sort. #> dd_ipis [ 23.100381] dyndbg: query 0: "module !virtio* +p " mod:* [ 23.103432] dyndbg: query 1: "-p" mod:* Delta-CAL (IPI): 242 Signed-off-by: Jim Cromie --- arch/x86/kernel/alternative.c | 42 ++++++++++++++++------------------- lib/dynamic_debug.c | 1 + 2 files changed, 20 insertions(+), 23 deletions(-) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index e87da25d1236..92987954b8aa 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2,6 +2,7 @@ #define pr_fmt(fmt) "SMP alternatives: " fmt =20 #include +#include #include #include #include @@ -2823,6 +2824,18 @@ static __always_inline int patch_cmp(const void *tpl= _a, const void *tpl_b) return 0; } =20 +static int text_poke_loc_cmp(const void *a, const void *b) +{ + const struct smp_text_poke_loc *tpl_a =3D a; + const struct smp_text_poke_loc *tpl_b =3D b; + + if (tpl_a->rel_addr < tpl_b->rel_addr) + return -1; + if (tpl_a->rel_addr > tpl_b->rel_addr) + return 1; + return 0; +} + noinstr int smp_text_poke_int3_handler(struct pt_regs *regs) { struct smp_text_poke_loc *tpl; @@ -2935,6 +2948,10 @@ void smp_text_poke_batch_finish(void) if (!text_poke_array.nr_entries) return; =20 + if (text_poke_array.nr_entries > 1) + sort(text_poke_array.vec, text_poke_array.nr_entries, + sizeof(struct smp_text_poke_loc), text_poke_loc_cmp, NULL); + lockdep_assert_held(&text_mutex); =20 /* @@ -3151,28 +3168,6 @@ static void __smp_text_poke_batch_add(void *addr, co= nst void *opcode, size_t len } } =20 -/* - * We hard rely on the text_poke_array.vec being ordered; ensure this is s= o by flushing - * early if needed. - */ -static bool text_poke_addr_ordered(void *addr) -{ - WARN_ON_ONCE(!addr); - - if (!text_poke_array.nr_entries) - return true; - - /* - * If the last current entry's address is higher than the - * new entry's address we'd like to add, then ordering - * is violated and we must first flush all pending patching - * requests: - */ - if (text_poke_addr(text_poke_array.vec + text_poke_array.nr_entries-1) > = addr) - return false; - - return true; -} =20 /** * smp_text_poke_batch_add() -- update instruction on live kernel on SMP, = batched @@ -3189,8 +3184,9 @@ static bool text_poke_addr_ordered(void *addr) */ void __ref smp_text_poke_batch_add(void *addr, const void *opcode, size_t = len, const void *emulate) { - if (text_poke_array.nr_entries =3D=3D TEXT_POKE_ARRAY_MAX || !text_poke_a= ddr_ordered(addr)) + if (text_poke_array.nr_entries =3D=3D TEXT_POKE_ARRAY_MAX) smp_text_poke_batch_finish(); + __smp_text_poke_batch_add(addr, opcode, len, emulate); } =20 diff --git a/lib/dynamic_debug.c b/lib/dynamic_debug.c index 18a71a9108d3..b5060749464e 100644 --- a/lib/dynamic_debug.c +++ b/lib/dynamic_debug.c @@ -264,6 +264,7 @@ static int ddebug_change(const struct ddebug_query *que= ry, } } mutex_unlock(&ddebug_lock); + v2pr_info("applied %d queued updates to sites in total\n", nfound); =20 if (!nfound && verbose) pr_info("no matches for query\n"); --=20 2.53.0 From nobody Tue Apr 7 12:37:03 2026 Received: from mail-ot1-f43.google.com (mail-ot1-f43.google.com [209.85.210.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6CBE3C0621 for ; Fri, 3 Apr 2026 14:24:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226253; cv=none; b=C//IfPPHRFGSPSI5l/TImUJxLi6964yslYQFGpmEbE7bxli9NtPj3oX0zgiqHlxiDFNCql0jWDexCiMciNoX+vJ620DgYfDmJ0m4xbhod2yHxwLVt65vSYKza+j4rrSw/0bbvx1EG6a8kgz1E8ptkn8FzJ0U1m+VonnXLjMwcso= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226253; c=relaxed/simple; bh=5buW3aqEpYxolPKaCtgfA2bWHIO+6C/i+cj9BP10w1Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Kuoltzz7zj8tfVVu4T3mv28WF877eZp7L0ZOgkG8Dt6aVmLGVyRunKgMcMyyipGarda3Kxl1bu/2MBhvv3vrNeww6FmVwJUwwQ8CCrX9NmOnsY1KytCYYAQjRfE3GPPlq5Jcc3U3VcXy8+jiW/hBETdI4RQyjuC3EiSm9toeCyQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YJWLCznO; arc=none smtp.client-ip=209.85.210.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YJWLCznO" Received: by mail-ot1-f43.google.com with SMTP id 46e09a7af769-7d7d4ebccf7so1797436a34.0 for ; Fri, 03 Apr 2026 07:24:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775226251; x=1775831051; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EwwuFEIzjNIpgcW71IJGOS6wh/gt1kuQeZxmGbOapRU=; b=YJWLCznOBE+X7K1u79l84Tuq4hKAlLzr2gxnwxGRfLyCkyn7dw5Kap1YXmmAU8IkR2 NbA2Q1wEEhD2B37ZuWU+UNCUZubrq3taWiuzIdbo2LtCILOibyYFV1w/QzDwGbT46XRu VUNDOKGkbTUh2NcUhA7tlMcwhFUfL+FtXAQgNtknNQHEnwvt2KtO2EGpx641DbTR9HYA 9bIOLBvfek02zSHMmAZIqS5ubp3zXi5cYUVf3mFcj4gdLLEH2eRS1+oLT6gxZGx45MH3 96qaHlJ29qUVEQzDThD7JlRSSEBXGcM/5kScP7uaSYV5U5rPlbuTCKfbLD3asHcqOuri /MDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775226251; x=1775831051; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=EwwuFEIzjNIpgcW71IJGOS6wh/gt1kuQeZxmGbOapRU=; b=YuSYwXgdsCiJidqQJqpFIwn2c6Jg9hM9lsrIg5KG/cAr2lYDjOVakYy/qjYX+7YXYB pj32wpT/5DBeCVwFAt8F9FSZeYBdi18YJmoxbgX62YAIiQ2sxsNvz+DziURAS9jEWzd4 gKgA7jnufVFKTO4DxXo6fbkDOMDGr/Wp2sDpipinKq/ECFtEDNKgmsJq6qdESd7rO0zw NR5dO5jpo/O2IwZwrkzKv1/yOFYsOhHmYMJjjptBGASmJlJRRNZwREOjbnszKpYxS7Hg QflNBQGcKsmfym0Rm9RrGgQEAVxQvbJiiFhmzZ8mnFO4j7Rg9sN9H6t3HL/Z1xDjQNE8 tJ9A== X-Forwarded-Encrypted: i=1; AJvYcCWeDU/rBnGtr8P6ZATDsjjvhmPXeCHQUB7vGgivn2YCIwqEWxvSp5ynHrOMxNjQ4BfSOEt4qrpGXAumGDU=@vger.kernel.org X-Gm-Message-State: AOJu0YyWly1af1NNviQULGZHBJ4lGuAldZH9Z96nfQi4hQSjKxFEpqws yzS5CqcQ/e2hwC3fJPbTUXvOtrHqbgY0fR/qyhJst7uKo225sJLsRXdA X-Gm-Gg: ATEYQzzxzE4yINyEeN0F5mmfIPrdsMmvGPxHoajzZjWr87aQ5WJ0Km+rjNZ4Vggk4je 6icNQjJmyrBD4/QdbTsC4AiOdCbfTEaMkmQodKI8X5pqUdhyf2Wf5BOBp0Blg7FgdrKLeK3IzBn nQLfPeupvP8oeAArkXWEl9glWkK/6++S60QcxKg0pqBEhh44puZOzAyjILaWTYnRHT5ytDu1WLw Qn7Gk0j1tSmEkIFzKBmTkZAnROkA0opMX6R+dQsvb+Kh5JPD7KnBNt2Je5bMwTA0gFAkaSoZbhI 05Kb7Tz94Gs/V7RCshaO8hocCcqN4uec828nA3qV1AJoSkcGpPJRN2hUsWQzZY+SHJFIKLOW5Rh 7KzFzF4cF8ES8M8EozuS4BZXEPwncES2vbxXuC9rICYmM4bGlymsyD38nazjeKysM943zGb98E5 VaLzYXQzK/jdkLB/VEurc4X9A6JkaB7SHr408dH/gMonS0UW4fvz7K2lNJq76DxwtOVjTjAfG6g 6o= X-Received: by 2002:a05:6830:6014:b0:7d9:b314:1452 with SMTP id 46e09a7af769-7dbb730994dmr2089068a34.7.1775226250948; Fri, 03 Apr 2026 07:24:10 -0700 (PDT) Received: from frodo.raven-morpho.ts.net (c-98-38-17-99.hsd1.co.comcast.net. [98.38.17.99]) by smtp.googlemail.com with ESMTPSA id 46e09a7af769-7dbb7c3a0c8sm1806671a34.12.2026.04.03.07.24.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Apr 2026 07:24:10 -0700 (PDT) From: Jim Cromie To: peterz@infradead.org, gregkh@linuxfoundation.org Cc: jpoimboe@kernel.org, jbaron@akamai.com, aliceryhl@google.com, rostedt@goodmis.org, ardb@kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, Jim Cromie Subject: [PATCH 3/5] dyndbg: use static-key queueing API in dynamic-debug Date: Fri, 3 Apr 2026 08:23:59 -0600 Message-ID: <20260403142401.1387033-4-jim.cromie@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260403142401.1387033-1-jim.cromie@gmail.com> References: <20260403142401.1387033-1-jim.cromie@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the new static-key queueing API in dynamic-debug to reduce IPIs. this replaces the 2 calls to static_branch_{enable.disable}() in ddebug_change() with their _queued() counterparts, and adds a call to static_branch_apply_queued() at the bottom. This gives a theoretical reduction of IPIs up to 256x, and a practical reduction by 120-180x. Signed-off-by: Jim Cromie --- lib/dynamic_debug.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/dynamic_debug.c b/lib/dynamic_debug.c index b5060749464e..9bee7dde5148 100644 --- a/lib/dynamic_debug.c +++ b/lib/dynamic_debug.c @@ -250,9 +250,9 @@ static int ddebug_change(const struct ddebug_query *que= ry, #ifdef CONFIG_JUMP_LABEL if (dp->flags & _DPRINTK_FLAGS_PRINT) { if (!(newflags & _DPRINTK_FLAGS_PRINT)) - static_branch_disable(&dp->key.dd_key_true); + static_branch_disable_queued(&dp->key.dd_key_true); } else if (newflags & _DPRINTK_FLAGS_PRINT) { - static_branch_enable(&dp->key.dd_key_true); + static_branch_enable_queued(&dp->key.dd_key_true); } #endif v4pr_info("changed %s:%d [%s]%s %s =3D> %s\n", @@ -263,6 +263,7 @@ static int ddebug_change(const struct ddebug_query *que= ry, dp->flags =3D newflags; } } + static_branch_apply_queued(); mutex_unlock(&ddebug_lock); v2pr_info("applied %d queued updates to sites in total\n", nfound); =20 --=20 2.53.0 From nobody Tue Apr 7 12:37:03 2026 Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FF473BED22 for ; Fri, 3 Apr 2026 14:24:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226254; cv=none; b=cnUiyvKb6tdC5oQrFsdVR0hBo0jAIlCLyO2uNizYeltMcGMgg48nuTnCckZ69/xX1jl4xwofVWzCk363CcIIdb845UeqYOy0DzmCUJYrRcZ1TPzHaiNowytUZAlUSmupFIzEbxZuBCDAJMqcluOdj9Aym54Wir3V40tt+h3BJGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226254; c=relaxed/simple; bh=/89vDYlBhMLpJ7u6S9FMe0zD93ir3EtKs+spl8omcbI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sezm2KCriwyBZeb6qsTBJUp1uVbEgfpTMuo5KFemHZb+bMgeQluiZx2TkKImOaQyBUTMNyFv0ZBfoxspROwVR3+zpdUXIzr32o2eFycvgyTLZOsiw19CRyrFrKnG5djTbHev0agJ6Y9u3U6HVEnfE/ALT1YbZuY1BZqjRtqTg3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gw5PdwFy; arc=none smtp.client-ip=209.85.210.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gw5PdwFy" Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-7d7f09aa39fso2554825a34.0 for ; Fri, 03 Apr 2026 07:24:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775226252; x=1775831052; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7Vh5jItPLrzs3QNiph18TB7iH3kbRO7YJxOr4c937Uw=; b=gw5PdwFyzYeYufT7Z2D0UwI+N4Du0p9PXhrivQ/Ofk9zwg9TG2EV4QwTiwJC90IsuE NOJ5QMzqDJztHxCGFIPTInejE0Jw8ZgPEbLb+jM1Q6OlEVhsAPKaqcAJFYimcVPrmZcr PbT6f3N7AfPHco+DH2aRkaITCNrS22fFoETGmoOyRYw80Rp6yQDtHtitySCdkcflSv55 rDyKvoK/D3dxDN/NqUyBVrVT9S1k6koUuuf/5tw8sGs25mhKrPQLDt+qU0WYDsQX/KKI 6cqXJqSprdzGHxC9hE3+G7fcBUBvI6bmMdLFAHosarkVPSMEcVhq3mZlSiWU3+BlMVfo hh8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775226252; x=1775831052; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=7Vh5jItPLrzs3QNiph18TB7iH3kbRO7YJxOr4c937Uw=; b=DLrmrIkzlc34RZ2QOH7u+Rjt0HdqoOloT9FD8ks8ElmocHSTgpQuhGLWD65Sy/UzkB J7Z/Q6f4xxzoMjcFuleljbi/QH0L+Ys0nUXk8aeimf8TepvpUTqq9h5cKMa6oV7wYVgW 6NMy19DWkgJ8kdnUQslSCRwdjqfxJREZpFzOg4JSg7gUnCkQaoUAfBImF2o+sfJ3f9Kf KJ5Wju5ehph918//IUNocakaYzKsTLrjGkj7gxNqRWVfaBfOzvyRyiNxrcC4gI54QqTM PKhdJhumlFx7LfCnRcE9k3OmRlOVlFSqsRGojD5y2kpitcFBb9H7nLL34iVmOJOPz8su zd9g== X-Forwarded-Encrypted: i=1; AJvYcCXqe+uTjCFvAMUQYNoGleWmyo83w8DKOWcJSfTPPN3G5d0OdXyaHF2L0FDeUEpftJC5yXZ6fh/Mcko+d38=@vger.kernel.org X-Gm-Message-State: AOJu0Yze3CBPYoVugrJ7aWie9g/P9YYjC/a/HNSo3Au8r1nPApOCwZxt wt9cPKmiXX+U6DxYjXIDDv4NiomEExyzaS/9thK+CeHcXh119fvzAXcW X-Gm-Gg: ATEYQzw4mTDw2W0NIera+M25nx5h0YRb00pnNjLrQ3Wy5GwuKkEjCeyuRDT6MPXaXDu +ssoLa5m5eOW1ZKTw8dfCfOzhjR/FFBkIgnrYaS/B1gne1VtQ7z7nAJZqzsAru5p1qLOk37GLY/ rh8oja1ATp3R6D92P+mMtqAA9zMACTiHZYqX45UrLQjvovu3PCUpv0PrcdJWtpQLiMIR4wD/P1B xzFUbXf8+8vW9OUvFk59siPru7luRPqQvrTV4Gwtn1JHRih1vik2Mo+GnqDvvuPDYfGhKZcVcu8 xkS35VOxB/qMrScquYGq4Q4PzVudyV8JRqQMV8lHvNHHO5Wo4PLHVPo4bNkh267Xt3FSY8gPodo qPZWgLQzrFSYcbI1/7yL+Ywm3OZItVOOE9OF/yUuqg81PBj6rQtDgqdjQyXL87pMuwx5qR96o/L maYxyHSZIFKSdfwyFZZhc99lvAS6sKdPfUyhKDBcvVv3bkEsvXp1LWzGWMvuPT8Hn3065dZwgMy zASwJXgZmfR0w== X-Received: by 2002:a9d:6014:0:b0:7db:c162:992c with SMTP id 46e09a7af769-7dbc1629d6bmr670050a34.16.1775226251970; Fri, 03 Apr 2026 07:24:11 -0700 (PDT) Received: from frodo.raven-morpho.ts.net (c-98-38-17-99.hsd1.co.comcast.net. [98.38.17.99]) by smtp.googlemail.com with ESMTPSA id 46e09a7af769-7dbb7c3a0c8sm1806671a34.12.2026.04.03.07.24.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Apr 2026 07:24:11 -0700 (PDT) From: Jim Cromie To: peterz@infradead.org, gregkh@linuxfoundation.org Cc: jpoimboe@kernel.org, jbaron@akamai.com, aliceryhl@google.com, rostedt@goodmis.org, ardb@kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, Jim Cromie , Louis Chauvet Subject: [PATCH 4/5] dyndbg: factor ddebug_match_desc out from ddebug_change (for demonstration only) Date: Fri, 3 Apr 2026 08:24:00 -0600 Message-ID: <20260403142401.1387033-5-jim.cromie@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260403142401.1387033-1-jim.cromie@gmail.com> References: <20260403142401.1387033-1-jim.cromie@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" ddebug_change() is a big (~100 lines) function with a nested for loop. The outer loop walks the per-module ddebug_tables list, and does module stuff: it filters on a query's "module FOO*" and "class BAR", failures here skip the entire inner loop. The inner loop (60 lines) scans a module's descriptors. It starts with a long block of filters on function, line, format, and the validated "BAR" class (or the legacy/_DPRINTK_CLASS_DFLT). These filters "continue" past pr_debugs that don't match the query criteria, before it falls through the code below that counts matches, then adjusts the flags and static-keys. This is unnecessarily hard to think about. So move the per-descriptor filter-block into a boolean function: ddebug_match_desc(desc), and change each "continue" to "return false". This puts a clear interface in place, so any future changes are either inside, outside, or across this interface. also fix checkpatch complaints about spaces and braces. Reviewed-by: Louis Chauvet Signed-off-by: Jim Cromie --- lib/dynamic_debug.c | 83 +++++++++++++++++++++++++-------------------- 1 file changed, 47 insertions(+), 36 deletions(-) diff --git a/lib/dynamic_debug.c b/lib/dynamic_debug.c index 9bee7dde5148..0b13810b03db 100644 --- a/lib/dynamic_debug.c +++ b/lib/dynamic_debug.c @@ -172,6 +172,52 @@ static struct ddebug_class_map *ddebug_find_valid_clas= s(struct ddebug_table cons * callsites, normally the same as number of changes. If verbose, * logs the changes. Takes ddebug_lock. */ +static bool ddebug_match_desc(const struct ddebug_query *query, + struct _ddebug *dp, + int valid_class) +{ + /* match site against query-class */ + if (dp->class_id !=3D valid_class) + return false; + + /* match against the source filename */ + if (query->filename && + !match_wildcard(query->filename, dp->filename) && + !match_wildcard(query->filename, + kbasename(dp->filename)) && + !match_wildcard(query->filename, + trim_prefix(dp->filename))) + return false; + + /* match against the function */ + if (query->function && + !match_wildcard(query->function, dp->function)) + return false; + + /* match against the format */ + if (query->format) { + if (*query->format =3D=3D '^') { + char *p; + /* anchored search. match must be at beginning */ + p =3D strstr(dp->format, query->format + 1); + if (p !=3D dp->format) + return false; + } else if (!strstr(dp->format, query->format)) { + return false; + } + } + + /* match against the line number range */ + if (query->first_lineno && + dp->lineno < query->first_lineno) + return false; + if (query->last_lineno && + dp->lineno > query->last_lineno) + return false; + + return true; +} + static int ddebug_change(const struct ddebug_query *query, struct flag_settings *modifiers) { @@ -204,42 +250,7 @@ static int ddebug_change(const struct ddebug_query *qu= ery, for (i =3D 0; i < dt->num_ddebugs; i++) { struct _ddebug *dp =3D &dt->ddebugs[i]; =20 - /* match site against query-class */ - if (dp->class_id !=3D valid_class) - continue; - - /* match against the source filename */ - if (query->filename && - !match_wildcard(query->filename, dp->filename) && - !match_wildcard(query->filename, - kbasename(dp->filename)) && - !match_wildcard(query->filename, - trim_prefix(dp->filename))) - continue; - - /* match against the function */ - if (query->function && - !match_wildcard(query->function, dp->function)) - continue; - - /* match against the format */ - if (query->format) { - if (*query->format =3D=3D '^') { - char *p; - /* anchored search. match must be at beginning */ - p =3D strstr(dp->format, query->format+1); - if (p !=3D dp->format) - continue; - } else if (!strstr(dp->format, query->format)) - continue; - } - - /* match against the line number range */ - if (query->first_lineno && - dp->lineno < query->first_lineno) - continue; - if (query->last_lineno && - dp->lineno > query->last_lineno) + if (!ddebug_match_desc(query, dp, valid_class)) continue; =20 nfound++; --=20 2.53.0 From nobody Tue Apr 7 12:37:03 2026 Received: from mail-oi1-f181.google.com (mail-oi1-f181.google.com [209.85.167.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33FA03C5529 for ; Fri, 3 Apr 2026 14:24:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226256; cv=none; b=We8PkCHSwh1ZF8JrKEPYRrWQcTBo/E1bE1nhh7ixNEnx/JvK1aUEunUyMCI519FCDhhh2ZHLWxJaE0AO8kZ2orflvjwynHPPePC1IE6Msvn0fpB0vV5xgwqelmeAqfnCHU8JhPQL/LtyV0mUZPRWlM7g9yv7F6zG1TCZc5V782U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775226256; c=relaxed/simple; bh=a5M7P5uLwE25ydXo7XlLk4Wdp4uQveWEuyZ/YP101ig=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bXaFDuhySrfKn/rXPw/fnUaiyazvl/d5vNEPINIa3wh2U8C3+T0IN6ta8zErup08Rb5s2BnPzT73dZzkcHmvicBQgdDmmpdFdN9VZ9D9rKCAT/GB4bjT8XYm4/a9l2PZc5iwHxdmr80s98oSerjS5IIrtw+Kof9PIpoEJ2qTuWQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bAAMPQc9; arc=none smtp.client-ip=209.85.167.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bAAMPQc9" Received: by mail-oi1-f181.google.com with SMTP id 5614622812f47-4670676ba03so723426b6e.1 for ; Fri, 03 Apr 2026 07:24:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775226253; x=1775831053; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wWGJKqvINZ2eLx5DVOX+1vlZFIzp1jhj4sKSWhds1pY=; b=bAAMPQc9ELyEhYA5iOMEzgkZAweOHzoDxIa0x6mqcVCiiW4Bg2L9NysJf/9nTHLG2X +kNz1LSs8Ov1UfQ+LDa+e2X0okc8+uKEmYz9I5STCb1nGZbmVS5oNFb/QK2TvqastL9r JogZpOB+F3wT1bFjmB1YXyUxpyX30LEjVX8vLM4MM3u7SlogbB/0qEOLfwgsmqA+QTvz u6pyWm0CMVcpmZceKPMTaM/y7jtbAYDswwvtkTiHgvQH/50t07SZ4sQffPqaeP4tCgXV yjWeWh4Jo+nB8PPqJngwVsUuhmkGZYQa88xvFZ3NGtwD4Zn92Xro2MhlE4EQw7JresJl VhVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775226253; x=1775831053; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=wWGJKqvINZ2eLx5DVOX+1vlZFIzp1jhj4sKSWhds1pY=; b=Guydrf1sZvaUXb/XXarGlxCD+sst32qJgiewKaQ+Y+fAhhb9DF0iqdfK80MBeEjd+a DrDpQZ4Pf1CEOvVT4Vb40T4/7gw231O2H76Q+LhJwbutLCAgi22HeX9hvApoaI4yyhB0 ePzPzCXvx7STaPAny/vATqjHUccifCWnGXGpiOpqVpOEA94i+s9MQO0F/tzRsTDz6gkP Qg0p0ZxLBbLTAZJT2csSqb5F8YktEyfqu7OAM8AWbD5Ul0dtlgult9OgQR5CUWFkUFYH PCN7PXTiis2aVSNDPN2OC8VTYC6+J/USXOSU6IhSZkvdsiAsyiif2I0jpZv3wD+OBB7z K6ag== X-Forwarded-Encrypted: i=1; AJvYcCXjh6lHWINlhn9di0Xmlaw4J1C3NEK8Q7i+NDs3lYHy0Y/N69d9S9jEp+wy9Jc7gkeH7AiG0Rc8rPMN6t8=@vger.kernel.org X-Gm-Message-State: AOJu0YyaEx3Z5pDIr4RaGbNx3ZRoHZsJlISifry9bavKC2t0+7kB9ORw AezH7vA0ejxA5t4M8ge5pIWj9d5YRBxv/YIymFtFURoMLCGTuihEEvjy X-Gm-Gg: ATEYQzxz+I8YUiJmTsJOQSiftpzVzD+4DG/HlfPaW2qtLGhYNSalnmaz+sA1Xx2BU/j o+D+v2bCxtOPEV/N2xypELA65VtbuJX/Z3PRTZP65F4LhwzEBK5cqUP0BFrz9UTh3o3e0faazU1 2nO+ovWfI934lPw11njSjBoHUoSR9mkBy9Lyuo267bHe/KZGqvvzYos4VbRFFlFuRNh8VV3RpIE Pyd12s2HHfX+r3dvZOLHm3kOv1bA7CCSZummhiSPiX1HGOfTbWnZNCh9m6B73wpEIScqFvfCYLU UxwXPM9hXZLpH4tLKK8PWWcDeJki6FSJM5Wk3i5ie7awJqGQ8uWwue7wkVsvUaMY9mmXsAIIsMT Dp6ik2FRF73c51IHqonw+KZ3R8psLvkbJlVQKwW7OmGRP7AaFKMBGhXQr7c9ot02DJCrZFo8zfm N4amelhcLtM/+80H09KGVyapQ+n7opQ9bT1F852j1Uc5lBEA68kdp0RYYom/ET53yfWpXARlt/t Xs= X-Received: by 2002:a05:6808:f8e:b0:463:8a49:ca7a with SMTP id 5614622812f47-46ef871d0demr1694677b6e.52.1775226253040; Fri, 03 Apr 2026 07:24:13 -0700 (PDT) Received: from frodo.raven-morpho.ts.net (c-98-38-17-99.hsd1.co.comcast.net. [98.38.17.99]) by smtp.googlemail.com with ESMTPSA id 46e09a7af769-7dbb7c3a0c8sm1806671a34.12.2026.04.03.07.24.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Apr 2026 07:24:12 -0700 (PDT) From: Jim Cromie To: peterz@infradead.org, gregkh@linuxfoundation.org Cc: jpoimboe@kernel.org, jbaron@akamai.com, aliceryhl@google.com, rostedt@goodmis.org, ardb@kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, Jim Cromie Subject: [PATCH 5/5] lib/dynamic_debug: add negation support to queries Date: Fri, 3 Apr 2026 08:24:01 -0600 Message-ID: <20260403142401.1387033-6-jim.cromie@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260403142401.1387033-1-jim.cromie@gmail.com> References: <20260403142401.1387033-1-jim.cromie@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This allow users to invert the selection of most keywords, excluding lineno and class. For example: echo "!module virtio* +p" > /proc/dynamic_debug/control When testing a kernel running in virtme-ng, this cmd prevents flooding the logs with virtio activity, and spoiling the the test scenario. NOTE: This is convenient, but not perfect; it cannot also avoid flooding from pr_debugs in serial_core or other potential sources. A more robust command is: echo "!module virtio* +p % module serial -p" > /proc/dynamic_debug/control It also "interferes" with evolution of the grammar; particularly around and/or combinations. For example this could become meaningful, though perhaps inadvisable: echo " !module virtio* !module serial +p " lineno is currently excluded, pending review of its usefulness vs ambiguities, for example " !lineno 1-20 " has straightforward meaning. !class FOO is unlikely to be justifiable; it defeats the "protection thru specificity" that classes are designed for. Signed-off-by: Jim Cromie --- lib/dynamic_debug.c | 81 +++++++++++++++++++++++++++++++-------------- 1 file changed, 57 insertions(+), 24 deletions(-) diff --git a/lib/dynamic_debug.c b/lib/dynamic_debug.c index 0b13810b03db..3addfa06e54d 100644 --- a/lib/dynamic_debug.c +++ b/lib/dynamic_debug.c @@ -58,6 +58,10 @@ struct ddebug_query { const char *format; const char *class_string; unsigned int first_lineno, last_lineno; + unsigned int filename_neg:1; + unsigned int module_neg:1; + unsigned int function_neg:1; + unsigned int format_neg:1; }; =20 struct ddebug_iter { @@ -139,11 +143,12 @@ static void vpr_info_dq(const struct ddebug_query *qu= ery, const char *msg) fmtlen--; } =20 - v3pr_info("%s: func=3D\"%s\" file=3D\"%s\" module=3D\"%s\" format=3D\"%.*= s\" lineno=3D%u-%u class=3D%s\n", + v3pr_info("%s: func%s=3D\"%s\" file%s=3D\"%s\" module%s=3D\"%s\" format%s= =3D\"%.*s\" lineno=3D%u-%u class=3D%s\n", msg, - query->function ?: "", - query->filename ?: "", - query->module ?: "", + query->function_neg ? "!" : "", query->function ?: "", + query->filename_neg ? "!" : "", query->filename ?: "", + query->module_neg ? "!" : "", query->module ?: "", + query->format_neg ? "!" : "", fmtlen, query->format ?: "", query->first_lineno, query->last_lineno, query->class_string); } @@ -176,35 +181,38 @@ static bool ddebug_match_desc(const struct ddebug_que= ry *query, struct _ddebug *dp, int valid_class) { + bool match; + /* match site against query-class */ if (dp->class_id !=3D valid_class) return false; =20 /* match against the source filename */ - if (query->filename && - !match_wildcard(query->filename, dp->filename) && - !match_wildcard(query->filename, - kbasename(dp->filename)) && - !match_wildcard(query->filename, - trim_prefix(dp->filename))) - return false; + if (query->filename) { + match =3D match_wildcard(query->filename, dp->filename) || + match_wildcard(query->filename, kbasename(dp->filename)) || + match_wildcard(query->filename, trim_prefix(dp->filename)); + if (match =3D=3D query->filename_neg) + return false; + } =20 /* match against the function */ - if (query->function && - !match_wildcard(query->function, dp->function)) - return false; + if (query->function) { + match =3D match_wildcard(query->function, dp->function); + if (match =3D=3D query->function_neg) + return false; + } =20 /* match against the format */ if (query->format) { if (*query->format =3D=3D '^') { - char *p; /* anchored search. match must be at beginning */ - p =3D strstr(dp->format, query->format + 1); - if (p !=3D dp->format) - return false; - } else if (!strstr(dp->format, query->format)) { - return false; + match =3D (strstr(dp->format, query->format + 1) =3D=3D dp->format); + } else { + match =3D !!strstr(dp->format, query->format); } + if (match =3D=3D query->format_neg) + return false; } =20 /* match against the line number range */ @@ -234,9 +242,11 @@ static int ddebug_change(const struct ddebug_query *qu= ery, list_for_each_entry(dt, &ddebug_tables, link) { =20 /* match against the module name */ - if (query->module && - !match_wildcard(query->module, dt->mod_name)) - continue; + if (query->module) { + bool match =3D match_wildcard(query->module, dt->mod_name); + if (match =3D=3D query->module_neg) + continue; + } =20 if (query->class_string) { map =3D ddebug_find_valid_class(dt, query->class_string, &valid_class); @@ -399,6 +409,16 @@ static int parse_linerange(struct ddebug_query *query,= const char *first) return 0; } =20 +static char *check_neg(char *src, unsigned int *neg) +{ + if (*src =3D=3D '!') { + *neg =3D 1; + return src + 1; + } + *neg =3D 0; + return src; +} + static int check_set(const char **dest, char *src, char *name) { int rc =3D 0; @@ -441,12 +461,15 @@ static int ddebug_parse_query(char *words[], int nwor= ds, } =20 for (i =3D 0; i < nwords; i +=3D 2) { - char *keyword =3D words[i]; + unsigned int neg; + char *keyword =3D check_neg(words[i], &neg); char *arg =3D words[i+1]; =20 if (!strcmp(keyword, "func")) { + query->function_neg =3D neg; rc =3D check_set(&query->function, arg, "func"); } else if (!strcmp(keyword, "file")) { + query->filename_neg =3D neg; if (check_set(&query->filename, arg, "file")) return -EINVAL; =20 @@ -464,16 +487,26 @@ static int ddebug_parse_query(char *words[], int nwor= ds, return -EINVAL; } } else if (!strcmp(keyword, "module")) { + query->module_neg =3D neg; rc =3D check_set(&query->module, arg, "module"); } else if (!strcmp(keyword, "format")) { string_unescape_inplace(arg, UNESCAPE_SPACE | UNESCAPE_OCTAL | UNESCAPE_SPECIAL); + query->format_neg =3D neg; rc =3D check_set(&query->format, arg, "format"); } else if (!strcmp(keyword, "line")) { + if (neg) { + pr_err("negation not supported for \"line\"\n"); + return -EINVAL; + } if (parse_linerange(query, arg)) return -EINVAL; } else if (!strcmp(keyword, "class")) { + if (neg) { + pr_err("negation not supported for \"class\"\n"); + return -EINVAL; + } rc =3D check_set(&query->class_string, arg, "class"); } else { pr_err("unknown keyword \"%s\"\n", keyword); --=20 2.53.0