From nobody Sun May 24 18:44:41 2026 Received: from www262.sakura.ne.jp (www262.sakura.ne.jp [202.181.97.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 132AE3CFF5D for ; Fri, 22 May 2026 11:10:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=202.181.97.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779448253; cv=none; b=SMGnCxnoKw8q4+LGBwV0Ha/V6GqXUrRycZ8Lm5M0j9w8qEjCU1laQi0NzfYMuffw5uidcz0I1DpPuyWPFnxh4m5NlCHTrNrug6q+gY1Y/Lz6oMk5UTfSy0v8jLowYg2Jwvw8KCTBl/lkvKjA5CH448ImYGq2+YQxUfQAWl8obkI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779448253; c=relaxed/simple; bh=FevLZ+L+tujlVsA55LQdamuHwvr6RKe94i4WRCRWneE=; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:References: In-Reply-To:Content-Type; b=QLn8/L+Jc0NZAtvuuj7bxZ15Ph9YDGcWKKhfbhTCkX6udvWcSG9Ufz7c15kQspwZ957tBnR9ywfTNbfMUq4MvhKS8n4D6nfJqdc3VLHFL9ZHgfH+q58FgHIyVWf7esGsB3ffGOFN7N0AkkWz0Ia3Yh3WH5/l15x6OZHwTLlP90E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=I-love.SAKURA.ne.jp; spf=pass smtp.mailfrom=I-love.SAKURA.ne.jp; arc=none smtp.client-ip=202.181.97.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=I-love.SAKURA.ne.jp Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=I-love.SAKURA.ne.jp Received: from www262.sakura.ne.jp (localhost [127.0.0.1]) by www262.sakura.ne.jp (8.15.2/8.15.2) with ESMTP id 64MBAmc6056709; Fri, 22 May 2026 20:10:48 +0900 (JST) (envelope-from penguin-kernel@I-love.SAKURA.ne.jp) Received: from [192.168.1.5] (M106072072000.v4.enabler.ne.jp [106.72.72.0]) (authenticated bits=0) by www262.sakura.ne.jp (8.15.2/8.15.2) with ESMTPSA id 64MBAlWF056706 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NO); Fri, 22 May 2026 20:10:48 +0900 (JST) (envelope-from penguin-kernel@I-love.SAKURA.ne.jp) Message-ID: <7aff4d71-a6a0-4898-9491-2a3973e9d0cc@I-love.SAKURA.ne.jp> Date: Fri, 22 May 2026 20:10:47 +0900 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: [PATCH v4] kcov: move kcov_remote_data to task_struct and remove local_lock From: Tetsuo Handa To: Sebastian Andrzej Siewior , Alexander Potapenko , Dmitry Vyukov Cc: kasan-dev , LKML , Alan Stern , Andrew Morton , Andrey Konovalov , Andrey Konovalov , Clark Williams , Greg Kroah-Hartman , Marco Elver References: <1956540a-e458-4a44-854b-11ff07cb1072@I-love.SAKURA.ne.jp> <20260521083848.UF_zf4xK@linutronix.de> <20260521181450.Ax-chxOE@linutronix.de> <88c3951e-f733-4733-ab81-4feeb5de6b51@I-love.SAKURA.ne.jp> Content-Language: en-US In-Reply-To: <88c3951e-f733-4733-ab81-4feeb5de6b51@I-love.SAKURA.ne.jp> Content-Transfer-Encoding: quoted-printable X-Anti-Virus-Server: fsav201.rs.sakura.ne.jp X-Virus-Status: clean Content-Type: text/plain; charset="utf-8" Softirq handling can be preempted in RT kernels. If we use per-CPU buffers for saving/restoring temporary KCOV contexts, various kcov-related warnings happen due to state corruption. This patch unconditionally moves temporary KCOV context buffers from per-CPU to per-task. Although it is safe to continue using per-CPU buffers in !RT kernels, this patch tries to unify the logic, for Dmitry Vyukov commented Is it possible to unify the logic between RT and non-RT kernels (not have any ifdef's)? The logic is already rather complicated, having 2 work modes, and limited testing will be problematic in the future. =2E The local_lock_t is removed, for per-CPU buffers are removed. But kcov_remote_lock and _irqsave/irqrestore remain in order to serialize access to kcov_remote_areas lists, for kcov_remote_{start,stop}() are called from both task context and softirq context. Analyzed-by: AI Mode in Google Search (no mail address) Signed-off-by: Tetsuo Handa Fixes: 5ff3b30ab57d ("kcov: collect coverage from interrupts") --- Only compile tested. include/linux/sched.h | 8 +++ kernel/kcov.c | 111 +++++++++++++++--------------------------- lib/Kconfig.debug | 5 +- 3 files changed, 51 insertions(+), 73 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index ee06cba5c6f5..eaf975138359 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1525,6 +1525,14 @@ struct task_struct { =20 /* Collect coverage from softirq context: */ unsigned int kcov_softirq; + + /* Temporary storage for preempting remote coverage collection: */ + unsigned int kcov_saved_mode; + unsigned int kcov_saved_size; + void *kcov_saved_area; + struct kcov *kcov_saved_kcov; + int kcov_saved_sequence; + #endif =20 #ifdef CONFIG_MEMCG_V1 diff --git a/kernel/kcov.c b/kernel/kcov.c index 0b369e88c7c9..b96d6ad68f92 100644 --- a/kernel/kcov.c +++ b/kernel/kcov.c @@ -86,21 +86,8 @@ struct kcov_remote { =20 static DEFINE_SPINLOCK(kcov_remote_lock); static DEFINE_HASHTABLE(kcov_remote_map, 4); -static struct list_head kcov_remote_areas =3D LIST_HEAD_INIT(kcov_remote_a= reas); - -struct kcov_percpu_data { - void *irq_area; - local_lock_t lock; - - unsigned int saved_mode; - unsigned int saved_size; - void *saved_area; - struct kcov *saved_kcov; - int saved_sequence; -}; - -static DEFINE_PER_CPU(struct kcov_percpu_data, kcov_percpu_data) =3D { - .lock =3D INIT_LOCAL_LOCK(lock), +static struct list_head kcov_remote_areas[2] =3D { + LIST_HEAD_INIT(kcov_remote_areas[0]), LIST_HEAD_INIT(kcov_remote_areas[1]) }; =20 /* Must be called with kcov_remote_lock locked. */ @@ -136,8 +123,9 @@ static struct kcov_remote_area *kcov_remote_area_get(un= signed int size) { struct kcov_remote_area *area; struct list_head *pos; + struct list_head *list =3D &kcov_remote_areas[size =3D=3D CONFIG_KCOV_IRQ= _AREA_SIZE]; =20 - list_for_each(pos, &kcov_remote_areas) { + list_for_each(pos, list) { area =3D list_entry(pos, struct kcov_remote_area, list); if (area->size =3D=3D size) { list_del(&area->list); @@ -153,7 +141,7 @@ static void kcov_remote_area_put(struct kcov_remote_are= a *area, { INIT_LIST_HEAD(&area->list); area->size =3D size; - list_add(&area->list, &kcov_remote_areas); + list_add(&area->list, &kcov_remote_areas[size =3D=3D CONFIG_KCOV_IRQ_AREA= _SIZE]); /* * KMSAN doesn't instrument this file, so it may not know area->list * is initialized. Unpoison it explicitly to avoid reports in @@ -824,37 +812,32 @@ static inline bool kcov_mode_enabled(unsigned int mod= e) } =20 static void kcov_remote_softirq_start(struct task_struct *t) - __must_hold(&kcov_percpu_data.lock) { - struct kcov_percpu_data *data =3D this_cpu_ptr(&kcov_percpu_data); unsigned int mode; =20 mode =3D READ_ONCE(t->kcov_mode); barrier(); if (kcov_mode_enabled(mode)) { - data->saved_mode =3D mode; - data->saved_size =3D t->kcov_size; - data->saved_area =3D t->kcov_area; - data->saved_sequence =3D t->kcov_sequence; - data->saved_kcov =3D t->kcov; + t->kcov_saved_mode =3D mode; + t->kcov_saved_size =3D t->kcov_size; + t->kcov_saved_area =3D t->kcov_area; + t->kcov_saved_sequence =3D t->kcov_sequence; + t->kcov_saved_kcov =3D t->kcov; kcov_stop(t); } } =20 static void kcov_remote_softirq_stop(struct task_struct *t) - __must_hold(&kcov_percpu_data.lock) { - struct kcov_percpu_data *data =3D this_cpu_ptr(&kcov_percpu_data); - - if (data->saved_kcov) { - kcov_start(t, data->saved_kcov, data->saved_size, - data->saved_area, data->saved_mode, - data->saved_sequence); - data->saved_mode =3D 0; - data->saved_size =3D 0; - data->saved_area =3D NULL; - data->saved_sequence =3D 0; - data->saved_kcov =3D NULL; + if (t->kcov_saved_kcov) { + kcov_start(t, t->kcov_saved_kcov, t->kcov_saved_size, + t->kcov_saved_area, t->kcov_saved_mode, + t->kcov_saved_sequence); + t->kcov_saved_mode =3D 0; + t->kcov_saved_size =3D 0; + t->kcov_saved_area =3D NULL; + t->kcov_saved_sequence =3D 0; + t->kcov_saved_kcov =3D NULL; } } =20 @@ -874,15 +857,12 @@ void kcov_remote_start(u64 handle) if (!in_task() && !in_softirq_really()) return; =20 - local_lock_irqsave(&kcov_percpu_data.lock, flags); - /* * Check that kcov_remote_start() is not called twice in background * threads nor called by user tasks (with enabled kcov). */ mode =3D READ_ONCE(t->kcov_mode); if (WARN_ON(in_task() && kcov_mode_enabled(mode))) { - local_unlock_irqrestore(&kcov_percpu_data.lock, flags); return; } /* @@ -891,15 +871,13 @@ void kcov_remote_start(u64 handle) * happened while collecting coverage from a background thread. */ if (WARN_ON(in_serving_softirq() && t->kcov_softirq)) { - local_unlock_irqrestore(&kcov_percpu_data.lock, flags); return; } =20 - spin_lock(&kcov_remote_lock); + spin_lock_irqsave(&kcov_remote_lock, flags); remote =3D kcov_remote_find(handle); if (!remote) { - spin_unlock(&kcov_remote_lock); - local_unlock_irqrestore(&kcov_percpu_data.lock, flags); + spin_unlock_irqrestore(&kcov_remote_lock, flags); return; } kcov_debug("handle =3D %llx, context: %s\n", handle, @@ -920,19 +898,17 @@ void kcov_remote_start(u64 handle) area =3D kcov_remote_area_get(size); } else { size =3D CONFIG_KCOV_IRQ_AREA_SIZE; - area =3D this_cpu_ptr(&kcov_percpu_data)->irq_area; + area =3D kcov_remote_area_get(size); } - spin_unlock(&kcov_remote_lock); + spin_unlock_irqrestore(&kcov_remote_lock, flags); =20 - /* Can only happen when in_task(). */ + /* Allocate new buffer if we can sleep. */ if (!area) { - local_unlock_irqrestore(&kcov_percpu_data.lock, flags); - area =3D vmalloc(size * sizeof(unsigned long)); + area =3D in_task() ? vmalloc(size * sizeof(unsigned long)) : NULL; if (!area) { kcov_put(kcov); return; } - local_lock_irqsave(&kcov_percpu_data.lock, flags); } =20 /* Reset coverage size. */ @@ -943,9 +919,6 @@ void kcov_remote_start(u64 handle) t->kcov_softirq =3D 1; } kcov_start(t, kcov, size, area, mode, sequence); - - local_unlock_irqrestore(&kcov_percpu_data.lock, flags); - } EXPORT_SYMBOL(kcov_remote_start); =20 @@ -1027,12 +1000,9 @@ void kcov_remote_stop(void) if (!in_task() && !in_softirq_really()) return; =20 - local_lock_irqsave(&kcov_percpu_data.lock, flags); - mode =3D READ_ONCE(t->kcov_mode); barrier(); if (!kcov_mode_enabled(mode)) { - local_unlock_irqrestore(&kcov_percpu_data.lock, flags); return; } /* @@ -1040,12 +1010,10 @@ void kcov_remote_stop(void) * actually found the remote handle and started collecting coverage. */ if (in_serving_softirq() && !t->kcov_softirq) { - local_unlock_irqrestore(&kcov_percpu_data.lock, flags); return; } /* Make sure that kcov_softirq is only set when in softirq. */ if (WARN_ON(!in_serving_softirq() && t->kcov_softirq)) { - local_unlock_irqrestore(&kcov_percpu_data.lock, flags); return; } =20 @@ -1069,13 +1037,9 @@ void kcov_remote_stop(void) kcov_move_area(kcov->mode, kcov->area, kcov->size, area); spin_unlock(&kcov->lock); =20 - if (in_task()) { - spin_lock(&kcov_remote_lock); - kcov_remote_area_put(area, size); - spin_unlock(&kcov_remote_lock); - } - - local_unlock_irqrestore(&kcov_percpu_data.lock, flags); + spin_lock_irqsave(&kcov_remote_lock, flags); + kcov_remote_area_put(area, size); + spin_unlock_irqrestore(&kcov_remote_lock, flags); =20 /* Get in kcov_remote_start(). */ kcov_put(kcov); @@ -1119,14 +1083,19 @@ static void __init selftest(void) =20 static int __init kcov_init(void) { - int cpu; + int cpu =3D num_possible_cpus(); =20 - for_each_possible_cpu(cpu) { - void *area =3D vmalloc_node(CONFIG_KCOV_IRQ_AREA_SIZE * - sizeof(unsigned long), cpu_to_node(cpu)); - if (!area) - return -ENOMEM; - per_cpu_ptr(&kcov_percpu_data, cpu)->irq_area =3D area; +#ifdef CONFIG_PREEMPT_RT + /* Allocate some extra buffers in order to prepare for softirq preemption= . */ + cpu =3D cpu >=3D 4 ? cpu * 2 : cpu + 4; +#endif + while (cpu--) { + void *area =3D vmalloc(CONFIG_KCOV_IRQ_AREA_SIZE * sizeof(unsigned long)= ); + unsigned long flags; + + spin_lock_irqsave(&kcov_remote_lock, flags); + kcov_remote_area_put(area, CONFIG_KCOV_IRQ_AREA_SIZE); + spin_unlock_irqrestore(&kcov_remote_lock, flags); } =20 /* diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 8ff5adcfe1e0..b9e6798dc66d 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2247,10 +2247,11 @@ config KCOV_INSTRUMENT_ALL config KCOV_IRQ_AREA_SIZE hex "Size of interrupt coverage collection area in words" depends on KCOV + range 0x80 0x1000000 default 0x40000 help - KCOV uses preallocated per-cpu areas to collect coverage from - soft interrupts. This specifies the size of those areas in the + KCOV uses preallocated areas to collect coverage from soft + interrupts. This specifies the size of those areas in the number of unsigned long words. =20 config KCOV_SELFTEST --=20 2.54.0