From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3ABE21EB1BB for ; Sun, 2 Mar 2025 19:36:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944209; cv=none; b=W6JYlIOlUXCLr0cc+m0C9KELIUV8DIHmaQc7PZzIgKzO2OLaNJ2WK8c259FKfSJVxBAqthX0wh1Xa9nAwzepMLwWGVPooZ/W4ChqEEPz8OMew/ECGmG7kPIgtJZTWHV0jx/zOyG8iF7D11fdoNyCN5DuHyHq+pSUzExklR0oAkY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944209; c=relaxed/simple; bh=Q9H5c4d2GSWvO0XsE5rchm4JPqe7txtl4JIBmyINDsE=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=GfWutE73UhtmX7+XlmC1/yISoAjFymMjcQOPy+cg9ECQh9DVHV0DXUEnpPxY5FFbR1SL3PbM1QHVHOLTKj2hg9f683HZHtDmiiJ8tf2UuyBkQbXkBiQv4C4ZuZkzG0UVruVBLPvQWAOWY006NIoZnb0kxWAzHSOAs+T9tSOomgw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=RuutOy3C; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=ieKfpvaA; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="RuutOy3C"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="ieKfpvaA" Message-ID: <20250302193626.974094734@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944205; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=aICCvsBZar4WrHQa6TttvV8MzfJyFFmhB1WgVQ9rg9c=; b=RuutOy3CehGQNvEeTHaCnzk6fxKDzO0CTUxGdv2Ia01IcmDdFJeyEqc3uSUteYUoMDsgPF xNIo/wlOH0Kvbk9I/xqjdMqz61Qn53wzNa1CMBpccfkfbHMHhtTSfXgaHMpcwxbPIauqVk jGNr7g78HP929mOomN8HLtaACrxJBv9tfi/IMKF3z6vOVMTBBOUus1Fsrk7g9/0Rr/fWYy Iwz8OU3SRFuAkC9Sc+ELAjWILIpIhX/QPRMUbkfgDB0DvnnTIYKfVUNZNl64oL8y8MOYse NwYI/KHhuHakkdKyL52HsLM5Rvf1DXUebXuSoemmkrrO7NSGaktxF083snCpdw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944205; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=aICCvsBZar4WrHQa6TttvV8MzfJyFFmhB1WgVQ9rg9c=; b=ieKfpvaAuO2NFlJjYaVyecWhthEqeZE2un4hjVLsEXk9Yt4nRq+r1Oyio43krTIl0LWDj4 T4zZiy9oggaqUvDg== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 01/17] posix-timers: Initialise timer before adding it to the hash table References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:44 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Dumazet A timer is only valid in the hashtable when both timer::it_signal and timer::it_id are set to their final values, but timers are added without those values being set. The timer ID is allocated when the timer is added to the hash in invalid state. The ID is taken from a monotonically increasing per process counter which wraps around after reaching INT_MAX. The hash insertion validates that there is no timer with the allocated ID in the hash table which belongs to the same process. That opens a mostly theoretical race condition: If other threads of the same process manage to create/delete timers in rapid succession before the newly created timer is fully initialized and wrap around to the timer ID which was handed out, then a duplicate timer ID will be inserted into the hash table. Prevent this by: 1) Setting timer::it_id before inserting the timer into the hashtable. =20 2) Storing the signal pointer in timer::it_signal with bit 0 set before inserting it into the hashtable. Bit 0 acts as a invalid bit, which means that the regular lookup for sys_timer_*() will fail the comparison with the signal pointer. But the lookup on insertion masks out bit 0 and can therefore detect a timer which is not yet valid, but allocated in the hash table. Bit 0 in the pointer is cleared once the initialization of the timer completed. [ tglx: Fold ID and signal iniitializaion into one patch and massage change log and comments. ] Signed-off-by: Eric Dumazet Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/all/20250219125522.2535263-3-edumazet@google.= com Reviewed-by: Frederic Weisbecker --- kernel/time/posix-timers.c | 56 +++++++++++++++++++++++++++++++++-------= ----- 1 file changed, 42 insertions(+), 14 deletions(-) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -72,13 +72,13 @@ static int hash(struct signal_struct *si return hash_32(hash32_ptr(sig) ^ nr, HASH_BITS(posix_timers_hashtable)); } =20 -static struct k_itimer *__posix_timers_find(struct hlist_head *head, - struct signal_struct *sig, - timer_t id) +static struct k_itimer *posix_timer_by_id(timer_t id) { + struct signal_struct *sig =3D current->signal; + struct hlist_head *head =3D &posix_timers_hashtable[hash(sig, id)]; struct k_itimer *timer; =20 - hlist_for_each_entry_rcu(timer, head, t_hash, lockdep_is_held(&hash_lock)= ) { + hlist_for_each_entry_rcu(timer, head, t_hash) { /* timer->it_signal can be set concurrently */ if ((READ_ONCE(timer->it_signal) =3D=3D sig) && (timer->it_id =3D=3D id)) return timer; @@ -86,12 +86,26 @@ static struct k_itimer *__posix_timers_f return NULL; } =20 -static struct k_itimer *posix_timer_by_id(timer_t id) +static inline struct signal_struct *posix_sig_owner(const struct k_itimer = *timer) { - struct signal_struct *sig =3D current->signal; - struct hlist_head *head =3D &posix_timers_hashtable[hash(sig, id)]; + unsigned long val =3D (unsigned long)timer->it_signal; + + /* + * Mask out bit 0, which acts as invalid marker to prevent + * posix_timer_by_id() detecting it as valid. + */ + return (struct signal_struct *)(val & ~1UL); +} + +static bool posix_timer_hashed(struct hlist_head *head, struct signal_stru= ct *sig, timer_t id) +{ + struct k_itimer *timer; =20 - return __posix_timers_find(head, sig, id); + hlist_for_each_entry_rcu(timer, head, t_hash, lockdep_is_held(&hash_lock)= ) { + if ((posix_sig_owner(timer) =3D=3D sig) && (timer->it_id =3D=3D id)) + return true; + } + return false; } =20 static int posix_timer_add(struct k_itimer *timer) @@ -112,7 +126,19 @@ static int posix_timer_add(struct k_itim sig->next_posix_timer_id =3D (id + 1) & INT_MAX; =20 head =3D &posix_timers_hashtable[hash(sig, id)]; - if (!__posix_timers_find(head, sig, id)) { + if (!posix_timer_hashed(head, sig, id)) { + /* + * Set the timer ID and the signal pointer to make + * it identifiable in the hash table. The signal + * pointer has bit 0 set to indicate that it is not + * yet fully initialized. posix_timer_hashed() + * masks this bit out, but the syscall lookup fails + * to match due to it being set. This guarantees + * that there can't be duplicate timer IDs handed + * out. + */ + timer->it_id =3D (timer_t)id; + timer->it_signal =3D (struct signal_struct *)((unsigned long)sig | 1UL); hlist_add_head_rcu(&timer->t_hash, head); spin_unlock(&hash_lock); return id; @@ -406,8 +432,7 @@ static int do_timer_create(clockid_t whi =20 /* * Add the timer to the hash table. The timer is not yet valid - * because new_timer::it_signal is still NULL. The timer id is also - * not yet visible to user space. + * after insertion, but has a unique ID allocated. */ new_timer_id =3D posix_timer_add(new_timer); if (new_timer_id < 0) { @@ -415,7 +440,6 @@ static int do_timer_create(clockid_t whi return new_timer_id; } =20 - new_timer->it_id =3D (timer_t) new_timer_id; new_timer->it_clock =3D which_clock; new_timer->kclock =3D kc; new_timer->it_overrun =3D -1LL; @@ -453,7 +477,7 @@ static int do_timer_create(clockid_t whi } /* * After succesful copy out, the timer ID is visible to user space - * now but not yet valid because new_timer::signal is still NULL. + * now but not yet valid because new_timer::signal low order bit is 1. * * Complete the initialization with the clock specific create * callback. @@ -463,7 +487,11 @@ static int do_timer_create(clockid_t whi goto out; =20 spin_lock_irq(¤t->sighand->siglock); - /* This makes the timer valid in the hash table */ + /* + * new_timer::it_signal contains the signal pointer with bit 0 set, + * which makes it invalid for syscall operations. Store the + * unmodified signal pointer to make it valid. + */ WRITE_ONCE(new_timer->it_signal, current->signal); hlist_add_head(&new_timer->list, ¤t->signal->posix_timers); spin_unlock_irq(¤t->sighand->siglock); From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B5511EC01B for ; Sun, 2 Mar 2025 19:36:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944209; cv=none; b=XAKo3qU9rTmQuVNV+G+lEqrxvyWVylqTnp9jrHvj1DHaGZXIewhEhKEqiOEVeVlslpYK8NNCM5sFyhMoGdX9EA1jlC1zAMC/XiCHNL0VdJ2qyoIhjptJBOIkJV+aWXy20b52gJ4aKO8D2hjga4SA0lBg/XDrmPaksdfVidh9H1A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944209; c=relaxed/simple; bh=oQlvzcCQFZGNU6jYrXc8Ejg3LVjNXwVuwT4rrg25Vc4=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=ZU4FkW7D/Jjx1oNRAEZ75Q43UC1yiQLg3JMakFXboWaP4fqDNE0SI9UTzPrDGHuF+MmlR04q+ge2Yj2nOWrsWICP7rlSU8LptvGyFjoOILEtA1jlepV9wgGB5czT83cO10wkVfCFYVzYn51o3hlLTuweP7bcMJwPTwWGciGaY4s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=xmgvyz3L; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=6T7NRgbA; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="xmgvyz3L"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="6T7NRgbA" Message-ID: <20250302193627.037768273@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944206; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=01lH/N7MkBCPfiP/7juACA911yRh9opjqKqKmt1gXEs=; b=xmgvyz3Lz2oqfV8XvLpQNPUczcFaLIp6LNooyT+yc49fKde1CuqDYvNyKNy6GkVxrlsnwK TNlAEQYDN8mRRgpcMqoiBMAcwiAqWjwbWeNwEbn3jPZXRl/6Aqpi6GHLpZk5BdnTF0xicd 5TvfHnufM0sBeGrzn7cfiglujka6yhvVwhY46Zyg9a9ZF8osTQ5dJ2U88JNTS2ZwX9oT89 fDQZgDJw2McX5k3rex28TK2g8oElz2fDWb/Q1iUraHTmqa8fVZuiC99ZuAoufl9BH8kicp Fo0+Ay+uvCavmKv0g1wQJgvMS7mM1t9YY37xyVVDO9BUPaKMvd2IJJmC9UIqHA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944206; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=01lH/N7MkBCPfiP/7juACA911yRh9opjqKqKmt1gXEs=; b=6T7NRgbApbr1ytTNyf7p6cTe+v5gPj75on1ynxjzknusjrHW8vR9AYWskNwZGwiYkftch6 oaBCHdiStkUtKoAQ== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 02/17] posix-timers: Add cond_resched() to posix_timer_add() search loop References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:46 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Dumazet With a large number of POSIX timers the search for a valid ID might cause a soft lockup on PREEMPT_NONE/VOLUNTARY kernels. Add cond_resched() to the loop to prevent that. [ tglx: Split out from Eric's series ] Signed-off-by: Eric Dumazet Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/all/20250214135911.2037402-2-edumazet@google.= com Reviewed-by: Frederic Weisbecker --- kernel/time/posix-timers.c | 1 + 1 file changed, 1 insertion(+) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -144,6 +144,7 @@ static int posix_timer_add(struct k_itim return id; } spin_unlock(&hash_lock); + cond_resched(); } /* POSIX return code when no timer ID could be allocated */ return -EAGAIN; From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 145701EDA28 for ; Sun, 2 Mar 2025 19:36:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944211; cv=none; b=eQcEYK8rufWLI5U6uQE3XZo5ZJt5NO0zwbfPXQ8CYqhFGJoySgjCFdqXc1e3W4aJKyo/MS3G6Tb8SZ104gJk0wThJsh6vojBrpiSIHG7MzWvl7e90NGOqocYVMpVh5j4/9EeS2niot/GgsJpVmgRZmAKcVF/js2ByzKr21z3i7Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944211; c=relaxed/simple; bh=YuCCqu2WJyvGIMJolIDYCOzTiBIB7QWQZ3Kkw08rbs0=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=B4lR/gtuENQ7b/+c3IwgfU+fqz66w3rzUkl7c+Or7APj/Nrjej2DJyO7w7lIkEkjq+IA1bkXVSFGj4cmO+/qiBAtvBSp8G7+ZrL1GyHf9oOjeMD1g21IvTAs2VHsbrW8f/WSO2aQbw6D4L84Fc0DGkvij2Hwevurj1IOkGfVoQ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=VQV7/27I; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=PIErsQBF; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="VQV7/27I"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="PIErsQBF" Message-ID: <20250302193627.103677445@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944208; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=06e/nLkYRQzPGWTt39F2KrcgBwQj9ajKa6Dm+qJCWzw=; b=VQV7/27IS1aIgst+en3u8DwatddEHRHHLXPInufdldiXqr6DNha3qgiFIElcJwGCrkLJoF kg20QH+ijJ9bB/T/okeDaQsZL5yWZaU5lYWihg0ELz7DwuSJtdCfiZwfknostrBv/zGWZj y5UVoQPsrV/oB+O3II5i0kdVMrRIF0ljQMUEWpHY8LsQF0p5vucwzazmh8X8k+AqczKrjm tVTP/kKTV8bUog0uNYvmZKEnLf3FpftMwO93tvMJhYF0eqjBqk2RuWuOjjK3dAGSYun3CX 3TuJPRKdbJldI6nslQuBHtuPeD+h62kenlDgkMU9LWmye1NBhrB7rxojTDdOaQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944208; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=06e/nLkYRQzPGWTt39F2KrcgBwQj9ajKa6Dm+qJCWzw=; b=PIErsQBFIYWmbY2G241NAoBPyFg8lmiXyqCgIyB7dhRRLut2lR3UbmCfpf9E0OPZ9RkMED 3MMSh4Of+CwH5gCQ== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 03/17] posix-timers: Cleanup includes References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:47 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove pointless includes and sort the remaining ones alphabetically. Signed-off-by: Thomas Gleixner Acked-by: Frederic Weisbecker --- kernel/time/posix-timers.c | 26 ++++++++++---------------- 1 file changed, 10 insertions(+), 16 deletions(-) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -9,28 +9,22 @@ * * These are all the functions necessary to implement POSIX clocks & timers */ -#include -#include -#include -#include -#include -#include - -#include -#include -#include +#include #include #include +#include +#include +#include +#include +#include #include #include +#include +#include #include -#include -#include -#include -#include -#include -#include +#include #include +#include =20 #include "timekeeping.h" #include "posix-timers.h" From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A60411EE019 for ; Sun, 2 Mar 2025 19:36:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944213; cv=none; b=hTA/9zZc6Lfpx9H45zPDQNDNQOOAj/nL67tpVu21gJar3MiIsyrO+wToIGsDcdYp3INQUF2tBigHhfw5jQEX24N3qc7mmelyCB1rW0uAY4k9luxC0b8O8s2/pQ1FJmepn6Jr6LpUv7+/b4UrQ/KjMzss03R96vmDxdohewG3W7M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944213; c=relaxed/simple; bh=59v/7acbCAwAzZHnQfrmUcEbnZngSTtR+DG+a0iztGA=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=fRCN48bxvKR/lclr6ffQgfhjT64zdSI9Ty/ZUMfdM5caRDPCPlkug7tH0IntgCKLmmm+zHjp2RA1wFEJYboLKONvvw1pIwnzyKENOO195gC0uV/cFCG61V7WApiLKQc+Zzy4X2pIrhZ+V/l3zfzJMaZdKIN1MFWKwm4m4EwCRHM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=HFoW7euW; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=sLoS4666; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="HFoW7euW"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="sLoS4666" Message-ID: <20250302193627.167378648@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944209; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=93tD0GtpcIGIW0gpAOkl8thP4K5eE8qN1mEpJfkAGDw=; b=HFoW7euWwQhoTZf6CFebTcMRgy3Uk/b1SQXeVyh+vr2ECFNO9RWke2aEfGLFi4d74ElvES f9r/T9ikU70vqGdVpwZlmBovLD1+Jr62c4/dyinVvFObmQstVQG4XDia3pK7t5LDCvqnZt DvHLLgR8u9vgvWocohgieEDcOIQfOZ79V0l78JXZ9b47jZ5sQh5Bb2EncF/52JnOqBgHVx sto4GZpE7+DsxqWkS8j7ahrv7rdvsY71HhC3rvngjDqWDkjX6F3SYzdgnrIl2vK0A+oUC+ riNbvKq9cqnMkhzoWgfoRi9n0zj7J3INBoHabm+ZIQ0QMBcvxMFeWMsfnjiTog== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944209; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=93tD0GtpcIGIW0gpAOkl8thP4K5eE8qN1mEpJfkAGDw=; b=sLoS4666GSV4BgRmf2FsmdT/zV0bappJMxwM//EU2Yyw48rqe1qtJBtxaSA3iLPVpe3JM5 M5lElbgeX7T8BvAA== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 04/17] posix-timers: Remove a few paranoid warnings References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:49 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Warnings about a non-initialized timer or non-existing callbacks are just useful for implementing new posix clocks, but there a NULL pointer dereference is expected anyway. :) Signed-off-by: Thomas Gleixner Reviewed-by: Frederic Weisbecker --- V2: New patch --- kernel/time/posix-timers.c | 37 ++++++++----------------------------- 1 file changed, 8 insertions(+), 29 deletions(-) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -675,7 +675,6 @@ void common_timer_get(struct k_itimer *t =20 static int do_timer_gettime(timer_t timer_id, struct itimerspec64 *settin= g) { - const struct k_clock *kc; struct k_itimer *timr; unsigned long flags; int ret =3D 0; @@ -685,11 +684,7 @@ static int do_timer_gettime(timer_t time return -EINVAL; =20 memset(setting, 0, sizeof(*setting)); - kc =3D timr->kclock; - if (WARN_ON_ONCE(!kc || !kc->timer_get)) - ret =3D -EINVAL; - else - kc->timer_get(timr, setting); + timr->kclock->timer_get(timr, setting); =20 unlock_timer(timr, flags); return ret; @@ -817,7 +812,6 @@ static void common_timer_wait_running(st static struct k_itimer *timer_wait_running(struct k_itimer *timer, unsigned long *flags) { - const struct k_clock *kc =3D READ_ONCE(timer->kclock); timer_t timer_id =3D READ_ONCE(timer->it_id); =20 /* Prevent kfree(timer) after dropping the lock */ @@ -828,8 +822,7 @@ static struct k_itimer *timer_wait_runni * kc->timer_wait_running() might drop RCU lock. So @timer * cannot be touched anymore after the function returns! */ - if (!WARN_ON_ONCE(!kc->timer_wait_running)) - kc->timer_wait_running(timer); + timer->kclock->timer_wait_running(timer); =20 rcu_read_unlock(); /* Relock the timer. It might be not longer hashed. */ @@ -892,7 +885,6 @@ static int do_timer_settime(timer_t time struct itimerspec64 *new_spec64, struct itimerspec64 *old_spec64) { - const struct k_clock *kc; struct k_itimer *timr; unsigned long flags; int error; @@ -915,11 +907,7 @@ static int do_timer_settime(timer_t time /* Prevent signal delivery and rearming. */ timr->it_signal_seq++; =20 - kc =3D timr->kclock; - if (WARN_ON_ONCE(!kc || !kc->timer_set)) - error =3D -EINVAL; - else - error =3D kc->timer_set(timr, tmr_flags, new_spec64, old_spec64); + error =3D timr->kclock->timer_set(timr, tmr_flags, new_spec64, old_spec64= ); =20 if (error =3D=3D TIMER_RETRY) { // We already got the old time... @@ -1001,18 +989,6 @@ static inline void posix_timer_cleanup_i } } =20 -static inline int timer_delete_hook(struct k_itimer *timer) -{ - const struct k_clock *kc =3D timer->kclock; - - /* Prevent signal delivery and rearming. */ - timer->it_signal_seq++; - - if (WARN_ON_ONCE(!kc || !kc->timer_del)) - return -EINVAL; - return kc->timer_del(timer); -} - /* Delete a POSIX.1b interval timer. */ SYSCALL_DEFINE1(timer_delete, timer_t, timer_id) { @@ -1025,7 +1001,10 @@ SYSCALL_DEFINE1(timer_delete, timer_t, t if (!timer) return -EINVAL; =20 - if (unlikely(timer_delete_hook(timer) =3D=3D TIMER_RETRY)) { + /* Prevent signal delivery and rearming. */ + timer->it_signal_seq++; + + if (unlikely(timer->kclock->timer_del(timer) =3D=3D TIMER_RETRY)) { /* Unlocks and relocks the timer if it still exists */ timer =3D timer_wait_running(timer, &flags); goto retry_delete; @@ -1071,7 +1050,7 @@ static void itimer_delete(struct k_itime * mechanism. Worse, that timer mechanism might run the expiry * function concurrently. */ - if (timer_delete_hook(timer) =3D=3D TIMER_RETRY) { + if (timer->kclock->timer_del(timer) =3D=3D TIMER_RETRY) { /* * Timer is expired concurrently, prevent livelocks * and pointless spinning on RT. From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C0761EE7C2 for ; Sun, 2 Mar 2025 19:36:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944214; cv=none; b=jZNaqWpdm9LS4wzFVTr67XocC64sO9zezQxRLlm83qIm7TTvP1Lsxkna+E9g10mk4IWZ5IFCwZ+b8z5bDjLWoHOKsYW9APwG3SK4Oy45U1t1X/CdDimLMIJgE6AQLLSfdfUuEnbYGfyeWI1d1Keer7RWffAkTI2VCFqFt5PNGyo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944214; c=relaxed/simple; bh=4W1lHUILucLe9BbR4U1iigk+d1qy1p2V8GPaddiITzI=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=nd+t+cvz6LXyimZege1WlKr1WbpdC/Ltl5zsTouvTqq6fVszrdb0MgJncGS4LeTFYuH3pq8QdEXHeeTroFbinaExdDZRMwHgStYb3ZWFXNeMghwPMHDZSLi0nuqysuWSh0G9HUcZqJEnGTaL34G/e4hsD9dvCbcl21oewtAAMNE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=N0XMIrhm; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=cNs6ObbK; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="N0XMIrhm"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="cNs6ObbK" Message-ID: <20250302193627.229366917@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QrbOx6uAQ5Ms/hSi33seKY8Lf7RBO9KA+TOAPdKMcCM=; b=N0XMIrhmSjms2dQDGK+S94UC7MCt8HJ6zMEo2UYyvkQSSoFGf3ePpba3XPDh4Xpey2oA8X bylo9GYw+194J7M9Ac3W41TOqP4Txc9bbiPswOrHk/JA/+/saGWq7Vuxh7qobbWeQj2ueW QkixES3wNoLUZYI2A9GipvIOlyUoFMCQD9voUerP6YHAOgZgpT39mux8BCJ1GxgxS5tifI H0Bk8Ft62sWN/hvwMpqZnk4EvqsbrSiw5VgJIj8xJ6+joNwVrsZ3AVkBqewqUUGKqZpufM KMXTmjYby7gew50S5HbpoC2Yi/ELJ8FHVFz3PZcwKCnasLLrYYsBQ6dAoUpFVw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QrbOx6uAQ5Ms/hSi33seKY8Lf7RBO9KA+TOAPdKMcCM=; b=cNs6ObbKUeqImfUJrtajq+i0oCbvjVKsGHM2bcq2nZy2jw1QsINpZbl+VtdDCM/U9lkkP/ CGPdG27E98h1zqDg== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 05/17] posix-timers: Remove SLAB_PANIC from kmem cache References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:51 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" There is no need to panic when the posix-timer kmem_cache can't be created. timer_create() will fail with -ENOMEM and that's it. Signed-off-by: Thomas Gleixner Reviewed-by: Frederic Weisbecker --- V2: New patch --- kernel/time/posix-timers.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -243,9 +243,8 @@ static int posix_get_hrtimer_res(clockid =20 static __init int init_posix_timers(void) { - posix_timers_cache =3D kmem_cache_create("posix_timers_cache", - sizeof(struct k_itimer), 0, - SLAB_PANIC | SLAB_ACCOUNT, NULL); + posix_timers_cache =3D kmem_cache_create("posix_timers_cache", sizeof(str= uct k_itimer), 0, + SLAB_ACCOUNT, NULL); return 0; } __initcall(init_posix_timers); @@ -371,8 +370,12 @@ static struct pid *good_sigevent(sigeven =20 static struct k_itimer *alloc_posix_timer(void) { - struct k_itimer *tmr =3D kmem_cache_zalloc(posix_timers_cache, GFP_KERNEL= ); + struct k_itimer *tmr; =20 + if (unlikely(!posix_timers_cache)) + return NULL; + + tmr =3D kmem_cache_zalloc(posix_timers_cache, GFP_KERNEL); if (!tmr) return tmr; From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99F0E1EEA28 for ; Sun, 2 Mar 2025 19:36:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944216; cv=none; b=LiYshKSXBQMz6XpCrAx18VNO1nTQ/9GmlwZiT0GW1D4jJfSCd42+YmOFy3XreVu2HHBoVl5pH35kvmQKltl5QASOss47YF7LtFnB2CItRjEwvpyr3h/lAxPreR8NGYSTMlHeOiIQME8r5zmm8rsylR1N1McllynduHij+cN2rcU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944216; c=relaxed/simple; bh=AkSUA35ExS6aBl8//QJiv+TiB475ODpJiF2ceG+f/74=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=VBHd5WUBStRkdtZEzBmz8eDoStmfpc/ZTlAMxNu77QAPAx0ywMkMe/74djMCCALKVPhE3j07w3C07y+yq1OrEShF0xHOLxxYiLPgJf/aN90soNPVXdUWOgoFPGU/ZYxK8CNkZjOOKx8I5189ndFWLMsZMGla0HBXwPeaN3B2Go8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Hmpnw/IG; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=2UDnbDy4; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Hmpnw/IG"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="2UDnbDy4" Message-ID: <20250302193627.291265233@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QCGoT9giFJLS+pv+a5LZeLjPMBj3VUG3X4dhDTU9960=; b=Hmpnw/IGghblgBcDAMxRa7nP2Dg5j7AlcDXn3gapPqbPKwmvemKkSsn2mZOSu/blkHXO4L QutgtgfWRdXQf84rATpG5iy46FgSUZYGYCMtdmWz88GoMgNkaJ8tf+vYV3cFb50m2+WTmr 8sIZFyFfjewNZvWP85fT1LdkBShpWi8qat6a/l09zcJ01DRKV9ySahxLhVNYwydgO5Y/GX dXeCmb22VOJ9hPSQ5OACvCaSXPic5+SCCMM3p5j0k7SDzqIzWsDsvavC25jehBisoaWeit pL8hdBv9K+HawrAVzGhk7oRVM/9QcaRfUB5FnWhG9T4SAzVT9M6W1fSfMquPxg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QCGoT9giFJLS+pv+a5LZeLjPMBj3VUG3X4dhDTU9960=; b=2UDnbDy4+U+fZX5H8s/mV4yIFBYaJpw0ltwviVkISk4WTV7XFz6bxMtux5cPMfK/7kIN3w 8qB4LrJNe+D8z0CQ== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 06/17] posix-timers: Use guards in a few places References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:52 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Switch locking and RCU to guards where applicable. Signed-off-by: Thomas Gleixner Reviewed-by: Frederic Weisbecker --- V2: New patch --- kernel/time/posix-timers.c | 86 ++++++++++++++++++++--------------------= ----- 1 file changed, 39 insertions(+), 47 deletions(-) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -397,9 +397,8 @@ void posixtimer_free_timer(struct k_itim =20 static void posix_timer_unhash_and_free(struct k_itimer *tmr) { - spin_lock(&hash_lock); - hlist_del_rcu(&tmr->t_hash); - spin_unlock(&hash_lock); + scoped_guard (spinlock, &hash_lock) + hlist_del_rcu(&tmr->t_hash); posixtimer_putref(tmr); } =20 @@ -443,9 +442,8 @@ static int do_timer_create(clockid_t whi new_timer->it_overrun =3D -1LL; =20 if (event) { - rcu_read_lock(); - new_timer->it_pid =3D get_pid(good_sigevent(event)); - rcu_read_unlock(); + scoped_guard (rcu) + new_timer->it_pid =3D get_pid(good_sigevent(event)); if (!new_timer->it_pid) { error =3D -EINVAL; goto out; @@ -484,15 +482,15 @@ static int do_timer_create(clockid_t whi if (error) goto out; =20 - spin_lock_irq(¤t->sighand->siglock); - /* - * new_timer::it_signal contains the signal pointer with bit 0 set, - * which makes it invalid for syscall operations. Store the - * unmodified signal pointer to make it valid. - */ - WRITE_ONCE(new_timer->it_signal, current->signal); - hlist_add_head(&new_timer->list, ¤t->signal->posix_timers); - spin_unlock_irq(¤t->sighand->siglock); + scoped_guard (spinlock_irq, ¤t->sighand->siglock) { + /* + * new_timer::it_signal contains the signal pointer with bit 0 set, + * which makes it invalid for syscall operations. Store the + * unmodified signal pointer to make it valid. + */ + WRITE_ONCE(new_timer->it_signal, current->signal); + hlist_add_head(&new_timer->list, ¤t->signal->posix_timers); + } /* * After unlocking sighand::siglock @new_timer is subject to * concurrent removal and cannot be touched anymore @@ -572,7 +570,7 @@ static struct k_itimer *__lock_timer(tim * can't change, but timr::it_signal becomes NULL during * destruction. */ - rcu_read_lock(); + guard(rcu)(); timr =3D posix_timer_by_id(timer_id); if (timr) { spin_lock_irqsave(&timr->it_lock, *flags); @@ -580,14 +578,10 @@ static struct k_itimer *__lock_timer(tim * Validate under timr::it_lock that timr::it_signal is * still valid. Pairs with #1 above. */ - if (timr->it_signal =3D=3D current->signal) { - rcu_read_unlock(); + if (timr->it_signal =3D=3D current->signal) return timr; - } spin_unlock_irqrestore(&timr->it_lock, *flags); } - rcu_read_unlock(); - return NULL; } =20 @@ -818,16 +812,15 @@ static struct k_itimer *timer_wait_runni timer_t timer_id =3D READ_ONCE(timer->it_id); =20 /* Prevent kfree(timer) after dropping the lock */ - rcu_read_lock(); - unlock_timer(timer, *flags); - - /* - * kc->timer_wait_running() might drop RCU lock. So @timer - * cannot be touched anymore after the function returns! - */ - timer->kclock->timer_wait_running(timer); + scoped_guard (rcu) { + unlock_timer(timer, *flags); + /* + * kc->timer_wait_running() might drop RCU lock. So @timer + * cannot be touched anymore after the function returns! + */ + timer->kclock->timer_wait_running(timer); + } =20 - rcu_read_unlock(); /* Relock the timer. It might be not longer hashed. */ return lock_timer(timer_id, flags); } @@ -1013,20 +1006,20 @@ SYSCALL_DEFINE1(timer_delete, timer_t, t goto retry_delete; } =20 - spin_lock(¤t->sighand->siglock); - hlist_del(&timer->list); - posix_timer_cleanup_ignored(timer); - /* - * A concurrent lookup could check timer::it_signal lockless. It - * will reevaluate with timer::it_lock held and observe the NULL. - * - * It must be written with siglock held so that the signal code - * observes timer->it_signal =3D=3D NULL in do_sigaction(SIG_IGN), - * which prevents it from moving a pending signal of a deleted - * timer to the ignore list. - */ - WRITE_ONCE(timer->it_signal, NULL); - spin_unlock(¤t->sighand->siglock); + scoped_guard (spinlock, ¤t->sighand->siglock) { + hlist_del(&timer->list); + posix_timer_cleanup_ignored(timer); + /* + * A concurrent lookup could check timer::it_signal lockless. It + * will reevaluate with timer::it_lock held and observe the NULL. + * + * It must be written with siglock held so that the signal code + * observes timer->it_signal =3D=3D NULL in do_sigaction(SIG_IGN), + * which prevents it from moving a pending signal of a deleted + * timer to the ignore list. + */ + WRITE_ONCE(timer->it_signal, NULL); + } =20 unlock_timer(timer, flags); posix_timer_unhash_and_free(timer); @@ -1099,9 +1092,8 @@ void exit_itimers(struct task_struct *ts return; =20 /* Protect against concurrent read via /proc/$PID/timers */ - spin_lock_irq(&tsk->sighand->siglock); - hlist_move_list(&tsk->signal->posix_timers, &timers); - spin_unlock_irq(&tsk->sighand->siglock); + scoped_guard (spinlock_irq, &tsk->sighand->siglock) + hlist_move_list(&tsk->signal->posix_timers, &timers); =20 /* The timers are not longer accessible via tsk::signal */ while (!hlist_empty(&timers)) { From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFC171EF092 for ; Sun, 2 Mar 2025 19:36:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944219; cv=none; b=t6xadA+MV46m9YbhExLEGCJKTL5omY6Eh7hNLs1MY2iI7KqUcXVErQQirpx0kIV18s65TUqqy35S7ybCINVjwu2A6qggJzDK9WuxFPnXamDpjTjW8C9RpraGfgAIBeAL/GQg/3hfYyJGB19f/ulDltZCiprD1PXk6zsRjpDjsyc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944219; c=relaxed/simple; bh=wuLQYqjk+6m09X5A+jf73uzZuGnLJwPNPzfnYqP66rA=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=DnAK4iniTZrK5SCDa3KkLr75wWLQ2n9XirtiA5wORCL2gOoAOsA7IOgjORm/zC44SJZQMmZFQoAAnFZZtXGw6VQMzvx3tC6Oh/Lc/LrNLE/4/W+BkmF234wV52wFeDvqgzKVK8NDwLujQ1LsplJD638GnWq+SsgBPrBoZIla8BI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=FLuzbFiK; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=0nngAjIX; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="FLuzbFiK"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="0nngAjIX" Message-ID: <20250302193627.354754536@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944215; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=FBa8Z5ux6kpNt1T/1dV+QnJKzr121MSpKu0kjmBjvdk=; b=FLuzbFiKVAIX4xHfqJgckAtr4/nLzmC/hIcVQ06whkiVWkq+SYnNmNTpDpXn3CaGZ11iX6 5UlNs2VJIp23YgaNuDX9y7gbjGamGeil4FzlIg/Yusm6pMEU4k9TE7bo0jTTejIq8OTmbs kDnmRqhNHtJpYiRa6lzUraGIh9fQ3qogouG5eX7lsI2DEg68/1llSmpAOHIkDiyNpvKLI+ cpp3YbVUyea8gwQgOUAHV9wrATI949sXY7BrGjV6a9iHotmkpGwQyo9/fgbBt5oWVgWhBV b4RyDv0+RRX6jdX9OsSrwAMHNut0E5d0cJhG1tN3H6fWktXbHSFaQ/Z2LlmAcg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944215; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=FBa8Z5ux6kpNt1T/1dV+QnJKzr121MSpKu0kjmBjvdk=; b=0nngAjIXaY6Jf6Ms63XDCaXGRbZWUPTZkED1PQ8gaNpHzZxMHpM0neoouBNgraSP8He06u rtaNXHIfhgZ/xICw== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 07/17] posix-timers: Simplify lock/unlock_timer() References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:54 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since the integration of sigqueue into the timer struct, lock_timer() is only used in task context. So taking the lock with irqsave() is not longer required. Convert it to use spin_[un]lock_irq(). Signed-off-by: Thomas Gleixner Reviewed-by: Frederic Weisbecker --- V2: New patch --- kernel/time/posix-timers.c | 70 ++++++++++++++++++----------------------= ----- 1 file changed, 29 insertions(+), 41 deletions(-) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -53,14 +53,19 @@ static const struct k_clock clock_realti #error "SIGEV_THREAD_ID must not share bit with other SIGEV values!" #endif =20 -static struct k_itimer *__lock_timer(timer_t timer_id, unsigned long *flag= s); +static struct k_itimer *__lock_timer(timer_t timer_id); =20 -#define lock_timer(tid, flags) \ -({ struct k_itimer *__timr; \ - __cond_lock(&__timr->it_lock, __timr =3D __lock_timer(tid, flags)); \ - __timr; \ +#define lock_timer(tid) \ +({ struct k_itimer *__timr; \ + __cond_lock(&__timr->it_lock, __timr =3D __lock_timer(tid)); \ + __timr; \ }) =20 +static inline void unlock_timer(struct k_itimer *timr) +{ + spin_unlock_irq(&timr->it_lock); +} + static int hash(struct signal_struct *sig, unsigned int nr) { return hash_32(hash32_ptr(sig) ^ nr, HASH_BITS(posix_timers_hashtable)); @@ -144,11 +149,6 @@ static int posix_timer_add(struct k_itim return -EAGAIN; } =20 -static inline void unlock_timer(struct k_itimer *timr, unsigned long flags) -{ - spin_unlock_irqrestore(&timr->it_lock, flags); -} - static int posix_get_realtime_timespec(clockid_t which_clock, struct times= pec64 *tp) { ktime_get_real_ts64(tp); @@ -531,7 +531,7 @@ COMPAT_SYSCALL_DEFINE3(timer_create, clo } #endif =20 -static struct k_itimer *__lock_timer(timer_t timer_id, unsigned long *flag= s) +static struct k_itimer *__lock_timer(timer_t timer_id) { struct k_itimer *timr; =20 @@ -573,14 +573,14 @@ static struct k_itimer *__lock_timer(tim guard(rcu)(); timr =3D posix_timer_by_id(timer_id); if (timr) { - spin_lock_irqsave(&timr->it_lock, *flags); + spin_lock_irq(&timr->it_lock); /* * Validate under timr::it_lock that timr::it_signal is * still valid. Pairs with #1 above. */ if (timr->it_signal =3D=3D current->signal) return timr; - spin_unlock_irqrestore(&timr->it_lock, *flags); + spin_unlock_irq(&timr->it_lock); } return NULL; } @@ -673,17 +673,16 @@ void common_timer_get(struct k_itimer *t static int do_timer_gettime(timer_t timer_id, struct itimerspec64 *settin= g) { struct k_itimer *timr; - unsigned long flags; int ret =3D 0; =20 - timr =3D lock_timer(timer_id, &flags); + timr =3D lock_timer(timer_id); if (!timr) return -EINVAL; =20 memset(setting, 0, sizeof(*setting)); timr->kclock->timer_get(timr, setting); =20 - unlock_timer(timr, flags); + unlock_timer(timr); return ret; } =20 @@ -739,15 +738,14 @@ SYSCALL_DEFINE2(timer_gettime32, timer_t SYSCALL_DEFINE1(timer_getoverrun, timer_t, timer_id) { struct k_itimer *timr; - unsigned long flags; int overrun; =20 - timr =3D lock_timer(timer_id, &flags); + timr =3D lock_timer(timer_id); if (!timr) return -EINVAL; =20 overrun =3D timer_overrun_to_int(timr); - unlock_timer(timr, flags); + unlock_timer(timr); =20 return overrun; } @@ -806,14 +804,13 @@ static void common_timer_wait_running(st * when the task which tries to delete or disarm the timer has preempted * the task which runs the expiry in task work context. */ -static struct k_itimer *timer_wait_running(struct k_itimer *timer, - unsigned long *flags) +static struct k_itimer *timer_wait_running(struct k_itimer *timer) { timer_t timer_id =3D READ_ONCE(timer->it_id); =20 /* Prevent kfree(timer) after dropping the lock */ scoped_guard (rcu) { - unlock_timer(timer, *flags); + unlock_timer(timer); /* * kc->timer_wait_running() might drop RCU lock. So @timer * cannot be touched anymore after the function returns! @@ -822,7 +819,7 @@ static struct k_itimer *timer_wait_runni } =20 /* Relock the timer. It might be not longer hashed. */ - return lock_timer(timer_id, flags); + return lock_timer(timer_id); } =20 /* @@ -882,7 +879,6 @@ static int do_timer_settime(timer_t time struct itimerspec64 *old_spec64) { struct k_itimer *timr; - unsigned long flags; int error; =20 if (!timespec64_valid(&new_spec64->it_interval) || @@ -892,7 +888,7 @@ static int do_timer_settime(timer_t time if (old_spec64) memset(old_spec64, 0, sizeof(*old_spec64)); =20 - timr =3D lock_timer(timer_id, &flags); + timr =3D lock_timer(timer_id); retry: if (!timr) return -EINVAL; @@ -909,10 +905,10 @@ static int do_timer_settime(timer_t time // We already got the old time... old_spec64 =3D NULL; /* Unlocks and relocks the timer if it still exists */ - timr =3D timer_wait_running(timr, &flags); + timr =3D timer_wait_running(timr); goto retry; } - unlock_timer(timr, flags); + unlock_timer(timr); =20 return error; } @@ -988,10 +984,7 @@ static inline void posix_timer_cleanup_i /* Delete a POSIX.1b interval timer. */ SYSCALL_DEFINE1(timer_delete, timer_t, timer_id) { - struct k_itimer *timer; - unsigned long flags; - - timer =3D lock_timer(timer_id, &flags); + struct k_itimer *timer =3D lock_timer(timer_id); =20 retry_delete: if (!timer) @@ -1002,7 +995,7 @@ SYSCALL_DEFINE1(timer_delete, timer_t, t =20 if (unlikely(timer->kclock->timer_del(timer) =3D=3D TIMER_RETRY)) { /* Unlocks and relocks the timer if it still exists */ - timer =3D timer_wait_running(timer, &flags); + timer =3D timer_wait_running(timer); goto retry_delete; } =20 @@ -1021,7 +1014,7 @@ SYSCALL_DEFINE1(timer_delete, timer_t, t WRITE_ONCE(timer->it_signal, NULL); } =20 - unlock_timer(timer, flags); + unlock_timer(timer); posix_timer_unhash_and_free(timer); return 0; } @@ -1032,12 +1025,7 @@ SYSCALL_DEFINE1(timer_delete, timer_t, t */ static void itimer_delete(struct k_itimer *timer) { - unsigned long flags; - - /* - * irqsave is required to make timer_wait_running() work. - */ - spin_lock_irqsave(&timer->it_lock, flags); + spin_lock_irq(&timer->it_lock); =20 retry_delete: /* @@ -1058,7 +1046,7 @@ static void itimer_delete(struct k_itime * do_exit() only for the last thread of the thread group. * So no other task can access and delete that timer. */ - if (WARN_ON_ONCE(timer_wait_running(timer, &flags) !=3D timer)) + if (WARN_ON_ONCE(timer_wait_running(timer) !=3D timer)) return; =20 goto retry_delete; @@ -1075,7 +1063,7 @@ static void itimer_delete(struct k_itime */ WRITE_ONCE(timer->it_signal, NULL); =20 - spin_unlock_irqrestore(&timer->it_lock, flags); + spin_unlock_irq(&timer->it_lock); posix_timer_unhash_and_free(timer); } From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0D2F1EF09B for ; Sun, 2 Mar 2025 19:36:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944220; cv=none; b=Gi9PH41C5nWoPEHti+YGAmj6dX5iOJjSIyO+dranYB+0Qn20/g+Ik8vfvfS2GSiFYEuGPnCwnrX1Xn5yQJcvuC340fmFEWjbOh7J8x6f2k3hRC1v2vITfwAKb4lBTgcCTKkw/CYt1MUYw3qOjpdLhe9YTQUGGjQFZA8xaDu8uWE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944220; c=relaxed/simple; bh=vR/TzYQakpYs1sHXKlUvtCoII5mBGoIDBKvXRHmj4mc=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=PkIKL7WFl9QYYwIKBKKAmmqXD5aVsswCmn+OMqMm8lTMJ9g5N16ZhGGcu/ieIjm2wc4QqMFARM5uYYW7D7rJPpMFjc1Pwa5JLis5kKR+DOooa2VC9nxVFMbLxgzLrpZXd3sMcxv5MjZl70/zqxvdONsk0HzzdlTqGA/qlP43iDU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=fl2XLSYk; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=cacJ0U03; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="fl2XLSYk"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="cacJ0U03" Message-ID: <20250302193627.416552300@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944216; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=gPOW6XpJD24jD66AeI2zsMmm3AcWYl09YPLa3iWKVj8=; b=fl2XLSYkJRrJMckb94PNHO2vWwCfMagJ4WkvRsw0yiUsgl21LPCgCD1pDEGoDzfMDLxjbr f5I17aDXWGBqVewAAqD5TbjxFMDYb/KMvGyBZtK1v8Hvc9D8ROwQV618A2guu7mFu38z8V wgX+0QhOHytS1ciquZ3IYEgPusnAprf9HnDIsqh3enyRGrYSD8cMOZFLSZ8X6qFyby0B8e qMYGT8GyPxQ5iBKZgLo+LDMfXISp5RNF1KD+oXu6gRquP4FSc43Ss3Lxa6vhaUsPnm6wKX ph4jdAL8chhoarSmVph78ADEU9CddL0jQ412D21+r//lKPyM6Ned64XGpb+jIw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944216; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=gPOW6XpJD24jD66AeI2zsMmm3AcWYl09YPLa3iWKVj8=; b=cacJ0U03Mf6Mzalb1oDc3xmLDaJQ9VzcjvJTxHE04TS7Qx6m8b8JMtNI/6FYviMQ8qN/v3 cceNyoo8cpvmpPCg== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 08/17] posix-timers: Rework timer removal References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:56 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" sys_timer_delete() and the do_exit() cleanup function itimer_delete() are doing the same thing, but have needlessly different implementations instead of sharing the code. The other oddity of timer deletion is the fact that the timer is not invalidated before the actual deletion happens, which allows concurrent lookups to succeed. That's wrong because a timer which is in the process of being deleted should not be visible and any actions like signal queueing, delivery and rearming should not happen once the task, which invoked timer_delete(), has the timer locked. Rework the code so that: 1) The signal queueing and delivery code ignore timers which are marked invalid 2) The deletion implementation between sys_timer_delete() and itimer_delete() is shared 3) The timer is invalidated and removed from the linked lists before the deletion callback of the relevant clock is invoked. That requires to rework timer_wait_running() as it does a lookup of the timer when relocking it at the end. In case of deletion this lookup would fail due to the preceding invalidation and the wait loop would terminate prematurely. But due to the preceding invalidation the timer cannot be accessed by other tasks anymore, so there is no way that the timer has been freed after the timer lock has been dropped. Move the re-validation out of timer_wait_running() and handle it at the only other usage site, timer_settime(). Signed-off-by: Thomas Gleixner --- V2: Simplify timer_wait_running() locking - PeterZ --- include/linux/posix-timers.h | 7 + kernel/signal.c | 2=20 kernel/time/posix-timers.c | 194 ++++++++++++++++++--------------------= ----- 3 files changed, 90 insertions(+), 113 deletions(-) --- a/include/linux/posix-timers.h +++ b/include/linux/posix-timers.h @@ -240,6 +240,13 @@ static inline void posixtimer_sigqueue_p =20 posixtimer_putref(tmr); } + +static inline bool posixtimer_valid(const struct k_itimer *timer) +{ + unsigned long val =3D (unsigned long)timer->it_signal; + + return !(val & 0x1UL); +} #else /* CONFIG_POSIX_TIMERS */ static inline void posixtimer_sigqueue_getref(struct sigqueue *q) { } static inline void posixtimer_sigqueue_putref(struct sigqueue *q) { } --- a/kernel/signal.c +++ b/kernel/signal.c @@ -2092,7 +2092,7 @@ static inline void posixtimer_sig_ignore * from a non-periodic timer, then just drop the reference * count. Otherwise queue it on the ignored list. */ - if (tmr->it_signal && tmr->it_sig_periodic) + if (posixtimer_valid(tmr) && tmr->it_sig_periodic) hlist_add_head(&tmr->ignored_list, &tsk->signal->ignored_posix_timers); else posixtimer_putref(tmr); --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -279,7 +279,7 @@ static bool __posixtimer_deliver_signal( * since the signal was queued. In either case, don't rearm and * drop the signal. */ - if (timr->it_signal_seq !=3D timr->it_sigqueue_seq || WARN_ON_ONCE(!timr-= >it_signal)) + if (timr->it_signal_seq !=3D timr->it_sigqueue_seq || !posixtimer_valid(t= imr)) return false; =20 if (!timr->it_interval || WARN_ON_ONCE(timr->it_status !=3D POSIX_TIMER_R= EQUEUE_PENDING)) @@ -324,6 +324,9 @@ void posix_timer_queue_signal(struct k_i { lockdep_assert_held(&timr->it_lock); =20 + if (!posixtimer_valid(timr)) + return; + timr->it_status =3D timr->it_interval ? POSIX_TIMER_REQUEUE_PENDING : POS= IX_TIMER_DISARMED; posixtimer_send_sigqueue(timr); } @@ -546,11 +549,11 @@ static struct k_itimer *__lock_timer(tim * The hash lookup and the timers are RCU protected. * * Timers are added to the hash in invalid state where - * timr::it_signal =3D=3D NULL. timer::it_signal is only set after the - * rest of the initialization succeeded. + * timr::it_signal is marked invalid. timer::it_signal is only set + * after the rest of the initialization succeeded. * * Timer destruction happens in steps: - * 1) Set timr::it_signal to NULL with timr::it_lock held + * 1) Set timr::it_signal marked invalid with timr::it_lock held * 2) Release timr::it_lock * 3) Remove from the hash under hash_lock * 4) Put the reference count. @@ -567,8 +570,8 @@ static struct k_itimer *__lock_timer(tim * * The lookup validates locklessly that timr::it_signal =3D=3D * current::it_signal and timr::it_id =3D=3D @timer_id. timr::it_id - * can't change, but timr::it_signal becomes NULL during - * destruction. + * can't change, but timr::it_signal can become invalid during + * destruction, which makes the locked check fail. */ guard(rcu)(); timr =3D posix_timer_by_id(timer_id); @@ -804,22 +807,13 @@ static void common_timer_wait_running(st * when the task which tries to delete or disarm the timer has preempted * the task which runs the expiry in task work context. */ -static struct k_itimer *timer_wait_running(struct k_itimer *timer) +static void timer_wait_running(struct k_itimer *timer) { - timer_t timer_id =3D READ_ONCE(timer->it_id); - - /* Prevent kfree(timer) after dropping the lock */ - scoped_guard (rcu) { - unlock_timer(timer); - /* - * kc->timer_wait_running() might drop RCU lock. So @timer - * cannot be touched anymore after the function returns! - */ - timer->kclock->timer_wait_running(timer); - } - - /* Relock the timer. It might be not longer hashed. */ - return lock_timer(timer_id); + /* + * kc->timer_wait_running() might drop RCU lock. So @timer + * cannot be touched anymore after the function returns! + */ + timer->kclock->timer_wait_running(timer); } =20 /* @@ -878,8 +872,7 @@ static int do_timer_settime(timer_t time struct itimerspec64 *new_spec64, struct itimerspec64 *old_spec64) { - struct k_itimer *timr; - int error; + int ret; =20 if (!timespec64_valid(&new_spec64->it_interval) || !timespec64_valid(&new_spec64->it_value)) @@ -888,29 +881,36 @@ static int do_timer_settime(timer_t time if (old_spec64) memset(old_spec64, 0, sizeof(*old_spec64)); =20 - timr =3D lock_timer(timer_id); -retry: - if (!timr) - return -EINVAL; + for (;;) { + struct k_itimer *timr =3D lock_timer(timer_id); =20 - if (old_spec64) - old_spec64->it_interval =3D ktime_to_timespec64(timr->it_interval); + if (!timr) + return -EINVAL; =20 - /* Prevent signal delivery and rearming. */ - timr->it_signal_seq++; + if (old_spec64) + old_spec64->it_interval =3D ktime_to_timespec64(timr->it_interval); =20 - error =3D timr->kclock->timer_set(timr, tmr_flags, new_spec64, old_spec64= ); + /* Prevent signal delivery and rearming. */ + timr->it_signal_seq++; + + ret =3D timr->kclock->timer_set(timr, tmr_flags, new_spec64, old_spec64); + if (ret !=3D TIMER_RETRY) { + unlock_timer(timr); + break; + } =20 - if (error =3D=3D TIMER_RETRY) { - // We already got the old time... + /* Read the old time only once */ old_spec64 =3D NULL; - /* Unlocks and relocks the timer if it still exists */ - timr =3D timer_wait_running(timr); - goto retry; + /* Protect the timer from being freed after the lock is dropped */ + guard(rcu)(); + unlock_timer(timr); + /* + * timer_wait_running() might drop RCU read side protection + * so the timer has to be looked up again! + */ + timer_wait_running(timr); } - unlock_timer(timr); - - return error; + return ret; } =20 /* Set a POSIX.1b interval timer */ @@ -981,90 +981,56 @@ static inline void posix_timer_cleanup_i } } =20 -/* Delete a POSIX.1b interval timer. */ -SYSCALL_DEFINE1(timer_delete, timer_t, timer_id) +static void posix_timer_delete(struct k_itimer *timer) { - struct k_itimer *timer =3D lock_timer(timer_id); - -retry_delete: - if (!timer) - return -EINVAL; - - /* Prevent signal delivery and rearming. */ + /* + * Invalidate the timer, remove it from the linked list and remove + * it from the ignored list if pending. + * + * The invalidation must be written with siglock held so that the + * signal code observes timer->it_valid =3D=3D false in do_sigaction(), + * which prevents it from moving a pending signal of a deleted + * timer to the ignore list. + * + * The invalidation also prevents signal queueing, signal delivery + * and therefore rearming from the signal delivery path. + * + * A concurrent lookup can still find the timer in the hash, but it + * will check timer::it_signal with timer::it_lock held and observe + * bit 0 set, which invalidates it. That also prevents the timer ID + * from being handed out before this timer is completely gone. + */ timer->it_signal_seq++; =20 - if (unlikely(timer->kclock->timer_del(timer) =3D=3D TIMER_RETRY)) { - /* Unlocks and relocks the timer if it still exists */ - timer =3D timer_wait_running(timer); - goto retry_delete; - } - scoped_guard (spinlock, ¤t->sighand->siglock) { + unsigned long sig =3D (unsigned long)timer->it_signal | 1UL; + + WRITE_ONCE(timer->it_signal, (struct signal_struct *)sig); hlist_del(&timer->list); posix_timer_cleanup_ignored(timer); - /* - * A concurrent lookup could check timer::it_signal lockless. It - * will reevaluate with timer::it_lock held and observe the NULL. - * - * It must be written with siglock held so that the signal code - * observes timer->it_signal =3D=3D NULL in do_sigaction(SIG_IGN), - * which prevents it from moving a pending signal of a deleted - * timer to the ignore list. - */ - WRITE_ONCE(timer->it_signal, NULL); } =20 - unlock_timer(timer); - posix_timer_unhash_and_free(timer); - return 0; + while (timer->kclock->timer_del(timer) =3D=3D TIMER_RETRY) { + guard(rcu)(); + spin_unlock_irq(&timer->it_lock); + timer_wait_running(timer); + spin_lock_irq(&timer->it_lock); + } } =20 -/* - * Delete a timer if it is armed, remove it from the hash and schedule it - * for RCU freeing. - */ -static void itimer_delete(struct k_itimer *timer) +/* Delete a POSIX.1b interval timer. */ +SYSCALL_DEFINE1(timer_delete, timer_t, timer_id) { - spin_lock_irq(&timer->it_lock); - -retry_delete: - /* - * Even if the timer is not longer accessible from other tasks - * it still might be armed and queued in the underlying timer - * mechanism. Worse, that timer mechanism might run the expiry - * function concurrently. - */ - if (timer->kclock->timer_del(timer) =3D=3D TIMER_RETRY) { - /* - * Timer is expired concurrently, prevent livelocks - * and pointless spinning on RT. - * - * timer_wait_running() drops timer::it_lock, which opens - * the possibility for another task to delete the timer. - * - * That's not possible here because this is invoked from - * do_exit() only for the last thread of the thread group. - * So no other task can access and delete that timer. - */ - if (WARN_ON_ONCE(timer_wait_running(timer) !=3D timer)) - return; - - goto retry_delete; - } - hlist_del(&timer->list); - - posix_timer_cleanup_ignored(timer); + struct k_itimer *timer =3D lock_timer(timer_id); =20 - /* - * Setting timer::it_signal to NULL is technically not required - * here as nothing can access the timer anymore legitimately via - * the hash table. Set it to NULL nevertheless so that all deletion - * paths are consistent. - */ - WRITE_ONCE(timer->it_signal, NULL); + if (!timer) + return -EINVAL; =20 - spin_unlock_irq(&timer->it_lock); + posix_timer_delete(timer); + unlock_timer(timer); + /* Remove it from the hash, which frees up the timer ID */ posix_timer_unhash_and_free(timer); + return 0; } =20 /* @@ -1075,6 +1041,8 @@ static void itimer_delete(struct k_itime void exit_itimers(struct task_struct *tsk) { struct hlist_head timers; + struct hlist_node *next; + struct k_itimer *timer; =20 if (hlist_empty(&tsk->signal->posix_timers)) return; @@ -1084,8 +1052,10 @@ void exit_itimers(struct task_struct *ts hlist_move_list(&tsk->signal->posix_timers, &timers); =20 /* The timers are not longer accessible via tsk::signal */ - while (!hlist_empty(&timers)) { - itimer_delete(hlist_entry(timers.first, struct k_itimer, list)); + hlist_for_each_entry_safe(timer, next, &timers, list) { + scoped_guard (spinlock_irq, &timer->it_lock) + posix_timer_delete(timer); + posix_timer_unhash_and_free(timer); cond_resched(); } From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E4951EF0B0 for ; Sun, 2 Mar 2025 19:36:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944222; cv=none; b=LftZBLCbsm/GW52AbHPOdVMxFgFfSSUl2qWspSGee5GzwoolYnhhIbwZILBbmTnKPyGZ/lKhrI0Jz+EEppHRwggepBtD9NQ8cMYjXEUJnh3jleE+1pCDf68OTr3hzHaIJyPF1zDqD6MSVxjZxk8GZD3g6ErIEKhKgw/N1j3pmjU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944222; c=relaxed/simple; bh=DAo6tqgpyFyCHP4VV4EihR4zoAKwOImm2nHPrHvXTDQ=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=bPYmFth1XIXa/OhiGuQT8gI5pcmZt3MqFTRIxCGCku5c9bY9sOKfuMT1u4TlZ+vTC+6X15JGE/quTcuodJlQRCBQcb4Vj3l+QuJXhd6Fq9i64jS9j0ZBsWOoqgD8FwVgHZ7lstybUk+pzJStmp5IDhlZELWGXg+quJ15Bu3froc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=iONGsM6p; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Vl0eo9wN; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="iONGsM6p"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Vl0eo9wN" Message-ID: <20250302193627.480949005@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=mH3NTv/8aQ+ZDpstJx5O7HCZL1s7rrY1IQNFX3q1COo=; b=iONGsM6pLuqi4m50bG+M8yyzyFw82ET+A8wwAAJ1k/A8gWo1oVoYjU4sga1fFU7VurFFJ6 Ou59AZ5s42UNkwxFycnKw39pdqHZlxzIDP9J6EG8mThXme9zyWOkZLB3fiEHEj52+ReIP8 72oAWKiHnhu2zG+esbQQR2+Yz0lzaPGX8xOx3C0vXa0w0/1jxCxtrpO+yB+EIUVRx+do9v 28ItWXr8F63qtWeUnpJl4woi95SJ/gUXScg3spKtY1bzWQwY/sSusKXQZXpEDS4yh4Nnhz XRrHzyUFZyejz+ZsKt6tZfmtF056mQknAUCdCJceTGZCJAp02+eHsx7JiyZHJA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=mH3NTv/8aQ+ZDpstJx5O7HCZL1s7rrY1IQNFX3q1COo=; b=Vl0eo9wNRLjrVDXoaXvTd28sZj1N7idbRccoyxkimkFVjsY618+l/6wgYh9JfvNX6IoBi3 uq68x3AHUaSuHRBQ== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 09/17] posix-timers: Make lock_timer() use guard() References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:58 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra The lookup and locking of posix timers requires the same repeating pattern at all usage sites: tmr =3D lock_timer(tiner_id); if (!tmr) return -EINVAL; .... unlock_timer(tmr); Solve this with a guard implementation, which works in most places out of the box except for those, which need to unlock the timer inside the guard scope. Though the only places where this matters are timer_delete() and timer_settime(). In both cases the timer pointer needs to be preserved across the end of the scope, which is solved by storing the pointer in a variable outside of the scope. timer_settime() also has to protect the timer with RCU before unlocking, which obviously can't use guard(rcu) before leaving the guard scope as that guard is cleaned up before the unlock. Solve this by providing the RCU protection open coded. [ tglx: Made it work and added change log ] Signed-off-by: Peter Zijlstra Signed-off-by-yet: Thomas Gleixner Link: https://lore.kernel.org/all/20250224162103.GD11590@noisy.programming.= kicks-ass.net --- V2: New patch --- include/linux/cleanup.h | 22 ++++++---- kernel/time/posix-timers.c | 91 +++++++++++++++++-----------------------= ----- 2 files changed, 49 insertions(+), 64 deletions(-) --- a/include/linux/cleanup.h +++ b/include/linux/cleanup.h @@ -291,11 +291,21 @@ static inline class_##_name##_t class_## #define __DEFINE_CLASS_IS_CONDITIONAL(_name, _is_cond) \ static __maybe_unused const bool class_##_name##_is_conditional =3D _is_co= nd =20 -#define DEFINE_GUARD(_name, _type, _lock, _unlock) \ +#define __DEFINE_GUARD_LOCK_PTR(_name, _exp) \ + static inline void * class_##_name##_lock_ptr(class_##_name##_t *_T) \ + { return (void *)(__force unsigned long)*(_exp); } + +#define DEFINE_CLASS_IS_GUARD(_name) \ __DEFINE_CLASS_IS_CONDITIONAL(_name, false); \ + __DEFINE_GUARD_LOCK_PTR(_name, _T) + +#define DEFINE_CLASS_IS_COND_GUARD(_name) \ + __DEFINE_CLASS_IS_CONDITIONAL(_name, true); \ + __DEFINE_GUARD_LOCK_PTR(_name, _T) + +#define DEFINE_GUARD(_name, _type, _lock, _unlock) \ DEFINE_CLASS(_name, _type, if (_T) { _unlock; }, ({ _lock; _T; }), _type = _T); \ - static inline void * class_##_name##_lock_ptr(class_##_name##_t *_T) \ - { return (void *)(__force unsigned long)*_T; } + DEFINE_CLASS_IS_GUARD(_name) =20 #define DEFINE_GUARD_COND(_name, _ext, _condlock) \ __DEFINE_CLASS_IS_CONDITIONAL(_name##_ext, true); \ @@ -375,11 +385,7 @@ static inline void class_##_name##_destr if (_T->lock) { _unlock; } \ } \ \ -static inline void *class_##_name##_lock_ptr(class_##_name##_t *_T) \ -{ \ - return (void *)(__force unsigned long)_T->lock; \ -} - +__DEFINE_GUARD_LOCK_PTR(_name, &_T->lock) =20 #define __DEFINE_LOCK_GUARD_1(_name, _type, _lock) \ static inline class_##_name##_t class_##_name##_constructor(_type *l) \ --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -66,6 +66,14 @@ static inline void unlock_timer(struct k spin_unlock_irq(&timr->it_lock); } =20 +#define scoped_timer_get_or_fail(_id) \ + scoped_cond_guard(lock_timer, return -EINVAL, _id) + +#define scoped_timer (scope) + +DEFINE_CLASS(lock_timer, struct k_itimer *, unlock_timer(_T), __lock_timer= (id), timer_t id); +DEFINE_CLASS_IS_COND_GUARD(lock_timer); + static int hash(struct signal_struct *sig, unsigned int nr) { return hash_32(hash32_ptr(sig) ^ nr, HASH_BITS(posix_timers_hashtable)); @@ -675,18 +683,10 @@ void common_timer_get(struct k_itimer *t =20 static int do_timer_gettime(timer_t timer_id, struct itimerspec64 *settin= g) { - struct k_itimer *timr; - int ret =3D 0; - - timr =3D lock_timer(timer_id); - if (!timr) - return -EINVAL; - memset(setting, 0, sizeof(*setting)); - timr->kclock->timer_get(timr, setting); - - unlock_timer(timr); - return ret; + scoped_timer_get_or_fail(timer_id) + scoped_timer->kclock->timer_get(scoped_timer, setting); + return 0; } =20 /* Get the time remaining on a POSIX.1b interval timer. */ @@ -740,17 +740,8 @@ SYSCALL_DEFINE2(timer_gettime32, timer_t */ SYSCALL_DEFINE1(timer_getoverrun, timer_t, timer_id) { - struct k_itimer *timr; - int overrun; - - timr =3D lock_timer(timer_id); - if (!timr) - return -EINVAL; - - overrun =3D timer_overrun_to_int(timr); - unlock_timer(timr); - - return overrun; + scoped_timer_get_or_fail(timer_id) + return timer_overrun_to_int(scoped_timer); } =20 static void common_hrtimer_arm(struct k_itimer *timr, ktime_t expires, @@ -868,12 +859,9 @@ int common_timer_set(struct k_itimer *ti return 0; } =20 -static int do_timer_settime(timer_t timer_id, int tmr_flags, - struct itimerspec64 *new_spec64, +static int do_timer_settime(timer_t timer_id, int tmr_flags, struct itimer= spec64 *new_spec64, struct itimerspec64 *old_spec64) { - int ret; - if (!timespec64_valid(&new_spec64->it_interval) || !timespec64_valid(&new_spec64->it_value)) return -EINVAL; @@ -881,36 +869,28 @@ static int do_timer_settime(timer_t time if (old_spec64) memset(old_spec64, 0, sizeof(*old_spec64)); =20 - for (;;) { - struct k_itimer *timr =3D lock_timer(timer_id); + for (; ; old_spec64 =3D NULL) { + struct k_itimer *timr; =20 - if (!timr) - return -EINVAL; + scoped_timer_get_or_fail(timer_id) { + timr =3D scoped_timer; =20 - if (old_spec64) - old_spec64->it_interval =3D ktime_to_timespec64(timr->it_interval); + if (old_spec64) + old_spec64->it_interval =3D ktime_to_timespec64(timr->it_interval); =20 - /* Prevent signal delivery and rearming. */ - timr->it_signal_seq++; - - ret =3D timr->kclock->timer_set(timr, tmr_flags, new_spec64, old_spec64); - if (ret !=3D TIMER_RETRY) { - unlock_timer(timr); - break; - } + /* Prevent signal delivery and rearming. */ + timr->it_signal_seq++; =20 - /* Read the old time only once */ - old_spec64 =3D NULL; - /* Protect the timer from being freed after the lock is dropped */ - guard(rcu)(); - unlock_timer(timr); - /* - * timer_wait_running() might drop RCU read side protection - * so the timer has to be looked up again! - */ + int ret =3D timr->kclock->timer_set(timr, tmr_flags, new_spec64, old_sp= ec64); + if (ret !=3D TIMER_RETRY) + return ret; + + /* Protect the timer from being freed when leaving the lock scope */ + rcu_read_lock(); + } timer_wait_running(timr); + rcu_read_unlock(); } - return ret; } =20 /* Set a POSIX.1b interval timer */ @@ -1021,13 +1001,12 @@ static void posix_timer_delete(struct k_ /* Delete a POSIX.1b interval timer. */ SYSCALL_DEFINE1(timer_delete, timer_t, timer_id) { - struct k_itimer *timer =3D lock_timer(timer_id); - - if (!timer) - return -EINVAL; + struct k_itimer *timer; =20 - posix_timer_delete(timer); - unlock_timer(timer); + scoped_timer_get_or_fail(timer_id) { + timer =3D scoped_timer; + posix_timer_delete(timer); + } /* Remove it from the hash, which frees up the timer ID */ posix_timer_unhash_and_free(timer); return 0; From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44B441EF38A for ; Sun, 2 Mar 2025 19:37:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944223; cv=none; b=QtrwkV1B5oU6+crcaYd05dggMaZFTYU8HaC8ASowreVG3qoSfuuPi/CjI64F1qRRUCNqO7wMmIu1Sp0R4zghwnjMECE6ppXCyszqCA18uys0djg5xDey0o0HkaZo9NNOJrsfoxMiQc3wKe7ETkXRHfKaqcaZVQYXAvC8s6RY0h4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944223; c=relaxed/simple; bh=ozLgWlnWW1qGcVv1VhQ91QXrKtsv7YuN2YdSlFVFPY0=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=pUZu8mB2vR0XAzgx53lnBaTTjHm57dY9PP7scmHQsiihJZ/IrCdhaHxIkYW44OPrR9JHl9jz7lVCXI65/gMqwSegy4oDTvYgMkOa55g8Wx3ZmsbQUdzRNc2WltAafMamHYNWmhCHQytGpv6H8Tu6oEFaros76cAY8elSAfPlENw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=TG6vcHbT; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=dCnS3G1V; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="TG6vcHbT"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="dCnS3G1V" Message-ID: <20250302193627.543399558@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944220; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=I3coZdNMcVaNaxvXTVu5YsAG2ik/WdCO4w3UUsUch0Y=; b=TG6vcHbTsLVOuNPf+tkBFMC67mI6CqwR9e3g+glfJYgs7XuDp7NvmDBmvQQSt9eFiQAjei naVGvhZpC8ipRKF78MrEoBYppbgYRcluJn8NjN168erZGT99RMCWdNyFErxvacZXLFSAy9 JFUEpgX3VNmE5sL+/U4sDTVb62BpK1j2EFBU69iJQtttCuSWjARfdkRHVns+mgIRYHS66d 1eCGy7u62LeFVnqy0XszlMtLb31CTbwFoP+q3Xs8ZzX7Tw3yrloSin9BuqgoJdi7mfUJsh GBIIsqRIH64tbQYtBmG5sIadka4hrZ7eDcPvALj8VR5IK/5+MMmzjeEMmrF+Cw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944220; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=I3coZdNMcVaNaxvXTVu5YsAG2ik/WdCO4w3UUsUch0Y=; b=dCnS3G1V42ksHT5eyAkf73lEpkrglzlupYSNfjFf9JEYe0tJo0BNxOSVPseaJheZVhKUjn GY0UDBcLNlicLVBw== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 10/17] posix-timers: Make signal_struct::next_posix_timer_id an atomic_t References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:36:59 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Dumazet The global hash_lock protecting the posix timer hash table can be heavily contended especially when there is an extensive linear search for a timer ID. Timer IDs are handed out by monotonically increasing next_posix_timer_id and then validating that there is no timer with the same ID in the hash table. Both operations happen with the global hash lock held. To reduce the hash lock contention the hash will be reworked to a scaled hash with per bucket locks, which requires to handle the ID counter lockless. Prepare for this by making next_posix_timer_id an atomic_t, which can be used lockless with atomic_inc_return(). [ tglx: Adopted from Eric's series, massaged change log and simplified it ] Signed-off-by: Eric Dumazet Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/all/20250219125522.2535263-2-edumazet@google.= com --- V2: Use atomic_fetch_inc() - PeterZ --- include/linux/sched/signal.h | 2 +- kernel/time/posix-timers.c | 14 +++++--------- 2 files changed, 6 insertions(+), 10 deletions(-) --- a/include/linux/sched/signal.h +++ b/include/linux/sched/signal.h @@ -136,7 +136,7 @@ struct signal_struct { #ifdef CONFIG_POSIX_TIMERS =20 /* POSIX.1b Interval Timers */ - unsigned int next_posix_timer_id; + atomic_t next_posix_timer_id; struct hlist_head posix_timers; struct hlist_head ignored_posix_timers; =20 --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -118,21 +118,17 @@ static bool posix_timer_hashed(struct hl static int posix_timer_add(struct k_itimer *timer) { struct signal_struct *sig =3D current->signal; - struct hlist_head *head; - unsigned int cnt, id; =20 /* * FIXME: Replace this by a per signal struct xarray once there is * a plan to handle the resulting CRIU regression gracefully. */ - for (cnt =3D 0; cnt <=3D INT_MAX; cnt++) { - spin_lock(&hash_lock); - id =3D sig->next_posix_timer_id; - - /* Write the next ID back. Clamp it to the positive space */ - sig->next_posix_timer_id =3D (id + 1) & INT_MAX; + for (unsigned int cnt =3D 0; cnt <=3D INT_MAX; cnt++) { + /* Get the next timer ID and clamp it to positive space */ + unsigned int id =3D atomic_fetch_inc(&sig->next_posix_timer_id) & INT_MA= X; + struct hlist_head *head =3D &posix_timers_hashtable[hash(sig, id)]; =20 - head =3D &posix_timers_hashtable[hash(sig, id)]; + spin_lock(&hash_lock); if (!posix_timer_hashed(head, sig, id)) { /* * Set the timer ID and the signal pointer to make From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE4461EFF96 for ; Sun, 2 Mar 2025 19:37:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944225; cv=none; b=EaT2m4mSKKMRTDAN3ku1a7YYMq9aX6TDsw34JWKs521zUmHUtOXmeaEaBMng5K5ShBYgcUSytJQixnmL55VvEl6Hu1rhStw4AsJE2mrZ6DDdvleUQlOQhTVXLmwTT1nObo5FP+qif0Hshn1BA3/LYPPUziTTVxNXvp/XDbSdeBs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944225; c=relaxed/simple; bh=YBQrTSxncls0OBxmM71iSE45YyZznb3x7bbDhZqGGAk=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=PK3zpKwFd2Z2pj6jVdL7jSuiowJhieF2rRHja6EqdCJGDufCBWBG5hTlxhbykmoVn3n3I4OA+L74fLMepuJkixgfZUTLzRNL+CBHUAn57S3ZX8b5CJSGoHtDWjeSlNKJe5VpMY0RIdcJdcLesd4Oh22oyeoguVCjCu9/viwlWTw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=buSHVS8H; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=m5ImAnZK; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="buSHVS8H"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="m5ImAnZK" Message-ID: <20250302193627.606672916@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944222; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Ko5OrhoHybTL1JZX2pFy2fb4X5fmnkg1bgBzYOv3lFY=; b=buSHVS8HwH8YXmF322HvWsWTs6vHWMmn/hJZONDE/U7OUwy8Mf1lU19yqAoSOWgb6vGara 9lNOTwUQ5cJhTX4qPvG+Z7CE3PdXIUpxA/IlmYGA9bvsVdJtZEXuCEhiL6XHoRFAG/vkX7 0KbsvuKPlVQ+HIJu3DbZ2q9QJ3TfLvVaRjenewTpebMIDEzNoVsZCR0r2fZKzGkILLdBLs n4q/6KDqaAWBdQ66+AxhuNk/Dj2jiFljSXwgd09YVLRlAJJXnkiJzrub5+wdYM2b37+MKb +910Cyyf7g7KEeRL01gH96B1iNwlKQu8d5rQimjX5k+7Hq36zzGYttlfghEp4A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944222; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Ko5OrhoHybTL1JZX2pFy2fb4X5fmnkg1bgBzYOv3lFY=; b=m5ImAnZKdv18UgVvTRUsQl90G4X9fjDJ0W7NFxL+//1EtsTqeEtykK5PxhS/46eQJHxm1J RFSfV88ZsHQ8nMCA== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 11/17] posix-timers: Improve hash table performance References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:37:01 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Eric and Ben reported a significant performance bottleneck on the global hash, which is used to store posix timers for lookup. Eric tried to do a lockless validation of a new timer ID before trying to insert the timer, but that does not solve the problem. For the non-contended case this is a pointless exercise and for the contended case this extra lookup just creates enough interleaving that all tasks can make progress. There are actually two real solutions to the problem: 1) Provide a per process (signal struct) xarray storage 2) Implement a smarter hash like the one in the futex code #1 works perfectly fine for most cases, but the fact that CRIU enforced a linear increasing timer ID to restore timers makes this problematic. It's easy enough to create a sparse timer ID space, which amounts very fast to a large junk of memory consumed for the xarray. 2048 timers with a ID offset of 512 consume more than one megabyte of memory for the xarray storage. #2 The main advantage of the futex hash is that it uses per hash bucket locks instead of a global hash lock. Aside of that it is scaled according to the number of CPUs at boot time. Experiments with artifical benchmarks have shown that a scaled hash with per bucket locks comes pretty close to the xarray performance and in some scenarios it performes better. Test 1: A single process creates 20000 timers and afterwards invokes timer_getoverrun(2) on each of them: mainline Eric newhash xarray create 23 ms 23 ms 9 ms 8 ms getoverrun 14 ms 14 ms 5 ms 4 ms Test 2: A single process creates 50000 timers and afterwards invokes timer_getoverrun(2) on each of them: mainline Eric newhash xarray create 98 ms 219 ms 20 ms 18 ms getoverrun 62 ms 62 ms 10 ms 9 ms Test 3: A single process creates 100000 timers and afterwards invokes timer_getoverrun(2) on each of them: mainline Eric newhash xarray create 313 ms 750 ms 48 ms 33 ms getoverrun 261 ms 260 ms 20 ms 14 ms Erics changes create quite some overhead in the create() path due to the double list walk, as the main issue according to perf is the list walk itself. With 100k timers each hash bucket contains ~200 timers, which in the worst case need to be all inspected. The same problem applies for getoverrun() where the lookup has to walk through the hash buckets to find the timer it is looking for. The scaled hash obviously reduces hash collisions and lock contention significantly. This becomes more prominent with concurrency. Test 4: A process creates 63 threads and all threads wait on a barrier before each instance creates 20000 timers and afterwards invokes timer_getoverrun(2) on each of them. The threads are pinned on seperate CPUs to achive maximum concurrency. The numbers are the average times per thread: mainline Eric newhash xarray create 180239 ms 38599 ms 579 ms 813 ms getoverrun 2645 ms 2642 ms 32 ms 7 ms Test 5: A process forks 63 times and all forks wait on a barrier before each instance creates 20000 timers and afterwards invokes timer_getoverrun(2) on each of them. The processes are pinned on seperate CPUs to achive maximum concurrency. The numbers are the average times per process: mainline eric newhash xarray create 157253 ms 40008 ms 83 ms 60 ms getoverrun 2611 ms 2614 ms 40 ms 4 ms So clearly the reduction of lock contention with Eric's changes makes a significant difference for the create() loop, but it does not mitigate the problem of long list walks, which is clearly visible on the getoverrun() side because that is purely dominated by the lookup itself. Once the timer is found, the syscall just reads from the timer structure with no other locks or code paths involved and returns. The reason for the difference between the thread and the fork case for the new hash and the xarray is that both suffer from contention on sighand::siglock and the xarray suffers additionally from contention on the xarray lock on insertion. The only case where the reworked hash slighly outperforms the xarray is a tight loop which creates and deletes timers. Test 4: A process creates 63 threads and all threads wait on a barrier before each instance runs a loop which creates and deletes a timer 100000 times in a row. The threads are pinned on seperate CPUs to achive maximum concurrency. The numbers are the average times per thread: mainline Eric newhash xarray loop 5917 ms 5897 ms 5473 ms 7846 ms Test 5: A process forks 63 times and all forks wait on a barrier before each each instance runs a loop which creates and deletes a timer 100000 times in a row. The processes are pinned on seperate CPUs to achive maximum concurrency. The numbers are the average times per process: mainline Eric newhash xarray loop 5137 ms 7828 ms 891 ms 872 ms In both test there is not much contention on the hash, but the ucount accounting for the signal and in the thread case the sighand::siglock contention (plus the xarray locking) contribute dominantly to the overhead. As the memory consumption of the xarray in the sparse ID case is significant, the scaled hash with per bucket locks seems to be the better overall option. While the xarray has faster lookup times for a large number of timers, the actual syscall usage, which requires the lookup is not an extreme hotpath. Most applications utilize signal delivery and all syscalls except timer_getoverrun(2) are all but cheap. So implement a scaled hash with per bucket locks, which offers the best tradeoff between performance and memory consumption. Reported-by: Eric Dumazet Reported-by: Benjamin Segall Signed-off-by: Thomas Gleixner --- V2: Replace hash() by hashbucket(), which returns the bucket pointer. --- kernel/time/posix-timers.c | 99 ++++++++++++++++++++++++++++++----------= ----- 1 file changed, 68 insertions(+), 31 deletions(-) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -12,10 +12,10 @@ #include #include #include -#include #include #include #include +#include #include #include #include @@ -40,8 +40,18 @@ static struct kmem_cache *posix_timers_c * This allows checkpoint/restore to reconstruct the exact timer IDs for * a process. */ -static DEFINE_HASHTABLE(posix_timers_hashtable, 9); -static DEFINE_SPINLOCK(hash_lock); +struct timer_hash_bucket { + spinlock_t lock; + struct hlist_head head; +}; + +static struct { + struct timer_hash_bucket *buckets; + unsigned long bits; +} __timer_data __ro_after_init __aligned(2*sizeof(long)); + +#define timer_buckets (__timer_data.buckets) +#define timer_hashbits (__timer_data.bits) =20 static const struct k_clock * const posix_clocks[]; static const struct k_clock *clockid_to_kclock(const clockid_t id); @@ -74,18 +84,18 @@ static inline void unlock_timer(struct k DEFINE_CLASS(lock_timer, struct k_itimer *, unlock_timer(_T), __lock_timer= (id), timer_t id); DEFINE_CLASS_IS_COND_GUARD(lock_timer); =20 -static int hash(struct signal_struct *sig, unsigned int nr) +static struct timer_hash_bucket *hash_bucket(struct signal_struct *sig, un= signed int nr) { - return hash_32(hash32_ptr(sig) ^ nr, HASH_BITS(posix_timers_hashtable)); + return &timer_buckets[hash_32(hash32_ptr(sig) ^ nr, timer_hashbits)]; } =20 static struct k_itimer *posix_timer_by_id(timer_t id) { struct signal_struct *sig =3D current->signal; - struct hlist_head *head =3D &posix_timers_hashtable[hash(sig, id)]; + struct timer_hash_bucket *bucket =3D hash_bucket(sig, id); struct k_itimer *timer; =20 - hlist_for_each_entry_rcu(timer, head, t_hash) { + hlist_for_each_entry_rcu(timer, &bucket->head, t_hash) { /* timer->it_signal can be set concurrently */ if ((READ_ONCE(timer->it_signal) =3D=3D sig) && (timer->it_id =3D=3D id)) return timer; @@ -104,11 +114,13 @@ static inline struct signal_struct *posi return (struct signal_struct *)(val & ~1UL); } =20 -static bool posix_timer_hashed(struct hlist_head *head, struct signal_stru= ct *sig, timer_t id) +static bool posix_timer_hashed(struct timer_hash_bucket *bucket, struct si= gnal_struct *sig, + timer_t id) { + struct hlist_head *head =3D &bucket->head; struct k_itimer *timer; =20 - hlist_for_each_entry_rcu(timer, head, t_hash, lockdep_is_held(&hash_lock)= ) { + hlist_for_each_entry_rcu(timer, head, t_hash, lockdep_is_held(&bucket->lo= ck)) { if ((posix_sig_owner(timer) =3D=3D sig) && (timer->it_id =3D=3D id)) return true; } @@ -119,34 +131,34 @@ static int posix_timer_add(struct k_itim { struct signal_struct *sig =3D current->signal; =20 - /* - * FIXME: Replace this by a per signal struct xarray once there is - * a plan to handle the resulting CRIU regression gracefully. - */ for (unsigned int cnt =3D 0; cnt <=3D INT_MAX; cnt++) { /* Get the next timer ID and clamp it to positive space */ unsigned int id =3D atomic_fetch_inc(&sig->next_posix_timer_id) & INT_MA= X; - struct hlist_head *head =3D &posix_timers_hashtable[hash(sig, id)]; + struct timer_hash_bucket *bucket =3D hash_bucket(sig, id); =20 - spin_lock(&hash_lock); - if (!posix_timer_hashed(head, sig, id)) { + scoped_guard (spinlock, &bucket->lock) { /* - * Set the timer ID and the signal pointer to make - * it identifiable in the hash table. The signal - * pointer has bit 0 set to indicate that it is not - * yet fully initialized. posix_timer_hashed() - * masks this bit out, but the syscall lookup fails - * to match due to it being set. This guarantees - * that there can't be duplicate timer IDs handed - * out. + * Validate under the lock as this could have raced + * against another thread ending up with the same + * ID, which is highly unlikely, but possible. */ - timer->it_id =3D (timer_t)id; - timer->it_signal =3D (struct signal_struct *)((unsigned long)sig | 1UL); - hlist_add_head_rcu(&timer->t_hash, head); - spin_unlock(&hash_lock); - return id; + if (!posix_timer_hashed(bucket, sig, id)) { + /* + * Set the timer ID and the signal pointer to make + * it identifiable in the hash table. The signal + * pointer has bit 0 set to indicate that it is not + * yet fully initialized. posix_timer_hashed() + * masks this bit out, but the syscall lookup fails + * to match due to it being set. This guarantees + * that there can't be duplicate timer IDs handed + * out. + */ + timer->it_id =3D (timer_t)id; + timer->it_signal =3D (struct signal_struct *)((unsigned long)sig | 1UL= ); + hlist_add_head_rcu(&timer->t_hash, &bucket->head); + return id; + } } - spin_unlock(&hash_lock); cond_resched(); } /* POSIX return code when no timer ID could be allocated */ @@ -404,7 +416,9 @@ void posixtimer_free_timer(struct k_itim =20 static void posix_timer_unhash_and_free(struct k_itimer *tmr) { - scoped_guard (spinlock, &hash_lock) + struct timer_hash_bucket *bucket =3D hash_bucket(posix_sig_owner(tmr), tm= r->it_id); + + scoped_guard (spinlock, &bucket->lock) hlist_del_rcu(&tmr->t_hash); posixtimer_putref(tmr); } @@ -1477,3 +1491,26 @@ static const struct k_clock *clockid_to_ =20 return posix_clocks[array_index_nospec(idx, ARRAY_SIZE(posix_clocks))]; } + +static int __init posixtimer_init(void) +{ + unsigned long i, size; + unsigned int shift; + + if (IS_ENABLED(CONFIG_BASE_SMALL)) + size =3D 512; + else + size =3D roundup_pow_of_two(512 * num_possible_cpus()); + + timer_buckets =3D alloc_large_system_hash("posixtimers", sizeof(*timer_bu= ckets), + size, 0, 0, &shift, NULL, size, size); + size =3D 1UL << shift; + timer_hashbits =3D ilog2(size); + + for (i =3D 0; i < size; i++) { + spin_lock_init(&timer_buckets[i].lock); + INIT_HLIST_HEAD(&timer_buckets[i].head); + } + return 0; +} +core_initcall(posixtimer_init); From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C72A11F03C8 for ; Sun, 2 Mar 2025 19:37:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944227; cv=none; b=I5Z+usthaeW0SLKQf4JKEPUJxIN4CnWdvM7lPKEGyDxe8DSa935vQTzqTb4KvxllyM9WiGIvMkcPPIaHxhEFFT3ZFYB1Db3n1GKVss/ibPFBWPnbc7QD8FtLHPQpkQFZVU6vscQnxWUtdI6gu151OiiJ2mYEdgDnqonxRnFrBys= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944227; c=relaxed/simple; bh=etY3ts9K3o7g29edq8/d7QUsN8eckbmfkidmCmhVHa0=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=UvzPuoBgEyiu9aZhWRw+1YVjDlHPbudlHhylq+zMJwhgu/6ovHUSZKI6tE1tTIJjsF+jenlA10mVu8UTXXAzE2x3mfzWF/UkvHDV4q/gn1GQMP3fJWSY93F80p3WJ+hzfX+z/AnMwB7KIZI7i84cOJ2sd+nPI046fCGC+E3taBw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=g30fDJVl; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=dMY7cQ8S; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="g30fDJVl"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="dMY7cQ8S" Message-ID: <20250302193627.669047127@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944224; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=JZzdR41mOA7M2Y9QSCZRrpukrbjjYu8nSktij2l6yxA=; b=g30fDJVlqyT2GqiEiki7hmlDMqvIyBJ93xesOrcmwOLTD9B6UmPKsUbdv6LLRtMuWaxz5w kubAlOamTfJI6mWwhNrJ1lJUak2+QMbNFwr3z/FOJlNXxP7FsB43fIhR482H+Is5gs4bY3 ufiWb1AOFd8WkJOZuIah5tSvV9bjIHVZ3FcFwJ2NQqSpXS10P9tjlM474H0RZixoRKKlVB 2O2ku7y0pdhdp5dklqsviTim+D4FsscUUXymWhnB7rgvGXBCM9HOTfIebI3duinDhOXLmx Vug/t/vCEdDstxwwCW5UGlG0+/KrZlmqTaR6ZLTpLS+EOglH0dGWTpLOzq2VXw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944224; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=JZzdR41mOA7M2Y9QSCZRrpukrbjjYu8nSktij2l6yxA=; b=dMY7cQ8SWggpfxCE3YKYzDtU8rz7A2qrpcqwM7TpNd9XFqCznsEjHCYrEtMoN5POlj4JjP 1z3lQEGWIGPJonCQ== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 12/17] posix-timers: Switch to jhash32() References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:37:03 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The hash distribution of hash_32() is suboptimal. jhash32() provides a way better distribution, which evens out the length of the hash bucket lists, which in turn avoids large outliers in list walk times. Due to the sparse ID space (thanks CRIU) there is no guarantee that the timers will be fully evenly distributed over the hash buckets, but the behaviour is way better than with hash_32() even for randomly sparse ID spaces. For a pathological test case with 64 processes creating and accessing 20000 timers each, this results in a runtime reduction of ~10% and a significantly reduced runtime variation. Signed-off-by: Thomas Gleixner --- V2: New patch --- kernel/time/posix-timers.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -11,8 +11,8 @@ */ #include #include -#include #include +#include #include #include #include @@ -47,11 +47,11 @@ struct timer_hash_bucket { =20 static struct { struct timer_hash_bucket *buckets; - unsigned long bits; + unsigned long mask; } __timer_data __ro_after_init __aligned(2*sizeof(long)); =20 #define timer_buckets (__timer_data.buckets) -#define timer_hashbits (__timer_data.bits) +#define timer_hashmask (__timer_data.mask) =20 static const struct k_clock * const posix_clocks[]; static const struct k_clock *clockid_to_kclock(const clockid_t id); @@ -86,7 +86,7 @@ DEFINE_CLASS_IS_COND_GUARD(lock_timer); =20 static struct timer_hash_bucket *hash_bucket(struct signal_struct *sig, un= signed int nr) { - return &timer_buckets[hash_32(hash32_ptr(sig) ^ nr, timer_hashbits)]; + return &timer_buckets[jhash2((u32 *)&sig, sizeof(sig) / sizeof(u32), nr) = & timer_hashmask]; } =20 static struct k_itimer *posix_timer_by_id(timer_t id) @@ -1505,7 +1505,7 @@ static int __init posixtimer_init(void) timer_buckets =3D alloc_large_system_hash("posixtimers", sizeof(*timer_bu= ckets), size, 0, 0, &shift, NULL, size, size); size =3D 1UL << shift; - timer_hashbits =3D ilog2(size); + timer_hashmask =3D size - 1; =20 for (i =3D 0; i < size; i++) { spin_lock_init(&timer_buckets[i].lock); From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44C131EDA32 for ; Sun, 2 Mar 2025 19:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944228; cv=none; b=cqnh+IMNLVGiUqRjVqwOF+ls5UhESs/HeMMuSOOBiCYZ8MmckwWpJs4y6freH3vSmEaUq/fq4X4wWg20Idqp2WNTbWEqmZ4V7l4g5iQccLsIJVEbsDAeXHynCZv4lNnNZd3DHm6/jx0kCviMMpn6jkRUGmslMUzJTbXzmorroEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944228; c=relaxed/simple; bh=JG1yW85eIxVDSozqC+oYG9lX+kCxaDGIRh7fRGvCJlQ=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=KU3KA/41jh9wvet0y8GzANAt+QzqxbCwbUWPqLCwNi3Tw8PrJ0JAH8iaWSSv/01nj1ECX5g6Jvd3hERtkFFVt7HvekbLA4yXFbEalIWx6oBnMze2HhsQwExS2T34h11GgKTXbwK4YK/dKzXkV+tZEVP6WZHPqygjpmbiqQOOGXw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=SZG1J4f+; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=2FCXbaqB; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="SZG1J4f+"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="2FCXbaqB" Message-ID: <20250302193627.730551737@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944226; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=KAkIcnagyxvTTCqc+Efy7bF6aips8b3Vv/tAhqWPhPQ=; b=SZG1J4f+6T+tA3lnmwjneVWbdhhJA6DrgmqYgNwKf8AB8goMumIJ67cEbdIhfIhaPsBPfm aKw9dAvxZRw9q/eGvcfrvW0FqUkPZ+Qcso3qxg7LwzTuhwqnkTudEHGodlR+JMf1DNDh9i Io/bQCqIiRNiDZIT48Ib+wQMDknbXklXEA6wx+Kor8dXbzZ8RORXoolpd6i0HZrRnKoNmM X0NrQmhF1FE7V7hyyFxRFkozx9fGjXCWx70Gy+v66SgM1OehSk3Jg6LvluVQBjbFa8SJtq CrNVzHRZdPyJBn8PWJ30bFv2OxE/97hjO1BG5+hM5TmeBTrvLetcWUBa3Wtv4g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944226; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=KAkIcnagyxvTTCqc+Efy7bF6aips8b3Vv/tAhqWPhPQ=; b=2FCXbaqB+DrejeIvbEltm3HzS4Ev3KZLQW1VZFcG1Ai3dhfEzBpBch44PF0P916KimzXLT ABo2chsJ0UbmHLCQ== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 13/17] posix-timers: Avoid false cacheline sharing References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:37:05 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" struct k_itimer has the hlist_node, which is used for lookup in the hash bucket, and the timer lock in the same cache line. That's obviously bad, if one CPU fiddles with a timer and the other is walking the hash bucket on which that timer is queued. Avoid this by restructuring struct k_itimer, so that the read mostly (only modified during setup and teardown) fields are in the first cache line and the lock and the rest of the fields which get written to are in cacheline 2-N. Reduces cacheline contention in a test case of 64 processes creating and accessing 20000 timers each by almost 30% according to perf. Signed-off-by: Thomas Gleixner --- V2: New patch --- include/linux/posix-timers.h | 21 ++++++++++++--------- kernel/time/posix-timers.c | 4 ++-- 2 files changed, 14 insertions(+), 11 deletions(-) --- a/include/linux/posix-timers.h +++ b/include/linux/posix-timers.h @@ -177,23 +177,26 @@ static inline void posix_cputimers_init_ * @rcu: RCU head for freeing the timer. */ struct k_itimer { - struct hlist_node list; - struct hlist_node ignored_list; + /* 1st cacheline contains read-mostly fields */ struct hlist_node t_hash; - spinlock_t it_lock; - const struct k_clock *kclock; - clockid_t it_clock; + struct hlist_node list; timer_t it_id; + clockid_t it_clock; + int it_sigev_notify; + enum pid_type it_pid_type; + struct signal_struct *it_signal; + const struct k_clock *kclock; + + /* 2nd cacheline and above contain fields which are modified regularly */ + spinlock_t it_lock; int it_status; bool it_sig_periodic; s64 it_overrun; s64 it_overrun_last; unsigned int it_signal_seq; unsigned int it_sigqueue_seq; - int it_sigev_notify; - enum pid_type it_pid_type; ktime_t it_interval; - struct signal_struct *it_signal; + struct hlist_node ignored_list; union { struct pid *it_pid; struct task_struct *it_process; @@ -210,7 +213,7 @@ struct k_itimer { } alarm; } it; struct rcu_head rcu; -}; +} ____cacheline_aligned_in_smp; =20 void run_posix_cpu_timers(void); void posix_cpu_timers_exit(struct task_struct *task); --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -259,8 +259,8 @@ static int posix_get_hrtimer_res(clockid =20 static __init int init_posix_timers(void) { - posix_timers_cache =3D kmem_cache_create("posix_timers_cache", sizeof(str= uct k_itimer), 0, - SLAB_ACCOUNT, NULL); + posix_timers_cache =3D kmem_cache_create("posix_timers_cache", sizeof(str= uct k_itimer), + __alignof__(struct k_itimer), SLAB_ACCOUNT, NULL); return 0; } __initcall(init_posix_timers); From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BB781F03F1 for ; Sun, 2 Mar 2025 19:37:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944230; cv=none; b=WOPG3jwQA11ii0q7eJw43zCKYk3imPmnQgJ8GvMyHeNWPARbwZSUOpW07O++xN/nOJ+uWNuOdQ+aEAhkTHe6He32KjIGzftBBmZMmYy0ipxwtZVYWN3rCDNq4MdjfcWsi+JS9bakGqnA8WyQYBmG83ylXec5CFQlfHGj8+6DQ68= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944230; c=relaxed/simple; bh=qXpFVgf2+2JUiKQvdmuu5fwT++cRBSrieXNRvQPzQzY=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=EyK2dwZEim+jTqJmtyu/5cwMzu3ini+vIDXigH6nzvFBH/lSHA+GOEixhBO/Yw3mxGkmkaRASk2j3GM/SelYwDoI1eZ9cbF9KFb3VmM0TZv0hNl+tF6JcgKJmV2lxOQr2LMYd46CP8q9JfLI9bY33JRGDbXhf49TDn+VI02vLY8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=W+VJcNjO; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=5RuDy35z; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="W+VJcNjO"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="5RuDy35z" Message-ID: <20250302193627.792792199@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944227; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=0n0+ojMJFh6Wn1lejikAH+TEQYW3S7ZwZSQhOekwXGE=; b=W+VJcNjOl96kL1LOMW29kgMDXo+XSY/5uqaskBojo3AdeAYXw4zyADdfQZ0VubCl1s+lxO rjqagKrdCT2gVmPx57+70sm1MUkTlIJy5F1/dNr7529obxiMtI+lF78jvuqNuPIse7RZou /F2KSRq29IH9WATs7lB1QfZykCd/XhlgpnTLwNPyNDv/Ne3t2kxJaQdMwd4KICYqNlKuDg MKZLnhR9q64uNnpRKJ4qBBzL4HKWYuMhe+FFLmSYnMDdp0rB9q3ckeFW/+GqOjE1YbjSE/ qNov+TG0duFFsXLOJ17hXwcHiS3gjpH0CdztVBn1vB4zP8Ww4wYoaBWGo06x1g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944227; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=0n0+ojMJFh6Wn1lejikAH+TEQYW3S7ZwZSQhOekwXGE=; b=5RuDy35zjS3mb33wZ+tNxQnuhHXQ9IK0/USfvBLcNkb9bUHaf6NBQGM35AlGFuWPx5HoCf 08FrJHcslLjqEsCw== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 14/17] posix-timers: Make per process list RCU safe References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:37:07 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Preparatory change to remove the sighand locking from the /proc/$PID/timers iterator. Signed-off-by: Thomas Gleixner --- kernel/time/posix-timers.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -510,7 +510,7 @@ static int do_timer_create(clockid_t whi * unmodified signal pointer to make it valid. */ WRITE_ONCE(new_timer->it_signal, current->signal); - hlist_add_head(&new_timer->list, ¤t->signal->posix_timers); + hlist_add_head_rcu(&new_timer->list, ¤t->signal->posix_timers); } /* * After unlocking sighand::siglock @new_timer is subject to @@ -996,7 +996,7 @@ static void posix_timer_delete(struct k_ unsigned long sig =3D (unsigned long)timer->it_signal | 1UL; =20 WRITE_ONCE(timer->it_signal, (struct signal_struct *)sig); - hlist_del(&timer->list); + hlist_del_rcu(&timer->list); posix_timer_cleanup_ignored(timer); } From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACF881F0E28 for ; Sun, 2 Mar 2025 19:37:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944232; cv=none; b=hP2HCKH+6JiAO1+mw9WLwURwEXazr2cPRAxlKSmcDNUBy5fIqeASLGTJwk9+zVSfO/8T2sBNUXqPk222rQUW3npegaOVbbTyHWGuAoqlV/dPRsmRgy3XcPjxNPy9nvV7sseIiUwj804Xm2AMatAlN+7eOrLHKUfi62PoNGtIkaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944232; c=relaxed/simple; bh=d3Y3ODPrIY1jp+7CJvmqQz5BLSj4jQPlqtFgLsgPGqQ=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=ZbTyCbXVWtTTGv1xWoJCTAtSS4TlMpa2DrqcJ1kqc5ylCefvyggaCwK4+UNoDORwWlCthnbUpN4uLEACuIBWKS3/cdQiLD2dzEM/BnInxtQjzqdrhXyy22rP/H4kYYGcv+baU48dqFV3ojNSnMTKrzgjmapsr12gnJrytSdd2Js= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=QJUIbIcC; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=1Jf0Rqto; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="QJUIbIcC"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="1Jf0Rqto" Message-ID: <20250302193627.854190870@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944229; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=jGU5P5nY7M5LQXxbj27aqdU32TjBwj9UsBnUCn2J8+Q=; b=QJUIbIcCkcYFCcNorkv/sWN5FfYXIsyRIqh2ZyxKMXhA0mRCo7l0PnDBraNj306nMcve2J WKVX130BUKNyWF5Y4HlM70CPFXxIuLLqN4nCOvuuaPbZAH+WRD4JjUjGRRoPuO5O3z4Ont /MOSClOKtyKl3D/RdwI4Dxm8NnOQ2XPhuATEgsFS2FrIi6ZLFFqHaIstd/KE/Wh1HyG8pv I81+F+RwRt9kS59zjqlV305uNxq0B4coVrHEwbLJmSZVqSuYdwJDutS+KtgA3ui7bjK649 1ZbvUEIwlfJWiZ89Pu9gQonH/orcHfXqVlZ/KAbLGxZJjGfVh5cV6VracIcOwA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944229; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=jGU5P5nY7M5LQXxbj27aqdU32TjBwj9UsBnUCn2J8+Q=; b=1Jf0RqtoBNFwCyKFDKX7BWHxZI7bZm4jIk987KzLFOK3Vyi15lYXM3gecK5zayGn/LHlrP gogYOnsruUF5k9Cg== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 15/17] posix-timers: Dont iterate /proc/$PID/timers with sighand::siglock held References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:37:08 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The readout of /proc/$PID/timers holds sighand::siglock with interrupts disabled. That is required to protect against concurrent modifications of the task::signal::posix_timers list because the list is not RCU safe. With the conversion of the timer storage to a RCU protected hlist, this is not longer required. The only requirement is to protect the returned entry against a concurrent free, which is trivial as the timers are RCU protected. Removing the trylock of sighand::siglock is benign because the life time of task_struct::signal is bound to the life time of the task_struct itself. There are two scenarios where this matters: 1) The process is life and not about to be checkpointed 2) The process is stopped via ptrace for checkpointing #1 is a racy snapshot of the armed timers and nothing can rely on it. It's not more than debug information and it has been that way before because sighand lock is dropped when the buffer is full and the restart of the iteration might find a completely different set of timers. The task and therefore task::signal cannot be freed as timers_start() acquired a reference count via get_pid_task(). #2 the process is stopped for checkpointing so nothing can delete or create timers at this point. Neither can the process exit during the traversal. If CRIU fails to observe an exit in progress prior to the dissimination of the timers, then there are more severe problems to solve in the CRIU mechanics as they can't rely on posix timers being enabled in the first place. Therefore replace the lock acquisition with rcu_read_lock() and switch the timer storage traversal over to seq_hlist_*_rcu(). Signed-off-by: Thomas Gleixner --- fs/proc/base.c | 48 ++++++++++++++++++++---------------------------- 1 file changed, 20 insertions(+), 28 deletions(-) --- a/fs/proc/base.c +++ b/fs/proc/base.c @@ -2497,11 +2497,9 @@ static const struct file_operations proc =20 #if defined(CONFIG_CHECKPOINT_RESTORE) && defined(CONFIG_POSIX_TIMERS) struct timers_private { - struct pid *pid; - struct task_struct *task; - struct sighand_struct *sighand; - struct pid_namespace *ns; - unsigned long flags; + struct pid *pid; + struct task_struct *task; + struct pid_namespace *ns; }; =20 static void *timers_start(struct seq_file *m, loff_t *pos) @@ -2512,54 +2510,48 @@ static void *timers_start(struct seq_fil if (!tp->task) return ERR_PTR(-ESRCH); =20 - tp->sighand =3D lock_task_sighand(tp->task, &tp->flags); - if (!tp->sighand) - return ERR_PTR(-ESRCH); - - return seq_hlist_start(&tp->task->signal->posix_timers, *pos); + rcu_read_lock(); + return seq_hlist_start_rcu(&tp->task->signal->posix_timers, *pos); } =20 static void *timers_next(struct seq_file *m, void *v, loff_t *pos) { struct timers_private *tp =3D m->private; - return seq_hlist_next(v, &tp->task->signal->posix_timers, pos); + + return seq_hlist_next_rcu(v, &tp->task->signal->posix_timers, pos); } =20 static void timers_stop(struct seq_file *m, void *v) { struct timers_private *tp =3D m->private; =20 - if (tp->sighand) { - unlock_task_sighand(tp->task, &tp->flags); - tp->sighand =3D NULL; - } - if (tp->task) { put_task_struct(tp->task); tp->task =3D NULL; + rcu_read_unlock(); } } =20 static int show_timer(struct seq_file *m, void *v) { - struct k_itimer *timer; - struct timers_private *tp =3D m->private; - int notify; static const char * const nstr[] =3D { - [SIGEV_SIGNAL] =3D "signal", - [SIGEV_NONE] =3D "none", - [SIGEV_THREAD] =3D "thread", + [SIGEV_SIGNAL] =3D "signal", + [SIGEV_NONE] =3D "none", + [SIGEV_THREAD] =3D "thread", }; =20 - timer =3D hlist_entry((struct hlist_node *)v, struct k_itimer, list); - notify =3D timer->it_sigev_notify; + struct k_itimer *timer =3D hlist_entry((struct hlist_node *)v, struct k_i= timer, list); + struct timers_private *tp =3D m->private; + int notify =3D timer->it_sigev_notify; + + guard(spinlock_irq)(&timer->it_lock); + if (!posixtimer_valid(timer)) + return 0; =20 seq_printf(m, "ID: %d\n", timer->it_id); - seq_printf(m, "signal: %d/%px\n", - timer->sigq.info.si_signo, + seq_printf(m, "signal: %d/%px\n", timer->sigq.info.si_signo, timer->sigq.info.si_value.sival_ptr); - seq_printf(m, "notify: %s/%s.%d\n", - nstr[notify & ~SIGEV_THREAD_ID], + seq_printf(m, "notify: %s/%s.%d\n", nstr[notify & ~SIGEV_THREAD_ID], (notify & SIGEV_THREAD_ID) ? "tid" : "pid", pid_nr_ns(timer->it_pid, tp->ns)); seq_printf(m, "ClockID: %d\n", timer->it_clock); From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D59FD1F0E4E for ; Sun, 2 Mar 2025 19:37:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944234; cv=none; b=DjZUDkAw5ed+LI6VrMEHf57A6ghG4r/O10teteQWA0UjQG0QoN9Txfnm/7dWIBwJ7aaOyNfmo2/btjdD5oX7Nb9r9uA9URp4OoeG53mRQuLtlgHp/pHRXYx8WMAF6JSJAiYJkG6dG++Wkpeni2xfygzBUVTTlo361MGHl96vBlA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944234; c=relaxed/simple; bh=LpxbSLqhmbCosjbhGGIcfWrq/2gzjkNoDHuXpWjPAkU=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=T/d/orCSeIoB3WiY37qXOUmKXDPBGN+A3m1TaZQx5HAtv4xIYnUDSa8WlloaSXQqZmJsDr1nz4elfIFGEtjVfqvS7HySiI4+Z9zSbKRcOJFfinhRIxDRTOJKX2RCRvUMYFrENnUH6A1sC/vrX+zJbJSGA8m98P5+B5eih1/SUrQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=GTIj+YSO; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=FQ1v2xAi; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="GTIj+YSO"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="FQ1v2xAi" Message-ID: <20250302193627.914275679@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944231; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=XOXn/XWdhruxzhP6MzL39Hfp7J3nNXYMPBVienLz+yY=; b=GTIj+YSOR1VdIm248fphpawShIM/Ruv7MGjj4HdpUIXDq2X7nVC3Tmi91Km35l2lX1OoLx 8P8FOtXe7ZJBKEMm2Rc+wqRaI1ETvpkjtkPsZ+OD1uW6eblxEoMzAuZTkKae97Vyr8RVHQ qSyYxA+y0YZNLQSGwpH4aRInp7D4q36h9GiiJMYyTW0nAfZXP7jdNqwPG9apuUOYxJZRYc rSMYLV/u/pY0KS1qVpc3sXMFzSWJLj0YTx2/01ZAnKzZKAlWW8mxBMOe7KdYCFhmmi3e6C ujqt4UNRnQWrqRSaafE2yzmOgm194BFGcSy/sGC/jMX0w5v6gBPzQOi55IB0tQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944231; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=XOXn/XWdhruxzhP6MzL39Hfp7J3nNXYMPBVienLz+yY=; b=FQ1v2xAin+ZPVZBTsDgKRPUE1LP4LVoQLYK4ml+fkGCGrA0d+ItZ5gDJ3/5X+sSpg/UDgy IXLVfjPOyN0H1rDw== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 16/17] posix-timers: Provide a mechanism to allocate a given timer ID References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:37:10 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Checkpoint/Restore in Userspace (CRIU) requires to reconstruct posix timers with the same timer ID on restore. It uses sys_timer_create() and relies on the monotonic increasing timer ID provided by this syscall. It creates and deletes timers until the desired ID is reached. This is can loop for a long time, when the checkpointed process had a very sparse timer ID range. It has been debated to implement a new syscall to allow the creation of timers with a given timer ID, but that's tideous due to the 32/64bit compat issues of sigevent_t and of dubious value. The restore mechanism of CRIU creates the timers in a state where all threads of the restored process are held on a barrier and cannot issue syscalls. That means the restorer task has exclusive control. This allows to address this issue with a prctl() so that the restorer thread can do: if (prctl(PR_TIMER_CREATE_RESTORE_IDS, PR_TIMER_CREATE_RESTORE_IDS_ON)) goto linear_mode; create_timers_with_explicit_ids(); prctl(PR_TIMER_CREATE_RESTORE_IDS, PR_TIMER_CREATE_RESTORE_IDS_OFF); =20 This is backwards compatible because the prctl() fails on older kernels and CRIU can fall back to the linear timer ID mechanism. CRIU versions which do not know about the prctl() just work as before. Implement the prctl() and modify timer_create() so that it copies the requested timer ID from userspace by utilizing the existing timer_t pointer, which is used to copy out the allocated timer ID on success. If the prctl() is disabled, which it is by default, timer_create() works as before and does not try to read from the userspace pointer. There is no problem when a broken or rogue user space application enables the prctl(). If the user space pointer does not contain a valid ID, then timer_create() fails. If the data is not initialized, but constains a random valid ID, timer_create() will create that random timer ID or fail if the ID is already given out.=20 =20 As CRIU must use the raw syscall to avoid manipulating the internal state of the restored process, this has no library dependencies and can be adopted by CRIU right away. Recreating two timers with IDs 1000000 and 2000000 takes 1.5 seconds with the create/delete method. With the prctl() it takes 3 microseconds. Signed-off-by: Thomas Gleixner --- V2: Move the ID counter ahead to avoid collisions after switching back to normal mode. --- include/linux/posix-timers.h | 2=20 include/linux/sched/signal.h | 1=20 include/uapi/linux/prctl.h | 10 ++++ kernel/sys.c | 5 ++ kernel/time/posix-timers.c | 97 +++++++++++++++++++++++++++++++-------= ----- 5 files changed, 89 insertions(+), 26 deletions(-) --- a/include/linux/posix-timers.h +++ b/include/linux/posix-timers.h @@ -114,6 +114,7 @@ bool posixtimer_init_sigqueue(struct sig void posixtimer_send_sigqueue(struct k_itimer *tmr); bool posixtimer_deliver_signal(struct kernel_siginfo *info, struct sigqueu= e *timer_sigq); void posixtimer_free_timer(struct k_itimer *timer); +long posixtimer_create_prctl(unsigned long ctrl); =20 /* Init task static initializer */ #define INIT_CPU_TIMERBASE(b) { \ @@ -140,6 +141,7 @@ static inline void posixtimer_rearm_itim static inline bool posixtimer_deliver_signal(struct kernel_siginfo *info, struct sigqueue *timer_sigq) { return false; } static inline void posixtimer_free_timer(struct k_itimer *timer) { } +static inline long posixtimer_create_prctl(unsigned long ctrl) { return -E= INVAL; } #endif =20 #ifdef CONFIG_POSIX_CPU_TIMERS_TASK_WORK --- a/include/linux/sched/signal.h +++ b/include/linux/sched/signal.h @@ -136,6 +136,7 @@ struct signal_struct { #ifdef CONFIG_POSIX_TIMERS =20 /* POSIX.1b Interval Timers */ + unsigned int timer_create_restore_ids:1; atomic_t next_posix_timer_id; struct hlist_head posix_timers; struct hlist_head ignored_posix_timers; --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -353,4 +353,14 @@ struct prctl_mm_map { */ #define PR_LOCK_SHADOW_STACK_STATUS 76 =20 +/* + * Controls the mode of timer_create() for CRIU restore operations. + * Enabling this allows CRIU to restore timers with explicit IDs. + * + * Don't use for normal operations as the result might be undefined. + */ +#define PR_TIMER_CREATE_RESTORE_IDS 77 +# define PR_TIMER_CREATE_RESTORE_IDS_OFF 0 +# define PR_TIMER_CREATE_RESTORE_IDS_ON 1 + #endif /* _LINUX_PRCTL_H */ --- a/kernel/sys.c +++ b/kernel/sys.c @@ -2811,6 +2811,11 @@ SYSCALL_DEFINE5(prctl, int, option, unsi return -EINVAL; error =3D arch_lock_shadow_stack_status(me, arg2); break; + case PR_TIMER_CREATE_RESTORE_IDS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error =3D posixtimer_create_prctl(arg2); + break; default: trace_task_prctl_unknown(option, arg2, arg3, arg4, arg5); error =3D -EINVAL; --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -57,6 +58,8 @@ static const struct k_clock * const posi static const struct k_clock *clockid_to_kclock(const clockid_t id); static const struct k_clock clock_realtime, clock_monotonic; =20 +#define TIMER_ANY_ID INT_MIN + /* SIGEV_THREAD_ID cannot share a bit with the other SIGEV values. */ #if SIGEV_THREAD_ID !=3D (SIGEV_THREAD_ID & \ ~(SIGEV_SIGNAL | SIGEV_NONE | SIGEV_THREAD)) @@ -127,38 +130,60 @@ static bool posix_timer_hashed(struct ti return false; } =20 -static int posix_timer_add(struct k_itimer *timer) +static bool posix_timer_add_at(struct k_itimer *timer, struct signal_struc= t *sig, unsigned int id) +{ + struct timer_hash_bucket *bucket =3D hash_bucket(sig, id); + + scoped_guard (spinlock, &bucket->lock) { + /* + * Validate under the lock as this could have raced against + * another thread ending up with the same ID, which is + * highly unlikely, but possible. + */ + if (!posix_timer_hashed(bucket, sig, id)) { + /* + * Set the timer ID and the signal pointer to make + * it identifiable in the hash table. The signal + * pointer has bit 0 set to indicate that it is not + * yet fully initialized. posix_timer_hashed() + * masks this bit out, but the syscall lookup fails + * to match due to it being set. This guarantees + * that there can't be duplicate timer IDs handed + * out. + */ + timer->it_id =3D (timer_t)id; + timer->it_signal =3D (struct signal_struct *)((unsigned long)sig | 1UL); + hlist_add_head_rcu(&timer->t_hash, &bucket->head); + return true; + } + } + return false; +} + +static int posix_timer_add(struct k_itimer *timer, int req_id) { struct signal_struct *sig =3D current->signal; =20 + if (unlikely(req_id !=3D TIMER_ANY_ID)) { + if (!posix_timer_add_at(timer, sig, req_id)) + return -EBUSY; + + /* + * Move the ID counter past the requested ID, so that after + * switching back to normal mode the IDs are outside of the + * exact allocated region. That avoids ID collisions on the + * next regular timer_create() invocations. + */ + atomic_set(&sig->next_posix_timer_id, req_id + 1); + return req_id; + } + for (unsigned int cnt =3D 0; cnt <=3D INT_MAX; cnt++) { /* Get the next timer ID and clamp it to positive space */ unsigned int id =3D atomic_fetch_inc(&sig->next_posix_timer_id) & INT_MA= X; - struct timer_hash_bucket *bucket =3D hash_bucket(sig, id); =20 - scoped_guard (spinlock, &bucket->lock) { - /* - * Validate under the lock as this could have raced - * against another thread ending up with the same - * ID, which is highly unlikely, but possible. - */ - if (!posix_timer_hashed(bucket, sig, id)) { - /* - * Set the timer ID and the signal pointer to make - * it identifiable in the hash table. The signal - * pointer has bit 0 set to indicate that it is not - * yet fully initialized. posix_timer_hashed() - * masks this bit out, but the syscall lookup fails - * to match due to it being set. This guarantees - * that there can't be duplicate timer IDs handed - * out. - */ - timer->it_id =3D (timer_t)id; - timer->it_signal =3D (struct signal_struct *)((unsigned long)sig | 1UL= ); - hlist_add_head_rcu(&timer->t_hash, &bucket->head); - return id; - } - } + if (posix_timer_add_at(timer, sig, id)) + return id; cond_resched(); } /* POSIX return code when no timer ID could be allocated */ @@ -363,6 +388,16 @@ static enum hrtimer_restart posix_timer_ return HRTIMER_NORESTART; } =20 +long posixtimer_create_prctl(unsigned long ctrl) +{ + if (ctrl > PR_TIMER_CREATE_RESTORE_IDS_ON) + return -EINVAL; + + guard(spinlock_irq)(¤t->sighand->siglock); + current->signal->timer_create_restore_ids =3D ctrl =3D=3D PR_TIMER_CREATE= _RESTORE_IDS_ON; + return 0; +} + static struct pid *good_sigevent(sigevent_t * event) { struct pid *pid =3D task_tgid(current); @@ -434,6 +469,7 @@ static int do_timer_create(clockid_t whi timer_t __user *created_timer_id) { const struct k_clock *kc =3D clockid_to_kclock(which_clock); + timer_t req_id =3D TIMER_ANY_ID; struct k_itimer *new_timer; int error, new_timer_id; =20 @@ -448,11 +484,20 @@ static int do_timer_create(clockid_t whi =20 spin_lock_init(&new_timer->it_lock); =20 + /* Special case for CRIU to restore timers with a given timer ID. */ + if (unlikely(current->signal->timer_create_restore_ids)) { + if (copy_from_user(&req_id, created_timer_id, sizeof(req_id))) + return -EFAULT; + /* Valid IDs are 0..INT_MAX */ + if ((unsigned int)req_id > INT_MAX) + return -EINVAL; + } + /* * Add the timer to the hash table. The timer is not yet valid * after insertion, but has a unique ID allocated. */ - new_timer_id =3D posix_timer_add(new_timer); + new_timer_id =3D posix_timer_add(new_timer, req_id); if (new_timer_id < 0) { posixtimer_free_timer(new_timer); return new_timer_id; From nobody Fri Dec 19 04:18:11 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC26A1EE01B for ; Sun, 2 Mar 2025 19:37:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944237; cv=none; b=puaMVPczD63JDJweKXlPmLrSE25DkggGnJwwoQjpS9ueNms/4nV/Q6SnnffthJEi2Ru75R3tkp3C+M0a/2lRlEmo8WweRYiCphm9/ifcwFxdXI7LMedmmBmxT7NX+BNSWEDxmtGEW6MFy+VqXqFcWR/VEpWIVG06UzRe3L9GXg0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740944237; c=relaxed/simple; bh=nm3jnHRda4HitifBptAWoGu1b+t/Cc/b1Cs2siInBQM=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=RholrtO9fkVKz0PoxNntvjz+dADGBtyOaC5DVn5fyLE6pUYhE2wFhn8OxII3FNTH4lt4SzQSk2GnZwyc9X7dAl47TYyoMDvNYVgZcQKFXKd1atYYO4U+SwLDONar5hyBnbfhRkV+araUJYbynlGSfK3j9gsDeOdO9C4v8vjLXSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=BLtC/kFN; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=LqI8fGvS; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="BLtC/kFN"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="LqI8fGvS" Message-ID: <20250302193627.975178175@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1740944233; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=5xK/l6LFqn7LeaJQi95qjBpThzZPtBacmbo+Vkd94eA=; b=BLtC/kFNkSe8NBPK98ozDVMimkgTCIR+vLD8HkvmdS+TszuPVsmNyWgcUjCl0DOVSifO3l 0ZQMXk064SAI4aEIpYDaZ27q4YXu/JhGEfBY3qzfFQP8lOJAiv/uMLSYelEPoTdB6rn++s badpjVvwkGZ16yw+XwHG346qNaUGBstkPrjeNseKoQOZJWhb3wly4lsnBYs+0ZFFGc704Z e0CDyB4INgj80KfOp53UukeOX+xhQAWxn2q/bEVs6DCjnH0yfPexorqspAiN7hCNgdz5oa E1yraRJJscxJ2QptUHlYeBVWaLMakATgMnJ8aTaUZC9UZDW0N3iYCL0yXVy2Hg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1740944233; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=5xK/l6LFqn7LeaJQi95qjBpThzZPtBacmbo+Vkd94eA=; b=LqI8fGvS5t3EP7sJwvB1OXntTiPoXYeDTaV+qjERry4zAJl/ThY7ci1s5urx6PZOfFYdtO h9L6SS5b46LLrWDQ== From: Thomas Gleixner To: LKML Cc: Anna-Maria Behnsen , Frederic Weisbecker , Benjamin Segall , Eric Dumazet , Andrey Vagin , Pavel Tikhomirov , Peter Zijlstra Subject: [patch V2 17/17] selftests/timers/posix-timers: Add a test for exact allocation mode References: <20250302185753.311903554@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sun, 2 Mar 2025 20:37:12 +0100 (CET) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The exact timer ID allocation mode is used by CRIU to restore timers with a given ID. Add a test case for it. It's skipped on older kernels when the prctl() fails. Signed-off-by: Thomas Gleixner --- V2: Adopt to the ID counter change in the exact mode case --- tools/testing/selftests/timers/posix_timers.c | 60 +++++++++++++++++++++= ++++- 1 file changed, 59 insertions(+), 1 deletion(-) --- a/tools/testing/selftests/timers/posix_timers.c +++ b/tools/testing/selftests/timers/posix_timers.c @@ -7,6 +7,7 @@ * Kernel loop code stolen from Steven Rostedt */ #define _GNU_SOURCE +#include #include #include #include @@ -599,14 +600,71 @@ static void check_overrun(int which, con "check_overrun %s\n", name); } =20 +#include + +static int do_timer_create(int *id) +{ + return syscall(__NR_timer_create, CLOCK_MONOTONIC, NULL, id); +} + +static int do_timer_delete(int id) +{ + return syscall(__NR_timer_delete, id); +} + +static void check_timer_create_exact(void) +{ + int id; + + if (prctl(77, 1, 0, 0, 0)) { + switch (errno) { + case EINVAL: + ksft_test_result_skip("check timer create exact, not supported\n"); + return; + default: + ksft_test_result_skip("check timer create exact, errno =3D %d\n", errno= ); + return; + } + } + + id =3D 8; + if (do_timer_create(&id) < 0) + fatal_error(NULL, "timer_create()"); + + if (do_timer_delete(id)) + fatal_error(NULL, "timer_delete()"); + + if (prctl(77, 0, 0, 0, 0)) + fatal_error(NULL, "prctl()"); + + if (id !=3D 8) { + ksft_test_result_fail("check timer create exact %d !=3D 8\n", id); + return; + } + + /* Validate that it went back to normal mode and allocates ID 9 */ + if (do_timer_create(&id) < 0) + fatal_error(NULL, "timer_create()"); + + if (do_timer_delete(id)) + fatal_error(NULL, "timer_delete()"); + + if (id =3D=3D 9) + ksft_test_result_pass("check timer create exact\n"); + else + ksft_test_result_fail("check timer create exact. Disabling failed.\n"); +} + int main(int argc, char **argv) { ksft_print_header(); - ksft_set_plan(18); + ksft_set_plan(19); =20 ksft_print_msg("Testing posix timers. False negative may happen on CPU ex= ecution \n"); ksft_print_msg("based timers if other threads run on the CPU...\n"); =20 + check_timer_create_exact(); + check_itimer(ITIMER_VIRTUAL, "ITIMER_VIRTUAL"); check_itimer(ITIMER_PROF, "ITIMER_PROF"); check_itimer(ITIMER_REAL, "ITIMER_REAL");