From nobody Sun Feb 8 20:45:17 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE4D31D95AB for ; Mon, 28 Oct 2024 12:19:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730117972; cv=none; b=MloILe2fhdDFjtMN3agAVD87HjP85V8F8VBUBMKtU2niKKOiRVRED9vOyPF73wU3eNse7A4QayGoWZmMyOMoJphTt71IQMdqrWOzHcsDDtVtCvTp7UxeLgMeWFzOXZqhwMgRL+qEUzvGKPxULkk1skrlbWAWIWzidvIyRmwNqD0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730117972; c=relaxed/simple; bh=wxTiCPgczYeasflXTB3Gnf/RVwmWOrJrFbwDdZDzhX4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uBHa1dnW7GAO4hWv4Tf/QqatBux3FsFkY6MJYa+KLHLytADJyjJ2buZoc8x9pCjaukRwT5WJJNRNcsS7xtVLftUyq1rJoVFxA+VDDjtCSFLdiPo+z2h8u44/KYoNjREv2b2kSi0l5zR7hsCmcLr9U/b5x3wkXrP5InUMYy9Ecag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=0Avv09th; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=EzCZRmyC; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="0Avv09th"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="EzCZRmyC" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1730117968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4mZlKRrG+DPnUxrk/55cv+455WXnVQzhH1V8fNj3nts=; b=0Avv09thCxTzw/EHsXH4o4jpe6LDkiSXGPP5UBrDLlZMlKk+p5aoUm476tIi6spKn54VJM 3SScuAQH3jbLLtDRxMjoGKOX1esdZGyUjyRu3fgf64SR+2NRpO1LxsE0mFy2O19SBCI+8u /Q61iwqY78rsqq4guXlewMG3VN1d/pgCSqBGhpXtX3liCX6VSk2QCIDc+2P+7IJe2f1H6k i1z4jlNoT3QFvJfKK8r3tBZ4HYQYhuFQ8VxfXmZQ1RIgHBulWi2YcJOhbihRSi7qxMA35T IpKKHGCyVFAnAkH3qw3/esgDndtm6Zeduvy0tfvTtvrOWsCP4dq+lSFeCFmmCQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1730117968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4mZlKRrG+DPnUxrk/55cv+455WXnVQzhH1V8fNj3nts=; b=EzCZRmyCicW2lvutwRArQOFBk6Izps7xLRpleRA/WfLnkR293QDN/8/H6yfXaXu9z6J27h 3iIEBz+s2/R1+wAQ== To: linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Andr=C3=A9=20Almeida?= , Darren Hart , Davidlohr Bueso , Ingo Molnar , Juri Lelli , Peter Zijlstra , Valentin Schneider , Waiman Long , Sebastian Andrzej Siewior Subject: [RFC PATCH v2 1/4] futex: Create helper function to initialize a hash slot. Date: Mon, 28 Oct 2024 13:13:55 +0100 Message-ID: <20241028121921.1264150-2-bigeasy@linutronix.de> In-Reply-To: <20241028121921.1264150-1-bigeasy@linutronix.de> References: <20241028121921.1264150-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Factor out the futex_hash_bucket initialisation into a helpr function. The helper function will be used in a follow up patch. Signed-off-by: Sebastian Andrzej Siewior --- kernel/futex/core.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/kernel/futex/core.c b/kernel/futex/core.c index 136768ae26375..de6d7f71961eb 100644 --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -1146,6 +1146,13 @@ void futex_exit_release(struct task_struct *tsk) futex_cleanup_end(tsk, FUTEX_STATE_DEAD); } =20 +static void futex_hash_bucket_init(struct futex_hash_bucket *fhb) +{ + atomic_set(&fhb->waiters, 0); + plist_head_init(&fhb->chain); + spin_lock_init(&fhb->lock); +} + static int __init futex_init(void) { unsigned int futex_shift; @@ -1163,11 +1170,8 @@ static int __init futex_init(void) futex_hashsize, futex_hashsize); futex_hashsize =3D 1UL << futex_shift; =20 - for (i =3D 0; i < futex_hashsize; i++) { - atomic_set(&futex_queues[i].waiters, 0); - plist_head_init(&futex_queues[i].chain); - spin_lock_init(&futex_queues[i].lock); - } + for (i =3D 0; i < futex_hashsize; i++) + futex_hash_bucket_init(&futex_queues[i]); =20 return 0; } --=20 2.45.2 From nobody Sun Feb 8 20:45:17 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE4771D95AA for ; Mon, 28 Oct 2024 12:19:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730117974; cv=none; b=PwWIRQfo03yEGGXvAuO5RUkkbaQDpupxnn3X7RsZZI1U2rx2jkfL5a0dnX33Tjgmz5cH4kd7niV5HbWLq9anQfh8+PdSEDanffHuNTg3Ec/XgihM47a/dpGlI3Ii7qH/ou4Tr1MXLVUewFGjCaAnlzPROevnlAJvpnUFdJ9Xqms= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730117974; c=relaxed/simple; bh=GDJFWglKeGAlkLsRllnwFDvntBAGxnxH7K5IBtlijRA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B5o6CymnwpZUdasTN3Ry6jxE3xk+Wv2KAQErDRXqepQ9JuSG3TCFO6u7dosSdtt95/MhECz5cyzZP6TqRcRB91sXp+h4nSkNbwZdFz5krLQLcmTYQmB4WN07Ke7cJn5HBUAMG08m4uTeBXihYGfUcHl+aZWSSxw7Ax4SwFeJd54= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Os1mNJ49; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=XiRmyz0X; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Os1mNJ49"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="XiRmyz0X" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1730117969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mr/K1ud3LEQQNYRmQNSOFkLG2DMCqjIqeeSROXeSODM=; b=Os1mNJ49LJJsB+l2C16/ZpM0Ex3j3ntam63H90/6VnGEeeD48YiP+kJJrwYy8Vs3WlkZyu wA+Tp5MKMPjfcwma1N21Jn6cuJngR5aY+73jXNIpqNTNL5hzF8JM5gjmTsC/nDbg79PDuR JSY72o9D9ulkl6hsLMlPmSc/YDM5CyO2X+1uyPf/DLprNhBhms4E5jQxBABm80IkpE0q/V R7w2oPluKdiiPzCTEwfGvAsjuK0CABp2SQt6HzJDb2ISvbIRoA/cwU2rNPpUMOfvZ333Uq FP4WaiiR59iRzquo+rdFBU1im9ooREtMMdy9Ipdtuj7hlJZR9CgqIGclSKHQNQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1730117969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mr/K1ud3LEQQNYRmQNSOFkLG2DMCqjIqeeSROXeSODM=; b=XiRmyz0Xgte/RVsCr5uj3U8iw9+EnfKe2TLaFL0Cn4y3hXXzf0DHiNBzD+ue3EVYkyPkil hkcfYBLnBrb/dzCA== To: linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Andr=C3=A9=20Almeida?= , Darren Hart , Davidlohr Bueso , Ingo Molnar , Juri Lelli , Peter Zijlstra , Valentin Schneider , Waiman Long , Sebastian Andrzej Siewior Subject: [RFC PATCH v2 2/4] futex: Add basic infrastructure for local task local hash. Date: Mon, 28 Oct 2024 13:13:56 +0100 Message-ID: <20241028121921.1264150-3-bigeasy@linutronix.de> In-Reply-To: <20241028121921.1264150-1-bigeasy@linutronix.de> References: <20241028121921.1264150-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The futex hashmap is system wide and shared by random tasks. Each slot is hashed based on its address and VMA. Due to randomized VMAs the same logical lock (pointer) can end up in a different hash bucket on each invocation of the application. This in turn means that different applications may share a hash bucket on each invocation and it is not always clear which applications will be involved. This can result in high latency's to acquire the futex_hash_bucket::lock especially if the lock owner is limited to a CPU and not be effectively PI boosted. Introduce a task local hash map. The hashmap can be allocated via prctl(PR_FUTEX_HASH, PR_FUTEX_HASH_ALLOCATE, 0) The `0' argument allocates a default number of 4 slots, a higher number can be specified if desired. The current uppoer limit is 16. The allocated hashmap is used by all threads within a process. A thread can check if the private map has been allocated via prctl(PR_FUTEX_HASH, PR_FUTEX_HASH_IS_SHARED); Which return the current number of slots. Signed-off-by: Sebastian Andrzej Siewior --- include/linux/futex.h | 7 ++++ include/linux/sched/signal.h | 4 +++ include/uapi/linux/prctl.h | 5 +++ kernel/fork.c | 1 + kernel/futex/core.c | 65 ++++++++++++++++++++++++++++++++++++ kernel/sys.c | 4 +++ 6 files changed, 86 insertions(+) diff --git a/include/linux/futex.h b/include/linux/futex.h index b70df27d7e85c..dad50173f70c4 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -77,6 +77,8 @@ void futex_exec_release(struct task_struct *tsk); =20 long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3); +int futex_hash_prctl(unsigned long arg2, unsigned long arg3, + unsigned long arg4, unsigned long arg5); #else static inline void futex_init_task(struct task_struct *tsk) { } static inline void futex_exit_recursive(struct task_struct *tsk) { } @@ -88,6 +90,11 @@ static inline long do_futex(u32 __user *uaddr, int op, u= 32 val, { return -EINVAL; } +static inline int futex_hash_prctl(unsigned long arg2, unsigned long arg3, + unsigned long arg4, unsigned long arg5) +{ + return -EINVAL; +} #endif =20 #endif diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h index c8ed09ac29ac5..3b8c8975cd493 100644 --- a/include/linux/sched/signal.h +++ b/include/linux/sched/signal.h @@ -14,6 +14,8 @@ #include #include =20 +struct futex_hash_bucket; + /* * Types defining task->signal and task->sighand and APIs using them: */ @@ -246,6 +248,8 @@ struct signal_struct { * and may have inconsistent * permissions. */ + unsigned int futex_hash_mask; + struct futex_hash_bucket *futex_hash_bucket; } __randomize_layout; =20 /* diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 35791791a879b..e912ce82de41f 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -328,4 +328,9 @@ struct prctl_mm_map { # define PR_PPC_DEXCR_CTRL_CLEAR_ONEXEC 0x10 /* Clear the aspect on exec */ # define PR_PPC_DEXCR_CTRL_MASK 0x1f =20 +/* FUTEX hash management */ +#define PR_FUTEX_HASH 74 +# define PR_FUTEX_HASH_ALLOCATE 1 +# define PR_FUTEX_HASH_IS_SHARED 2 + #endif /* _LINUX_PRCTL_H */ diff --git a/kernel/fork.c b/kernel/fork.c index 89ceb4a68af25..0d2b0a5299bbc 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -949,6 +949,7 @@ static inline void free_signal_struct(struct signal_str= uct *sig) { taskstats_tgid_free(sig); sched_autogroup_exit(sig); + kfree(sig->futex_hash_bucket); /* * __mmdrop is not safe to call from softirq context on x86 due to * pgd_dtor so postpone it to the async context diff --git a/kernel/futex/core.c b/kernel/futex/core.c index de6d7f71961eb..14e4cb5ccd722 100644 --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -39,6 +39,7 @@ #include #include #include +#include =20 #include "futex.h" #include "../locking/rtmutex_common.h" @@ -1153,6 +1154,70 @@ static void futex_hash_bucket_init(struct futex_hash= _bucket *fhb) spin_lock_init(&fhb->lock); } =20 +static int futex_hash_allocate(unsigned long arg3, unsigned long arg4, + unsigned long arg5) +{ + unsigned int hash_slots =3D arg3; + struct futex_hash_bucket *fhb; + int i; + + if (!thread_group_leader(current)) + return -EINVAL; + + if (current->signal->futex_hash_bucket) + return -EALREADY; + + if (hash_slots =3D=3D 0) + hash_slots =3D 4; + if (hash_slots < 2) + hash_slots =3D 2; + if (hash_slots > 16) + hash_slots =3D 16; + if (!is_power_of_2(hash_slots)) + hash_slots =3D rounddown_pow_of_two(hash_slots); + + fhb =3D kmalloc_array(hash_slots, sizeof(struct futex_hash_bucket), GFP_K= ERNEL); + if (!fhb) + return -ENOMEM; + + current->signal->futex_hash_mask =3D hash_slots - 1; + + for (i =3D 0; i < hash_slots; i++) + futex_hash_bucket_init(&fhb[i]); + + current->signal->futex_hash_bucket =3D fhb; + return 0; +} + +static int futex_hash_is_shared(unsigned long arg3, unsigned long arg4, + unsigned long arg5) +{ + if (current->signal->futex_hash_bucket) + return current->signal->futex_hash_mask + 1; + return 0; +} + +int futex_hash_prctl(unsigned long arg2, unsigned long arg3, + unsigned long arg4, unsigned long arg5) +{ + int ret; + + switch (arg2) { + case PR_FUTEX_HASH_ALLOCATE: + ret =3D futex_hash_allocate(arg3, arg4, arg5); + break; + + case PR_FUTEX_HASH_IS_SHARED: + ret =3D futex_hash_is_shared(arg3, arg4, arg5); + break; + + default: + ret =3D -EINVAL; + break; + } + return ret; +} + static int __init futex_init(void) { unsigned int futex_shift; diff --git a/kernel/sys.c b/kernel/sys.c index 4da31f28fda81..0dcbb8ce9f19d 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -52,6 +52,7 @@ #include #include #include +#include =20 #include #include @@ -2784,6 +2785,9 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, ar= g2, unsigned long, arg3, case PR_RISCV_SET_ICACHE_FLUSH_CTX: error =3D RISCV_SET_ICACHE_FLUSH_CTX(arg2, arg3); break; + case PR_FUTEX_HASH: + error =3D futex_hash_prctl(arg2, arg3, arg4, arg5); + break; default: error =3D -EINVAL; break; --=20 2.45.2 From nobody Sun Feb 8 20:45:17 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 523561D9660 for ; Mon, 28 Oct 2024 12:19:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730117973; cv=none; b=Tpcs3j0oTxufCvb+92p60+UCJNQMtQwIiaE4FsSWgTVZteOnY6e/XuYkU0zAZ7B9UEPWSmYl5SLS7JRfAGVtOIQHk1WYr0DJtDRqrDBenV7tb2IWviGXq7wmEQvHB/BeCwdffFvI1jzCpvP/FIGXe5JIC1jWEWxYDXVit9RPNHk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730117973; c=relaxed/simple; bh=5JdtUbK9JRTP/7wNU4slhmmPugwMyf/XIiP/M4WwZZo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pDJgBH8R+BzjWgemteyTdRYdXj5SkiZ4Z2b33adm0Uq88ikWRXjm+Jl0KUWx/wQAacM035VRBDttnWb8/ygeleAP44v1RB2B4C78BJENALUT9YY/Tcq78vFaA/mrQw0cQQhIAhUUCtmVtBkDSrHFP1TLq8Om1c/VyV457OM3It4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=YMUoDIvd; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=yoNTYwf4; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="YMUoDIvd"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="yoNTYwf4" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1730117969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oblyvANSZqruSqs+9bBfCTS3ayrNqKZAV/NQ1ZYwdp8=; b=YMUoDIvd4HsFu8ZDoDyXTWetTGAEe6T2FYlB32uJQOVqgXmrCOEj57uUdV6cC0pIyz/RYY TEAOV45fYbG+woDlvZBsa7uSOG4MmNyVMzP8XZtCABG13RYzkHbP3zFbyUi+io9P5Uf2jq sT1Vo7oLW7AbwIuW3O73IhhozZZ9S7yi9E4MAcSL9uf0euNrJRcf4jMA/sEonuSmpvcLw5 oM7oxr+iHKRnKLfXXRbtZKCQzHySwkc2rxcyZ8KLCnhd1N0mC4BfpwqS+JW5+vn6XTI0VE kjxegAg7La3yH/3ZP0DrzBNajDk6H8TN4vOgt1kMoMemHDe8hBz5jQ7akFWUzg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1730117969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oblyvANSZqruSqs+9bBfCTS3ayrNqKZAV/NQ1ZYwdp8=; b=yoNTYwf4FjKV0LjT2m8Hax2uFzr5w1RueUUwQFyS/bjilranO6eEEu6UIpJ9XxH8T1iBjF A8q40WyvDCEUVaDQ== To: linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Andr=C3=A9=20Almeida?= , Darren Hart , Davidlohr Bueso , Ingo Molnar , Juri Lelli , Peter Zijlstra , Valentin Schneider , Waiman Long , Sebastian Andrzej Siewior Subject: [RFC PATCH v2 3/4] futex: Use the task local hashmap. Date: Mon, 28 Oct 2024 13:13:57 +0100 Message-ID: <20241028121921.1264150-4-bigeasy@linutronix.de> In-Reply-To: <20241028121921.1264150-1-bigeasy@linutronix.de> References: <20241028121921.1264150-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the hashlocal hashmap if provided. Signed-off-by: Sebastian Andrzej Siewior --- kernel/futex/core.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/kernel/futex/core.c b/kernel/futex/core.c index 14e4cb5ccd722..3ef4cbd5cfa72 100644 --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -108,18 +108,33 @@ late_initcall(fail_futex_debugfs); =20 #endif /* CONFIG_FAIL_FUTEX */ =20 +static inline bool futex_key_is_private(union futex_key *key) +{ + /* + * Relies on get_futex_key() to set either bit for shared + * futexes -- see comment with union futex_key. + */ + return !(key->both.offset & (FUT_OFF_INODE | FUT_OFF_MMSHARED)); +} + /** * futex_hash - Return the hash bucket in the global hash * @key: Pointer to the futex key for which the hash is calculated * * We hash on the keys returned from get_futex_key (see below) and return = the - * corresponding hash bucket in the global hash. + * corresponding hash bucket in the global hash. If the FUTEX is private a= nd + * a local hash table is privated then this one is used. */ struct futex_hash_bucket *futex_hash(union futex_key *key) { + struct futex_hash_bucket *fhb; u32 hash =3D jhash2((u32 *)key, offsetof(typeof(*key), both.offset) / 4, key->both.offset); =20 + fhb =3D current->signal->futex_hash_bucket; + if (fhb && futex_key_is_private(key)) + return &fhb[hash & current->signal->futex_hash_mask]; + return &futex_queues[hash & (futex_hashsize - 1)]; } =20 --=20 2.45.2 From nobody Sun Feb 8 20:45:17 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 718CF1D7982 for ; Mon, 28 Oct 2024 12:19:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730117973; cv=none; b=apqpVFa/wjAruB4YVYq4R6KNZHNyBEt0OWzayKiEhOqkm66z45Gh38wyP+hwQkLLXDbSjw6WMj0VreLC98bBVrJUOrdyH1D10n4fAhnnGp3n+Oh+HP+x6pqzcREHmxVdIv2Xy0wR5MVdKqOhXstLwJItCg/XXAB8ruvheJAz5C0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730117973; c=relaxed/simple; bh=jk2kAjhswdvBih6es6xNEVf85QmUfesQHzKT7xwG7U0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IYzvd+kiI4LXn/OZ3Bm/P6aZ/hk0Oj998GlbUO4Z1PTOi1L96zn2dXpiTMp3RVWXhBc1PRb+aGnD7wttAlOlzNvxA0VQmQC+xx2otNjir51GUo2JADACqMOtfdXpZP34nBwVd5GD8bvaEDVdFUuQgoikrTHXUrlHXAPQlVscjyM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=AYg5sqSw; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=v2YWKZo2; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="AYg5sqSw"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="v2YWKZo2" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1730117969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5FAWWfDjSfgC8ZGoqMvtbJfesZHFbNMw6hYb8tQ1YbY=; b=AYg5sqSwo+/at9RqtCPh9BQM79djhDHxQROa1f54zUDxuWhYDvGYR3vYP0kyxsTcgx9BP6 m4JB7EUFaLz14lOYOZrsta0UfE6r5PDVhhbUyYhZ9QHfXgFDF4OGAojZcJ+6EcriBdXwCb a5UOvu3CjlBLVsgZcu5vwH8rpJPPw9F4l4sOs2RUubgJEPMgXJjgxsQqossh1n8HrGGlJP 9i06UMXPblhWCUe37fE1r7cJkfCBDBtb81s9anxrBfnR+JsXsX7tzNwSmFZbTrHcevfZ40 TbURT0vo9KNsycblYlkHIp8GdDA8u2yrxiZRVClVjil9VZY0/xI0MfYk+rV9rw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1730117969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5FAWWfDjSfgC8ZGoqMvtbJfesZHFbNMw6hYb8tQ1YbY=; b=v2YWKZo2FF/lYPBdN5JOgIuaZNAbrzBiRmqcZQuxJKBAk+GqVx5Ctig8OyBNpkR6BRM4ws WIEm+wecYWF7zXAA== To: linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Andr=C3=A9=20Almeida?= , Darren Hart , Davidlohr Bueso , Ingo Molnar , Juri Lelli , Peter Zijlstra , Valentin Schneider , Waiman Long , Sebastian Andrzej Siewior Subject: [RFC PATCH v2 4/4] futex: Allow automatic allocation of process wide futex hash. Date: Mon, 28 Oct 2024 13:13:58 +0100 Message-ID: <20241028121921.1264150-5-bigeasy@linutronix.de> In-Reply-To: <20241028121921.1264150-1-bigeasy@linutronix.de> References: <20241028121921.1264150-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allocate a default futex hash if a task forks its first thread. Signed-off-by: Sebastian Andrzej Siewior --- include/linux/futex.h | 6 ++++++ kernel/fork.c | 28 ++++++++++++++++++++++++++++ kernel/futex/core.c | 5 +++++ 3 files changed, 39 insertions(+) diff --git a/include/linux/futex.h b/include/linux/futex.h index dad50173f70c4..c0f90dda6a295 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -79,6 +79,7 @@ long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t= *timeout, u32 __user *uaddr2, u32 val2, u32 val3); int futex_hash_prctl(unsigned long arg2, unsigned long arg3, unsigned long arg4, unsigned long arg5); +int futex_hash_allocate_default(void); #else static inline void futex_init_task(struct task_struct *tsk) { } static inline void futex_exit_recursive(struct task_struct *tsk) { } @@ -95,6 +96,11 @@ static inline int futex_hash_prctl(unsigned long arg2, u= nsigned long arg3, { return -EINVAL; } +static inline int futex_hash_allocate_default(void) +{ + return 0; +} + #endif =20 #endif diff --git a/kernel/fork.c b/kernel/fork.c index 0d2b0a5299bbc..21dccdc8a1f6c 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -2107,6 +2107,19 @@ static void rv_task_fork(struct task_struct *p) #define rv_task_fork(p) do {} while (0) #endif =20 +static bool need_futex_hash_allocate_default(u64 clone_flags) +{ + if ((clone_flags & (CLONE_THREAD | CLONE_VM)) !=3D (CLONE_THREAD | CLONE_= VM)) + return false; + if (!thread_group_leader(current)) + return false; + if (current->signal->nr_threads !=3D 1) + return false; + if (current->signal->futex_hash_bucket) + return false; + return true; +} + /* * This creates a new process as a copy of the old one, * but does not actually start it yet. @@ -2483,6 +2496,21 @@ __latent_entropy struct task_struct *copy_process( if (retval) goto bad_fork_cancel_cgroup; =20 + /* + * Allocate a default futex hash for the user process once the first + * thread spawns. + */ + if (need_futex_hash_allocate_default(clone_flags)) { + retval =3D futex_hash_allocate_default(); + if (retval) + goto bad_fork_core_free; + /* + * If we fail beyond this point we don't free the allocated + * futex hash map. We assume that another thread will created + * and makes use of it The hash map will be freed once the main + * thread terminates. + */ + } /* * From this point on we must avoid any synchronous user-space * communication until we take the tasklist-lock. In particular, we do diff --git a/kernel/futex/core.c b/kernel/futex/core.c index 3ef4cbd5cfa72..8896ade418b4a 100644 --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -1204,6 +1204,11 @@ static int futex_hash_allocate(unsigned long arg3, u= nsigned long arg4, return 0; } =20 +int futex_hash_allocate_default(void) +{ + return futex_hash_allocate(0, 0, 0); +} + static int futex_hash_is_shared(unsigned long arg3, unsigned long arg4, unsigned long arg5) { --=20 2.45.2