From nobody Thu Apr 9 21:52:26 2026 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 143BB3ECBC9 for ; Thu, 5 Mar 2026 19:55:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772740557; cv=none; b=V/U68Kg+tOyKP2Df29oazDakfS/BQBjEkAzjuWMn1Rnpv8q27lOSQURW+6PcWH47g4/q7HIdWQ5nOo/k8Y64HdYr2ZLHxLdWFlamPk0oFYk2GnepCXVEl/yGFn5hOp4o+c036of+ew4eqHgAXtRmGweJpPSyetMSojXfufsrp4k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772740557; c=relaxed/simple; bh=/lZeqjqQZax3bUcO84V9r/my8kQ4QKbCXhEmGg8Cw4M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qNgx9CG1kKSH+LY55SOKTxxjj648+f6KkSoLt67a05sMGI8YkIcznQyN8E7PLKJyq4coHN9ZJUrkgEUM093zXzmg6iubOgkG6uEEd8g4ToqSuiFfBRr4cEAI+goxrUqZ3NRzZncQIhJPH8NVg4iN6dXKc4amt0niWASYr3pqIpc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=OxC9nrzk; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="OxC9nrzk" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=jG2jyA9BuQlTAXqpUX1aWUfapy2uCq+TR/xn+DRwXC0=; b=OxC9nrzkXtBvr2Fh+d6PLSCxSg 5ORfgj76VdPhbt5EF2sQcIDGKN6duiIiBPgMzB1oINDmElWkojXvYlO52aj+Mtlm8XkzB3tWNvELV tr2TfR+tYkpHJ1Hb6v2gjOJHGlP9MQDSfGZAKayNh7NOrzidaklzNMHS54lHbTTAH+E5vgpvbT6lD +N+YX5o2TgB6WrUjnMtb1HLsVOLgH2yHAH6GbP3NCCFQXt/9Br6ralmsAjkhbWy7HFXRefal0quSP mWDqb41zcbrFicspEDN2YdCnmb7PkR5Mvayy7MApNIGtrbNga/8WpzDRmXFmaOQMIRGA9Vxyh/gLF /GD+cRGw==; Received: from willy by casper.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1vyEnT-0000000FYWE-3xjd; Thu, 05 Mar 2026 19:55:47 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra Cc: "Matthew Wilcox (Oracle)" , Ingo Molnar , Will Deacon , Boqun Feng , Waiman Long , linux-kernel@vger.kernel.org Subject: [PATCH 2/3] semaphore: Remove the list_head from struct semaphore Date: Thu, 5 Mar 2026 19:55:42 +0000 Message-ID: <20260305195545.3707590-3-willy@infradead.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260305195545.3707590-1-willy@infradead.org> References: <20260305195545.3707590-1-willy@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Instead of embedding a list_head in struct semaphore, store a pointer to the first waiter. The list of waiters remains a doubly linked list so we can efficiently add to the tail of the list and remove from the front (or middle) of the list. Some of the list manipulation becomes more complicated, but it's a reasonable tradeoff on the slow paths to shrink data structures which embed a semaphore. Signed-off-by: Matthew Wilcox (Oracle) --- drivers/acpi/osl.c | 2 +- include/linux/semaphore.h | 4 ++-- kernel/locking/semaphore.c | 41 ++++++++++++++++++++++++++++---------- 3 files changed, 34 insertions(+), 13 deletions(-) diff --git a/drivers/acpi/osl.c b/drivers/acpi/osl.c index 05393a7315fe..782dd93b891e 100644 --- a/drivers/acpi/osl.c +++ b/drivers/acpi/osl.c @@ -1257,7 +1257,7 @@ acpi_status acpi_os_delete_semaphore(acpi_handle hand= le) =20 ACPI_DEBUG_PRINT((ACPI_DB_MUTEX, "Deleting semaphore[%p].\n", handle)); =20 - BUG_ON(!list_empty(&sem->wait_list)); + BUG_ON(sem->first_waiter); kfree(sem); sem =3D NULL; =20 diff --git a/include/linux/semaphore.h b/include/linux/semaphore.h index 89706157e622..a4c8651ef021 100644 --- a/include/linux/semaphore.h +++ b/include/linux/semaphore.h @@ -15,7 +15,7 @@ struct semaphore { raw_spinlock_t lock; unsigned int count; - struct list_head wait_list; + struct semaphore_waiter *first_waiter; =20 #ifdef CONFIG_DETECT_HUNG_TASK_BLOCKER unsigned long last_holder; @@ -33,7 +33,7 @@ struct semaphore { { \ .lock =3D __RAW_SPIN_LOCK_UNLOCKED((name).lock), \ .count =3D n, \ - .wait_list =3D LIST_HEAD_INIT((name).wait_list) \ + .first_waiter =3D NULL \ __LAST_HOLDER_SEMAPHORE_INITIALIZER \ } =20 diff --git a/kernel/locking/semaphore.c b/kernel/locking/semaphore.c index 3ef032e22f7e..cb9eae819e64 100644 --- a/kernel/locking/semaphore.c +++ b/kernel/locking/semaphore.c @@ -21,7 +21,7 @@ * too. * * The ->count variable represents how many more tasks can acquire this - * semaphore. If it's zero, there may be tasks waiting on the wait_list. + * semaphore. If it's zero, there may be waiters. */ =20 #include @@ -226,7 +226,7 @@ void __sched up(struct semaphore *sem) =20 hung_task_sem_clear_if_holder(sem); =20 - if (likely(list_empty(&sem->wait_list))) + if (likely(!sem->first_waiter)) sem->count++; else __up(sem, &wake_q); @@ -244,6 +244,21 @@ struct semaphore_waiter { bool up; }; =20 +static inline +void sem_del_waiter(struct semaphore *sem, struct semaphore_waiter *waiter) +{ + if (list_empty(&waiter->list)) { + sem->first_waiter =3D NULL; + return; + } + + if (sem->first_waiter =3D=3D waiter) { + sem->first_waiter =3D list_first_entry(&waiter->list, + struct semaphore_waiter, list); + } + list_del(&waiter->list); +} + /* * Because this function is inlined, the 'state' parameter will be * constant, and thus optimised away by the compiler. Likewise the @@ -252,9 +267,15 @@ struct semaphore_waiter { static inline int __sched ___down_common(struct semaphore *sem, long state, long timeout) { - struct semaphore_waiter waiter; - - list_add_tail(&waiter.list, &sem->wait_list); + struct semaphore_waiter waiter, *first; + + first =3D sem->first_waiter; + if (first) { + list_add_tail(&waiter.list, &first->list); + } else { + INIT_LIST_HEAD(&waiter.list); + sem->first_waiter =3D &waiter; + } waiter.task =3D current; waiter.up =3D false; =20 @@ -274,11 +295,11 @@ static inline int __sched ___down_common(struct semap= hore *sem, long state, } =20 timed_out: - list_del(&waiter.list); + sem_del_waiter(sem, &waiter); return -ETIME; =20 interrupted: - list_del(&waiter.list); + sem_del_waiter(sem, &waiter); return -EINTR; } =20 @@ -321,9 +342,9 @@ static noinline int __sched __down_timeout(struct semap= hore *sem, long timeout) static noinline void __sched __up(struct semaphore *sem, struct wake_q_head *wake_q) { - struct semaphore_waiter *waiter =3D list_first_entry(&sem->wait_list, - struct semaphore_waiter, list); - list_del(&waiter->list); + struct semaphore_waiter *waiter =3D sem->first_waiter; + + sem_del_waiter(sem, waiter); waiter->up =3D true; wake_q_add(wake_q, waiter->task); } --=20 2.47.3