From nobody Fri Nov 29 03:55:56 2024 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1F05155732 for ; Wed, 25 Sep 2024 05:34:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242477; cv=none; b=Q1a7WoJ3P8OReGqEs20sh5HwkOmmOfcw439GaYfmBaIfjCCeWWhuP+/BNWLcNb4XHSQuvH0hQQBlZF32uqKNe6AScO4yjBHH8u43Xj5OK9ezDdcjdFfaLGFo9y5Y6Rs2kMvVWRDl/d0xHyaRe8lkUuxBp7v1gjnc7tLQLZ/hzx4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242477; c=relaxed/simple; bh=r3sxpMMlRKaoiiDUtmPbRteoeqVWw1zcEr5ctVzUO9U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Go6S+ZNaphcTBAdhqMMfuud1nJtJcXz98GYwipX7lhy+6r/6qoQAVBeYz22x+WOHcqeO9NY9MEtucJoH3S0goJVORod3M5Gs//1YoMAYIbOCvvC32wb4SVxCtmVZYE5eZ3nMa5AHp1uoEijGlIQYmlDLvKXPNGjWB47FoD8DUFY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=IfEcE97H; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=c9BDKYbU; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=IfEcE97H; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=c9BDKYbU; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="IfEcE97H"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="c9BDKYbU"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="IfEcE97H"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="c9BDKYbU" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id CF82A1FD30; Wed, 25 Sep 2024 05:34:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242467; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jto+WAjJOCZLH4emYxzPTw5OFHfbNqohY3iXFmMl6WU=; b=IfEcE97HG/d6WokmYODXmSPcjTCWNLQWIbxBJfszRzDii0jtZy+M8aTexOx4zIzaPOER+h 9Qm3Qhz4FypVWrifsZHvmKJ/C+kb5h/8brz1gbEpYxHXbws0dM8EXmflBIqLKQmfvtgRiS PLLUG7Zq4J1/jky236s85NtOjkyHfI8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242467; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jto+WAjJOCZLH4emYxzPTw5OFHfbNqohY3iXFmMl6WU=; b=c9BDKYbUGH/I1eP3oxKJaf6FV/3gZbPO4RUnlAtcggXCajbvY5guA9jqmo9Je53VUdV88z 2yIaSWfuaYbGqNBA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242467; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jto+WAjJOCZLH4emYxzPTw5OFHfbNqohY3iXFmMl6WU=; b=IfEcE97HG/d6WokmYODXmSPcjTCWNLQWIbxBJfszRzDii0jtZy+M8aTexOx4zIzaPOER+h 9Qm3Qhz4FypVWrifsZHvmKJ/C+kb5h/8brz1gbEpYxHXbws0dM8EXmflBIqLKQmfvtgRiS PLLUG7Zq4J1/jky236s85NtOjkyHfI8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242467; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jto+WAjJOCZLH4emYxzPTw5OFHfbNqohY3iXFmMl6WU=; b=c9BDKYbUGH/I1eP3oxKJaf6FV/3gZbPO4RUnlAtcggXCajbvY5guA9jqmo9Je53VUdV88z 2yIaSWfuaYbGqNBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4A5C413A66; Wed, 25 Sep 2024 05:34:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id V1KOAOKg82YiUQAAD6G6ig (envelope-from ); Wed, 25 Sep 2024 05:34:26 +0000 From: NeilBrown To: Ingo Molnar , Peter Zijlstra , Linus Torvalds Cc: linux-kernel@vger.kernel.org Subject: [PATCH 1/8] block: change wait on bd_claiming to use a var_waitqueue, not a bit_waitqueue Date: Wed, 25 Sep 2024 15:24:04 +1000 Message-ID: <20240925053118.3956520-2-neilb@suse.de> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240925053118.3956520-1-neilb@suse.de> References: <20240925053118.3956520-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Level: X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid,imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 X-Spam-Flag: NO Content-Type: text/plain; charset="utf-8" bd_prepare_to_claim() waits for a var to change, not for a bit to be cleared. So change from bit_waitqueue() to __var_waitqueue() and correspondingly use wake_up_var(). This will allow a future patch which change the "bit" function to expect an "unsigned long *" instead of "void *". Signed-off-by: NeilBrown --- block/bdev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/bdev.c b/block/bdev.c index 33f9c4605e3a..738e3c8457e7 100644 --- a/block/bdev.c +++ b/block/bdev.c @@ -555,7 +555,7 @@ int bd_prepare_to_claim(struct block_device *bdev, void= *holder, =20 /* if claiming is already in progress, wait for it to finish */ if (whole->bd_claiming) { - wait_queue_head_t *wq =3D bit_waitqueue(&whole->bd_claiming, 0); + wait_queue_head_t *wq =3D __var_waitqueue(&whole->bd_claiming); DEFINE_WAIT(wait); =20 prepare_to_wait(wq, &wait, TASK_UNINTERRUPTIBLE); @@ -578,7 +578,7 @@ static void bd_clear_claiming(struct block_device *whol= e, void *holder) /* tell others that we're done */ BUG_ON(whole->bd_claiming !=3D holder); whole->bd_claiming =3D NULL; - wake_up_bit(&whole->bd_claiming, 0); + wake_up_var(&whole->bd_claiming); } =20 /** --=20 2.46.0 From nobody Fri Nov 29 03:55:56 2024 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B77A714D29C for ; Wed, 25 Sep 2024 05:34:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242480; cv=none; b=ZN51GYfUYA/BfQDRM9+01FJWT2OIDLbigALKwiAScXLfR+InzElMuO/LWbyTDMqyAyympD8sb3HsNyDnVYHP+kKL0614JMorqnNVYVRT89Mhbf2ciV/690y8SZorufrVTXbyYEHFFVZOt5E85Efs23YVUhAZ7oJsjuYHomw6POE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242480; c=relaxed/simple; bh=UXxvTx1TeCLXwdn7Ve5EZeCAjiffj2i3+zPa9jKyQ+M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FxqgWw81dZKpeC8TmecGyM4/tXgniFHScuMa5dagIOLgmSRZrwVay4uuBoUn4WwhzR+5V0Jggf7r6NwfKQpsVOPbbg+Y9/1+LOp2YnWlwfmGCNjJmbPhB9LH24MCj+cwyJpisFmSDYXd7/ptv0DNgBFgNru7Gzh2OAskVltqFco= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=WaOTmgiu; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=Fu7T08bn; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=WaOTmgiu; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=Fu7T08bn; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="WaOTmgiu"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="Fu7T08bn"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="WaOTmgiu"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="Fu7T08bn" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id B15D91FD32; Wed, 25 Sep 2024 05:34:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242476; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WGE3jRZ/Xz3OFuyHWXitti4fsw2duTAvaS4bioTEIn0=; b=WaOTmgiut9PAdvUgTeQelc1xfee/wuA8oD+OX7RxmRHIqrS6GE3bHCx1TSPDzDKzz8sZpZ lkTT96S81TWFoNZyMzdkoWyhfe1jSibhPln6pWRzUrhMKtUH0SmOJisa/OzunFWDjp3rxa 3WUDZukSwbsW3vg602DmTdOZn5Zy8L4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242476; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WGE3jRZ/Xz3OFuyHWXitti4fsw2duTAvaS4bioTEIn0=; b=Fu7T08bnkmjKIlbnz2C96yTMjDjuYGpUzX4ZKeKsdeLCRgKcqYAfG1KgSLy8YRWbiOtqUE 0GDVG8p5p1OTlaBA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242476; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WGE3jRZ/Xz3OFuyHWXitti4fsw2duTAvaS4bioTEIn0=; b=WaOTmgiut9PAdvUgTeQelc1xfee/wuA8oD+OX7RxmRHIqrS6GE3bHCx1TSPDzDKzz8sZpZ lkTT96S81TWFoNZyMzdkoWyhfe1jSibhPln6pWRzUrhMKtUH0SmOJisa/OzunFWDjp3rxa 3WUDZukSwbsW3vg602DmTdOZn5Zy8L4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242476; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WGE3jRZ/Xz3OFuyHWXitti4fsw2duTAvaS4bioTEIn0=; b=Fu7T08bnkmjKIlbnz2C96yTMjDjuYGpUzX4ZKeKsdeLCRgKcqYAfG1KgSLy8YRWbiOtqUE 0GDVG8p5p1OTlaBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 36EA313A66; Wed, 25 Sep 2024 05:34:34 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 6JloN+qg82YrUQAAD6G6ig (envelope-from ); Wed, 25 Sep 2024 05:34:34 +0000 From: NeilBrown To: Ingo Molnar , Peter Zijlstra , Linus Torvalds Cc: linux-kernel@vger.kernel.org Subject: [PATCH 2/8] sched: change wake_up_bit() and related function to expect unsigned long * Date: Wed, 25 Sep 2024 15:24:05 +1000 Message-ID: <20240925053118.3956520-3-neilb@suse.de> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240925053118.3956520-1-neilb@suse.de> References: <20240925053118.3956520-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Level: X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid,imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 X-Spam-Flag: NO Content-Type: text/plain; charset="utf-8" wake_up_bit() currently allows a "void *". While this isn't strictly a problem as the address is never dereferenced, it is inconsistent with the corresponding wait_on_bit() which requires "unsigned long *" and does dereference the pointer. Any code that needs to wait for a change in something other than an unsigned long would be better served by wake_up_var()/wait_var_event(). This patch changes all related "void *" to "unsigned long *". Reported-by: Linus Torvalds Signed-off-by: NeilBrown --- include/linux/wait_bit.h | 16 ++++++++-------- kernel/sched/wait_bit.c | 12 ++++++------ 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 7725b7579b78..48e123839892 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -8,7 +8,7 @@ #include =20 struct wait_bit_key { - void *flags; + unsigned long *flags; int bit_nr; unsigned long timeout; }; @@ -23,14 +23,14 @@ struct wait_bit_queue_entry { =20 typedef int wait_bit_action_f(struct wait_bit_key *key, int mode); =20 -void __wake_up_bit(struct wait_queue_head *wq_head, void *word, int bit); +void __wake_up_bit(struct wait_queue_head *wq_head, unsigned long *word, i= nt bit); int __wait_on_bit(struct wait_queue_head *wq_head, struct wait_bit_queue_e= ntry *wbq_entry, wait_bit_action_f *action, unsigned int mode); int __wait_on_bit_lock(struct wait_queue_head *wq_head, struct wait_bit_qu= eue_entry *wbq_entry, wait_bit_action_f *action, unsigned int mode); -void wake_up_bit(void *word, int bit); -int out_of_line_wait_on_bit(void *word, int, wait_bit_action_f *action, un= signed int mode); -int out_of_line_wait_on_bit_timeout(void *word, int, wait_bit_action_f *ac= tion, unsigned int mode, unsigned long timeout); -int out_of_line_wait_on_bit_lock(void *word, int, wait_bit_action_f *actio= n, unsigned int mode); -struct wait_queue_head *bit_waitqueue(void *word, int bit); +void wake_up_bit(unsigned long *word, int bit); +int out_of_line_wait_on_bit(unsigned long *word, int, wait_bit_action_f *a= ction, unsigned int mode); +int out_of_line_wait_on_bit_timeout(unsigned long *word, int, wait_bit_act= ion_f *action, unsigned int mode, unsigned long timeout); +int out_of_line_wait_on_bit_lock(unsigned long *word, int, wait_bit_action= _f *action, unsigned int mode); +struct wait_queue_head *bit_waitqueue(unsigned long *word, int bit); extern void __init wait_bit_init(void); =20 int wake_bit_function(struct wait_queue_entry *wq_entry, unsigned mode, in= t sync, void *key); @@ -327,7 +327,7 @@ do { \ * You can use this helper if bitflags are manipulated atomically rather t= han * non-atomically under a lock. */ -static inline void clear_and_wake_up_bit(int bit, void *word) +static inline void clear_and_wake_up_bit(int bit, unsigned long *word) { clear_bit_unlock(bit, word); /* See wake_up_bit() for which memory barrier you need to use. */ diff --git a/kernel/sched/wait_bit.c b/kernel/sched/wait_bit.c index 134d7112ef71..058b0e18727e 100644 --- a/kernel/sched/wait_bit.c +++ b/kernel/sched/wait_bit.c @@ -9,7 +9,7 @@ =20 static wait_queue_head_t bit_wait_table[WAIT_TABLE_SIZE] __cacheline_align= ed; =20 -wait_queue_head_t *bit_waitqueue(void *word, int bit) +wait_queue_head_t *bit_waitqueue(unsigned long *word, int bit) { const int shift =3D BITS_PER_LONG =3D=3D 32 ? 5 : 6; unsigned long val =3D (unsigned long)word << shift | bit; @@ -55,7 +55,7 @@ __wait_on_bit(struct wait_queue_head *wq_head, struct wai= t_bit_queue_entry *wbq_ } EXPORT_SYMBOL(__wait_on_bit); =20 -int __sched out_of_line_wait_on_bit(void *word, int bit, +int __sched out_of_line_wait_on_bit(unsigned long *word, int bit, wait_bit_action_f *action, unsigned mode) { struct wait_queue_head *wq_head =3D bit_waitqueue(word, bit); @@ -66,7 +66,7 @@ int __sched out_of_line_wait_on_bit(void *word, int bit, EXPORT_SYMBOL(out_of_line_wait_on_bit); =20 int __sched out_of_line_wait_on_bit_timeout( - void *word, int bit, wait_bit_action_f *action, + unsigned long *word, int bit, wait_bit_action_f *action, unsigned mode, unsigned long timeout) { struct wait_queue_head *wq_head =3D bit_waitqueue(word, bit); @@ -108,7 +108,7 @@ __wait_on_bit_lock(struct wait_queue_head *wq_head, str= uct wait_bit_queue_entry } EXPORT_SYMBOL(__wait_on_bit_lock); =20 -int __sched out_of_line_wait_on_bit_lock(void *word, int bit, +int __sched out_of_line_wait_on_bit_lock(unsigned long *word, int bit, wait_bit_action_f *action, unsigned mode) { struct wait_queue_head *wq_head =3D bit_waitqueue(word, bit); @@ -118,7 +118,7 @@ int __sched out_of_line_wait_on_bit_lock(void *word, in= t bit, } EXPORT_SYMBOL(out_of_line_wait_on_bit_lock); =20 -void __wake_up_bit(struct wait_queue_head *wq_head, void *word, int bit) +void __wake_up_bit(struct wait_queue_head *wq_head, unsigned long *word, i= nt bit) { struct wait_bit_key key =3D __WAIT_BIT_KEY_INITIALIZER(word, bit); =20 @@ -144,7 +144,7 @@ EXPORT_SYMBOL(__wake_up_bit); * may need to use a less regular barrier, such fs/inode.c's smp_mb(), * because spin_unlock() does not guarantee a memory barrier. */ -void wake_up_bit(void *word, int bit) +void wake_up_bit(unsigned long *word, int bit) { __wake_up_bit(bit_waitqueue(word, bit), word, bit); } --=20 2.46.0 From nobody Fri Nov 29 03:55:56 2024 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCBC0149C4F for ; Wed, 25 Sep 2024 05:34:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242494; cv=none; b=XEUqGWzL4g4e2Q+6Ifn4RU+OObJkRDAm8dDs48uaWVqxNGAbaxpW7aWqjhPi3sutY0D/4YaaZ0AgICU1XMBrlOnjQlqgbXhVruVRhdbyy2FSkOMjFbeVlKZ3W8ni0Okg0PpvtIqkej79t4eQ2ZXsAqFl7Ef8Gr2W3TqEgr1xDkw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242494; c=relaxed/simple; bh=5G8drIKcKc9eVEXrKmaP6lcL1lKGSHzxebmMrtAz1EY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ER4ylbYWisNkPTpyUbQpdRxmpIRLe6xv2aUKsL/JiMK4wMovU7tE7KdYCFs7lmQ/tHUuJKqJqkgNaBQvrVuZHlBhIPoMFetT+E74RauLpFWo6dhxMZf5W0BhEI4gk/VM9AEFv5S02f9JInBYnC8e60F7YLAGi1bQYX4XeLTHVBE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=QOmsihOI; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=yLu+JxSu; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=QOmsihOI; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=yLu+JxSu; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="QOmsihOI"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="yLu+JxSu"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="QOmsihOI"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="yLu+JxSu" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0E47E21A6A; Wed, 25 Sep 2024 05:34:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242490; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vNdiD1XMKNrft2fFnwEWO+4kNLwMK+cCeQRN0tFRGyQ=; b=QOmsihOIvpSGd6q2vgJZ9WPecotlbDMPvRgsEzq/5gQMSULLIMkaezUE3COYxv7FAhwewW 44mwisAloE5cUJ4EOcg3tlgYUPkw+WMrIakS20K2bG8vL8iwBL09tzMQvuZ54sE/OU3FtQ Pfo9G7QbkgzrY81gfFh/IlhZgsaUY3A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242490; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vNdiD1XMKNrft2fFnwEWO+4kNLwMK+cCeQRN0tFRGyQ=; b=yLu+JxSujyTv6JWQA5h0xhYJ3eU3k0NncAUjaXkBWa8QftPQsjrFE6UM9nXcLtSR+HZ2ee YXb05FYJ/aFzkJAw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=QOmsihOI; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=yLu+JxSu DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242490; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vNdiD1XMKNrft2fFnwEWO+4kNLwMK+cCeQRN0tFRGyQ=; b=QOmsihOIvpSGd6q2vgJZ9WPecotlbDMPvRgsEzq/5gQMSULLIMkaezUE3COYxv7FAhwewW 44mwisAloE5cUJ4EOcg3tlgYUPkw+WMrIakS20K2bG8vL8iwBL09tzMQvuZ54sE/OU3FtQ Pfo9G7QbkgzrY81gfFh/IlhZgsaUY3A= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242490; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vNdiD1XMKNrft2fFnwEWO+4kNLwMK+cCeQRN0tFRGyQ=; b=yLu+JxSujyTv6JWQA5h0xhYJ3eU3k0NncAUjaXkBWa8QftPQsjrFE6UM9nXcLtSR+HZ2ee YXb05FYJ/aFzkJAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3436813A66; Wed, 25 Sep 2024 05:34:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id R4WyNveg82ZCUQAAD6G6ig (envelope-from ); Wed, 25 Sep 2024 05:34:47 +0000 From: NeilBrown To: Ingo Molnar , Peter Zijlstra , Linus Torvalds Cc: linux-kernel@vger.kernel.org Subject: [PATCH 3/8] sched: Improve documentation for wake_up_bit/wait_on_bit family of functions Date: Wed, 25 Sep 2024 15:24:06 +1000 Message-ID: <20240925053118.3956520-4-neilb@suse.de> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240925053118.3956520-1-neilb@suse.de> References: <20240925053118.3956520-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 0E47E21A6A X-Spam-Level: X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:dkim,suse.de:mid,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 X-Spam-Flag: NO Content-Type: text/plain; charset="utf-8" This patch revises the documention for wake_up_bit(), clear_and_wake_up_bit(), and all the wait_on_bit() family of functions. The new documentation places less emphasis on the pool of waitqueues used (an implementation detail) and focuses instead on details of how the functions behave. The barriers included in the wait functions and clear_and_wake_up_bit() and those required for wake_up_bit() are spelled out more clearly. The error statuses returned are given explicitly. The fact that the wait_on_bit_lock() function sets the bit is made more obvious. Signed-off-by: NeilBrown --- include/linux/wait_bit.h | 159 +++++++++++++++++++++------------------ kernel/sched/wait_bit.c | 34 +++++---- 2 files changed, 107 insertions(+), 86 deletions(-) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 48e123839892..723e7bf35747 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -53,19 +53,21 @@ extern int bit_wait_io_timeout(struct wait_bit_key *key= , int mode); =20 /** * wait_on_bit - wait for a bit to be cleared - * @word: the word being waited on, a kernel virtual address - * @bit: the bit of the word being waited on + * @word: the address containing the bit being waited on + * @bit: the bit at that address being waited on * @mode: the task state to sleep in * - * There is a standard hashed waitqueue table for generic use. This - * is the part of the hashtable's accessor API that waits on a bit. - * For instance, if one were to have waiters on a bitflag, one would - * call wait_on_bit() in threads waiting for the bit to clear. - * One uses wait_on_bit() where one is waiting for the bit to clear, - * but has no intention of setting it. - * Returned value will be zero if the bit was cleared, or non-zero - * if the process received a signal and the mode permitted wakeup - * on that signal. + * Wait for the given bit in an unsigned long or bitmap (see DECLARE_BITMA= P()) + * to be cleared. The clearing of the bit must be signalled with + * wake_up_bit(), often as clear_and_wake_up_bit(). + * + * The process will wait on a waitqueue selected by hash from a shared + * pool. It will only be woken on a wake_up for the target bit, even + * if other processes on the same queue are waiting for other bits. + * + * Returned value will be zero if the bit was cleared in which case the + * call has ACQUIRE semantics, or %-EINTR if the process received a + * signal and the mode permitted wake up on that signal. */ static inline int wait_on_bit(unsigned long *word, int bit, unsigned mode) @@ -80,17 +82,20 @@ wait_on_bit(unsigned long *word, int bit, unsigned mode) =20 /** * wait_on_bit_io - wait for a bit to be cleared - * @word: the word being waited on, a kernel virtual address - * @bit: the bit of the word being waited on + * @word: the address containing the bit being waited on + * @bit: the bit at that address being waited on * @mode: the task state to sleep in * - * Use the standard hashed waitqueue table to wait for a bit - * to be cleared. This is similar to wait_on_bit(), but calls - * io_schedule() instead of schedule() for the actual waiting. + * Wait for the given bit in an unsigned long or bitmap (see DECLARE_BITMA= P()) + * to be cleared. The clearing of the bit must be signalled with + * wake_up_bit(), often as clear_and_wake_up_bit(). + * + * This is similar to wait_on_bit(), but calls io_schedule() instead of + * schedule() for the actual waiting. * - * Returned value will be zero if the bit was cleared, or non-zero - * if the process received a signal and the mode permitted wakeup - * on that signal. + * Returned value will be zero if the bit was cleared in which case the + * call has ACQUIRE semantics, or %-EINTR if the process received a + * signal and the mode permitted wake up on that signal. */ static inline int wait_on_bit_io(unsigned long *word, int bit, unsigned mode) @@ -104,19 +109,24 @@ wait_on_bit_io(unsigned long *word, int bit, unsigned= mode) } =20 /** - * wait_on_bit_timeout - wait for a bit to be cleared or a timeout elapses - * @word: the word being waited on, a kernel virtual address - * @bit: the bit of the word being waited on + * wait_on_bit_timeout - wait for a bit to be cleared or a timeout to elap= se + * @word: the address containing the bit being waited on + * @bit: the bit at that address being waited on * @mode: the task state to sleep in * @timeout: timeout, in jiffies * - * Use the standard hashed waitqueue table to wait for a bit - * to be cleared. This is similar to wait_on_bit(), except also takes a - * timeout parameter. + * Wait for the given bit in an unsigned long or bitmap (see + * DECLARE_BITMAP()) to be cleared, or for a timeout to expire. The + * clearing of the bit must be signalled with wake_up_bit(), often as + * clear_and_wake_up_bit(). * - * Returned value will be zero if the bit was cleared before the - * @timeout elapsed, or non-zero if the @timeout elapsed or process - * received a signal and the mode permitted wakeup on that signal. + * This is similar to wait_on_bit(), except it also takes a timeout + * parameter. + * + * Returned value will be zero if the bit was cleared in which case the + * call has ACQUIRE semantics, or %-EINTR if the process received a + * signal and the mode permitted wake up on that signal, or %-EAGAIN if the + * timeout elapsed. */ static inline int wait_on_bit_timeout(unsigned long *word, int bit, unsigned mode, @@ -132,19 +142,21 @@ wait_on_bit_timeout(unsigned long *word, int bit, uns= igned mode, =20 /** * wait_on_bit_action - wait for a bit to be cleared - * @word: the word being waited on, a kernel virtual address - * @bit: the bit of the word being waited on + * @word: the address containing the bit waited on + * @bit: the bit at that address being waited on * @action: the function used to sleep, which may take special actions * @mode: the task state to sleep in * - * Use the standard hashed waitqueue table to wait for a bit - * to be cleared, and allow the waiting action to be specified. - * This is like wait_on_bit() but allows fine control of how the waiting - * is done. + * Wait for the given bit in an unsigned long or bitmap (see DECLARE_BITMA= P()) + * to be cleared. The clearing of the bit must be signalled with + * wake_up_bit(), often as clear_and_wake_up_bit(). + * + * This is similar to wait_on_bit(), but calls @action() instead of + * schedule() for the actual waiting. * - * Returned value will be zero if the bit was cleared, or non-zero - * if the process received a signal and the mode permitted wakeup - * on that signal. + * Returned value will be zero if the bit was cleared in which case the + * call has ACQUIRE semantics, or the error code returned by @action if + * that call returned non-zero. */ static inline int wait_on_bit_action(unsigned long *word, int bit, wait_bit_action_f *action, @@ -157,23 +169,22 @@ wait_on_bit_action(unsigned long *word, int bit, wait= _bit_action_f *action, } =20 /** - * wait_on_bit_lock - wait for a bit to be cleared, when wanting to set it - * @word: the word being waited on, a kernel virtual address - * @bit: the bit of the word being waited on + * wait_on_bit_lock - wait for a bit to be cleared, then set it + * @word: the address containing the bit being waited on + * @bit: the bit of the word being waited on and set * @mode: the task state to sleep in * - * There is a standard hashed waitqueue table for generic use. This - * is the part of the hashtable's accessor API that waits on a bit - * when one intends to set it, for instance, trying to lock bitflags. - * For instance, if one were to have waiters trying to set bitflag - * and waiting for it to clear before setting it, one would call - * wait_on_bit() in threads waiting to be able to set the bit. - * One uses wait_on_bit_lock() where one is waiting for the bit to - * clear with the intention of setting it, and when done, clearing it. + * Wait for the given bit in an unsigned long or bitmap (see + * DECLARE_BITMAP()) to be cleared. The clearing of the bit must be + * signalled with wake_up_bit(), often as clear_and_wake_up_bit(). As + * soon as it is clear, atomically set it and return. * - * Returns zero if the bit was (eventually) found to be clear and was - * set. Returns non-zero if a signal was delivered to the process and - * the @mode allows that signal to wake the process. + * This is similar to wait_on_bit(), but sets the bit before returning. + * + * Returned value will be zero if the bit was successfully set in which + * case the call has the same memory sequencing semantics as + * test_and_clear_bit(), or %-EINTR if the process received a signal and + * the mode permitted wake up on that signal. */ static inline int wait_on_bit_lock(unsigned long *word, int bit, unsigned mode) @@ -185,15 +196,18 @@ wait_on_bit_lock(unsigned long *word, int bit, unsign= ed mode) } =20 /** - * wait_on_bit_lock_io - wait for a bit to be cleared, when wanting to set= it - * @word: the word being waited on, a kernel virtual address - * @bit: the bit of the word being waited on + * wait_on_bit_lock_io - wait for a bit to be cleared, then set it + * @word: the address containing the bit being waited on + * @bit: the bit of the word being waited on and set * @mode: the task state to sleep in * - * Use the standard hashed waitqueue table to wait for a bit - * to be cleared and then to atomically set it. This is similar - * to wait_on_bit(), but calls io_schedule() instead of schedule() - * for the actual waiting. + * Wait for the given bit in an unsigned long or bitmap (see + * DECLARE_BITMAP()) to be cleared. The clearing of the bit must be + * signalled with wake_up_bit(), often as clear_and_wake_up_bit(). As + * soon as it is clear, atomically set it and return. + * + * This is similar to wait_on_bit_lock(), but calls io_schedule() instead + * of schedule(). * * Returns zero if the bit was (eventually) found to be clear and was * set. Returns non-zero if a signal was delivered to the process and @@ -209,21 +223,19 @@ wait_on_bit_lock_io(unsigned long *word, int bit, uns= igned mode) } =20 /** - * wait_on_bit_lock_action - wait for a bit to be cleared, when wanting to= set it - * @word: the word being waited on, a kernel virtual address - * @bit: the bit of the word being waited on + * wait_on_bit_lock_action - wait for a bit to be cleared, then set it + * @word: the address containing the bit being waited on + * @bit: the bit of the word being waited on and set * @action: the function used to sleep, which may take special actions * @mode: the task state to sleep in * - * Use the standard hashed waitqueue table to wait for a bit - * to be cleared and then to set it, and allow the waiting action - * to be specified. - * This is like wait_on_bit() but allows fine control of how the waiting - * is done. + * This is similar to wait_on_bit_lock(), but calls @action() instead of + * schedule() for the actual waiting. * - * Returns zero if the bit was (eventually) found to be clear and was - * set. Returns non-zero if a signal was delivered to the process and - * the @mode allows that signal to wake the process. + * Returned value will be zero if the bit was successfully set in which + * case the call has the same memory sequencing semantics as + * test_and_clear_bit(), or the error code returned by @action if that + * call returned non-zero. */ static inline int wait_on_bit_lock_action(unsigned long *word, int bit, wait_bit_action_f *a= ction, @@ -320,12 +332,13 @@ do { \ =20 /** * clear_and_wake_up_bit - clear a bit and wake up anyone waiting on that = bit - * * @bit: the bit of the word being waited on - * @word: the word being waited on, a kernel virtual address + * @word: the address containing the bit being waited on * - * You can use this helper if bitflags are manipulated atomically rather t= han - * non-atomically under a lock. + * The designated bit is cleared and any tasks waiting in wait_on_bit() + * or similar will be woken. This call has RELEASE semantics so that + * any changes to memory made before this call are guaranteed to be visible + * after the corresponding wait_on_bit() completes. */ static inline void clear_and_wake_up_bit(int bit, unsigned long *word) { diff --git a/kernel/sched/wait_bit.c b/kernel/sched/wait_bit.c index 058b0e18727e..bd2fc750fb1f 100644 --- a/kernel/sched/wait_bit.c +++ b/kernel/sched/wait_bit.c @@ -128,21 +128,29 @@ void __wake_up_bit(struct wait_queue_head *wq_head, u= nsigned long *word, int bit EXPORT_SYMBOL(__wake_up_bit); =20 /** - * wake_up_bit - wake up a waiter on a bit - * @word: the word being waited on, a kernel virtual address - * @bit: the bit of the word being waited on + * wake_up_bit - wake up waiters on a bit + * @word: the address containing the bit being waited on + * @bit: the bit at that address being waited on * - * There is a standard hashed waitqueue table for generic use. This - * is the part of the hash-table's accessor API that wakes up waiters - * on a bit. For instance, if one were to have waiters on a bitflag, - * one would call wake_up_bit() after clearing the bit. + * Wake up any process waiting in wait_on_bit() or similar for the + * given bit to be cleared. * - * In order for this to function properly, as it uses waitqueue_active() - * internally, some kind of memory barrier must be done prior to calling - * this. Typically, this will be smp_mb__after_atomic(), but in some - * cases where bitflags are manipulated non-atomically under a lock, one - * may need to use a less regular barrier, such fs/inode.c's smp_mb(), - * because spin_unlock() does not guarantee a memory barrier. + * The wake-up is sent to tasks in a waitqueue selected by hash from a + * shared pool. Only those tasks on that queue which have requested + * wake_up on this specific address and bit will be woken, and only if the + * bit is clear. + * + * In order for this to function properly there must be a full memory + * barrier after the bit is cleared and before this function is called. + * If the bit was cleared atomically, such as a by clear_bit() then + * smb_mb__after_atomic() can be used, othwewise smb_mb() is needed. + * If the bit was cleared with a fully-ordered operation, no further + * barrier is required. + * + * Normally the bit should be cleared by an operation with RELEASE + * semantics so that any changes to memory made before the bit is + * cleared are guaranteed to be visible after the matching wait_on_bit() + * completes. */ void wake_up_bit(unsigned long *word, int bit) { --=20 2.46.0 From nobody Fri Nov 29 03:55:56 2024 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 858B2154BEB for ; Wed, 25 Sep 2024 05:34:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242499; cv=none; b=Fw7cmWXEj9BmOIzzMQZucUHiM0x2OmYd1Lvqnbp3uDCXl75EmsDUgtrF9rlsxkrlFxPyMcHBeVyP7YC2A9r1NFGPlKAzwrK2DTkYyndKc4nmD3JOYcVov5P+3ev9rIXG7IBoNJJz+yjHTiM9UB8mVedDzRvfTWXqW5yW9oJaPxo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242499; c=relaxed/simple; bh=3qiBvSTL5KVMcgRfJKD7rY9QeKLgUOpk2C3iJy5L2SM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=J4yHCtYvql33dnEWHH18s8wgw7FbRKDP1fa8xFiulOSzYhcD2sEEuBmAV35s1ESRQiZPt0iYSQufVTpnJJs9zC31X4UwP723r2csrVA4gV3UHT5Q9WgoPM9dYBTqoO9tJMim7bL+Utjm6P9/2wPU0JDdQL595zy3/2XfIHq02E0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=G83FfE97; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=fZnHGQfb; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=VrDGNt1M; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=bXfcs86a; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="G83FfE97"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="fZnHGQfb"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="VrDGNt1M"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="bXfcs86a" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id E71B11FD30; Wed, 25 Sep 2024 05:34:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242496; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lm78a9I/y5Jhx1MOAKRxaG8h/PjpFbsM89I1+Rnyl4w=; b=G83FfE97F9zkrY9sPg+xTO+RjbeWnKSK7OLV2NWdpM9wJoM18JUKTfckeaY/SbDaFZdi1w zakFhbayryViT+aMHd0N/VPcweSDg3+IGJUICWtx/ZXKs8eXyoLo0USF160gN5BX3nqobs r925IkeN162QD/fBrrsWW7EVRaWN8a4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242496; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lm78a9I/y5Jhx1MOAKRxaG8h/PjpFbsM89I1+Rnyl4w=; b=fZnHGQfbwEKbvSGxmE+1CKvJd73PzXFjZ0Ek15iDouG6H7+GuMkvMgQJHhVqFR322phEdr 5UOzj+kJHrSt3jAg== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=VrDGNt1M; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=bXfcs86a DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242494; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lm78a9I/y5Jhx1MOAKRxaG8h/PjpFbsM89I1+Rnyl4w=; b=VrDGNt1MwR37E3Pba8mhas7Fw36DAMflp9r/xEuklo2RSwNO1TRbgAOY6njvWuf0LlfyxE ufVuvbwKCtnr6aRssADOdxrAnSDyz6HUEOXMwQBeJPSXO4/QaXe9l7iE8bvQWGDgg1kZEn IoER6wgdjAq/MnIzsiGjVKHQ7Fvo048= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242494; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lm78a9I/y5Jhx1MOAKRxaG8h/PjpFbsM89I1+Rnyl4w=; b=bXfcs86a6E/HTIO6H0N8IMXi0YkWzROHlJmzOJkRFjtGmszk6FeuCQITNYRDofJ4pGwejb CEM5m9qMvsaZWqCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6BD3B13A66; Wed, 25 Sep 2024 05:34:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id oS+3CP2g82ZIUQAAD6G6ig (envelope-from ); Wed, 25 Sep 2024 05:34:53 +0000 From: NeilBrown To: Ingo Molnar , Peter Zijlstra , Linus Torvalds Cc: linux-kernel@vger.kernel.org Subject: [PATCH 4/8] sched: Document wait_var_event() family of functions and wake_up_var() Date: Wed, 25 Sep 2024 15:24:07 +1000 Message-ID: <20240925053118.3956520-5-neilb@suse.de> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240925053118.3956520-1-neilb@suse.de> References: <20240925053118.3956520-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: E71B11FD30 X-Spam-Score: -5.01 X-Rspamd-Action: no action X-Spamd-Result: default: False [-5.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; DWL_DNSWL_MED(-2.00)[suse.de:dkim]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCPT_COUNT_THREE(0.00)[4]; ARC_NA(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo,suse.de:dkim,suse.de:mid,suse.de:email] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spam-Flag: NO X-Spam-Level: Content-Type: text/plain; charset="utf-8" wake_up_var(), wait_var_event() and related interfaces are not documented but have important ordering requirements. This patch adds documentation and makes these requirements explicit. The return values for those wait_var_event_* functions which return a value are documented. Note that these are, perhaps surprisingly, sometimes different from comparable wait_on_bit() functions. Signed-off-by: NeilBrown --- include/linux/wait_bit.h | 71 ++++++++++++++++++++++++++++++++++++++++ kernel/sched/wait_bit.c | 30 +++++++++++++++++ 2 files changed, 101 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 723e7bf35747..06ec99b90bf3 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -282,6 +282,22 @@ __out: __ret; \ ___wait_var_event(var, condition, TASK_UNINTERRUPTIBLE, 0, 0, \ schedule()) =20 +/** + * wait_var_event - wait for a variable to be updated and notified + * @var: the address of variable being waited on + * @condition: the condition to wait for + * + * Wait for a @condition to be true, only re-checking when a wake up is + * received for the given @var (an arbitrary kernel address which need + * not be directly related to the given condition, but usually is). + * + * The process will wait on a waitqueue selected by hash from a shared + * pool. It will only be woken on a wake_up for the given address. + * + * The condition should normally use smp_load_acquire() or a similarly + * ordered access to ensure that any changes to memory made before the + * condition became true will be visible after the wait completes. + */ #define wait_var_event(var, condition) \ do { \ might_sleep(); \ @@ -294,6 +310,24 @@ do { \ ___wait_var_event(var, condition, TASK_KILLABLE, 0, 0, \ schedule()) =20 +/** + * wait_var_event_killable - wait for a variable to be updated and notified + * @var: the address of variable being waited on + * @condition: the condition to wait for + * + * Wait for a @condition to be true or a fatal signal to be received, + * only re-checking the condition when a wake up is received for the given + * @var (an arbitrary kernel address which need not be directly related + * to the given condition, but usually is). + * + * This is similar to wait_var_event() but returns a value which is + * 0 if the condition became true, or %-ERESTARTSYS if a fatal signal + * was received. + * + * The condition should normally use smp_load_acquire() or a similarly + * ordered access to ensure that any changes to memory made before the + * condition became true will be visible after the wait completes. + */ #define wait_var_event_killable(var, condition) \ ({ \ int __ret =3D 0; \ @@ -308,6 +342,26 @@ do { \ TASK_UNINTERRUPTIBLE, 0, timeout, \ __ret =3D schedule_timeout(__ret)) =20 +/** + * wait_var_event_timeout - wait for a variable to be updated or a timeout= to expire + * @var: the address of variable being waited on + * @condition: the condition to wait for + * @timeout: maximum time to wait in jiffies + * + * Wait for a @condition to be true or a timeout to expire, only + * re-checking the condition when a wake up is received for the given + * @var (an arbitrary kernel address which need not be directly related + * to the given condition, but usually is). + * + * This is similar to wait_var_event() but returns a value which is 0 if + * the timeout expired and the condition was still false, or the + * remaining time left in the timeout (but at least 1) if the condition + * was found to be true. + * + * The condition should normally use smp_load_acquire() or a similarly + * ordered access to ensure that any changes to memory made before the + * condition became true will be visible after the wait completes. + */ #define wait_var_event_timeout(var, condition, timeout) \ ({ \ long __ret =3D timeout; \ @@ -321,6 +375,23 @@ do { \ ___wait_var_event(var, condition, TASK_INTERRUPTIBLE, 0, 0, \ schedule()) =20 +/** + * wait_var_event_killable - wait for a variable to be updated and notified + * @var: the address of variable being waited on + * @condition: the condition to wait for + * + * Wait for a @condition to be true or a signal to be received, only + * re-checking the condition when a wake up is received for the given + * @var (an arbitrary kernel address which need not be directly related + * to the given condition, but usually is). + * + * This is similar to wait_var_event() but returns a value which is 0 if + * the condition became true, or %-ERESTARTSYS if a signal was received. + * + * The condition should normally use smp_load_acquire() or a similarly + * ordered access to ensure that any changes to memory made before the + * condition became true will be visible after the wait completes. + */ #define wait_var_event_interruptible(var, condition) \ ({ \ int __ret =3D 0; \ diff --git a/kernel/sched/wait_bit.c b/kernel/sched/wait_bit.c index bd2fc750fb1f..22ec270f5ab5 100644 --- a/kernel/sched/wait_bit.c +++ b/kernel/sched/wait_bit.c @@ -196,6 +196,36 @@ void init_wait_var_entry(struct wait_bit_queue_entry *= wbq_entry, void *var, int } EXPORT_SYMBOL(init_wait_var_entry); =20 +/** + * wake_up_var - wake up waiters on a variable (kernel address) + * @var: the address of the variable being waited on + * + * Wake up any process waiting in wait_var_event() or similar for the + * given variable to change. wait_var_event() can be waiting for an + * arbitrary condition to be true and associates that condition with an + * address. Calling wake_up_var() suggests that the condition has been + * made true, but does not strictly require the condtion to use the + * address given. + * + * The wake-up is sent to tasks in a waitqueue selected by hash from a + * shared pool. Only those tasks on that queue which have requested + * wake_up on this specific address will be woken. + * + * In order for this to function properly there must be a full memory + * barrier after the variable is updated (or more accurately, after the + * condition waited on has been made to be true) and before this function + * is called. If the variable was updated atomically, such as a by + * atomic_dec() then smb_mb__after_atomic() can be used. If the + * variable was updated by a fully ordered operation such as + * atomic_dec_and_test() then no extra barrier is required. Otherwise + * smb_mb() is needed. + * + * Normally the variable should be updated (the condition should be made + * to be true) by an operation with RELEASE semantics such as + * smp_store_release() so that any changes to memory made before the + * variable was updated are guaranteed to be visible after the matching + * wait_var_event() completes. + */ void wake_up_var(void *var) { __wake_up_bit(__var_waitqueue(var), var, -1); --=20 2.46.0 From nobody Fri Nov 29 03:55:56 2024 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 994E6155751 for ; Wed, 25 Sep 2024 05:35:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242503; cv=none; b=s0twwbxwxoVCbBIHFctabO6K+NP0xzJyJnaaqTdhDsAOiIPm63euR7FncybpN6kEzuYtXiUa8Hog7hcHS0mqOZK8DXjcN6qMvnIRQUHLjAvEpeJkBCEyaAl4DFteXbm1zEeo2r+Px0ekiFXPy6mBN59443lyiAnd50BGWyUp2vc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242503; c=relaxed/simple; bh=d06xgWgkQbMZwtnKU5C5cXUH1HsFJbDXxFWLbHzTy1A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PGAypktOlECKoeh5At01IOz98ZNEXxSdt+wtObVw/3u0bGocL+xNcNxAsyMIxewNwYdH0p5fs8GMh6yW17GnZvEXjaBEQ65Kpn+4RJcGtRFhOj/3QxdZvOUvX+aZDCrDv6oyOAwlZPCyYeSqUGO5CxT3q8V1iPLbPxMwAYKkgYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=bPgo0c4w; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=g0ZlonWw; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=bPgo0c4w; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=g0ZlonWw; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="bPgo0c4w"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="g0ZlonWw"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="bPgo0c4w"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="g0ZlonWw" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D4A3321A6A; Wed, 25 Sep 2024 05:34:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242499; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vqEuJGyXxJQ9CYo6juFxRH005TpzRgjAm+F33eybZko=; b=bPgo0c4wtDUK9W+7bFDJWIGjec8i27VBA3wAq2cD0gUoZMrLPbq2kdPoASmrtSvyLZ3hVz c7JrzFREgkLJpfQqdLY8PUjiJ9+foPmvV2A2kcr3OrOXWJMPSUHQH2QRcMhQtcXHoA0EMj XaHvRqWUx3Bt6yxJssxlCEMK5UgdFdg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242499; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vqEuJGyXxJQ9CYo6juFxRH005TpzRgjAm+F33eybZko=; b=g0ZlonWwW7VmVaLtxN2xoNWfOw23G2l3bUfdDdfu6d+9s9hYI8oPRZsCrKdYZlG2gC1iGv XG+4hfjoWuUWEWCA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=bPgo0c4w; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=g0ZlonWw DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242499; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vqEuJGyXxJQ9CYo6juFxRH005TpzRgjAm+F33eybZko=; b=bPgo0c4wtDUK9W+7bFDJWIGjec8i27VBA3wAq2cD0gUoZMrLPbq2kdPoASmrtSvyLZ3hVz c7JrzFREgkLJpfQqdLY8PUjiJ9+foPmvV2A2kcr3OrOXWJMPSUHQH2QRcMhQtcXHoA0EMj XaHvRqWUx3Bt6yxJssxlCEMK5UgdFdg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242499; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vqEuJGyXxJQ9CYo6juFxRH005TpzRgjAm+F33eybZko=; b=g0ZlonWwW7VmVaLtxN2xoNWfOw23G2l3bUfdDdfu6d+9s9hYI8oPRZsCrKdYZlG2gC1iGv XG+4hfjoWuUWEWCA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5A0E713A66; Wed, 25 Sep 2024 05:34:58 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id Pd9bBAKh82ZRUQAAD6G6ig (envelope-from ); Wed, 25 Sep 2024 05:34:58 +0000 From: NeilBrown To: Ingo Molnar , Peter Zijlstra , Linus Torvalds Cc: linux-kernel@vger.kernel.org Subject: [PATCH 5/8] sched: Add test_and_clear_wake_up_bit() and atomic_dec_and_wake_up() Date: Wed, 25 Sep 2024 15:24:08 +1000 Message-ID: <20240925053118.3956520-6-neilb@suse.de> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240925053118.3956520-1-neilb@suse.de> References: <20240925053118.3956520-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: D4A3321A6A X-Spam-Level: X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_TLS_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:email,suse.de:dkim,suse.de:mid] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 X-Spam-Flag: NO Content-Type: text/plain; charset="utf-8" There are common patterns in the kernel of using test_and_clear_bit() before wake_up_bit(), and atomic_dec_and_test() before wake_up_var(). These combinations don't need extra barriers but sometimes include them unnecessarily. To help avoid the unnecessary barriers and to help discourage the general use of wake_up_bit/var (which is a fragile interface) introduce two combined functions which implement these patterns. Also add store_release_wake_up() which supports the task of simply setting a non-atomic variable and sending a wakeup. This pattern requires barriers which are often omitted. Signed-off-by: NeilBrown --- include/linux/wait_bit.h | 60 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 06ec99b90bf3..0272629b590a 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -419,4 +419,64 @@ static inline void clear_and_wake_up_bit(int bit, unsi= gned long *word) wake_up_bit(word, bit); } =20 +/** + * test_and_clear_wake_up_bit - clear a bit if it was set: wake up anyone = waiting on that bit + * @bit: the bit of the word being waited on + * @word: the address of memory containing that bit + * + * If the bit is set and can be atomically cleared, any tasks waiting in + * wait_on_bit() or similar will be woken. This call has the same + * complete ordering semantics as test_and_clear_bit(). Any changes to + * memory made before this call are guaranteed to be visible after the + * corresponding wait_on_bit() completes. + * + * Returns %true if the bit was successfully set and the wake up was sent. + */ +static inline bool test_and_clear_wake_up_bit(int bit, unsigned long *word) +{ + if (!test_and_clear_bit(bit, word)) + return false; + /* no extra barrier required */ + wake_up_bit(word, bit); + return true; +} + +/** + * atomic_dec_and_wake_up - decrement an atomic_t and if zero, wake up wai= ters + * @var: the variable to dec and test + * + * Decrements the atomic variable and if it reaches zero, send a wake_up t= o any + * processes waiting on the variable. + * + * This function has the same complete ordering semantics as atomic_dec_an= d_test. + * + * Returns %true is the variable reaches zero and the wake up was sent. + */ + +static inline bool atomic_dec_and_wake_up(atomic_t *var) +{ + if (!atomic_dec_and_test(var)) + return false; + /* No extra barrier required */ + wake_up_var(var); + return true; +} + +/** + * store_release_wake_up - update a variable and send a wake_up + * @var: the address of the variable to be updated and woken + * @val: the value to store in the variable. + * + * Store the given value in the variable send a wake up to any tasks + * waiting on the variable. All necessary barriers are included to ensure + * the task calling wait_var_event() sees the new value and all values + * written to memory before this call. + */ +#define store_release_wake_up(var, val) \ +do { \ + smp_store_release(var, val); \ + smp_mb(); \ + wake_up_var(var); \ +} while (0) + #endif /* _LINUX_WAIT_BIT_H */ --=20 2.46.0 From nobody Fri Nov 29 03:55:56 2024 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FFCF14C5A7 for ; Wed, 25 Sep 2024 05:35:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242512; cv=none; b=cOyHLb/8H2LZ4/f0PX2C4UVq+iVtRc5myTRzD2c7xIY1kkSWK+aW5n23GRyrBZTLtjz4MhO85/Rr1f9s5E/GroMCC5N2PucqlBJdPRWQq0i47DMMIgb/CQMi6DpG/jU6QVbDalShQcmOHXiWUb0v1f1BNqgcasiVEnjUcgd781Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242512; c=relaxed/simple; bh=Ok0FBSdPwSnmqEJ/Nk9GrxxOsJzZZpXDENy218IFJ0U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XEfUsc0Iw1Rq9F1SP0LxvYW60YNnVGq37+zfhxA+D2FHFq/00tIhQzvEegTL+hP/EQ23gRZoLsG+GiH7SG8AcE7r/ZJKg0xUgAhi3qGzRlTuig0inH1+cueWVbDQT92u09vsytHfHJdyI7HF3PCnDlYrEVx3bTmzaZe0K+kKLCs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=k5Z38Qdo; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=c6RM9qOl; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=k5Z38Qdo; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=c6RM9qOl; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="k5Z38Qdo"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="c6RM9qOl"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="k5Z38Qdo"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="c6RM9qOl" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D35DF1FD32; Wed, 25 Sep 2024 05:35:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242508; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lKw1JB4J5UZuj0DJxVaFneJSr819xZJRg3iiAi5BrmE=; b=k5Z38Qdo3p44GQ5JTm+Rw13A4aORP/0F4owOOdOK+Hx2ZcCh0tryh//cMpgoEc18znee6F K2ZF3LM0k4HxzVbfoiB5hUPNNh1en5IzqObICUWmfQWy2sxZinxFRShjWeKkG+SJ8mbgK6 R2vlRzKsZqFGfcKqDyhqcwbj1VTkeYQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242508; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lKw1JB4J5UZuj0DJxVaFneJSr819xZJRg3iiAi5BrmE=; b=c6RM9qOlUHrxe4FkuwXEkXP5SryRc54nf79pn0D6YqHu/nIXdJdgm7VzMTNhOHBKEXSkBV LwcKVJ8xNtCaNeAQ== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=k5Z38Qdo; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=c6RM9qOl DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242508; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lKw1JB4J5UZuj0DJxVaFneJSr819xZJRg3iiAi5BrmE=; b=k5Z38Qdo3p44GQ5JTm+Rw13A4aORP/0F4owOOdOK+Hx2ZcCh0tryh//cMpgoEc18znee6F K2ZF3LM0k4HxzVbfoiB5hUPNNh1en5IzqObICUWmfQWy2sxZinxFRShjWeKkG+SJ8mbgK6 R2vlRzKsZqFGfcKqDyhqcwbj1VTkeYQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242508; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lKw1JB4J5UZuj0DJxVaFneJSr819xZJRg3iiAi5BrmE=; b=c6RM9qOlUHrxe4FkuwXEkXP5SryRc54nf79pn0D6YqHu/nIXdJdgm7VzMTNhOHBKEXSkBV LwcKVJ8xNtCaNeAQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5826413A66; Wed, 25 Sep 2024 05:35:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id EHSKAwuh82ZZUQAAD6G6ig (envelope-from ); Wed, 25 Sep 2024 05:35:07 +0000 From: NeilBrown To: Ingo Molnar , Peter Zijlstra , Linus Torvalds Cc: linux-kernel@vger.kernel.org Subject: [PATCH 6/8] sched: Add wait/wake interface for variable updated under a lock. Date: Wed, 25 Sep 2024 15:24:09 +1000 Message-ID: <20240925053118.3956520-7-neilb@suse.de> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240925053118.3956520-1-neilb@suse.de> References: <20240925053118.3956520-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: D35DF1FD32 X-Spam-Level: X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:email,suse.de:dkim,suse.de:mid]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 X-Spam-Flag: NO Content-Type: text/plain; charset="utf-8" Sometimes we need to wait for a condition to be true which must be testing while holding a lock. Correspondingly the condition is made true while holding the lock and the wake up is sent under the lock. This patch provides wake and wait interfaces which can be used for this situation when the lock is a mutex or a spinlock, or any other lock for which there are foo_lock() and foo_unlock() functions. Signed-off-by: NeilBrown --- include/linux/wait_bit.h | 106 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 0272629b590a..6aea10efca3d 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -401,6 +401,112 @@ do { \ __ret; \ }) =20 +/** + * wait_var_event_any_lock - wait for a variable to be updated under a lock + * @var: the address of the variable being waited on + * @condition: condition to wait for + * @lock: the object that is locked to protect updates to the variable + * @type: prefix on lock and unlock operations + * @state: waiting state, %TASK_UNINTERRUPTIBLE etc. + * + * Wait for a condition which can only be reliably tested while holding + * a lock. The variables assessed in the condition will normal be updated + * under the same lock, and the wake up should be signalled with + * wake_up_var_locked() under the same lock. + * + * This is similar to wait_var_event(), but assumes a lock is held + * while calling this function and while updating the variable. + * + * This must be called while the given lock is held and the lock will be + * dropped when schedule() is called to wait for a wake up, and will be + * reclaimed before testing the condition again. The functions used to + * unlock and lock the object are constructed by appending _unlock and _lo= ck + * to @type. + * + * Return %-ERESTARTSYS if a signal arrives which is allowed to interrupt + * the wait according to @state. + */ +#define wait_var_event_any_lock(var, condition, lock, type, state) \ +({ \ + int __ret =3D 0; \ + if (!(condition)) \ + __ret =3D ___wait_var_event(var, condition, state, 0, 0, \ + type ## _unlock(lock); \ + schedule(); \ + type ## _lock(lock)); \ + __ret; \ +}) + +/** + * wait_var_event_spinlock - wait for a variable to be updated under a spi= nlock + * @var: the address of the variable being waited on + * @condition: condition to wait for + * @lock: the spinlock which protects updates to the variable + * + * Wait for a condition which can only be reliably tested while holding + * a spinlock. The variables assessed in the condition will normal be upd= ated + * under the same spinlock, and the wake up should be signalled with + * wake_up_var_locked() under the same spinlock. + * + * This is similar to wait_var_event(), but assumes a spinlock is held + * while calling this function and while updating the variable. + * + * This must be called while the given lock is held and the lock will be + * dropped when schedule() is called to wait for a wake up, and will be + * reclaimed before testing the condition again. + */ +#define wait_var_event_spinlock(var, condition, lock) \ + wait_var_event_any_lock(var, condition, lock, spin, TASK_UNINTERRUPTIBLE) + +/** + * wait_var_event_mutex - wait for a variable to be updated under a mutex + * @var: the address of the variable being waited on + * @condition: condition to wait for + * @mutex: the mutex which protects updates to the variable + * + * Wait for a condition which can only be reliably tested while holding + * a mutex. The variables assessed in the condition will normal be + * updated under the same mutex, and the wake up should be signalled + * with wake_up_var_locked() under the same mutex. + * + * This is similar to wait_var_event(), but assumes a mutex is held + * while calling this function and while updating the variable. + * + * This must be called while the given mutex is held and the mutex will be + * dropped when schedule() is called to wait for a wake up, and will be + * reclaimed before testing the condition again. + */ +#define wait_var_event_mutex(var, condition, lock) \ + wait_var_event_any_lock(var, condition, lock, mutex, TASK_UNINTERRUPTIBLE) + +/** + * wake_up_var_protected - wake up waiters for a variable asserting that i= t is safe + * @var: the address of the variable being waited on + * @cond: the condition which afirms this is safe + * + * When waking waiters which use wait_var_event_any_lock() the waker must = be + * holding the reelvant lock to avoid races. This version of wake_up_var() + * asserts that the relevant lock is held and so no barrier is needed. + * The @cond is only tested when CONFIG_LOCKDEP is enabled. + */ +#define wake_up_var_protected(var, cond) \ +do { \ + lockdep_assert(cond); \ + wake_up_var(var); \ +} while (0) + +/** + * wake_up_var_locked - wake up waiters for a variable while holding a spi= nlock or mutex + * @var: the address of the variable being waited on + * @lock: The spinlock or mutex what protects the variable + * + * Send a wake up for the given variable which should be waited for with + * wait_var_event_spinlock() or wait_var_event_mutex(). Unlike wake_up_va= r(), + * no extra barriers are needed as the locking provides sufficient sequenc= ing. + */ +#define wake_up_var_locked(var, lock) \ + wake_up_var_protected(var, lockdep_is_held(lock)) + /** * clear_and_wake_up_bit - clear a bit and wake up anyone waiting on that = bit * @bit: the bit of the word being waited on --=20 2.46.0 From nobody Fri Nov 29 03:55:56 2024 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A59661B85D1 for ; Wed, 25 Sep 2024 05:35:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242525; cv=none; b=LN1nd1jZFsB8BhQSvECWUlzDAW1Q2lPnAcSForMiM0XrSIMy49eRpOrwFCjaWibMJehmXfTt8LvbBI4XqQKFHScs1s+83tKMhQRqYzfchzDb5DpChRnjSEgJYhJoRoh48+XH2+kBCRT8OV037Xji/ivxDPyrNCCZ8Ci05mMDMjc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242525; c=relaxed/simple; bh=ES55q1qO4ScgIBk1GsInz47oQLLXboIg+8vX743mNxs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AoepK7tdN8gubadkT4UvatHwYnTrCECfveI0DmZceZa0K8085LkvL97IsuUfCPqa2Y+/oHMnVOx+xt/2ge1CGhoan925S7OeFyxb0tv6+wLcnW9PddjK7updc/CEVY7941tKf5HY24nZYazKWsSjFVznDPYHWXmFC8e9ziTv36w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=saYCNXRZ; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=H3vGIEw3; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=saYCNXRZ; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=H3vGIEw3; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="saYCNXRZ"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="H3vGIEw3"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="saYCNXRZ"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="H3vGIEw3" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B81B121A6A; Wed, 25 Sep 2024 05:35:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RtTb28IU1JK+M/CDCe9gHZ0UxDCFT041G0Qqg0/6YrU=; b=saYCNXRZvvMQUrvsOBScAwR8ofkxxpBVkreRQNU4rTUfF4FJeBIwdZlb9BEYEonEM/ruPm SOdlVutQ0pfTk2sRMJf36RmPwvarL9CpmMazjz1us/UI09IBha2QI1F62IfctqrCgF7Pxe L2yTR87LuAYfBua/RxKfaLTzFVvT/hI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RtTb28IU1JK+M/CDCe9gHZ0UxDCFT041G0Qqg0/6YrU=; b=H3vGIEw3xkcELjcRKbDwcae/XjvIdV+8FDwpGH+X56cMQphpC839AAizGsAu9o2HmVJNb6 baW4Wqqr+1mMuzDg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RtTb28IU1JK+M/CDCe9gHZ0UxDCFT041G0Qqg0/6YrU=; b=saYCNXRZvvMQUrvsOBScAwR8ofkxxpBVkreRQNU4rTUfF4FJeBIwdZlb9BEYEonEM/ruPm SOdlVutQ0pfTk2sRMJf36RmPwvarL9CpmMazjz1us/UI09IBha2QI1F62IfctqrCgF7Pxe L2yTR87LuAYfBua/RxKfaLTzFVvT/hI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242521; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RtTb28IU1JK+M/CDCe9gHZ0UxDCFT041G0Qqg0/6YrU=; b=H3vGIEw3xkcELjcRKbDwcae/XjvIdV+8FDwpGH+X56cMQphpC839AAizGsAu9o2HmVJNb6 baW4Wqqr+1mMuzDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3E08313A66; Wed, 25 Sep 2024 05:35:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id JHApOReh82ZlUQAAD6G6ig (envelope-from ); Wed, 25 Sep 2024 05:35:19 +0000 From: NeilBrown To: Ingo Molnar , Peter Zijlstra , Linus Torvalds Cc: linux-kernel@vger.kernel.org Subject: [PATCH 7/8] sched: add wait_var_event_io() Date: Wed, 25 Sep 2024 15:24:10 +1000 Message-ID: <20240925053118.3956520-8-neilb@suse.de> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240925053118.3956520-1-neilb@suse.de> References: <20240925053118.3956520-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Level: X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid,imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 X-Spam-Flag: NO Content-Type: text/plain; charset="utf-8" It is not currently possible to wait wait_var_event for an io_schedule() style wait. This patch adds wait_var_event_io() for that purpose. Signed-off-by: NeilBrown --- include/linux/wait_bit.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 6aea10efca3d..6346e26fbfd1 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -281,6 +281,9 @@ __out: __ret; \ #define __wait_var_event(var, condition) \ ___wait_var_event(var, condition, TASK_UNINTERRUPTIBLE, 0, 0, \ schedule()) +#define __wait_var_event_io(var, condition) \ + ___wait_var_event(var, condition, TASK_UNINTERRUPTIBLE, 0, 0, \ + io_schedule()) =20 /** * wait_var_event - wait for a variable to be updated and notified @@ -306,6 +309,34 @@ do { \ __wait_var_event(var, condition); \ } while (0) =20 +/** + * wait_var_event_io - wait for a variable to be updated and notified + * @var: the address of variable being waited on + * @condition: the condition to wait for + * + * Wait for an IO related @condition to be true, only re-checking when a + * wake up is received for the given @var (an arbitrary kernel address + * which need not be directly related to the given condition, but + * usually is). + * + * The process will wait on a waitqueue selected by hash from a shared + * pool. It will only be woken on a wake_up for the given address. + * + * This is similar to wait_var_event(), but calls io_schedule() instead + * of schedule(). + * + * The condition should normally use smp_load_acquire() or a similarly + * ordered access to ensure that any changes to memory made before the + * condition became true will be visible after the wait completes. + */ +#define wait_var_event_io(var, condition) \ +do { \ + might_sleep(); \ + if (condition) \ + break; \ + __wait_var_event_io(var, condition); \ +} while (0) + #define __wait_var_event_killable(var, condition) \ ___wait_var_event(var, condition, TASK_KILLABLE, 0, 0, \ schedule()) --=20 2.46.0 From nobody Fri Nov 29 03:55:56 2024 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A36EB657 for ; Wed, 25 Sep 2024 05:35:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242534; cv=none; b=WtopiP0DhaOk7aF0DMXe3LsJHN9ZQHjpoml7/+zdMfU827FGl3PfkiVTt0V2gbvlHrdcyuhktBn2dKOfXxeiOv8LqywLhs2INS7i1SDKdQWnb+fqMuYN5iUmE8dshZL6pYT2mzqxqEnil3vKCUHq7FQ4eSl8emCzkwMFm/LrD0k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727242534; c=relaxed/simple; bh=bAE66nT3MP0O4MwpXHlyXPxPQuTR1dP9n3i/h6bl0rw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KkrWBfS6jQKA01t9HzkBQ/2W0yGJzRGNpq7Hwg2Tk5g3ddCch/ir3i5q99jctItL+2l5o+YcyzUfuinpTGnCH8sULUBTP15FtINUFQaVTXJ/MbQNW9gA81U21cDtk3gyOhzIOCl5NUhy7++ZLUUJAYpG70C3eyavFrWFltIqF1I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=wEN2x5tE; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=Q3yBlT+n; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=wEN2x5tE; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=Q3yBlT+n; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="wEN2x5tE"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="Q3yBlT+n"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="wEN2x5tE"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="Q3yBlT+n" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id AED5421A6A; Wed, 25 Sep 2024 05:35:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1+1KIOVKEQInYKPGDI1t7SxUOTRlcwDrGtA7ysyVBWk=; b=wEN2x5tELFD1L7rZGFEZ6FUj0L4rhYN8tro4rZ3kdgwi1MTDVuTgX16EAeti5wpuUKd9/3 9hT6F2UZ5mhFtGsv7CVUAvnBbxFeQrSOscR+6DxjikbvvaM7OLxuawg7ekVWdaF2zY+lHl mmLC/X/zWOGRGldyjeZo9RiMlmsuL+Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1+1KIOVKEQInYKPGDI1t7SxUOTRlcwDrGtA7ysyVBWk=; b=Q3yBlT+nAA52jgAGJHdM8H7b06fmLpudnGlprLc3PoCG3oTUets9pStQ2oYun/8KCsmTzC 7DgoV1EIvR0YPvDg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1727242530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1+1KIOVKEQInYKPGDI1t7SxUOTRlcwDrGtA7ysyVBWk=; b=wEN2x5tELFD1L7rZGFEZ6FUj0L4rhYN8tro4rZ3kdgwi1MTDVuTgX16EAeti5wpuUKd9/3 9hT6F2UZ5mhFtGsv7CVUAvnBbxFeQrSOscR+6DxjikbvvaM7OLxuawg7ekVWdaF2zY+lHl mmLC/X/zWOGRGldyjeZo9RiMlmsuL+Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1727242530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1+1KIOVKEQInYKPGDI1t7SxUOTRlcwDrGtA7ysyVBWk=; b=Q3yBlT+nAA52jgAGJHdM8H7b06fmLpudnGlprLc3PoCG3oTUets9pStQ2oYun/8KCsmTzC 7DgoV1EIvR0YPvDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 355D013A66; Wed, 25 Sep 2024 05:35:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 9zAGNyCh82ZvUQAAD6G6ig (envelope-from ); Wed, 25 Sep 2024 05:35:28 +0000 From: NeilBrown To: Ingo Molnar , Peter Zijlstra , Linus Torvalds Cc: linux-kernel@vger.kernel.org Subject: [PATCH 8/8] softirq: use bit waits instead of var waits. Date: Wed, 25 Sep 2024 15:24:11 +1000 Message-ID: <20240925053118.3956520-9-neilb@suse.de> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240925053118.3956520-1-neilb@suse.de> References: <20240925053118.3956520-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Level: X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 X-Spam-Flag: NO Content-Type: text/plain; charset="utf-8" The waiting in softirq.c is always waiting for a bit to be cleared. This makes the bit wait functions seem more suitable. By switching over we can rid of all explicit barriers. We also use wait_on_bit_lock() to avoid an explicit loop. Signed-off-by: NeilBrown --- kernel/softirq.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/kernel/softirq.c b/kernel/softirq.c index d082e7840f88..b756d6b3fd09 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -748,10 +748,8 @@ EXPORT_SYMBOL(__tasklet_hi_schedule); =20 static bool tasklet_clear_sched(struct tasklet_struct *t) { - if (test_and_clear_bit(TASKLET_STATE_SCHED, &t->state)) { - wake_up_var(&t->state); + if (test_and_clear_wake_up_bit(TASKLET_STATE_SCHED, &t->state)) return true; - } =20 WARN_ONCE(1, "tasklet SCHED state not set: %s %pS\n", t->use_callback ? "callback" : "func", @@ -871,8 +869,7 @@ void tasklet_kill(struct tasklet_struct *t) if (in_interrupt()) pr_notice("Attempt to kill tasklet from interrupt\n"); =20 - while (test_and_set_bit(TASKLET_STATE_SCHED, &t->state)) - wait_var_event(&t->state, !test_bit(TASKLET_STATE_SCHED, &t->state)); + wait_on_bit_lock(&t->state, TASKLET_STATE_SCHED, TASK_UNINTERRUPTIBLE); =20 tasklet_unlock_wait(t); tasklet_clear_sched(t); @@ -882,16 +879,13 @@ EXPORT_SYMBOL(tasklet_kill); #if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT) void tasklet_unlock(struct tasklet_struct *t) { - smp_mb__before_atomic(); - clear_bit(TASKLET_STATE_RUN, &t->state); - smp_mb__after_atomic(); - wake_up_var(&t->state); + clear_and_wake_up_bit(TASKLET_STATE_RUN, &t->state); } EXPORT_SYMBOL_GPL(tasklet_unlock); =20 void tasklet_unlock_wait(struct tasklet_struct *t) { - wait_var_event(&t->state, !test_bit(TASKLET_STATE_RUN, &t->state)); + wait_on_bit(&t->state, TASKLET_STATE_RUN, TASK_UNINTERRUPTIBLE); } EXPORT_SYMBOL_GPL(tasklet_unlock_wait); #endif --=20 2.46.0