This commit expands on the ordering properties of rcu_assign_pointer()
and rcu_dereference(), outlining their constraints on CPUs and compilers.

Reported-by: Rao Shoaib <rao.shoaib@oracle.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 Documentation/RCU/whatisRCU.rst | 30 +++++++++++++++++++-----------
 1 file changed, 19 insertions(+), 11 deletions(-)

diff --git a/Documentation/RCU/whatisRCU.rst b/Documentation/RCU/whatisRCU.rst
index 94838c65c7d97..d585a5490aeec 100644
--- a/Documentation/RCU/whatisRCU.rst
+++ b/Documentation/RCU/whatisRCU.rst
@@ -250,21 +250,25 @@ rcu_assign_pointer()
 ^^^^^^^^^^^^^^^^^^^^
 	void rcu_assign_pointer(p, typeof(p) v);
 
-	Yes, rcu_assign_pointer() **is** implemented as a macro, though it
-	would be cool to be able to declare a function in this manner.
-	(Compiler experts will no doubt disagree.)
+	Yes, rcu_assign_pointer() **is** implemented as a macro, though
+	it would be cool to be able to declare a function in this manner.
+	(And there has been some discussion of adding overloaded functions
+	to the C language, so who knows?)
 
 	The updater uses this spatial macro to assign a new value to an
 	RCU-protected pointer, in order to safely communicate the change
 	in value from the updater to the reader.  This is a spatial (as
 	opposed to temporal) macro.  It does not evaluate to an rvalue,
-	but it does execute any memory-barrier instructions required
-	for a given CPU architecture.  Its ordering properties are that
-	of a store-release operation.
-
-	Perhaps just as important, it serves to document (1) which
-	pointers are protected by RCU and (2) the point at which a
-	given structure becomes accessible to other CPUs.  That said,
+	but it does provide any compiler directives and memory-barrier
+	instructions required for a given compile or CPU architecture.
+	Its ordering properties are that of a store-release operation,
+	that is, any prior loads and stores required to initialize the
+	structure are ordered before the store that publishes the pointer
+	to that structure.
+
+	Perhaps just as important, rcu_assign_pointer() serves to document
+	(1) which pointers are protected by RCU and (2) the point at which
+	a given structure becomes accessible to other CPUs.  That said,
 	rcu_assign_pointer() is most frequently used indirectly, via
 	the _rcu list-manipulation primitives such as list_add_rcu().
 
@@ -283,7 +287,11 @@ rcu_dereference()
 	executes any needed memory-barrier instructions for a given
 	CPU architecture.  Currently, only Alpha needs memory barriers
 	within rcu_dereference() -- on other CPUs, it compiles to a
-	volatile load.
+	volatile load.	However, no mainstream C compilers respect
+	address dependencies, so rcu_dereference() uses volatile casts,
+	which, in combination with the coding guidelines listed in
+	rcu_dereference.rst, prevent current compilers from breaking
+	these dependencies.
 
 	Common coding practice uses rcu_dereference() to copy an
 	RCU-protected pointer to a local variable, then dereferences
-- 
2.40.1

Le Tue, Jun 04, 2024 at 03:21:55PM -0700, Paul E. McKenney a écrit :
> This commit expands on the ordering properties of rcu_assign_pointer()
> and rcu_dereference(), outlining their constraints on CPUs and compilers.
> 
> Reported-by: Rao Shoaib <rao.shoaib@oracle.com>
> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> ---
>  Documentation/RCU/whatisRCU.rst | 30 +++++++++++++++++++-----------
>  1 file changed, 19 insertions(+), 11 deletions(-)
> 
> diff --git a/Documentation/RCU/whatisRCU.rst b/Documentation/RCU/whatisRCU.rst
> index 94838c65c7d97..d585a5490aeec 100644
> --- a/Documentation/RCU/whatisRCU.rst
> +++ b/Documentation/RCU/whatisRCU.rst
> @@ -250,21 +250,25 @@ rcu_assign_pointer()
>  ^^^^^^^^^^^^^^^^^^^^
>  	void rcu_assign_pointer(p, typeof(p) v);
>  
> -	Yes, rcu_assign_pointer() **is** implemented as a macro, though it
> -	would be cool to be able to declare a function in this manner.
> -	(Compiler experts will no doubt disagree.)
> +	Yes, rcu_assign_pointer() **is** implemented as a macro, though
> +	it would be cool to be able to declare a function in this manner.
> +	(And there has been some discussion of adding overloaded functions
> +	to the C language, so who knows?)
>  
>  	The updater uses this spatial macro to assign a new value to an
>  	RCU-protected pointer, in order to safely communicate the change
>  	in value from the updater to the reader.  This is a spatial (as
>  	opposed to temporal) macro.  It does not evaluate to an rvalue,
> -	but it does execute any memory-barrier instructions required
> -	for a given CPU architecture.  Its ordering properties are that
> -	of a store-release operation.
> -
> -	Perhaps just as important, it serves to document (1) which
> -	pointers are protected by RCU and (2) the point at which a
> -	given structure becomes accessible to other CPUs.  That said,
> +	but it does provide any compiler directives and memory-barrier
> +	instructions required for a given compile or CPU architecture.
> +	Its ordering properties are that of a store-release operation,
> +	that is, any prior loads and stores required to initialize the
> +	structure are ordered before the store that publishes the pointer
> +	to that structure.

About that, why rcu_dereference() isn't a matching load-acquire?

Thanks.

On Wed, Jun 05, 2024 at 01:56:23PM +0200, Frederic Weisbecker wrote:
> Le Tue, Jun 04, 2024 at 03:21:55PM -0700, Paul E. McKenney a écrit :
> > This commit expands on the ordering properties of rcu_assign_pointer()
> > and rcu_dereference(), outlining their constraints on CPUs and compilers.
> > 
> > Reported-by: Rao Shoaib <rao.shoaib@oracle.com>
> > Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> > ---
> >  Documentation/RCU/whatisRCU.rst | 30 +++++++++++++++++++-----------
> >  1 file changed, 19 insertions(+), 11 deletions(-)
> > 
> > diff --git a/Documentation/RCU/whatisRCU.rst b/Documentation/RCU/whatisRCU.rst
> > index 94838c65c7d97..d585a5490aeec 100644
> > --- a/Documentation/RCU/whatisRCU.rst
> > +++ b/Documentation/RCU/whatisRCU.rst
> > @@ -250,21 +250,25 @@ rcu_assign_pointer()
> >  ^^^^^^^^^^^^^^^^^^^^
> >  	void rcu_assign_pointer(p, typeof(p) v);
> >  
> > -	Yes, rcu_assign_pointer() **is** implemented as a macro, though it
> > -	would be cool to be able to declare a function in this manner.
> > -	(Compiler experts will no doubt disagree.)
> > +	Yes, rcu_assign_pointer() **is** implemented as a macro, though
> > +	it would be cool to be able to declare a function in this manner.
> > +	(And there has been some discussion of adding overloaded functions
> > +	to the C language, so who knows?)
> >  
> >  	The updater uses this spatial macro to assign a new value to an
> >  	RCU-protected pointer, in order to safely communicate the change
> >  	in value from the updater to the reader.  This is a spatial (as
> >  	opposed to temporal) macro.  It does not evaluate to an rvalue,
> > -	but it does execute any memory-barrier instructions required
> > -	for a given CPU architecture.  Its ordering properties are that
> > -	of a store-release operation.
> > -
> > -	Perhaps just as important, it serves to document (1) which
> > -	pointers are protected by RCU and (2) the point at which a
> > -	given structure becomes accessible to other CPUs.  That said,
> > +	but it does provide any compiler directives and memory-barrier
> > +	instructions required for a given compile or CPU architecture.
> > +	Its ordering properties are that of a store-release operation,
> > +	that is, any prior loads and stores required to initialize the
> > +	structure are ordered before the store that publishes the pointer
> > +	to that structure.
> 
> About that, why rcu_dereference() isn't a matching load-acquire?

Here is an example showing the difference:

	p = rcu_dereference(gp);
	r1 = READ_ONCE(x);
	r2 = p->a;

The READ_ONCE() is not ordered against the rcu_dereference(), only the
read from p->a.  In contrast, if that rcu_dereference() was instead an
smp_load_acquire(), both of the two later statements would be ordered.

Ah.  You are suggesting that this be added to the description of
rcu_dereference()?

Or are you asking that this documentation state that an rcu_dereference()
memory-barrier-pairs with an rcu_assign_pointer()?

Or something else completely?

							Thanx, Paul