random: avoid reading two cache lines on irq randomness

[PATCH] random: avoid reading two cache lines on irq randomness

Posted by Jason A. Donenfeld 3 years, 6 months ago

In order to avoid reading and dirtying two cache lines on every IRQ, move
the work_struct to the bottom of the fast_pool struct. add_interrupt_
randomness() always touches .pool and .count, which are currently split,
because .mix pushes everything down. Instead, move .mix to the bottom,
so that .pool and .count are always in the first cache line, since the
.mix is only accessed when the pool is full.

Fixes: 58340f8e952b ("random: defer fast pool mixing to worker")
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Sultan Alsawaf <sultan@kerneltoast.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
 drivers/char/random.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 2f370aa248b2..a90d96f4b3bb 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -918,10 +918,10 @@ EXPORT_SYMBOL_GPL(unregister_random_vmfork_notifier);
 #endif

 struct fast_pool {
-	struct work_struct mix;
 	unsigned long pool[4];
 	unsigned long last;
 	unsigned int count;
+	struct work_struct mix;
 };

 static DEFINE_PER_CPU(struct fast_pool, irq_randomness) = {
-- 
2.37.3

Re: [PATCH] random: avoid reading two cache lines on irq randomness

Posted by Sebastian Andrzej Siewior 3 years, 6 months ago

On 2022-09-23 20:28:49 [+0200], Jason A. Donenfeld wrote:
> In order to avoid reading and dirtying two cache lines on every IRQ, move
> the work_struct to the bottom of the fast_pool struct. add_interrupt_
> randomness() always touches .pool and .count, which are currently split,
> because .mix pushes everything down. Instead, move .mix to the bottom,
> so that .pool and .count are always in the first cache line, since the
> .mix is only accessed when the pool is full.
> 
> Fixes: 58340f8e952b ("random: defer fast pool mixing to worker")
> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> Cc: Sultan Alsawaf <sultan@kerneltoast.com>
> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>

Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>

Yup, makes sense.

Sebastian