coroutine_pool_refill_local() acquires global_pool_lock on every
call even when the global pool is empty (global_pool_size == 0).
Under high I/O concurrency many threads simultaneously attempt to
refill from an empty global pool, causing unnecessary mutex
contention.

Add a fast-path check: read global_pool_size with qatomic_read()
before acquiring the mutex.  If zero, the global pool is empty and
we return immediately.  This is a racy read but correctness is
preserved: the only consequence of a stale read is a missed refill
opportunity, which will be retried on the next coroutine allocation.

Signed-off-by: Bin Guo <guobin@linux.alibaba.com>
---
 util/qemu-coroutine.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c
index d17135f585..0f8c3a23eb 100644
--- a/util/qemu-coroutine.c
+++ b/util/qemu-coroutine.c
@@ -138,6 +138,15 @@ static void coroutine_pool_refill_local(void)
     CoroutinePool *local_pool = get_ptr_local_pool();
     CoroutinePoolBatch *batch = NULL;
 
+    /*
+     * Fast path: skip the lock when the global pool is obviously empty.
+     * The read is racy but harmless -- worst case we miss a concurrent
+     * put and retry on the next allocation.
+     */
+    if (qatomic_read(&global_pool_size) == 0) {
+        return;
+    }
+
     WITH_QEMU_LOCK_GUARD(&global_pool_lock) {
         batch = QSLIST_FIRST(&global_pool);
 
-- 
2.50.1 (Apple Git-155)

Am 29.05.2026 um 03:54 hat Bin Guo geschrieben:
> coroutine_pool_refill_local() acquires global_pool_lock on every
> call even when the global pool is empty (global_pool_size == 0).
> Under high I/O concurrency many threads simultaneously attempt to
> refill from an empty global pool, causing unnecessary mutex
> contention.
> 
> Add a fast-path check: read global_pool_size with qatomic_read()
> before acquiring the mutex.  If zero, the global pool is empty and
> we return immediately.  This is a racy read but correctness is
> preserved: the only consequence of a stale read is a missed refill
> opportunity, which will be retried on the next coroutine allocation.
> 
> Signed-off-by: Bin Guo <guobin@linux.alibaba.com>

If you have any benchmark results, putting them in the commit message
would be a good idea. (If you don't have them, you should probably get
them for a patch that is motivated by performance.)

Kevin

On 5/29/26 03:54, Bin Guo wrote:
> coroutine_pool_refill_local() acquires global_pool_lock on every
> call even when the global pool is empty (global_pool_size == 0).
> Under high I/O concurrency many threads simultaneously attempt to
> refill from an empty global pool, causing unnecessary mutex
> contention.
> 
> Add a fast-path check: read global_pool_size with qatomic_read()
> before acquiring the mutex.  If zero, the global pool is empty and
> we return immediately.  This is a racy read but correctness is
> preserved: the only consequence of a stale read is a missed refill
> opportunity, which will be retried on the next coroutine allocation.
> 
> Signed-off-by: Bin Guo <guobin@linux.alibaba.com>
> ---
>   util/qemu-coroutine.c | 9 +++++++++
>   1 file changed, 9 insertions(+)
> 
> diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c
> index d17135f585..0f8c3a23eb 100644
> --- a/util/qemu-coroutine.c
> +++ b/util/qemu-coroutine.c
> @@ -138,6 +138,15 @@ static void coroutine_pool_refill_local(void)
>       CoroutinePool *local_pool = get_ptr_local_pool();
>       CoroutinePoolBatch *batch = NULL;
>   
> +    /*
> +     * Fast path: skip the lock when the global pool is obviously empty.
> +     * The read is racy but harmless -- worst case we miss a concurrent
> +     * put and retry on the next allocation.
> +     */
> +    if (qatomic_read(&global_pool_size) == 0) {
> +        return;
> +    }

Reads are fine within the lock, but you still need to use qatomic_set 
for writes now, i.e.

-            global_pool_size -= batch->size;
+            qatomic_set(&global_pool_size,
+                        global_pool_size - batch->size);

-            global_pool_size += batch->size;
+            qatomic_set(&global_pool_size,
+                        global_pool_size + batch->size);

Thanks,

Paolo

> +
>       WITH_QEMU_LOCK_GUARD(&global_pool_lock) {
>           batch = QSLIST_FIRST(&global_pool);
>