v5:
- Added explicit "#include "qemu/coroutine-tls.h" in patch 4 [Philippe]
- Updated patch 1 commit description and comments to describe the current
  noinline plus asm volatile approach [Peter]
v4:
- Dropped '[RFC]'.
- Dropped inline asm for now. -fPIC versions of the code are missing and I
  hit several issues including a clang LTO bug where thread local variables are
  incorrectly discarded because inline asm is not analyzed to find symbol
  dependencies (Serge Guelton is aware).
- Fixed CI failures.
v3:
- Added __attribute__((weak)) to get_ptr_*() [Florian]
- Replace rdfsbase with mov %%fs:0,%0 [Florian]

This patch series solves the coroutines TLS problem. Coroutines re-entered from
another thread sometimes see stale TLS values. This happens because compilers
may cache values across yield points, so a value from the previous thread will
be used when the coroutine is re-entered in another thread.

Serge Guelton developed a portable technique, see the first patch for details.

I have audited all __thread variables in QEMU and converted those that can be
used from coroutines. Most actually look safe to me.

Stefan Hajnoczi (4):
  tls: add macros for coroutine-safe TLS variables
  util/async: replace __thread with QEMU TLS macros
  rcu: use coroutine TLS macros
  cpus: use coroutine TLS macros for iothread_locked

 include/qemu/coroutine-tls.h | 165 +++++++++++++++++++++++++++++++++++
 include/qemu/rcu.h           |   7 +-
 softmmu/cpus.c               |   9 +-
 tests/unit/rcutorture.c      |  10 +--
 tests/unit/test-rcu-list.c   |   4 +-
 util/async.c                 |  12 +--
 util/rcu.c                   |  10 +--
 7 files changed, 193 insertions(+), 24 deletions(-)
 create mode 100644 include/qemu/coroutine-tls.h

-- 
2.34.1

On Tue, Feb 22, 2022 at 02:01:46PM +0000, Stefan Hajnoczi wrote:
> v5:
> - Added explicit "#include "qemu/coroutine-tls.h" in patch 4 [Philippe]
> - Updated patch 1 commit description and comments to describe the current
>   noinline plus asm volatile approach [Peter]
> v4:
> - Dropped '[RFC]'.
> - Dropped inline asm for now. -fPIC versions of the code are missing and I
>   hit several issues including a clang LTO bug where thread local variables are
>   incorrectly discarded because inline asm is not analyzed to find symbol
>   dependencies (Serge Guelton is aware).
> - Fixed CI failures.
> v3:
> - Added __attribute__((weak)) to get_ptr_*() [Florian]
> - Replace rdfsbase with mov %%fs:0,%0 [Florian]
> 
> This patch series solves the coroutines TLS problem. Coroutines re-entered from
> another thread sometimes see stale TLS values. This happens because compilers
> may cache values across yield points, so a value from the previous thread will
> be used when the coroutine is re-entered in another thread.
> 
> Serge Guelton developed a portable technique, see the first patch for details.
> 
> I have audited all __thread variables in QEMU and converted those that can be
> used from coroutines. Most actually look safe to me.
> 
> Stefan Hajnoczi (4):
>   tls: add macros for coroutine-safe TLS variables
>   util/async: replace __thread with QEMU TLS macros
>   rcu: use coroutine TLS macros
>   cpus: use coroutine TLS macros for iothread_locked
> 
>  include/qemu/coroutine-tls.h | 165 +++++++++++++++++++++++++++++++++++
>  include/qemu/rcu.h           |   7 +-
>  softmmu/cpus.c               |   9 +-
>  tests/unit/rcutorture.c      |  10 +--
>  tests/unit/test-rcu-list.c   |   4 +-
>  util/async.c                 |  12 +--
>  util/rcu.c                   |  10 +--
>  7 files changed, 193 insertions(+), 24 deletions(-)
>  create mode 100644 include/qemu/coroutine-tls.h

Ping.

Stefan