It is too easy to mis-use tb_flush().  For instance, because of
the cpu argument, some parts assumed that it needed to call the
global flush function for every cpu.  It is easy to forget that
the flush is not complete when the call returns: we have merely
queued work to the cpu run loop.

(Phil, I suspect this second case is what's biting split-accel.)

So: remove tb_flush and expose only the core as tb_flush__exclusive,
to be used only when we are already within an exclusive context.

In some cases (gdbstub, alpha, riscv, ppc spapr), we can eliminate
the need for tb_flush completely.

Lightly tested so far, and I'm off on holiday next, but I thought
this might help others working on split-accel in the meantime.


r~


Richard Henderson (11):
  gdbstub: Remove tb_flush uses
  accel/tcg: Split out tb_flush__exclusive
  target/alpha: Simplify call_pal implementation
  target/riscv: Record misa_ext in TCGTBCPUState.cs_base
  accel/tcg: Move post-load tb_flush to vm_change_state hook
  hw/ppc/spapr: Use tb_invalidate_phys_range in h_page_init
  linux-user: Use tb_flush_exclusive to start second thread
  plugins: Use tb_flush__exclusive
  accel/tcg: Introduce EXCP_TB_FLUSH
  accel/tcg: Use EXCP_TB_FLUSH in tb_gen_code
  accel/tcg: Remove tb_flush

 include/exec/cpu-common.h       |  1 +
 include/exec/tb-flush.h         | 18 +++++++---------
 target/alpha/helper.h           |  1 -
 accel/tcg/tb-maint.c            | 38 +++++++--------------------------
 accel/tcg/tcg-accel-ops-mttcg.c |  7 ++++++
 accel/tcg/tcg-accel-ops-rr.c    |  9 ++++++--
 accel/tcg/tcg-all.c             | 21 ++++++++++++++++++
 accel/tcg/translate-all.c       |  5 +----
 gdbstub/system.c                |  4 ----
 gdbstub/user.c                  |  3 ---
 hw/core/cpu-system.c            |  8 -------
 hw/ppc/spapr_hcall.c            |  4 ++--
 linux-user/mmap.c               |  4 ++--
 linux-user/syscall.c            |  2 +-
 plugins/core.c                  |  6 ++----
 plugins/loader.c                |  2 +-
 target/alpha/sys_helper.c       |  6 ------
 target/alpha/translate.c        | 21 ++++++------------
 target/riscv/csr.c              |  3 ---
 target/riscv/tcg/tcg-cpu.c      |  3 ++-
 20 files changed, 69 insertions(+), 97 deletions(-)

-- 
2.43.0

On 2025-09-06 07:18, Richard Henderson wrote:
> It is too easy to mis-use tb_flush().  For instance, because of
> the cpu argument, some parts assumed that it needed to call the
> global flush function for every cpu.  It is easy to forget that
> the flush is not complete when the call returns: we have merely
> queued work to the cpu run loop.
>

That's a good idea, and will remove the confusion that sometimes the 
tb_flush is immediate, and sometimes queued to run asynchronously, 
depending on current context.

> (Phil, I suspect this second case is what's biting split-accel.)
> 
> So: remove tb_flush and expose only the core as tb_flush__exclusive,
> to be used only when we are already within an exclusive context.
> 
> In some cases (gdbstub, alpha, riscv, ppc spapr), we can eliminate
> the need for tb_flush completely.
> 
> Lightly tested so far, and I'm off on holiday next, but I thought
> this might help others working on split-accel in the meantime.
> 
> 
> r~
> 
> 
> Richard Henderson (11):
>    gdbstub: Remove tb_flush uses
>    accel/tcg: Split out tb_flush__exclusive
>    target/alpha: Simplify call_pal implementation
>    target/riscv: Record misa_ext in TCGTBCPUState.cs_base
>    accel/tcg: Move post-load tb_flush to vm_change_state hook
>    hw/ppc/spapr: Use tb_invalidate_phys_range in h_page_init
>    linux-user: Use tb_flush_exclusive to start second thread
>    plugins: Use tb_flush__exclusive
>    accel/tcg: Introduce EXCP_TB_FLUSH
>    accel/tcg: Use EXCP_TB_FLUSH in tb_gen_code
>    accel/tcg: Remove tb_flush
> 
>   include/exec/cpu-common.h       |  1 +
>   include/exec/tb-flush.h         | 18 +++++++---------
>   target/alpha/helper.h           |  1 -
>   accel/tcg/tb-maint.c            | 38 +++++++--------------------------
>   accel/tcg/tcg-accel-ops-mttcg.c |  7 ++++++
>   accel/tcg/tcg-accel-ops-rr.c    |  9 ++++++--
>   accel/tcg/tcg-all.c             | 21 ++++++++++++++++++
>   accel/tcg/translate-all.c       |  5 +----
>   gdbstub/system.c                |  4 ----
>   gdbstub/user.c                  |  3 ---
>   hw/core/cpu-system.c            |  8 -------
>   hw/ppc/spapr_hcall.c            |  4 ++--
>   linux-user/mmap.c               |  4 ++--
>   linux-user/syscall.c            |  2 +-
>   plugins/core.c                  |  6 ++----
>   plugins/loader.c                |  2 +-
>   target/alpha/sys_helper.c       |  6 ------
>   target/alpha/translate.c        | 21 ++++++------------
>   target/riscv/csr.c              |  3 ---
>   target/riscv/tcg/tcg-cpu.c      |  3 ++-
>   20 files changed, 69 insertions(+), 97 deletions(-)
>

+cc target and migration maintainers

On 9/6/25 07:18, Richard Henderson wrote:
> It is too easy to mis-use tb_flush().  For instance, because of
> the cpu argument, some parts assumed that it needed to call the
> global flush function for every cpu.  It is easy to forget that
> the flush is not complete when the call returns: we have merely
> queued work to the cpu run loop.
> 
> (Phil, I suspect this second case is what's biting split-accel.)
> 
> So: remove tb_flush and expose only the core as tb_flush__exclusive,
> to be used only when we are already within an exclusive context.
> 
> In some cases (gdbstub, alpha, riscv, ppc spapr), we can eliminate
> the need for tb_flush completely.
> 
> Lightly tested so far, and I'm off on holiday next, but I thought
> this might help others working on split-accel in the meantime.
> 
> 
> r~
> 
> 
> Richard Henderson (11):
>    gdbstub: Remove tb_flush uses
>    accel/tcg: Split out tb_flush__exclusive
>    target/alpha: Simplify call_pal implementation
>    target/riscv: Record misa_ext in TCGTBCPUState.cs_base
>    accel/tcg: Move post-load tb_flush to vm_change_state hook
>    hw/ppc/spapr: Use tb_invalidate_phys_range in h_page_init
>    linux-user: Use tb_flush_exclusive to start second thread
>    plugins: Use tb_flush__exclusive
>    accel/tcg: Introduce EXCP_TB_FLUSH
>    accel/tcg: Use EXCP_TB_FLUSH in tb_gen_code
>    accel/tcg: Remove tb_flush
> 
>   include/exec/cpu-common.h       |  1 +
>   include/exec/tb-flush.h         | 18 +++++++---------
>   target/alpha/helper.h           |  1 -
>   accel/tcg/tb-maint.c            | 38 +++++++--------------------------
>   accel/tcg/tcg-accel-ops-mttcg.c |  7 ++++++
>   accel/tcg/tcg-accel-ops-rr.c    |  9 ++++++--
>   accel/tcg/tcg-all.c             | 21 ++++++++++++++++++
>   accel/tcg/translate-all.c       |  5 +----
>   gdbstub/system.c                |  4 ----
>   gdbstub/user.c                  |  3 ---
>   hw/core/cpu-system.c            |  8 -------
>   hw/ppc/spapr_hcall.c            |  4 ++--
>   linux-user/mmap.c               |  4 ++--
>   linux-user/syscall.c            |  2 +-
>   plugins/core.c                  |  6 ++----
>   plugins/loader.c                |  2 +-
>   target/alpha/sys_helper.c       |  6 ------
>   target/alpha/translate.c        | 21 ++++++------------
>   target/riscv/csr.c              |  3 ---
>   target/riscv/tcg/tcg-cpu.c      |  3 ++-
>   20 files changed, 69 insertions(+), 97 deletions(-)
>