Series comparison

 [PULL 00/20] tcg patch queue
-TCG patch queue, plus one target/sh4 patch that
+The following changes since commit 9c6c079bc6723da8061ccfb44361d67b1dd785dd:
 Yoshinori Sato asked me to process.
+  Merge tag 'pull-target-arm-20240430' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2024-04-30 09:58:54 -0700)
 r~
 The following changes since commit efbf38d73e5dcc4d5f8b98c6e7a12be1f3b91745:
   Merge tag 'for-upstream' of git://repo.or.cz/qemu/kevin into staging (2022-10-03 15:06:07 -0400)
 are available in the Git repository at:
-  https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20221004
+  https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20240501
-for you to fetch changes up to ab419fd8a035a65942de4e63effcd55ccbf1a9fe:
+for you to fetch changes up to 917d7f8d948d706e275c9f33169b9dd0149ded1e:
-  target/sh4: Fix TB_FLAG_UNALIGN (2022-10-04 12:33:05 -0700)
+  plugins: Update the documentation block for plugin-gen.c (2024-04-30 16:12:05 -0700)
 ----------------------------------------------------------------
-Cache CPUClass for use in hot code paths.
+plugins: Rewrite plugin tcg expansion
 Add CPUTLBEntryFull, probe_access_full, tlb_set_page_full.
 Add generic support for TARGET_TB_PCREL.
 tcg/ppc: Optimize 26-bit jumps using STQ for POWER 2.07
 target/sh4: Fix TB_FLAG_UNALIGN
 ----------------------------------------------------------------
-Alex Bennée (3):
+Richard Henderson (20):
-      cpu: cache CPUClass in CPUState for hot code paths
+      tcg: Make tcg/helper-info.h self-contained
-      hw/core/cpu-sysemu: used cached class in cpu_asidx_from_attrs
+      tcg: Pass function pointer to tcg_gen_call*
-      cputlb: used cached CPUClass in our hot-paths
+      plugins: Zero new qemu_plugin_dyn_cb entries
       plugins: Move function pointer in qemu_plugin_dyn_cb
       plugins: Create TCGHelperInfo for all out-of-line callbacks
       plugins: Use emit_before_op for PLUGIN_GEN_AFTER_INSN
       plugins: Use emit_before_op for PLUGIN_GEN_FROM_TB
       plugins: Add PLUGIN_GEN_AFTER_TB
       plugins: Use emit_before_op for PLUGIN_GEN_FROM_INSN
       plugins: Use emit_before_op for PLUGIN_GEN_FROM_MEM
       plugins: Remove plugin helpers
       tcg: Remove TCG_CALL_PLUGIN
       tcg: Remove INDEX_op_plugin_cb_{start,end}
       plugins: Simplify callback queues
       plugins: Introduce PLUGIN_CB_MEM_REGULAR
       plugins: Replace pr_ops with a proper debug dump flag
       plugins: Split out common cb expanders
       plugins: Merge qemu_plugin_tb_insn_get to plugin-gen.c
       plugins: Inline plugin_gen_empty_callback
       plugins: Update the documentation block for plugin-gen.c
-Leandro Lupori (1):
+ accel/tcg/plugin-helpers.h         |    5 -
-      tcg/ppc: Optimize 26-bit jumps
+ include/exec/helper-gen-common.h   |    4 -
+ include/exec/helper-proto-common.h |    4 -
-Richard Henderson (16):
+ include/exec/plugin-gen.h          |    4 -
-      accel/tcg: Rename CPUIOTLBEntry to CPUTLBEntryFull
+ include/qemu/log.h                 |    1 +
-      accel/tcg: Drop addr member from SavedIOTLB
+ include/qemu/plugin.h              |   67 +--
-      accel/tcg: Suppress auto-invalidate in probe_access_internal
+ include/tcg/helper-info.h          |    3 +
-      accel/tcg: Introduce probe_access_full
+ include/tcg/tcg-op-common.h        |    4 +-
-      accel/tcg: Introduce tlb_set_page_full
+ include/tcg/tcg-opc.h              |    4 +-
-      include/exec: Introduce TARGET_PAGE_ENTRY_EXTRA
+ include/tcg/tcg.h                  |   26 +-
-      accel/tcg: Remove PageDesc code_bitmap
+ include/exec/helper-gen.h.inc      |   24 +-
-      accel/tcg: Use bool for page_find_alloc
+ accel/tcg/plugin-gen.c             | 1007 +++++++++---------------------------
-      accel/tcg: Use DisasContextBase in plugin_gen_tb_start
+ plugins/api.c                      |   26 +-
-      accel/tcg: Do not align tb->page_addr[0]
+ plugins/core.c                     |   61 ++-
-      accel/tcg: Inline tb_flush_jmp_cache
+ tcg/tcg-op-ldst.c                  |    6 +-
-      include/hw/core: Create struct CPUJumpCache
+ tcg/tcg-op.c                       |    8 +-
-      hw/core: Add CPUClass.get_pc
+ tcg/tcg.c                          |   78 ++-
-      accel/tcg: Introduce tb_pc and log_pc
+ tcg/tci.c                          |    1 +
-      accel/tcg: Introduce TARGET_TB_PCREL
+ util/log.c                         |    4 +
-      target/sh4: Fix TB_FLAG_UNALIGN
+files changed, 399 insertions(+), 938 deletions(-)
+ delete mode 100644 accel/tcg/plugin-helpers.h
  accel/tcg/internal.h                    |  10 ++
  accel/tcg/tb-hash.h                     |   1 +
  accel/tcg/tb-jmp-cache.h                |  65 ++++++++
  include/exec/cpu-common.h               |   1 +
  include/exec/cpu-defs.h                 |  48 ++++--
  include/exec/exec-all.h                 |  75 ++++++++-
  include/exec/plugin-gen.h               |   7 +-
  include/hw/core/cpu.h                   |  28 ++--
  include/qemu/typedefs.h                 |   2 +
  include/tcg/tcg.h                       |   2 +-
  target/sh4/cpu.h                        |  56 ++++---
  accel/stubs/tcg-stub.c                  |   4 +
  accel/tcg/cpu-exec.c                    |  80 +++++-----
  accel/tcg/cputlb.c                      | 259 ++++++++++++++++++--------------
  accel/tcg/plugin-gen.c                  |  22 +--
  accel/tcg/translate-all.c               | 214 ++++++++++++--------------
  accel/tcg/translator.c                  |   2 +-
  cpu.c                                   |   9 +-
  hw/core/cpu-common.c                    |   3 +-
  hw/core/cpu-sysemu.c                    |   5 +-
  linux-user/sh4/signal.c                 |   6 +-
  plugins/core.c                          |   2 +-
  target/alpha/cpu.c                      |   9 ++
  target/arm/cpu.c                        |  17 ++-
  target/arm/mte_helper.c                 |  14 +-
  target/arm/sve_helper.c                 |   4 +-
  target/arm/translate-a64.c              |   2 +-
  target/avr/cpu.c                        |  10 +-
  target/cris/cpu.c                       |   8 +
  target/hexagon/cpu.c                    |  10 +-
  target/hppa/cpu.c                       |  12 +-
  target/i386/cpu.c                       |   9 ++
  target/i386/tcg/tcg-cpu.c               |   2 +-
  target/loongarch/cpu.c                  |  11 +-
  target/m68k/cpu.c                       |   8 +
  target/microblaze/cpu.c                 |  10 +-
  target/mips/cpu.c                       |   8 +
  target/mips/tcg/exception.c             |   2 +-
  target/mips/tcg/sysemu/special_helper.c |   2 +-
  target/nios2/cpu.c                      |   9 ++
  target/openrisc/cpu.c                   |  10 +-
  target/ppc/cpu_init.c                   |   8 +
  target/riscv/cpu.c                      |  17 ++-
  target/rx/cpu.c                         |  10 +-
  target/s390x/cpu.c                      |   8 +
  target/s390x/tcg/mem_helper.c           |   4 -
  target/sh4/cpu.c                        |  18 ++-
  target/sh4/helper.c                     |   6 +-
  target/sh4/translate.c                  |  90 +++++------
  target/sparc/cpu.c                      |  10 +-
  target/tricore/cpu.c                    |  11 +-
  target/xtensa/cpu.c                     |   8 +
  tcg/tcg.c                               |   8 +-
  trace/control-target.c                  |   2 +-
  tcg/ppc/tcg-target.c.inc                | 119 +++++++++++----
 files changed, 915 insertions(+), 462 deletions(-)
  create mode 100644 accel/tcg/tb-jmp-cache.h

-[PULL 06/20] accel/tcg: Suppress auto-invalidate in probe_access_internal
+[PULL 01/20] tcg: Make tcg/helper-info.h self-contained
-When PAGE_WRITE_INV is set when calling tlb_set_page,
+Move MAX_CALL_IARGS from tcg.h and include for
-we immediately set TLB_INVALID_MASK in order to force
+the define of TCG_TARGET_REG_BITS.
 tlb_fill to be called on the next lookup.  Here in
 probe_access_internal, we have just called tlb_fill
 and eliminated true misses, thus the lookup must be valid.
 This allows us to remove a warning comment from s390x.
 There doesn't seem to be a reason to change the code though.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Reviewed-by: David Hildenbrand <david@redhat.com>
+Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
 Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/cputlb.c            | 10 +++++++++-
+ include/tcg/helper-info.h | 3 +++
- target/s390x/tcg/mem_helper.c |  4 ----
+ include/tcg/tcg.h         | 2 --
-files changed, 9 insertions(+), 5 deletions(-)
+ tcg/tci.c                 | 1 +
 files changed, 4 insertions(+), 2 deletions(-)
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
+diff --git a/include/tcg/helper-info.h b/include/tcg/helper-info.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
+--- a/include/tcg/helper-info.h
-+++ b/accel/tcg/cputlb.c
++++ b/include/tcg/helper-info.h
-@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
+@@ -XXX,XX +XXX,XX @@
-     }
+ #ifdef CONFIG_TCG_INTERPRETER
-     tlb_addr = tlb_read_ofs(entry, elt_ofs);
+ #include <ffi.h>
+ #endif
-+    flags = TLB_FLAGS_MASK;
++#include "tcg-target-reg-bits.h"
      page_addr = addr & TARGET_PAGE_MASK;
      if (!tlb_hit_page(tlb_addr, page_addr)) {
          if (!victim_tlb_hit(env, mmu_idx, index, elt_ofs, page_addr)) {
@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
              /* TLB resize via tlb_fill may have moved the entry.  */
              entry = tlb_entry(env, mmu_idx, addr);
 +
-+            /*
++#define MAX_CALL_IARGS  7
-+             * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately,
-+             * to force the next access through tlb_fill.  We've just
+ /*
-+             * called tlb_fill, so we know that this entry *is* valid.
+  * Describe the calling convention of a given argument type.
-+             */
+diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
 +            flags &= ~TLB_INVALID_MASK;
          }
          tlb_addr = tlb_read_ofs(entry, elt_ofs);
      }
 -    flags = tlb_addr & TLB_FLAGS_MASK;
 +    flags &= tlb_addr;
      /* Fold all "mmio-like" bits into TLB_MMIO.  This is not RAM.  */
      if (unlikely(flags & ~(TLB_WATCHPOINT | TLB_NOTDIRTY))) {
 diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c
 index XXXXXXX..XXXXXXX 100644
---- a/target/s390x/tcg/mem_helper.c
+--- a/include/tcg/tcg.h
-+++ b/target/s390x/tcg/mem_helper.c
++++ b/include/tcg/tcg.h
-@@ -XXX,XX +XXX,XX @@ static int s390_probe_access(CPUArchState *env, target_ulong addr, int size,
+@@ -XXX,XX +XXX,XX @@
- #else
+ /* XXX: make safe guess about sizes */
-     int flags;
+ #define MAX_OP_PER_INSTR 266
--    /*
+-#define MAX_CALL_IARGS  7
--     * For !CONFIG_USER_ONLY, we cannot rely on TLB_INVALID_MASK or haddr==NULL
+-
--     * to detect if there was an exception during tlb_fill().
+ #define CPU_TEMP_BUF_NLONGS 128
--     */
+ #define TCG_STATIC_FRAME_SIZE  (CPU_TEMP_BUF_NLONGS * sizeof(long))
-     env->tlb_fill_exc = 0;
-     flags = probe_access_flags(env, addr, access_type, mmu_idx, nonfault, phost,
+diff --git a/tcg/tci.c b/tcg/tci.c
-                                ra);
+index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tci.c
 +++ b/tcg/tci.c
@@ -XXX,XX +XXX,XX @@
  #include "qemu/osdep.h"
  #include "tcg/tcg.h"
 +#include "tcg/helper-info.h"
  #include "tcg/tcg-ldst.h"
  #include <ffi.h>
 --
 .34.1

-[PULL 10/20] accel/tcg: Remove PageDesc code_bitmap
+[PULL 02/20] tcg: Pass function pointer to tcg_gen_call*
-This bitmap is created and discarded immediately.
+For normal helpers, read the function pointer from the
-We gain nothing by its existence.
+structure earlier.  For plugins, this will allow the
 function pointer to come from elsewhere.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
-Message-Id: <20220822232338.1727934-2-richard.henderson@linaro.org>
 ---
- accel/tcg/translate-all.c | 78 ++-------------------------------------
+ include/tcg/tcg.h             | 21 +++++++++-------
-file changed, 4 insertions(+), 74 deletions(-)
+ include/exec/helper-gen.h.inc | 24 ++++++++++++-------
+ tcg/tcg.c                     | 45 +++++++++++++++++++----------------
-diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
+files changed, 52 insertions(+), 38 deletions(-)
 diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/translate-all.c
+--- a/include/tcg/tcg.h
-+++ b/accel/tcg/translate-all.c
++++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ typedef struct TCGTargetOpDef {
  bool tcg_op_supported(TCGOpcode op);
 -void tcg_gen_call0(TCGHelperInfo *, TCGTemp *ret);
 -void tcg_gen_call1(TCGHelperInfo *, TCGTemp *ret, TCGTemp *);
 -void tcg_gen_call2(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *);
 -void tcg_gen_call3(TCGHelperInfo *, TCGTemp *ret, TCGTemp *,
 +void tcg_gen_call0(void *func, TCGHelperInfo *, TCGTemp *ret);
 +void tcg_gen_call1(void *func, TCGHelperInfo *, TCGTemp *ret, TCGTemp *);
 +void tcg_gen_call2(void *func, TCGHelperInfo *, TCGTemp *ret,
                     TCGTemp *, TCGTemp *);
 -void tcg_gen_call4(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *,
 -                   TCGTemp *, TCGTemp *);
 -void tcg_gen_call5(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *,
 +void tcg_gen_call3(void *func, TCGHelperInfo *, TCGTemp *ret,
                     TCGTemp *, TCGTemp *, TCGTemp *);
 -void tcg_gen_call6(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *,
 +void tcg_gen_call4(void *func, TCGHelperInfo *, TCGTemp *ret,
                     TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *);
 -void tcg_gen_call7(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *,
 +void tcg_gen_call5(void *func, TCGHelperInfo *, TCGTemp *ret,
                     TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *);
 +void tcg_gen_call6(void *func, TCGHelperInfo *, TCGTemp *ret,
 +                   TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *,
 +                   TCGTemp *, TCGTemp *);
 +void tcg_gen_call7(void *func, TCGHelperInfo *, TCGTemp *ret,
 +                   TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *,
 +                   TCGTemp *, TCGTemp *, TCGTemp *);
  TCGOp *tcg_emit_op(TCGOpcode opc, unsigned nargs);
  void tcg_op_remove(TCGContext *s, TCGOp *op);
 diff --git a/include/exec/helper-gen.h.inc b/include/exec/helper-gen.h.inc
 index XXXXXXX..XXXXXXX 100644
 --- a/include/exec/helper-gen.h.inc
 +++ b/include/exec/helper-gen.h.inc
 @@ -XXX,XX +XXX,XX @@
- #define assert_memory_lock() tcg_debug_assert(have_mmap_lock())
+ extern TCGHelperInfo glue(helper_info_, name);                          \
- #endif
+ static inline void glue(gen_helper_, name)(dh_retvar_decl0(ret))        \
+ {                                                                       \
--#define SMC_BITMAP_USE_THRESHOLD 10
+-    tcg_gen_call0(&glue(helper_info_, name), dh_retvar(ret));           \
--
++    tcg_gen_call0(glue(helper_info_,name).func,                         \
- typedef struct PageDesc {
++                  &glue(helper_info_,name), dh_retvar(ret));            \
-     /* list of TBs intersecting this ram page */
+ }
-     uintptr_t first_tb;
--#ifdef CONFIG_SOFTMMU
+ #define DEF_HELPER_FLAGS_1(name, flags, ret, t1)                        \
--    /* in order to optimize self modifying code, we count the number
+@@ -XXX,XX +XXX,XX @@ extern TCGHelperInfo glue(helper_info_, name);                          \
--       of lookups we do to a given page to use a bitmap */
+ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
--    unsigned long *code_bitmap;
+     dh_arg_decl(t1, 1))                                                 \
--    unsigned int code_write_count;
+ {                                                                       \
--#else
+-    tcg_gen_call1(&glue(helper_info_, name), dh_retvar(ret),            \
-+#ifdef CONFIG_USER_ONLY
++    tcg_gen_call1(glue(helper_info_,name).func,                         \
-     unsigned long flags;
++                  &glue(helper_info_,name), dh_retvar(ret),             \
-     void *target_data;
+                   dh_arg(t1, 1));                                       \
- #endif
+ }
--#ifndef CONFIG_USER_ONLY
-+#ifdef CONFIG_SOFTMMU
+@@ -XXX,XX +XXX,XX @@ extern TCGHelperInfo glue(helper_info_, name);                          \
-     QemuSpin lock;
+ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
- #endif
+     dh_arg_decl(t1, 1), dh_arg_decl(t2, 2))                             \
- } PageDesc;
+ {                                                                       \
-@@ -XXX,XX +XXX,XX @@ void tb_htable_init(void)
+-    tcg_gen_call2(&glue(helper_info_, name), dh_retvar(ret),            \
-     qht_init(&tb_ctx.htable, tb_cmp, CODE_GEN_HTABLE_SIZE, mode);
++    tcg_gen_call2(glue(helper_info_,name).func,                         \
- }
++                  &glue(helper_info_,name), dh_retvar(ret),             \
+                   dh_arg(t1, 1), dh_arg(t2, 2));                        \
--/* call with @p->lock held */
+ }
--static inline void invalidate_page_bitmap(PageDesc *p)
--{
+@@ -XXX,XX +XXX,XX @@ extern TCGHelperInfo glue(helper_info_, name);                          \
--    assert_page_locked(p);
+ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
--#ifdef CONFIG_SOFTMMU
+     dh_arg_decl(t1, 1), dh_arg_decl(t2, 2), dh_arg_decl(t3, 3))         \
--    g_free(p->code_bitmap);
+ {                                                                       \
--    p->code_bitmap = NULL;
+-    tcg_gen_call3(&glue(helper_info_, name), dh_retvar(ret),            \
--    p->code_write_count = 0;
++    tcg_gen_call3(glue(helper_info_,name).func,                         \
--#endif
++                  &glue(helper_info_,name), dh_retvar(ret),             \
--}
+                   dh_arg(t1, 1), dh_arg(t2, 2), dh_arg(t3, 3));         \
--
+ }
- /* Set to NULL all the 'first_tb' fields in all PageDescs. */
- static void page_flush_tb_1(int level, void **lp)
+@@ -XXX,XX +XXX,XX @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
- {
+     dh_arg_decl(t1, 1), dh_arg_decl(t2, 2),                             \
-@@ -XXX,XX +XXX,XX @@ static void page_flush_tb_1(int level, void **lp)
+     dh_arg_decl(t3, 3), dh_arg_decl(t4, 4))                             \
-         for (i = 0; i < V_L2_SIZE; ++i) {
+ {                                                                       \
-             page_lock(&pd[i]);
+-    tcg_gen_call4(&glue(helper_info_, name), dh_retvar(ret),            \
-             pd[i].first_tb = (uintptr_t)NULL;
++    tcg_gen_call4(glue(helper_info_,name).func,                         \
--            invalidate_page_bitmap(pd + i);
++                  &glue(helper_info_,name), dh_retvar(ret),             \
-             page_unlock(&pd[i]);
+                   dh_arg(t1, 1), dh_arg(t2, 2),                         \
-         }
+                   dh_arg(t3, 3), dh_arg(t4, 4));                        \
-     } else {
+ }
-@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
+@@ -XXX,XX +XXX,XX @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
-     if (rm_from_page_list) {
+     dh_arg_decl(t1, 1), dh_arg_decl(t2, 2), dh_arg_decl(t3, 3),         \
-         p = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS);
+     dh_arg_decl(t4, 4), dh_arg_decl(t5, 5))                             \
-         tb_page_remove(p, tb);
+ {                                                                       \
--        invalidate_page_bitmap(p);
+-    tcg_gen_call5(&glue(helper_info_, name), dh_retvar(ret),            \
-         if (tb->page_addr[1] != -1) {
++    tcg_gen_call5(glue(helper_info_,name).func,                         \
-             p = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS);
++                  &glue(helper_info_,name), dh_retvar(ret),             \
-             tb_page_remove(p, tb);
+                   dh_arg(t1, 1), dh_arg(t2, 2), dh_arg(t3, 3),          \
--            invalidate_page_bitmap(p);
+                   dh_arg(t4, 4), dh_arg(t5, 5));                        \
  }
@@ -XXX,XX +XXX,XX @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
      dh_arg_decl(t1, 1), dh_arg_decl(t2, 2), dh_arg_decl(t3, 3),         \
      dh_arg_decl(t4, 4), dh_arg_decl(t5, 5), dh_arg_decl(t6, 6))         \
  {                                                                       \
 -    tcg_gen_call6(&glue(helper_info_, name), dh_retvar(ret),            \
 +    tcg_gen_call6(glue(helper_info_,name).func,                         \
 +                  &glue(helper_info_,name), dh_retvar(ret),             \
                    dh_arg(t1, 1), dh_arg(t2, 2), dh_arg(t3, 3),          \
                    dh_arg(t4, 4), dh_arg(t5, 5), dh_arg(t6, 6));         \
  }
@@ -XXX,XX +XXX,XX @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
      dh_arg_decl(t4, 4), dh_arg_decl(t5, 5), dh_arg_decl(t6, 6),         \
      dh_arg_decl(t7, 7))                                                 \
  {                                                                       \
 -    tcg_gen_call7(&glue(helper_info_, name), dh_retvar(ret),            \
 +    tcg_gen_call7(glue(helper_info_,name).func,                         \
 +                  &glue(helper_info_,name), dh_retvar(ret),             \
                    dh_arg(t1, 1), dh_arg(t2, 2), dh_arg(t3, 3),          \
                    dh_arg(t4, 4), dh_arg(t5, 5), dh_arg(t6, 6),          \
                    dh_arg(t7, 7));                                       \
 diff --git a/tcg/tcg.c b/tcg/tcg.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg.c
 +++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ bool tcg_op_supported(TCGOpcode op)
  static TCGOp *tcg_op_alloc(TCGOpcode opc, unsigned nargs);
 -static void tcg_gen_callN(TCGHelperInfo *info, TCGTemp *ret, TCGTemp **args)
 +static void tcg_gen_callN(void *func, TCGHelperInfo *info,
 +                          TCGTemp *ret, TCGTemp **args)
  {
      TCGv_i64 extend_free[MAX_CALL_IARGS];
      int n_extend = 0;
@@ -XXX,XX +XXX,XX @@ static void tcg_gen_callN(TCGHelperInfo *info, TCGTemp *ret, TCGTemp **args)
              g_assert_not_reached();
          }
      }
+-    op->args[pi++] = (uintptr_t)info->func;
-@@ -XXX,XX +XXX,XX @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
++    op->args[pi++] = (uintptr_t)func;
      op->args[pi++] = (uintptr_t)info;
      tcg_debug_assert(pi == total_args);
@@ -XXX,XX +XXX,XX @@ static void tcg_gen_callN(TCGHelperInfo *info, TCGTemp *ret, TCGTemp **args)
      }
  }
--#ifdef CONFIG_SOFTMMU
+-void tcg_gen_call0(TCGHelperInfo *info, TCGTemp *ret)
--/* call with @p->lock held */
++void tcg_gen_call0(void *func, TCGHelperInfo *info, TCGTemp *ret)
--static void build_page_bitmap(PageDesc *p)
+ {
--{
+-    tcg_gen_callN(info, ret, NULL);
--    int n, tb_start, tb_end;
++    tcg_gen_callN(func, info, ret, NULL);
--    TranslationBlock *tb;
+ }
--
--    assert_page_locked(p);
+-void tcg_gen_call1(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1)
--    p->code_bitmap = bitmap_new(TARGET_PAGE_SIZE);
++void tcg_gen_call1(void *func, TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1)
--
+ {
--    PAGE_FOR_EACH_TB(p, tb, n) {
+-    tcg_gen_callN(info, ret, &t1);
--        /* NOTE: this is subtle as a TB may span two physical pages */
++    tcg_gen_callN(func, info, ret, &t1);
--        if (n == 0) {
+ }
--            /* NOTE: tb_end may be after the end of the page, but
--               it is not a problem */
+-void tcg_gen_call2(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1, TCGTemp *t2)
--            tb_start = tb->pc & ~TARGET_PAGE_MASK;
++void tcg_gen_call2(void *func, TCGHelperInfo *info, TCGTemp *ret,
--            tb_end = tb_start + tb->size;
++                   TCGTemp *t1, TCGTemp *t2)
--            if (tb_end > TARGET_PAGE_SIZE) {
+ {
--                tb_end = TARGET_PAGE_SIZE;
+     TCGTemp *args[2] = { t1, t2 };
--             }
+-    tcg_gen_callN(info, ret, args);
--        } else {
++    tcg_gen_callN(func, info, ret, args);
--            tb_start = 0;
+ }
--            tb_end = ((tb->pc + tb->size) & ~TARGET_PAGE_MASK);
--        }
+-void tcg_gen_call3(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
--        bitmap_set(p->code_bitmap, tb_start, tb_end - tb_start);
+-                   TCGTemp *t2, TCGTemp *t3)
--    }
++void tcg_gen_call3(void *func, TCGHelperInfo *info, TCGTemp *ret,
--}
++                   TCGTemp *t1, TCGTemp *t2, TCGTemp *t3)
--#endif
+ {
--
+     TCGTemp *args[3] = { t1, t2, t3 };
- /* add the tb in the target page and protect it if necessary
+-    tcg_gen_callN(info, ret, args);
-  *
++    tcg_gen_callN(func, info, ret, args);
-  * Called with mmap_lock held for user-mode emulation.
+ }
-@@ -XXX,XX +XXX,XX @@ static inline void tb_page_add(PageDesc *p, TranslationBlock *tb,
-     page_already_protected = p->first_tb != (uintptr_t)NULL;
+-void tcg_gen_call4(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
- #endif
+-                   TCGTemp *t2, TCGTemp *t3, TCGTemp *t4)
-     p->first_tb = (uintptr_t)tb | n;
++void tcg_gen_call4(void *func, TCGHelperInfo *info, TCGTemp *ret,
--    invalidate_page_bitmap(p);
++                   TCGTemp *t1, TCGTemp *t2, TCGTemp *t3, TCGTemp *t4)
+ {
- #if defined(CONFIG_USER_ONLY)
+     TCGTemp *args[4] = { t1, t2, t3, t4 };
-     /* translator_loop() must have made all TB pages non-writable */
+-    tcg_gen_callN(info, ret, args);
-@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
++    tcg_gen_callN(func, info, ret, args);
-     /* remove TB from the page(s) if we couldn't insert it */
+ }
-     if (unlikely(existing_tb)) {
-         tb_page_remove(p, tb);
+-void tcg_gen_call5(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
--        invalidate_page_bitmap(p);
++void tcg_gen_call5(void *func, TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
-         if (p2) {
+                    TCGTemp *t2, TCGTemp *t3, TCGTemp *t4, TCGTemp *t5)
-             tb_page_remove(p2, tb);
+ {
--            invalidate_page_bitmap(p2);
+     TCGTemp *args[5] = { t1, t2, t3, t4, t5 };
-         }
+-    tcg_gen_callN(info, ret, args);
-         tb = existing_tb;
++    tcg_gen_callN(func, info, ret, args);
-     }
+ }
-@@ -XXX,XX +XXX,XX @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
- #if !defined(CONFIG_USER_ONLY)
+-void tcg_gen_call6(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1, TCGTemp *t2,
-     /* if no code remaining, no need to continue to use slow writes */
+-                   TCGTemp *t3, TCGTemp *t4, TCGTemp *t5, TCGTemp *t6)
-     if (!p->first_tb) {
++void tcg_gen_call6(void *func, TCGHelperInfo *info, TCGTemp *ret,
--        invalidate_page_bitmap(p);
++                   TCGTemp *t1, TCGTemp *t2, TCGTemp *t3,
-         tlb_unprotect_code(start);
++                   TCGTemp *t4, TCGTemp *t5, TCGTemp *t6)
-     }
+ {
- #endif
+     TCGTemp *args[6] = { t1, t2, t3, t4, t5, t6 };
-@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_page_fast(struct page_collection *pages,
+-    tcg_gen_callN(info, ret, args);
-     }
++    tcg_gen_callN(func, info, ret, args);
+ }
-     assert_page_locked(p);
--    if (!p->code_bitmap &&
+-void tcg_gen_call7(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
--        ++p->code_write_count >= SMC_BITMAP_USE_THRESHOLD) {
++void tcg_gen_call7(void *func, TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
--        build_page_bitmap(p);
+                    TCGTemp *t2, TCGTemp *t3, TCGTemp *t4,
--    }
+                    TCGTemp *t5, TCGTemp *t6, TCGTemp *t7)
--    if (p->code_bitmap) {
+ {
--        unsigned int nr;
+     TCGTemp *args[7] = { t1, t2, t3, t4, t5, t6, t7 };
--        unsigned long b;
+-    tcg_gen_callN(info, ret, args);
--
++    tcg_gen_callN(func, info, ret, args);
--        nr = start & ~TARGET_PAGE_MASK;
+ }
--        b = p->code_bitmap[BIT_WORD(nr)] >> (nr & (BITS_PER_LONG - 1));
--        if (b & ((1 << len) - 1)) {
+ static void tcg_reg_alloc_start(TCGContext *s)
 -            goto do_invalidate;
 -        }
 -    } else {
 -    do_invalidate:
 -        tb_invalidate_phys_page_range__locked(pages, p, start, start + len,
 -                                              retaddr);
 -    }
 +    tb_invalidate_phys_page_range__locked(pages, p, start, start + len,
 +                                          retaddr);
  }
  #else
  /* Called with mmap_lock held. If pc is not 0 then it indicates the
 --
 .34.1

-[PULL 13/20] accel/tcg: Do not align tb->page_addr[0]
+[PULL 03/20] plugins: Zero new qemu_plugin_dyn_cb entries
-Let tb->page_addr[0] contain the address of the first byte of the
-translated block, rather than the address of the page containing the
-start of the translated block.  We need to recover this value anyway
-at various points, and it is easier to discard a page offset when it
-is not needed, which happens naturally via the existing find_page shift.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/cpu-exec.c      | 16 ++++++++--------
+ plugins/core.c | 2 +-
- accel/tcg/cputlb.c        |  3 ++-
+file changed, 1 insertion(+), 1 deletion(-)
  accel/tcg/translate-all.c |  9 +++++----
 files changed, 15 insertions(+), 13 deletions(-)
-diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
+diff --git a/plugins/core.c b/plugins/core.c
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cpu-exec.c
+--- a/plugins/core.c
-+++ b/accel/tcg/cpu-exec.c
++++ b/plugins/core.c
-@@ -XXX,XX +XXX,XX @@ struct tb_desc {
+@@ -XXX,XX +XXX,XX @@ static struct qemu_plugin_dyn_cb *plugin_get_dyn_cb(GArray **arr)
-     target_ulong pc;
+     GArray *cbs = *arr;
-     target_ulong cs_base;
-     CPUArchState *env;
+     if (!cbs) {
--    tb_page_addr_t phys_page1;
+-        cbs = g_array_sized_new(false, false,
-+    tb_page_addr_t page_addr0;
++        cbs = g_array_sized_new(false, true,
-     uint32_t flags;
+                                 sizeof(struct qemu_plugin_dyn_cb), 1);
-     uint32_t cflags;
+         *arr = cbs;
      uint32_t trace_vcpu_dstate;
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
      const struct tb_desc *desc = d;
      if (tb->pc == desc->pc &&
 -        tb->page_addr[0] == desc->phys_page1 &&
 +        tb->page_addr[0] == desc->page_addr0 &&
          tb->cs_base == desc->cs_base &&
          tb->flags == desc->flags &&
          tb->trace_vcpu_dstate == desc->trace_vcpu_dstate &&
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
          if (tb->page_addr[1] == -1) {
              return true;
          } else {
 -            tb_page_addr_t phys_page2;
 -            target_ulong virt_page2;
 +            tb_page_addr_t phys_page1;
 +            target_ulong virt_page1;
              /*
               * We know that the first page matched, and an otherwise valid TB
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
               * is different for the new TB.  Therefore any exception raised
               * here by the faulting lookup is not premature.
               */
 -            virt_page2 = TARGET_PAGE_ALIGN(desc->pc);
 -            phys_page2 = get_page_addr_code(desc->env, virt_page2);
 -            if (tb->page_addr[1] == phys_page2) {
 +            virt_page1 = TARGET_PAGE_ALIGN(desc->pc);
 +            phys_page1 = get_page_addr_code(desc->env, virt_page1);
 +            if (tb->page_addr[1] == phys_page1) {
                  return true;
              }
          }
@@ -XXX,XX +XXX,XX @@ static TranslationBlock *tb_htable_lookup(CPUState *cpu, target_ulong pc,
      if (phys_pc == -1) {
          return NULL;
      }
--    desc.phys_page1 = phys_pc & TARGET_PAGE_MASK;
-+    desc.page_addr0 = phys_pc;
-     h = tb_hash_func(phys_pc, pc, flags, cflags, *cpu->trace_dstate);
-     return qht_lookup_custom(&tb_ctx.htable, &desc, h, tb_lookup_cmp);
- }
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
-index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
-+++ b/accel/tcg/cputlb.c
-@@ -XXX,XX +XXX,XX @@ void tlb_flush_page_bits_by_mmuidx_all_cpus_synced(CPUState *src_cpu,
-    can be detected */
- void tlb_protect_code(ram_addr_t ram_addr)
- {
--    cpu_physical_memory_test_and_clear_dirty(ram_addr, TARGET_PAGE_SIZE,
-+    cpu_physical_memory_test_and_clear_dirty(ram_addr & TARGET_PAGE_MASK,
-+                                             TARGET_PAGE_SIZE,
-                                              DIRTY_MEMORY_CODE);
- }
-diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
-index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/translate-all.c
-+++ b/accel/tcg/translate-all.c
-@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
-     qemu_spin_unlock(&tb->jmp_lock);
-     /* remove the TB from the hash list */
--    phys_pc = tb->page_addr[0] + (tb->pc & ~TARGET_PAGE_MASK);
-+    phys_pc = tb->page_addr[0];
-     h = tb_hash_func(phys_pc, tb->pc, tb->flags, orig_cflags,
-                      tb->trace_vcpu_dstate);
-     if (!qht_remove(&tb_ctx.htable, tb, h)) {
-@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
-      * we can only insert TBs that are fully initialized.
-      */
-     page_lock_pair(&p, phys_pc, &p2, phys_page2, true);
--    tb_page_add(p, tb, 0, phys_pc & TARGET_PAGE_MASK);
-+    tb_page_add(p, tb, 0, phys_pc);
-     if (p2) {
-         tb_page_add(p2, tb, 1, phys_page2);
-     } else {
-@@ -XXX,XX +XXX,XX @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
-         if (n == 0) {
-             /* NOTE: tb_end may be after the end of the page, but
-                it is not a problem */
--            tb_start = tb->page_addr[0] + (tb->pc & ~TARGET_PAGE_MASK);
-+            tb_start = tb->page_addr[0];
-             tb_end = tb_start + tb->size;
-         } else {
-             tb_start = tb->page_addr[1];
--            tb_end = tb_start + ((tb->pc + tb->size) & ~TARGET_PAGE_MASK);
-+            tb_end = tb_start + ((tb->page_addr[0] + tb->size)
-+                                 & ~TARGET_PAGE_MASK);
-         }
-         if (!(tb_end <= start || tb_start >= end)) {
- #ifdef TARGET_HAS_PRECISE_SMC
 --
 .34.1

-[PULL 15/20] include/hw/core: Create struct CPUJumpCache
+[PULL 04/20] plugins: Move function pointer in qemu_plugin_dyn_cb
-Wrap the bare TranslationBlock pointer into a structure.
+The out-of-line function pointer is mutually exclusive
 with inline expansion, so move it into the union.
 Wrap the pointer in a structure named 'regular' to match
 PLUGIN_CB_REGULAR.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/tb-hash.h       |  1 +
+ include/qemu/plugin.h  | 4 +++-
- accel/tcg/tb-jmp-cache.h  | 24 ++++++++++++++++++++++++
+ accel/tcg/plugin-gen.c | 4 ++--
- include/exec/cpu-common.h |  1 +
+ plugins/core.c         | 8 ++++----
- include/hw/core/cpu.h     | 15 +--------------
+files changed, 9 insertions(+), 7 deletions(-)
  include/qemu/typedefs.h   |  1 +
  accel/stubs/tcg-stub.c    |  4 ++++
  accel/tcg/cpu-exec.c      | 10 +++++++---
  accel/tcg/cputlb.c        |  9 +++++----
  accel/tcg/translate-all.c | 28 +++++++++++++++++++++++++---
  hw/core/cpu-common.c      |  3 +--
  plugins/core.c            |  2 +-
  trace/control-target.c    |  2 +-
 files changed, 72 insertions(+), 28 deletions(-)
  create mode 100644 accel/tcg/tb-jmp-cache.h
-diff --git a/accel/tcg/tb-hash.h b/accel/tcg/tb-hash.h
+diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/tb-hash.h
+--- a/include/qemu/plugin.h
-+++ b/accel/tcg/tb-hash.h
++++ b/include/qemu/plugin.h
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ enum plugin_dyn_cb_subtype {
- #include "exec/cpu-defs.h"
+  * instance of a callback to be called upon the execution of a particular TB.
- #include "exec/exec-all.h"
+  */
- #include "qemu/xxhash.h"
+ struct qemu_plugin_dyn_cb {
-+#include "tb-jmp-cache.h"
+-    union qemu_plugin_cb_sig f;
+     void *userp;
- #ifdef CONFIG_SOFTMMU
+     enum plugin_dyn_cb_subtype type;
+     /* @rw applies to mem callbacks only (both regular and inline) */
-diff --git a/accel/tcg/tb-jmp-cache.h b/accel/tcg/tb-jmp-cache.h
+     enum qemu_plugin_mem_rw rw;
-new file mode 100644
+     /* fields specific to each dyn_cb type go here */
-index XXXXXXX..XXXXXXX
+     union {
---- /dev/null
++        struct {
-+++ b/accel/tcg/tb-jmp-cache.h
++            union qemu_plugin_cb_sig f;
-@@ -XXX,XX +XXX,XX @@
++        } regular;
-+/*
+         struct {
-+ * The per-CPU TranslationBlock jump cache.
+             qemu_plugin_u64 entry;
-+ *
+             enum qemu_plugin_op op;
-+ *  Copyright (c) 2003 Fabrice Bellard
+diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 + *
 + * SPDX-License-Identifier: GPL-2.0-or-later
 + */
 +
 +#ifndef ACCEL_TCG_TB_JMP_CACHE_H
 +#define ACCEL_TCG_TB_JMP_CACHE_H
 +
 +#define TB_JMP_CACHE_BITS 12
 +#define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS)
 +
 +/*
 + * Accessed in parallel; all accesses to 'tb' must be atomic.
 + */
 +struct CPUJumpCache {
 +    struct {
 +        TranslationBlock *tb;
 +    } array[TB_JMP_CACHE_SIZE];
 +};
 +
 +#endif /* ACCEL_TCG_TB_JMP_CACHE_H */
 diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
 index XXXXXXX..XXXXXXX 100644
---- a/include/exec/cpu-common.h
+--- a/accel/tcg/plugin-gen.c
-+++ b/include/exec/cpu-common.h
++++ b/accel/tcg/plugin-gen.c
-@@ -XXX,XX +XXX,XX @@ void cpu_list_unlock(void);
+@@ -XXX,XX +XXX,XX @@ static TCGOp *append_udata_cb(const struct qemu_plugin_dyn_cb *cb,
- unsigned int cpu_list_generation_id_get(void);
+     }
- void tcg_flush_softmmu_tlb(CPUState *cs);
+     /* call */
-+void tcg_flush_jmp_cache(CPUState *cs);
+-    op = copy_call(&begin_op, op, cb->f.vcpu_udata, cb_idx);
++    op = copy_call(&begin_op, op, cb->regular.f.vcpu_udata, cb_idx);
- void tcg_iommu_init_notifier_list(CPUState *cpu);
- void tcg_iommu_free_notifier_list(CPUState *cpu);
+     return op;
 diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/hw/core/cpu.h
 +++ b/include/hw/core/cpu.h
@@ -XXX,XX +XXX,XX @@ struct kvm_run;
  struct hax_vcpu_state;
  struct hvf_vcpu_state;
 -#define TB_JMP_CACHE_BITS 12
 -#define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS)
 -
  /* work queue */
  /* The union type allows passing of 64 bit target pointers on 32 bit
@@ -XXX,XX +XXX,XX @@ struct CPUState {
      CPUArchState *env_ptr;
      IcountDecr *icount_decr_ptr;
 -    /* Accessed in parallel; all accesses must be atomic */
 -    TranslationBlock *tb_jmp_cache[TB_JMP_CACHE_SIZE];
 +    CPUJumpCache *tb_jmp_cache;
      struct GDBRegisterState *gdb_regs;
      int gdb_num_regs;
@@ -XXX,XX +XXX,XX @@ extern CPUTailQ cpus;
  extern __thread CPUState *current_cpu;
 -static inline void cpu_tb_jmp_cache_clear(CPUState *cpu)
 -{
 -    unsigned int i;
 -
 -    for (i = 0; i < TB_JMP_CACHE_SIZE; i++) {
 -        qatomic_set(&cpu->tb_jmp_cache[i], NULL);
 -    }
 -}
 -
  /**
   * qemu_tcg_mttcg_enabled:
   * Check whether we are running MultiThread TCG or not.
 diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/qemu/typedefs.h
 +++ b/include/qemu/typedefs.h
@@ -XXX,XX +XXX,XX @@ typedef struct CoMutex CoMutex;
  typedef struct ConfidentialGuestSupport ConfidentialGuestSupport;
  typedef struct CPUAddressSpace CPUAddressSpace;
  typedef struct CPUArchState CPUArchState;
 +typedef struct CPUJumpCache CPUJumpCache;
  typedef struct CPUState CPUState;
  typedef struct CPUTLBEntryFull CPUTLBEntryFull;
  typedef struct DeviceListener DeviceListener;
 diff --git a/accel/stubs/tcg-stub.c b/accel/stubs/tcg-stub.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/stubs/tcg-stub.c
 +++ b/accel/stubs/tcg-stub.c
@@ -XXX,XX +XXX,XX @@ void tlb_set_dirty(CPUState *cpu, target_ulong vaddr)
  {
  }
+@@ -XXX,XX +XXX,XX @@ static TCGOp *append_mem_cb(const struct qemu_plugin_dyn_cb *cb,
-+void tcg_flush_jmp_cache(CPUState *cpu)
-+{
+     if (type == PLUGIN_GEN_CB_MEM) {
-+}
+         /* call */
-+
+-        op = copy_call(&begin_op, op, cb->f.vcpu_udata, cb_idx);
- int probe_access_flags(CPUArchState *env, target_ulong addr,
++        op = copy_call(&begin_op, op, cb->regular.f.vcpu_udata, cb_idx);
                         MMUAccessType access_type, int mmu_idx,
                         bool nonfault, void **phost, uintptr_t retaddr)
 diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cpu-exec.c
 +++ b/accel/tcg/cpu-exec.c
@@ -XXX,XX +XXX,XX @@
  #include "sysemu/replay.h"
  #include "sysemu/tcg.h"
  #include "exec/helper-proto.h"
 +#include "tb-jmp-cache.h"
  #include "tb-hash.h"
  #include "tb-context.h"
  #include "internal.h"
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
      tcg_debug_assert(!(cflags & CF_INVALID));
      hash = tb_jmp_cache_hash_func(pc);
 -    tb = qatomic_rcu_read(&cpu->tb_jmp_cache[hash]);
 +    tb = qatomic_rcu_read(&cpu->tb_jmp_cache->array[hash].tb);
      if (likely(tb &&
                 tb->pc == pc &&
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
      if (tb == NULL) {
          return NULL;
      }
--    qatomic_set(&cpu->tb_jmp_cache[hash], tb);
-+    qatomic_set(&cpu->tb_jmp_cache->array[hash].tb, tb);
+     return op;
      return tb;
  }
@@ -XXX,XX +XXX,XX @@ int cpu_exec(CPUState *cpu)
              tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
              if (tb == NULL) {
 +                uint32_t h;
 +
                  mmap_lock();
                  tb = tb_gen_code(cpu, pc, cs_base, flags, cflags);
                  mmap_unlock();
@@ -XXX,XX +XXX,XX @@ int cpu_exec(CPUState *cpu)
                   * We add the TB in the virtual pc hash table
                   * for the fast lookup
                   */
 -                qatomic_set(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)], tb);
 +                h = tb_jmp_cache_hash_func(pc);
 +                qatomic_set(&cpu->tb_jmp_cache->array[h].tb, tb);
              }
  #ifndef CONFIG_USER_ONLY
 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cputlb.c
 +++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
  static void tb_jmp_cache_clear_page(CPUState *cpu, target_ulong page_addr)
  {
 -    unsigned int i, i0 = tb_jmp_cache_hash_page(page_addr);
 +    int i, i0 = tb_jmp_cache_hash_page(page_addr);
 +    CPUJumpCache *jc = cpu->tb_jmp_cache;
      for (i = 0; i < TB_JMP_PAGE_SIZE; i++) {
 -        qatomic_set(&cpu->tb_jmp_cache[i0 + i], NULL);
 +        qatomic_set(&jc->array[i0 + i].tb, NULL);
      }
  }
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
      qemu_spin_unlock(&env_tlb(env)->c.lock);
 -    cpu_tb_jmp_cache_clear(cpu);
 +    tcg_flush_jmp_cache(cpu);
      if (to_clean == ALL_MMUIDX_BITS) {
          qatomic_set(&env_tlb(env)->c.full_flush_count,
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_by_mmuidx_async_0(CPUState *cpu,
       * longer to clear each entry individually than it will to clear it all.
       */
      if (d.len >= (TARGET_PAGE_SIZE * TB_JMP_CACHE_SIZE)) {
 -        cpu_tb_jmp_cache_clear(cpu);
 +        tcg_flush_jmp_cache(cpu);
          return;
      }
 diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/translate-all.c
 +++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@
  #include "sysemu/tcg.h"
  #include "qapi/error.h"
  #include "hw/core/tcg-cpu-ops.h"
 +#include "tb-jmp-cache.h"
  #include "tb-hash.h"
  #include "tb-context.h"
  #include "internal.h"
@@ -XXX,XX +XXX,XX @@ static void do_tb_flush(CPUState *cpu, run_on_cpu_data tb_flush_count)
      }
      CPU_FOREACH(cpu) {
 -        cpu_tb_jmp_cache_clear(cpu);
 +        tcg_flush_jmp_cache(cpu);
      }
      qht_reset_size(&tb_ctx.htable, CODE_GEN_HTABLE_SIZE);
@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
      /* remove the TB from the hash list */
      h = tb_jmp_cache_hash_func(tb->pc);
      CPU_FOREACH(cpu) {
 -        if (qatomic_read(&cpu->tb_jmp_cache[h]) == tb) {
 -            qatomic_set(&cpu->tb_jmp_cache[h], NULL);
 +        CPUJumpCache *jc = cpu->tb_jmp_cache;
 +        if (qatomic_read(&jc->array[h].tb) == tb) {
 +            qatomic_set(&jc->array[h].tb, NULL);
          }
      }
@@ -XXX,XX +XXX,XX @@ int page_unprotect(target_ulong address, uintptr_t pc)
  }
  #endif /* CONFIG_USER_ONLY */
 +/*
 + * Called by generic code at e.g. cpu reset after cpu creation,
 + * therefore we must be prepared to allocate the jump cache.
 + */
 +void tcg_flush_jmp_cache(CPUState *cpu)
 +{
 +    CPUJumpCache *jc = cpu->tb_jmp_cache;
 +
 +    if (likely(jc)) {
 +        for (int i = 0; i < TB_JMP_CACHE_SIZE; i++) {
 +            qatomic_set(&jc->array[i].tb, NULL);
 +        }
 +    } else {
 +        /* This should happen once during realize, and thus never race. */
 +        jc = g_new0(CPUJumpCache, 1);
 +        jc = qatomic_xchg(&cpu->tb_jmp_cache, jc);
 +        assert(jc == NULL);
 +    }
 +}
 +
  /* This is a wrapper for common code that can not use CONFIG_SOFTMMU */
  void tcg_flush_softmmu_tlb(CPUState *cs)
  {
 diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
 index XXXXXXX..XXXXXXX 100644
 --- a/hw/core/cpu-common.c
 +++ b/hw/core/cpu-common.c
@@ -XXX,XX +XXX,XX @@ static void cpu_common_reset(DeviceState *dev)
      cpu->cflags_next_tb = -1;
      if (tcg_enabled()) {
 -        cpu_tb_jmp_cache_clear(cpu);
 -
 +        tcg_flush_jmp_cache(cpu);
          tcg_flush_softmmu_tlb(cpu);
      }
  }
 diff --git a/plugins/core.c b/plugins/core.c
 index XXXXXXX..XXXXXXX 100644
 --- a/plugins/core.c
 +++ b/plugins/core.c
-@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_ctx *plugin_id_to_ctx_locked(qemu_plugin_id_t id)
+@@ -XXX,XX +XXX,XX @@ void plugin_register_dyn_cb__udata(GArray **arr,
- static void plugin_cpu_update__async(CPUState *cpu, run_on_cpu_data data)
- {
+     dyn_cb->userp = udata;
-     bitmap_copy(cpu->plugin_mask, &data.host_ulong, QEMU_PLUGIN_EV_MAX);
+     /* Note flags are discarded as unused. */
--    cpu_tb_jmp_cache_clear(cpu);
+-    dyn_cb->f.vcpu_udata = cb;
-+    tcg_flush_jmp_cache(cpu);
++    dyn_cb->regular.f.vcpu_udata = cb;
      dyn_cb->type = PLUGIN_CB_REGULAR;
  }
- static void plugin_cpu_update__locked(gpointer k, gpointer v, gpointer udata)
+@@ -XXX,XX +XXX,XX @@ void plugin_register_vcpu_mem_cb(GArray **arr,
-diff --git a/trace/control-target.c b/trace/control-target.c
+     /* Note flags are discarded as unused. */
-index XXXXXXX..XXXXXXX 100644
+     dyn_cb->type = PLUGIN_CB_REGULAR;
---- a/trace/control-target.c
+     dyn_cb->rw = rw;
-+++ b/trace/control-target.c
+-    dyn_cb->f.generic = cb;
-@@ -XXX,XX +XXX,XX @@ static void trace_event_synchronize_vcpu_state_dynamic(
++    dyn_cb->regular.f.vcpu_mem = cb;
  {
      bitmap_copy(vcpu->trace_dstate, vcpu->trace_dstate_delayed,
                  CPU_TRACE_DSTATE_MAX_EVENTS);
 -    cpu_tb_jmp_cache_clear(vcpu);
 +    tcg_flush_jmp_cache(vcpu);
  }
- void trace_event_set_vcpu_state_dynamic(CPUState *vcpu,
+ /*
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_vcpu_mem_cb(CPUState *cpu, uint64_t vaddr,
          }
          switch (cb->type) {
          case PLUGIN_CB_REGULAR:
 -            cb->f.vcpu_mem(cpu->cpu_index, make_plugin_meminfo(oi, rw),
 -                           vaddr, cb->userp);
 +            cb->regular.f.vcpu_mem(cpu->cpu_index, make_plugin_meminfo(oi, rw),
 +                                   vaddr, cb->userp);
              break;
          case PLUGIN_CB_INLINE:
              exec_inline_op(cb, cpu->cpu_index);
 --
 .34.1

-[PULL 09/20] include/exec: Introduce TARGET_PAGE_ENTRY_EXTRA
+[PULL 05/20] plugins: Create TCGHelperInfo for all out-of-line callbacks
-Allow the target to cache items from the guest page tables.
+TCGHelperInfo includes the ABI for every function call.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/cpu-defs.h | 9 +++++++++
+ include/qemu/plugin.h |  1 +
-file changed, 9 insertions(+)
+ plugins/core.c        | 51 ++++++++++++++++++++++++++++++++++++++-----
 files changed, 46 insertions(+), 6 deletions(-)
-diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
+diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
 index XXXXXXX..XXXXXXX 100644
---- a/include/exec/cpu-defs.h
+--- a/include/qemu/plugin.h
-+++ b/include/exec/cpu-defs.h
++++ b/include/qemu/plugin.h
-@@ -XXX,XX +XXX,XX @@ typedef struct CPUTLBEntryFull {
+@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_dyn_cb {
+     union {
-     /* @lg_page_size contains the log2 of the page size. */
+         struct {
-     uint8_t lg_page_size;
+             union qemu_plugin_cb_sig f;
 +            TCGHelperInfo *info;
          } regular;
          struct {
              qemu_plugin_u64 entry;
 diff --git a/plugins/core.c b/plugins/core.c
 index XXXXXXX..XXXXXXX 100644
 --- a/plugins/core.c
 +++ b/plugins/core.c
@@ -XXX,XX +XXX,XX @@ void plugin_register_dyn_cb__udata(GArray **arr,
                                     enum qemu_plugin_cb_flags flags,
                                     void *udata)
  {
 -    struct qemu_plugin_dyn_cb *dyn_cb = plugin_get_dyn_cb(arr);
 +    static TCGHelperInfo info[3] = {
 +        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG | TCG_CALL_PLUGIN,
 +        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG | TCG_CALL_PLUGIN,
 +        [QEMU_PLUGIN_CB_RW_REGS].flags = TCG_CALL_PLUGIN,
 +        /*
 +         * Match qemu_plugin_vcpu_udata_cb_t:
 +         *   void (*)(uint32_t, void *)
 +         */
 +        [0 ... 2].typemask = (dh_typemask(void, 0) |
 +                              dh_typemask(i32, 1) |
 +                              dh_typemask(ptr, 2))
 +    };
 +    struct qemu_plugin_dyn_cb *dyn_cb = plugin_get_dyn_cb(arr);
      dyn_cb->userp = udata;
 -    /* Note flags are discarded as unused. */
 -    dyn_cb->regular.f.vcpu_udata = cb;
      dyn_cb->type = PLUGIN_CB_REGULAR;
 +    dyn_cb->regular.f.vcpu_udata = cb;
 +
++    assert((unsigned)flags < ARRAY_SIZE(info));
++    dyn_cb->regular.info = &info[flags];
+ }
+ void plugin_register_vcpu_mem_cb(GArray **arr,
+@@ -XXX,XX +XXX,XX @@ void plugin_register_vcpu_mem_cb(GArray **arr,
+                                  enum qemu_plugin_mem_rw rw,
+                                  void *udata)
+ {
+-    struct qemu_plugin_dyn_cb *dyn_cb;
 +    /*
-+     * Allow target-specific additions to this structure.
++     * Expect that the underlying type for enum qemu_plugin_meminfo_t
-+     * This may be used to cache items from the guest cpu
++     * is either int32_t or uint32_t, aka int or unsigned int.
 +     * page tables for later use by the implementation.
 +     */
-+#ifdef TARGET_PAGE_ENTRY_EXTRA
++    QEMU_BUILD_BUG_ON(
-+    TARGET_PAGE_ENTRY_EXTRA
++        !__builtin_types_compatible_p(qemu_plugin_meminfo_t, uint32_t) &&
-+#endif
++        !__builtin_types_compatible_p(qemu_plugin_meminfo_t, int32_t));
- } CPUTLBEntryFull;
 -    dyn_cb = plugin_get_dyn_cb(arr);
 +    static TCGHelperInfo info[3] = {
 +        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG | TCG_CALL_PLUGIN,
 +        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG | TCG_CALL_PLUGIN,
 +        [QEMU_PLUGIN_CB_RW_REGS].flags = TCG_CALL_PLUGIN,
 +        /*
 +         * Match qemu_plugin_vcpu_mem_cb_t:
 +         *   void (*)(uint32_t, qemu_plugin_meminfo_t, uint64_t, void *)
 +         */
 +        [0 ... 2].typemask =
 +            (dh_typemask(void, 0) |
 +             dh_typemask(i32, 1) |
 +             (__builtin_types_compatible_p(qemu_plugin_meminfo_t, uint32_t)
 +              ? dh_typemask(i32, 2) : dh_typemask(s32, 2)) |
 +             dh_typemask(i64, 3) |
 +             dh_typemask(ptr, 4))
 +    };
 +
 +    struct qemu_plugin_dyn_cb *dyn_cb = plugin_get_dyn_cb(arr);
      dyn_cb->userp = udata;
 -    /* Note flags are discarded as unused. */
      dyn_cb->type = PLUGIN_CB_REGULAR;
      dyn_cb->rw = rw;
      dyn_cb->regular.f.vcpu_mem = cb;
 +
 +    assert((unsigned)flags < ARRAY_SIZE(info));
 +    dyn_cb->regular.info = &info[flags];
  }
  /*
 --
 .34.1

-[PULL 11/20] accel/tcg: Use bool for page_find_alloc
+[PULL 06/20] plugins: Use emit_before_op for PLUGIN_GEN_AFTER_INSN
-Bool is more appropriate type for the alloc parameter.
+Introduce a new plugin_cb op and migrate one operation.
 By using emit_before_op, we do not need to emit opcodes
 early and modify them later -- we can simply emit the
 final set of opcodes once.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/translate-all.c | 14 +++++++-------
+ include/tcg/tcg-op-common.h |  1 +
-file changed, 7 insertions(+), 7 deletions(-)
+ include/tcg/tcg-opc.h       |  1 +
  accel/tcg/plugin-gen.c      | 74 +++++++++++++++++++++----------------
  tcg/tcg-op.c                |  5 +++
 files changed, 50 insertions(+), 31 deletions(-)
-diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
+diff --git a/include/tcg/tcg-op-common.h b/include/tcg/tcg-op-common.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/translate-all.c
+--- a/include/tcg/tcg-op-common.h
-+++ b/accel/tcg/translate-all.c
++++ b/include/tcg/tcg-op-common.h
-@@ -XXX,XX +XXX,XX @@ void page_init(void)
+@@ -XXX,XX +XXX,XX @@ void tcg_gen_goto_tb(unsigned idx);
- #endif
+  */
  void tcg_gen_lookup_and_goto_ptr(void);
 +void tcg_gen_plugin_cb(unsigned from);
  void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr);
  void tcg_gen_plugin_cb_end(void);
 diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg-opc.h
 +++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(exit_tb, 0, 0, 1, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
  DEF(goto_tb, 0, 0, 1, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
  DEF(goto_ptr, 0, 1, 0, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
 +DEF(plugin_cb, 0, 0, 1, TCG_OPF_NOT_PRESENT)
  DEF(plugin_cb_start, 0, 0, 3, TCG_OPF_NOT_PRESENT)
  DEF(plugin_cb_end, 0, 0, 0, TCG_OPF_NOT_PRESENT)
 diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/plugin-gen.c
 +++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_empty_callback(enum plugin_gen_from from)
  {
      switch (from) {
      case PLUGIN_GEN_AFTER_INSN:
 -        gen_wrapped(from, PLUGIN_GEN_DISABLE_MEM_HELPER,
 -                    gen_empty_mem_helper);
 +        tcg_gen_plugin_cb(from);
          break;
      case PLUGIN_GEN_FROM_INSN:
          /*
@@ -XXX,XX +XXX,XX @@ static void inject_mem_enable_helper(struct qemu_plugin_tb *ptb,
      inject_mem_helper(begin_op, arr);
  }
--static PageDesc *page_find_alloc(tb_page_addr_t index, int alloc)
+-static void inject_mem_disable_helper(struct qemu_plugin_insn *plugin_insn,
-+static PageDesc *page_find_alloc(tb_page_addr_t index, bool alloc)
+-                                      TCGOp *begin_op)
 -{
 -    if (likely(!plugin_insn->mem_helper)) {
 -        rm_ops(begin_op);
 -        return;
 -    }
 -    inject_mem_helper(begin_op, NULL);
 -}
 -
  /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
  void plugin_gen_disable_mem_helpers(void)
  {
-     PageDesc *pd;
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
-     void **lp;
+     inject_mem_enable_helper(ptb, insn, begin_op);
-@@ -XXX,XX +XXX,XX @@ static PageDesc *page_find_alloc(tb_page_addr_t index, int alloc)
+ }
- static inline PageDesc *page_find(tb_page_addr_t index)
+-static void plugin_gen_disable_mem_helper(struct qemu_plugin_tb *ptb,
 -                                          TCGOp *begin_op, int insn_idx)
 +static void gen_disable_mem_helper(struct qemu_plugin_tb *ptb,
 +                                   struct qemu_plugin_insn *insn)
  {
--    return page_find_alloc(index, 0);
+-    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
-+    return page_find_alloc(index, false);
+-    inject_mem_disable_helper(insn, begin_op);
 +    if (insn->mem_helper) {
 +        tcg_gen_st_ptr(tcg_constant_ptr(0), tcg_env,
 +                       offsetof(CPUState, plugin_mem_cbs) -
 +                       offsetof(ArchCPU, env));
 +    }
  }
- static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
+ /* #define DEBUG_PLUGIN_GEN_OPS */
--                           PageDesc **ret_p2, tb_page_addr_t phys2, int alloc);
+@@ -XXX,XX +XXX,XX @@ static void pr_ops(void)
-+                           PageDesc **ret_p2, tb_page_addr_t phys2, bool alloc);
+ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
  /* In user-mode page locks aren't used; mmap_lock is enough */
  #ifdef CONFIG_USER_ONLY
@@ -XXX,XX +XXX,XX @@ static inline void page_unlock(PageDesc *pd)
  /* lock the page(s) of a TB in the correct acquisition order */
  static inline void page_lock_tb(const TranslationBlock *tb)
  {
--    page_lock_pair(NULL, tb->page_addr[0], NULL, tb->page_addr[1], 0);
+-    TCGOp *op;
-+    page_lock_pair(NULL, tb->page_addr[0], NULL, tb->page_addr[1], false);
++    TCGOp *op, *next;
      int insn_idx = -1;
      pr_ops();
 -    QTAILQ_FOREACH(op, &tcg_ctx->ops, link) {
 +    /*
 +     * While injecting code, we cannot afford to reuse any ebb temps
 +     * that might be live within the existing opcode stream.
 +     * The simplest solution is to release them all and create new.
 +     */
 +    memset(tcg_ctx->free_temps, 0, sizeof(tcg_ctx->free_temps));
 +
 +    QTAILQ_FOREACH_SAFE(op, &tcg_ctx->ops, link, next) {
          switch (op->opc) {
          case INDEX_op_insn_start:
              insn_idx++;
              break;
 +
 +        case INDEX_op_plugin_cb:
 +        {
 +            enum plugin_gen_from from = op->args[0];
 +            struct qemu_plugin_insn *insn = NULL;
 +
 +            if (insn_idx >= 0) {
 +                insn = g_ptr_array_index(plugin_tb->insns, insn_idx);
 +            }
 +
 +            tcg_ctx->emit_before_op = op;
 +
 +            switch (from) {
 +            case PLUGIN_GEN_AFTER_INSN:
 +                assert(insn != NULL);
 +                gen_disable_mem_helper(plugin_tb, insn);
 +                break;
 +            default:
 +                g_assert_not_reached();
 +            }
 +
 +            tcg_ctx->emit_before_op = NULL;
 +            tcg_op_remove(tcg_ctx, op);
 +            break;
 +        }
 +
          case INDEX_op_plugin_cb_start:
          {
              enum plugin_gen_from from = op->args[0];
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
                  break;
              }
 -            case PLUGIN_GEN_AFTER_INSN:
 -            {
 -                g_assert(insn_idx >= 0);
 -
 -                switch (type) {
 -                case PLUGIN_GEN_DISABLE_MEM_HELPER:
 -                    plugin_gen_disable_mem_helper(plugin_tb, op, insn_idx);
 -                    break;
 -                default:
 -                    g_assert_not_reached();
 -                }
 -                break;
 -            }
              default:
                  g_assert_not_reached();
              }
 diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg-op.c
 +++ b/tcg/tcg-op.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_mb(TCGBar mb_type)
      }
  }
- static inline void page_unlock_tb(const TranslationBlock *tb)
++void tcg_gen_plugin_cb(unsigned from)
-@@ -XXX,XX +XXX,XX @@ void page_collection_unlock(struct page_collection *set)
++{
- #endif /* !CONFIG_USER_ONLY */
++    tcg_gen_op1(INDEX_op_plugin_cb, from);
++}
- static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
++
--                           PageDesc **ret_p2, tb_page_addr_t phys2, int alloc)
+ void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr)
 +                           PageDesc **ret_p2, tb_page_addr_t phys2, bool alloc)
  {
-     PageDesc *p1, *p2;
+     tcg_gen_op3(INDEX_op_plugin_cb_start, from, type, wr);
      tb_page_addr_t page1;
@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
       * Note that inserting into the hash table first isn't an option, since
       * we can only insert TBs that are fully initialized.
       */
 -    page_lock_pair(&p, phys_pc, &p2, phys_page2, 1);
 +    page_lock_pair(&p, phys_pc, &p2, phys_page2, true);
      tb_page_add(p, tb, 0, phys_pc & TARGET_PAGE_MASK);
      if (p2) {
          tb_page_add(p2, tb, 1, phys_page2);
@@ -XXX,XX +XXX,XX @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
      for (addr = start, len = end - start;
           len != 0;
           len -= TARGET_PAGE_SIZE, addr += TARGET_PAGE_SIZE) {
 -        PageDesc *p = page_find_alloc(addr >> TARGET_PAGE_BITS, 1);
 +        PageDesc *p = page_find_alloc(addr >> TARGET_PAGE_BITS, true);
          /* If the write protection bit is set, then we invalidate
             the code inside.  */
 --
 .34.1

-[PULL 16/20] hw/core: Add CPUClass.get_pc
+[PULL 07/20] plugins: Use emit_before_op for PLUGIN_GEN_FROM_TB
-Populate this new method for all targets.  Always match
+By having the qemu_plugin_cb_flags be recorded in the TCGHelperInfo,
-the result that would be given by cpu_get_tb_cpu_state,
+we no longer need to distinguish PLUGIN_CB_REGULAR from
-as we will want these values to correspond in the logs.
+PLUGIN_CB_REGULAR_R, so place all TB callbacks in the same queue.
-Reviewed-by: Taylor Simpson <tsimpson@quicinc.com>
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> (target/sparc)
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
-Cc: Eduardo Habkost <eduardo@habkost.net> (supporter:Machine core)
+ accel/tcg/plugin-gen.c | 96 +++++++++++++++++++++++++-----------------
-Cc: Marcel Apfelbaum <marcel.apfelbaum@gmail.com> (supporter:Machine core)
+ plugins/api.c          |  6 +--
-Cc: "Philippe Mathieu-Daudé" <f4bug@amsat.org> (reviewer:Machine core)
+files changed, 58 insertions(+), 44 deletions(-)
 Cc: Yanan Wang <wangyanan55@huawei.com> (reviewer:Machine core)
 Cc: Michael Rolnik <mrolnik@gmail.com> (maintainer:AVR TCG CPUs)
 Cc: "Edgar E. Iglesias" <edgar.iglesias@gmail.com> (maintainer:CRIS TCG CPUs)
 Cc: Taylor Simpson <tsimpson@quicinc.com> (supporter:Hexagon TCG CPUs)
 Cc: Song Gao <gaosong@loongson.cn> (maintainer:LoongArch TCG CPUs)
 Cc: Xiaojuan Yang <yangxiaojuan@loongson.cn> (maintainer:LoongArch TCG CPUs)
 Cc: Laurent Vivier <laurent@vivier.eu> (maintainer:M68K TCG CPUs)
 Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> (reviewer:MIPS TCG CPUs)
 Cc: Aleksandar Rikalo <aleksandar.rikalo@syrmia.com> (reviewer:MIPS TCG CPUs)
 Cc: Chris Wulff <crwulff@gmail.com> (maintainer:NiosII TCG CPUs)
 Cc: Marek Vasut <marex@denx.de> (maintainer:NiosII TCG CPUs)
 Cc: Stafford Horne <shorne@gmail.com> (odd fixer:OpenRISC TCG CPUs)
 Cc: Yoshinori Sato <ysato@users.sourceforge.jp> (reviewer:RENESAS RX CPUs)
 Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> (maintainer:SPARC TCG CPUs)
 Cc: Bastian Koppelmann <kbastian@mail.uni-paderborn.de> (maintainer:TriCore TCG CPUs)
 Cc: Max Filippov <jcmvbkbc@gmail.com> (maintainer:Xtensa TCG CPUs)
 Cc: qemu-arm@nongnu.org (open list:ARM TCG CPUs)
 Cc: qemu-ppc@nongnu.org (open list:PowerPC TCG CPUs)
 Cc: qemu-riscv@nongnu.org (open list:RISC-V TCG CPUs)
 Cc: qemu-s390x@nongnu.org (open list:S390 TCG CPUs)
 ---
  include/hw/core/cpu.h   |  3 +++
  target/alpha/cpu.c      |  9 +++++++++
  target/arm/cpu.c        | 13 +++++++++++++
  target/avr/cpu.c        |  8 ++++++++
  target/cris/cpu.c       |  8 ++++++++
  target/hexagon/cpu.c    |  8 ++++++++
  target/hppa/cpu.c       |  8 ++++++++
  target/i386/cpu.c       |  9 +++++++++
  target/loongarch/cpu.c  |  9 +++++++++
  target/m68k/cpu.c       |  8 ++++++++
  target/microblaze/cpu.c |  8 ++++++++
  target/mips/cpu.c       |  8 ++++++++
  target/nios2/cpu.c      |  9 +++++++++
  target/openrisc/cpu.c   |  8 ++++++++
  target/ppc/cpu_init.c   |  8 ++++++++
  target/riscv/cpu.c      | 13 +++++++++++++
  target/rx/cpu.c         |  8 ++++++++
  target/s390x/cpu.c      |  8 ++++++++
  target/sh4/cpu.c        |  8 ++++++++
  target/sparc/cpu.c      |  8 ++++++++
  target/tricore/cpu.c    |  9 +++++++++
  target/xtensa/cpu.c     |  8 ++++++++
 files changed, 186 insertions(+)
-diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
+diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
---- a/include/hw/core/cpu.h
+--- a/accel/tcg/plugin-gen.c
-+++ b/include/hw/core/cpu.h
++++ b/accel/tcg/plugin-gen.c
-@@ -XXX,XX +XXX,XX @@ struct SysemuCPUOps;
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_empty_callback(enum plugin_gen_from from)
-  *       If the target behaviour here is anything other than "set
+ {
-  *       the PC register to the value passed in" then the target must
+     switch (from) {
-  *       also implement the synchronize_from_tb hook.
+     case PLUGIN_GEN_AFTER_INSN:
-+ * @get_pc: Callback for getting the Program Counter register.
++    case PLUGIN_GEN_FROM_TB:
-+ *       As above, with the semantics of the target architecture.
+         tcg_gen_plugin_cb(from);
-  * @gdb_read_register: Callback for letting GDB read a register.
+         break;
-  * @gdb_write_register: Callback for letting GDB write a register.
+     case PLUGIN_GEN_FROM_INSN:
-  * @gdb_adjust_breakpoint: Callback for adjusting the address of a
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_empty_callback(enum plugin_gen_from from)
-@@ -XXX,XX +XXX,XX @@ struct CPUClass {
+          */
-     void (*dump_state)(CPUState *cpu, FILE *, int flags);
+         gen_wrapped(from, PLUGIN_GEN_ENABLE_MEM_HELPER,
-     int64_t (*get_arch_id)(CPUState *cpu);
+                     gen_empty_mem_helper);
-     void (*set_pc)(CPUState *cpu, vaddr value);
+-        /* fall through */
-+    vaddr (*get_pc)(CPUState *cpu);
+-    case PLUGIN_GEN_FROM_TB:
-     int (*gdb_read_register)(CPUState *cpu, GByteArray *buf, int reg);
+         gen_wrapped(from, PLUGIN_GEN_CB_UDATA, gen_empty_udata_cb_no_rwg);
-     int (*gdb_write_register)(CPUState *cpu, uint8_t *buf, int reg);
+         gen_wrapped(from, PLUGIN_GEN_CB_UDATA_R, gen_empty_udata_cb_no_wg);
-     vaddr (*gdb_adjust_breakpoint)(CPUState *cpu, vaddr addr);
+         gen_wrapped(from, PLUGIN_GEN_CB_INLINE, gen_empty_inline_cb);
-diff --git a/target/alpha/cpu.c b/target/alpha/cpu.c
+@@ -XXX,XX +XXX,XX @@ void plugin_gen_disable_mem_helpers(void)
-index XXXXXXX..XXXXXXX 100644
+                    offsetof(CPUState, plugin_mem_cbs) - offsetof(ArchCPU, env));
 --- a/target/alpha/cpu.c
 +++ b/target/alpha/cpu.c
@@ -XXX,XX +XXX,XX @@ static void alpha_cpu_set_pc(CPUState *cs, vaddr value)
      cpu->env.pc = value;
  }
-+static vaddr alpha_cpu_get_pc(CPUState *cs)
+-static void plugin_gen_tb_udata(const struct qemu_plugin_tb *ptb,
 -                                TCGOp *begin_op)
 -{
 -    inject_udata_cb(ptb->cbs[PLUGIN_CB_REGULAR], begin_op);
 -}
 -
 -static void plugin_gen_tb_udata_r(const struct qemu_plugin_tb *ptb,
 -                                  TCGOp *begin_op)
 -{
 -    inject_udata_cb(ptb->cbs[PLUGIN_CB_REGULAR_R], begin_op);
 -}
 -
 -static void plugin_gen_tb_inline(const struct qemu_plugin_tb *ptb,
 -                                 TCGOp *begin_op)
 -{
 -    inject_inline_cb(ptb->cbs[PLUGIN_CB_INLINE], begin_op, op_ok);
 -}
 -
  static void plugin_gen_insn_udata(const struct qemu_plugin_tb *ptb,
                                    TCGOp *begin_op, int insn_idx)
  {
@@ -XXX,XX +XXX,XX @@ static void gen_disable_mem_helper(struct qemu_plugin_tb *ptb,
      }
  }
 +static void gen_udata_cb(struct qemu_plugin_dyn_cb *cb)
 +{
-+    AlphaCPU *cpu = ALPHA_CPU(cs);
++    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
 +
-+    return cpu->env.pc;
++    tcg_gen_ld_i32(cpu_index, tcg_env,
 +                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
 +    tcg_gen_call2(cb->regular.f.vcpu_udata, cb->regular.info, NULL,
 +                  tcgv_i32_temp(cpu_index),
 +                  tcgv_ptr_temp(tcg_constant_ptr(cb->userp)));
 +    tcg_temp_free_i32(cpu_index);
 +}
 +
++static void gen_inline_cb(struct qemu_plugin_dyn_cb *cb)
++{
++    GArray *arr = cb->inline_insn.entry.score->data;
++    size_t offset = cb->inline_insn.entry.offset;
++    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
++    TCGv_i64 val = tcg_temp_ebb_new_i64();
++    TCGv_ptr ptr = tcg_temp_ebb_new_ptr();
 +
- static bool alpha_cpu_has_work(CPUState *cs)
++    tcg_gen_ld_i32(cpu_index, tcg_env,
 +                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
 +    tcg_gen_muli_i32(cpu_index, cpu_index, g_array_get_element_size(arr));
 +    tcg_gen_ext_i32_ptr(ptr, cpu_index);
 +    tcg_temp_free_i32(cpu_index);
 +
 +    tcg_gen_addi_ptr(ptr, ptr, (intptr_t)arr->data);
 +    tcg_gen_ld_i64(val, ptr, offset);
 +    tcg_gen_addi_i64(val, val, cb->inline_insn.imm);
 +    tcg_gen_st_i64(val, ptr, offset);
 +
 +    tcg_temp_free_i64(val);
 +    tcg_temp_free_ptr(ptr);
 +}
 +
  /* #define DEBUG_PLUGIN_GEN_OPS */
  static void pr_ops(void)
  {
-     /* Here we are checking to see if the CPU should wake up from HALT.
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
-@@ -XXX,XX +XXX,XX @@ static void alpha_cpu_class_init(ObjectClass *oc, void *data)
+         {
-     cc->has_work = alpha_cpu_has_work;
+             enum plugin_gen_from from = op->args[0];
-     cc->dump_state = alpha_cpu_dump_state;
+             struct qemu_plugin_insn *insn = NULL;
-     cc->set_pc = alpha_cpu_set_pc;
++            const GArray *cbs;
-+    cc->get_pc = alpha_cpu_get_pc;
++            int i, n;
-     cc->gdb_read_register = alpha_cpu_gdb_read_register;
-     cc->gdb_write_register = alpha_cpu_gdb_write_register;
+             if (insn_idx >= 0) {
- #ifndef CONFIG_USER_ONLY
+                 insn = g_ptr_array_index(plugin_tb->insns, insn_idx);
-diff --git a/target/arm/cpu.c b/target/arm/cpu.c
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
                  assert(insn != NULL);
                  gen_disable_mem_helper(plugin_tb, insn);
                  break;
 +
 +            case PLUGIN_GEN_FROM_TB:
 +                assert(insn == NULL);
 +
 +                cbs = plugin_tb->cbs[PLUGIN_CB_REGULAR];
 +                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 +                    struct qemu_plugin_dyn_cb *cb =
 +                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 +                    gen_udata_cb(cb);
 +                }
 +
 +                cbs = plugin_tb->cbs[PLUGIN_CB_INLINE];
 +                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 +                    struct qemu_plugin_dyn_cb *cb =
 +                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 +                    gen_inline_cb(cb);
 +                }
 +                break;
 +
              default:
                  g_assert_not_reached();
              }
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
              enum plugin_gen_cb type = op->args[1];
              switch (from) {
 -            case PLUGIN_GEN_FROM_TB:
 -            {
 -                g_assert(insn_idx == -1);
 -
 -                switch (type) {
 -                case PLUGIN_GEN_CB_UDATA:
 -                    plugin_gen_tb_udata(plugin_tb, op);
 -                    break;
 -                case PLUGIN_GEN_CB_UDATA_R:
 -                    plugin_gen_tb_udata_r(plugin_tb, op);
 -                    break;
 -                case PLUGIN_GEN_CB_INLINE:
 -                    plugin_gen_tb_inline(plugin_tb, op);
 -                    break;
 -                default:
 -                    g_assert_not_reached();
 -                }
 -                break;
 -            }
              case PLUGIN_GEN_FROM_INSN:
              {
                  g_assert(insn_idx >= 0);
 diff --git a/plugins/api.c b/plugins/api.c
 index XXXXXXX..XXXXXXX 100644
---- a/target/arm/cpu.c
+--- a/plugins/api.c
-+++ b/target/arm/cpu.c
++++ b/plugins/api.c
-@@ -XXX,XX +XXX,XX @@ static void arm_cpu_set_pc(CPUState *cs, vaddr value)
+@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_tb_exec_cb(struct qemu_plugin_tb *tb,
                                            void *udata)
  {
      if (!tb->mem_only) {
 -        int index = flags == QEMU_PLUGIN_CB_R_REGS ||
 -                    flags == QEMU_PLUGIN_CB_RW_REGS ?
 -                    PLUGIN_CB_REGULAR_R : PLUGIN_CB_REGULAR;
 -
 -        plugin_register_dyn_cb__udata(&tb->cbs[index],
 +        plugin_register_dyn_cb__udata(&tb->cbs[PLUGIN_CB_REGULAR],
                                        cb, flags, udata);
      }
  }
-+static vaddr arm_cpu_get_pc(CPUState *cs)
-+{
-+    ARMCPU *cpu = ARM_CPU(cs);
-+    CPUARMState *env = &cpu->env;
-+
-+    if (is_a64(env)) {
-+        return env->pc;
-+    } else {
-+        return env->regs[15];
-+    }
-+}
-+
- #ifdef CONFIG_TCG
- void arm_cpu_synchronize_from_tb(CPUState *cs,
-                                  const TranslationBlock *tb)
-@@ -XXX,XX +XXX,XX @@ static void arm_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = arm_cpu_has_work;
-     cc->dump_state = arm_cpu_dump_state;
-     cc->set_pc = arm_cpu_set_pc;
-+    cc->get_pc = arm_cpu_get_pc;
-     cc->gdb_read_register = arm_cpu_gdb_read_register;
-     cc->gdb_write_register = arm_cpu_gdb_write_register;
- #ifndef CONFIG_USER_ONLY
-diff --git a/target/avr/cpu.c b/target/avr/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/avr/cpu.c
-+++ b/target/avr/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void avr_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.pc_w = value / 2; /* internally PC points to words */
- }
-+static vaddr avr_cpu_get_pc(CPUState *cs)
-+{
-+    AVRCPU *cpu = AVR_CPU(cs);
-+
-+    return cpu->env.pc_w * 2;
-+}
-+
- static bool avr_cpu_has_work(CPUState *cs)
- {
-     AVRCPU *cpu = AVR_CPU(cs);
-@@ -XXX,XX +XXX,XX @@ static void avr_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = avr_cpu_has_work;
-     cc->dump_state = avr_cpu_dump_state;
-     cc->set_pc = avr_cpu_set_pc;
-+    cc->get_pc = avr_cpu_get_pc;
-     dc->vmsd = &vms_avr_cpu;
-     cc->sysemu_ops = &avr_sysemu_ops;
-     cc->disas_set_info = avr_cpu_disas_set_info;
-diff --git a/target/cris/cpu.c b/target/cris/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/cris/cpu.c
-+++ b/target/cris/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void cris_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.pc = value;
- }
-+static vaddr cris_cpu_get_pc(CPUState *cs)
-+{
-+    CRISCPU *cpu = CRIS_CPU(cs);
-+
-+    return cpu->env.pc;
-+}
-+
- static bool cris_cpu_has_work(CPUState *cs)
- {
-     return cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_NMI);
-@@ -XXX,XX +XXX,XX @@ static void cris_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = cris_cpu_has_work;
-     cc->dump_state = cris_cpu_dump_state;
-     cc->set_pc = cris_cpu_set_pc;
-+    cc->get_pc = cris_cpu_get_pc;
-     cc->gdb_read_register = cris_cpu_gdb_read_register;
-     cc->gdb_write_register = cris_cpu_gdb_write_register;
- #ifndef CONFIG_USER_ONLY
-diff --git a/target/hexagon/cpu.c b/target/hexagon/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/hexagon/cpu.c
-+++ b/target/hexagon/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void hexagon_cpu_set_pc(CPUState *cs, vaddr value)
-     env->gpr[HEX_REG_PC] = value;
- }
-+static vaddr hexagon_cpu_get_pc(CPUState *cs)
-+{
-+    HexagonCPU *cpu = HEXAGON_CPU(cs);
-+    CPUHexagonState *env = &cpu->env;
-+    return env->gpr[HEX_REG_PC];
-+}
-+
- static void hexagon_cpu_synchronize_from_tb(CPUState *cs,
-                                             const TranslationBlock *tb)
- {
-@@ -XXX,XX +XXX,XX @@ static void hexagon_cpu_class_init(ObjectClass *c, void *data)
-     cc->has_work = hexagon_cpu_has_work;
-     cc->dump_state = hexagon_dump_state;
-     cc->set_pc = hexagon_cpu_set_pc;
-+    cc->get_pc = hexagon_cpu_get_pc;
-     cc->gdb_read_register = hexagon_gdb_read_register;
-     cc->gdb_write_register = hexagon_gdb_write_register;
-     cc->gdb_num_core_regs = TOTAL_PER_THREAD_REGS + NUM_VREGS + NUM_QREGS;
-diff --git a/target/hppa/cpu.c b/target/hppa/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/hppa/cpu.c
-+++ b/target/hppa/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void hppa_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.iaoq_b = value + 4;
- }
-+static vaddr hppa_cpu_get_pc(CPUState *cs)
-+{
-+    HPPACPU *cpu = HPPA_CPU(cs);
-+
-+    return cpu->env.iaoq_f;
-+}
-+
- static void hppa_cpu_synchronize_from_tb(CPUState *cs,
-                                          const TranslationBlock *tb)
- {
-@@ -XXX,XX +XXX,XX @@ static void hppa_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = hppa_cpu_has_work;
-     cc->dump_state = hppa_cpu_dump_state;
-     cc->set_pc = hppa_cpu_set_pc;
-+    cc->get_pc = hppa_cpu_get_pc;
-     cc->gdb_read_register = hppa_cpu_gdb_read_register;
-     cc->gdb_write_register = hppa_cpu_gdb_write_register;
- #ifndef CONFIG_USER_ONLY
-diff --git a/target/i386/cpu.c b/target/i386/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/i386/cpu.c
-+++ b/target/i386/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void x86_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.eip = value;
- }
-+static vaddr x86_cpu_get_pc(CPUState *cs)
-+{
-+    X86CPU *cpu = X86_CPU(cs);
-+
-+    /* Match cpu_get_tb_cpu_state. */
-+    return cpu->env.eip + cpu->env.segs[R_CS].base;
-+}
-+
- int x86_cpu_pending_interrupt(CPUState *cs, int interrupt_request)
- {
-     X86CPU *cpu = X86_CPU(cs);
-@@ -XXX,XX +XXX,XX @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
-     cc->has_work = x86_cpu_has_work;
-     cc->dump_state = x86_cpu_dump_state;
-     cc->set_pc = x86_cpu_set_pc;
-+    cc->get_pc = x86_cpu_get_pc;
-     cc->gdb_read_register = x86_cpu_gdb_read_register;
-     cc->gdb_write_register = x86_cpu_gdb_write_register;
-     cc->get_arch_id = x86_cpu_get_arch_id;
-diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/loongarch/cpu.c
-+++ b/target/loongarch/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void loongarch_cpu_set_pc(CPUState *cs, vaddr value)
-     env->pc = value;
- }
-+static vaddr loongarch_cpu_get_pc(CPUState *cs)
-+{
-+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
-+    CPULoongArchState *env = &cpu->env;
-+
-+    return env->pc;
-+}
-+
- #ifndef CONFIG_USER_ONLY
- #include "hw/loongarch/virt.h"
-@@ -XXX,XX +XXX,XX @@ static void loongarch_cpu_class_init(ObjectClass *c, void *data)
-     cc->has_work = loongarch_cpu_has_work;
-     cc->dump_state = loongarch_cpu_dump_state;
-     cc->set_pc = loongarch_cpu_set_pc;
-+    cc->get_pc = loongarch_cpu_get_pc;
- #ifndef CONFIG_USER_ONLY
-     dc->vmsd = &vmstate_loongarch_cpu;
-     cc->sysemu_ops = &loongarch_sysemu_ops;
-diff --git a/target/m68k/cpu.c b/target/m68k/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/m68k/cpu.c
-+++ b/target/m68k/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void m68k_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.pc = value;
- }
-+static vaddr m68k_cpu_get_pc(CPUState *cs)
-+{
-+    M68kCPU *cpu = M68K_CPU(cs);
-+
-+    return cpu->env.pc;
-+}
-+
- static bool m68k_cpu_has_work(CPUState *cs)
- {
-     return cs->interrupt_request & CPU_INTERRUPT_HARD;
-@@ -XXX,XX +XXX,XX @@ static void m68k_cpu_class_init(ObjectClass *c, void *data)
-     cc->has_work = m68k_cpu_has_work;
-     cc->dump_state = m68k_cpu_dump_state;
-     cc->set_pc = m68k_cpu_set_pc;
-+    cc->get_pc = m68k_cpu_get_pc;
-     cc->gdb_read_register = m68k_cpu_gdb_read_register;
-     cc->gdb_write_register = m68k_cpu_gdb_write_register;
- #if defined(CONFIG_SOFTMMU)
-diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/microblaze/cpu.c
-+++ b/target/microblaze/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void mb_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.iflags = 0;
- }
-+static vaddr mb_cpu_get_pc(CPUState *cs)
-+{
-+    MicroBlazeCPU *cpu = MICROBLAZE_CPU(cs);
-+
-+    return cpu->env.pc;
-+}
-+
- static void mb_cpu_synchronize_from_tb(CPUState *cs,
-                                        const TranslationBlock *tb)
- {
-@@ -XXX,XX +XXX,XX @@ static void mb_cpu_class_init(ObjectClass *oc, void *data)
-     cc->dump_state = mb_cpu_dump_state;
-     cc->set_pc = mb_cpu_set_pc;
-+    cc->get_pc = mb_cpu_get_pc;
-     cc->gdb_read_register = mb_cpu_gdb_read_register;
-     cc->gdb_write_register = mb_cpu_gdb_write_register;
-diff --git a/target/mips/cpu.c b/target/mips/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/mips/cpu.c
-+++ b/target/mips/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void mips_cpu_set_pc(CPUState *cs, vaddr value)
-     mips_env_set_pc(&cpu->env, value);
- }
-+static vaddr mips_cpu_get_pc(CPUState *cs)
-+{
-+    MIPSCPU *cpu = MIPS_CPU(cs);
-+
-+    return cpu->env.active_tc.PC;
-+}
-+
- static bool mips_cpu_has_work(CPUState *cs)
- {
-     MIPSCPU *cpu = MIPS_CPU(cs);
-@@ -XXX,XX +XXX,XX @@ static void mips_cpu_class_init(ObjectClass *c, void *data)
-     cc->has_work = mips_cpu_has_work;
-     cc->dump_state = mips_cpu_dump_state;
-     cc->set_pc = mips_cpu_set_pc;
-+    cc->get_pc = mips_cpu_get_pc;
-     cc->gdb_read_register = mips_cpu_gdb_read_register;
-     cc->gdb_write_register = mips_cpu_gdb_write_register;
- #ifndef CONFIG_USER_ONLY
-diff --git a/target/nios2/cpu.c b/target/nios2/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/nios2/cpu.c
-+++ b/target/nios2/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void nios2_cpu_set_pc(CPUState *cs, vaddr value)
-     env->pc = value;
- }
-+static vaddr nios2_cpu_get_pc(CPUState *cs)
-+{
-+    Nios2CPU *cpu = NIOS2_CPU(cs);
-+    CPUNios2State *env = &cpu->env;
-+
-+    return env->pc;
-+}
-+
- static bool nios2_cpu_has_work(CPUState *cs)
- {
-     return cs->interrupt_request & CPU_INTERRUPT_HARD;
-@@ -XXX,XX +XXX,XX @@ static void nios2_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = nios2_cpu_has_work;
-     cc->dump_state = nios2_cpu_dump_state;
-     cc->set_pc = nios2_cpu_set_pc;
-+    cc->get_pc = nios2_cpu_get_pc;
-     cc->disas_set_info = nios2_cpu_disas_set_info;
- #ifndef CONFIG_USER_ONLY
-     cc->sysemu_ops = &nios2_sysemu_ops;
-diff --git a/target/openrisc/cpu.c b/target/openrisc/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/openrisc/cpu.c
-+++ b/target/openrisc/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void openrisc_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.dflag = 0;
- }
-+static vaddr openrisc_cpu_get_pc(CPUState *cs)
-+{
-+    OpenRISCCPU *cpu = OPENRISC_CPU(cs);
-+
-+    return cpu->env.pc;
-+}
-+
- static void openrisc_cpu_synchronize_from_tb(CPUState *cs,
-                                              const TranslationBlock *tb)
- {
-@@ -XXX,XX +XXX,XX @@ static void openrisc_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = openrisc_cpu_has_work;
-     cc->dump_state = openrisc_cpu_dump_state;
-     cc->set_pc = openrisc_cpu_set_pc;
-+    cc->get_pc = openrisc_cpu_get_pc;
-     cc->gdb_read_register = openrisc_cpu_gdb_read_register;
-     cc->gdb_write_register = openrisc_cpu_gdb_write_register;
- #ifndef CONFIG_USER_ONLY
-diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/ppc/cpu_init.c
-+++ b/target/ppc/cpu_init.c
-@@ -XXX,XX +XXX,XX @@ static void ppc_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.nip = value;
- }
-+static vaddr ppc_cpu_get_pc(CPUState *cs)
-+{
-+    PowerPCCPU *cpu = POWERPC_CPU(cs);
-+
-+    return cpu->env.nip;
-+}
-+
- static bool ppc_cpu_has_work(CPUState *cs)
- {
-     PowerPCCPU *cpu = POWERPC_CPU(cs);
-@@ -XXX,XX +XXX,XX @@ static void ppc_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = ppc_cpu_has_work;
-     cc->dump_state = ppc_cpu_dump_state;
-     cc->set_pc = ppc_cpu_set_pc;
-+    cc->get_pc = ppc_cpu_get_pc;
-     cc->gdb_read_register = ppc_cpu_gdb_read_register;
-     cc->gdb_write_register = ppc_cpu_gdb_write_register;
- #ifndef CONFIG_USER_ONLY
-diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/riscv/cpu.c
-+++ b/target/riscv/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void riscv_cpu_set_pc(CPUState *cs, vaddr value)
-     }
- }
-+static vaddr riscv_cpu_get_pc(CPUState *cs)
-+{
-+    RISCVCPU *cpu = RISCV_CPU(cs);
-+    CPURISCVState *env = &cpu->env;
-+
-+    /* Match cpu_get_tb_cpu_state. */
-+    if (env->xl == MXL_RV32) {
-+        return env->pc & UINT32_MAX;
-+    }
-+    return env->pc;
-+}
-+
- static void riscv_cpu_synchronize_from_tb(CPUState *cs,
-                                           const TranslationBlock *tb)
- {
-@@ -XXX,XX +XXX,XX @@ static void riscv_cpu_class_init(ObjectClass *c, void *data)
-     cc->has_work = riscv_cpu_has_work;
-     cc->dump_state = riscv_cpu_dump_state;
-     cc->set_pc = riscv_cpu_set_pc;
-+    cc->get_pc = riscv_cpu_get_pc;
-     cc->gdb_read_register = riscv_cpu_gdb_read_register;
-     cc->gdb_write_register = riscv_cpu_gdb_write_register;
-     cc->gdb_num_core_regs = 33;
-diff --git a/target/rx/cpu.c b/target/rx/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/rx/cpu.c
-+++ b/target/rx/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void rx_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.pc = value;
- }
-+static vaddr rx_cpu_get_pc(CPUState *cs)
-+{
-+    RXCPU *cpu = RX_CPU(cs);
-+
-+    return cpu->env.pc;
-+}
-+
- static void rx_cpu_synchronize_from_tb(CPUState *cs,
-                                        const TranslationBlock *tb)
- {
-@@ -XXX,XX +XXX,XX @@ static void rx_cpu_class_init(ObjectClass *klass, void *data)
-     cc->has_work = rx_cpu_has_work;
-     cc->dump_state = rx_cpu_dump_state;
-     cc->set_pc = rx_cpu_set_pc;
-+    cc->get_pc = rx_cpu_get_pc;
- #ifndef CONFIG_USER_ONLY
-     cc->sysemu_ops = &rx_sysemu_ops;
-diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/s390x/cpu.c
-+++ b/target/s390x/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void s390_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.psw.addr = value;
- }
-+static vaddr s390_cpu_get_pc(CPUState *cs)
-+{
-+    S390CPU *cpu = S390_CPU(cs);
-+
-+    return cpu->env.psw.addr;
-+}
-+
- static bool s390_cpu_has_work(CPUState *cs)
- {
-     S390CPU *cpu = S390_CPU(cs);
-@@ -XXX,XX +XXX,XX @@ static void s390_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = s390_cpu_has_work;
-     cc->dump_state = s390_cpu_dump_state;
-     cc->set_pc = s390_cpu_set_pc;
-+    cc->get_pc = s390_cpu_get_pc;
-     cc->gdb_read_register = s390_cpu_gdb_read_register;
-     cc->gdb_write_register = s390_cpu_gdb_write_register;
- #ifndef CONFIG_USER_ONLY
-diff --git a/target/sh4/cpu.c b/target/sh4/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/sh4/cpu.c
-+++ b/target/sh4/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void superh_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.pc = value;
- }
-+static vaddr superh_cpu_get_pc(CPUState *cs)
-+{
-+    SuperHCPU *cpu = SUPERH_CPU(cs);
-+
-+    return cpu->env.pc;
-+}
-+
- static void superh_cpu_synchronize_from_tb(CPUState *cs,
-                                            const TranslationBlock *tb)
- {
-@@ -XXX,XX +XXX,XX @@ static void superh_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = superh_cpu_has_work;
-     cc->dump_state = superh_cpu_dump_state;
-     cc->set_pc = superh_cpu_set_pc;
-+    cc->get_pc = superh_cpu_get_pc;
-     cc->gdb_read_register = superh_cpu_gdb_read_register;
-     cc->gdb_write_register = superh_cpu_gdb_write_register;
- #ifndef CONFIG_USER_ONLY
-diff --git a/target/sparc/cpu.c b/target/sparc/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/sparc/cpu.c
-+++ b/target/sparc/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void sparc_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.npc = value + 4;
- }
-+static vaddr sparc_cpu_get_pc(CPUState *cs)
-+{
-+    SPARCCPU *cpu = SPARC_CPU(cs);
-+
-+    return cpu->env.pc;
-+}
-+
- static void sparc_cpu_synchronize_from_tb(CPUState *cs,
-                                           const TranslationBlock *tb)
- {
-@@ -XXX,XX +XXX,XX @@ static void sparc_cpu_class_init(ObjectClass *oc, void *data)
-     cc->memory_rw_debug = sparc_cpu_memory_rw_debug;
- #endif
-     cc->set_pc = sparc_cpu_set_pc;
-+    cc->get_pc = sparc_cpu_get_pc;
-     cc->gdb_read_register = sparc_cpu_gdb_read_register;
-     cc->gdb_write_register = sparc_cpu_gdb_write_register;
- #ifndef CONFIG_USER_ONLY
-diff --git a/target/tricore/cpu.c b/target/tricore/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/tricore/cpu.c
-+++ b/target/tricore/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void tricore_cpu_set_pc(CPUState *cs, vaddr value)
-     env->PC = value & ~(target_ulong)1;
- }
-+static vaddr tricore_cpu_get_pc(CPUState *cs)
-+{
-+    TriCoreCPU *cpu = TRICORE_CPU(cs);
-+    CPUTriCoreState *env = &cpu->env;
-+
-+    return env->PC;
-+}
-+
- static void tricore_cpu_synchronize_from_tb(CPUState *cs,
-                                             const TranslationBlock *tb)
- {
-@@ -XXX,XX +XXX,XX @@ static void tricore_cpu_class_init(ObjectClass *c, void *data)
-     cc->dump_state = tricore_cpu_dump_state;
-     cc->set_pc = tricore_cpu_set_pc;
-+    cc->get_pc = tricore_cpu_get_pc;
-     cc->sysemu_ops = &tricore_sysemu_ops;
-     cc->tcg_ops = &tricore_tcg_ops;
- }
-diff --git a/target/xtensa/cpu.c b/target/xtensa/cpu.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/xtensa/cpu.c
-+++ b/target/xtensa/cpu.c
-@@ -XXX,XX +XXX,XX @@ static void xtensa_cpu_set_pc(CPUState *cs, vaddr value)
-     cpu->env.pc = value;
- }
-+static vaddr xtensa_cpu_get_pc(CPUState *cs)
-+{
-+    XtensaCPU *cpu = XTENSA_CPU(cs);
-+
-+    return cpu->env.pc;
-+}
-+
- static bool xtensa_cpu_has_work(CPUState *cs)
- {
- #ifndef CONFIG_USER_ONLY
-@@ -XXX,XX +XXX,XX @@ static void xtensa_cpu_class_init(ObjectClass *oc, void *data)
-     cc->has_work = xtensa_cpu_has_work;
-     cc->dump_state = xtensa_cpu_dump_state;
-     cc->set_pc = xtensa_cpu_set_pc;
-+    cc->get_pc = xtensa_cpu_get_pc;
-     cc->gdb_read_register = xtensa_cpu_gdb_read_register;
-     cc->gdb_write_register = xtensa_cpu_gdb_write_register;
-     cc->gdb_stop_before_watchpoint = true;
 --
 .34.1

-[PULL 04/20] accel/tcg: Rename CPUIOTLBEntry to CPUTLBEntryFull
+[PULL 08/20] plugins: Add PLUGIN_GEN_AFTER_TB
-This structure will shortly contain more than just
+Delay test of plugin_tb->mem_helper until the inject pass.
 data for accessing MMIO.  Rename the 'addr' member
 to 'xlat_section' to more clearly indicate its purpose.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/cpu-defs.h    |  22 ++++----
+ accel/tcg/plugin-gen.c | 37 ++++++++++++++++---------------------
- accel/tcg/cputlb.c         | 102 +++++++++++++++++++------------------
+file changed, 16 insertions(+), 21 deletions(-)
  target/arm/mte_helper.c    |  14 ++---
  target/arm/sve_helper.c    |   4 +-
  target/arm/translate-a64.c |   2 +-
 files changed, 73 insertions(+), 71 deletions(-)
-diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
+diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
---- a/include/exec/cpu-defs.h
+--- a/accel/tcg/plugin-gen.c
-+++ b/include/exec/cpu-defs.h
++++ b/accel/tcg/plugin-gen.c
-@@ -XXX,XX +XXX,XX @@ typedef uint64_t target_ulong;
+@@ -XXX,XX +XXX,XX @@ enum plugin_gen_from {
- #  endif
+     PLUGIN_GEN_FROM_INSN,
- # endif
+     PLUGIN_GEN_FROM_MEM,
+     PLUGIN_GEN_AFTER_INSN,
-+/* Minimalized TLB entry for use by TCG fast path. */
++    PLUGIN_GEN_AFTER_TB,
- typedef struct CPUTLBEntry {
+     PLUGIN_GEN_N_FROMS,
-     /* bit TARGET_LONG_BITS to TARGET_PAGE_BITS : virtual address
+ };
-        bit TARGET_PAGE_BITS-1..4  : Nonzero for accesses that should not
-@@ -XXX,XX +XXX,XX @@ typedef struct CPUTLBEntry {
+@@ -XXX,XX +XXX,XX @@ static void inject_mem_enable_helper(struct qemu_plugin_tb *ptb,
+ /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
- QEMU_BUILD_BUG_ON(sizeof(CPUTLBEntry) != (1 << CPU_TLB_ENTRY_BITS));
+ void plugin_gen_disable_mem_helpers(void)
+ {
--/* The IOTLB is not accessed directly inline by generated TCG code,
+-    /*
-- * so the CPUIOTLBEntry layout is not as critical as that of the
+-     * We could emit the clearing unconditionally and be done. However, this can
-- * CPUTLBEntry. (This is also why we don't want to combine the two
+-     * be wasteful if for instance plugins don't track memory accesses, or if
-- * structs into one.)
+-     * most TBs don't use helpers. Instead, emit the clearing iff the TB calls
-+/*
+-     * helpers that might access guest memory.
-+ * The full TLB entry, which is not accessed by generated TCG code,
+-     *
-+ * so the layout is not as critical as that of CPUTLBEntry. This is
+-     * Note: we do not reset plugin_tb->mem_helper here; a TB might have several
-+ * also why we don't want to combine the two structs.
+-     * exit points, and we want to emit the clearing from all of them.
-  */
+-     */
--typedef struct CPUIOTLBEntry {
+-    if (!tcg_ctx->plugin_tb->mem_helper) {
-+typedef struct CPUTLBEntryFull {
+-        return;
-     /*
++    if (tcg_ctx->plugin_insn) {
--     * @addr contains:
++        tcg_gen_plugin_cb(PLUGIN_GEN_AFTER_TB);
 +     * @xlat_section contains:
       *  - in the lower TARGET_PAGE_BITS, a physical section number
       *  - with the lower TARGET_PAGE_BITS masked off, an offset which
       *    must be added to the virtual address to obtain:
@@ -XXX,XX +XXX,XX @@ typedef struct CPUIOTLBEntry {
       *       number is PHYS_SECTION_NOTDIRTY or PHYS_SECTION_ROM)
       *     + the offset within the target MemoryRegion (otherwise)
       */
 -    hwaddr addr;
 +    hwaddr xlat_section;
      MemTxAttrs attrs;
 -} CPUIOTLBEntry;
 +} CPUTLBEntryFull;
  /*
   * Data elements that are per MMU mode, minus the bits accessed by
@@ -XXX,XX +XXX,XX @@ typedef struct CPUTLBDesc {
      size_t vindex;
      /* The tlb victim table, in two parts.  */
      CPUTLBEntry vtable[CPU_VTLB_SIZE];
 -    CPUIOTLBEntry viotlb[CPU_VTLB_SIZE];
 -    /* The iotlb.  */
 -    CPUIOTLBEntry *iotlb;
 +    CPUTLBEntryFull vfulltlb[CPU_VTLB_SIZE];
 +    CPUTLBEntryFull *fulltlb;
  } CPUTLBDesc;
  /*
 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cputlb.c
 +++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
      }
+-    tcg_gen_st_ptr(tcg_constant_ptr(NULL), tcg_env,
-     g_free(fast->table);
+-                   offsetof(CPUState, plugin_mem_cbs) - offsetof(ArchCPU, env));
 -    g_free(desc->iotlb);
 +    g_free(desc->fulltlb);
      tlb_window_reset(desc, now, 0);
      /* desc->n_used_entries is cleared by the caller */
      fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
      fast->table = g_try_new(CPUTLBEntry, new_size);
 -    desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
 +    desc->fulltlb = g_try_new(CPUTLBEntryFull, new_size);
      /*
       * If the allocations fail, try smaller sizes. We just freed some
@@ -XXX,XX +XXX,XX @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
       * allocations to fail though, so we progressively reduce the allocation
       * size, aborting if we cannot even allocate the smallest TLB we support.
       */
 -    while (fast->table == NULL || desc->iotlb == NULL) {
 +    while (fast->table == NULL || desc->fulltlb == NULL) {
          if (new_size == (1 << CPU_TLB_DYN_MIN_BITS)) {
              error_report("%s: %s", __func__, strerror(errno));
              abort();
@@ -XXX,XX +XXX,XX @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
          fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
          g_free(fast->table);
 -        g_free(desc->iotlb);
 +        g_free(desc->fulltlb);
          fast->table = g_try_new(CPUTLBEntry, new_size);
 -        desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
 +        desc->fulltlb = g_try_new(CPUTLBEntryFull, new_size);
      }
  }
-@@ -XXX,XX +XXX,XX @@ static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now)
+ static void plugin_gen_insn_udata(const struct qemu_plugin_tb *ptb,
-     desc->n_used_entries = 0;
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
-     fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
+     inject_mem_enable_helper(ptb, insn, begin_op);
      fast->table = g_new(CPUTLBEntry, n_entries);
 -    desc->iotlb = g_new(CPUIOTLBEntry, n_entries);
 +    desc->fulltlb = g_new(CPUTLBEntryFull, n_entries);
      tlb_mmu_flush_locked(desc, fast);
  }
-@@ -XXX,XX +XXX,XX @@ void tlb_destroy(CPUState *cpu)
+-static void gen_disable_mem_helper(struct qemu_plugin_tb *ptb,
-         CPUTLBDescFast *fast = &env_tlb(env)->f[i];
+-                                   struct qemu_plugin_insn *insn)
++static void gen_disable_mem_helper(void)
-         g_free(fast->table);
+ {
--        g_free(desc->iotlb);
+-    if (insn->mem_helper) {
-+        g_free(desc->fulltlb);
+-        tcg_gen_st_ptr(tcg_constant_ptr(0), tcg_env,
-     }
+-                       offsetof(CPUState, plugin_mem_cbs) -
 -                       offsetof(ArchCPU, env));
 -    }
 +    tcg_gen_st_ptr(tcg_constant_ptr(0), tcg_env,
 +                   offsetof(CPUState, plugin_mem_cbs) -
 +                   offsetof(ArchCPU, env));
  }
-@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
+ static void gen_udata_cb(struct qemu_plugin_dyn_cb *cb)
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
-         /* Evict the old entry into the victim tlb.  */
+             tcg_ctx->emit_before_op = op;
-         copy_tlb_helper_locked(tv, te);
--        desc->viotlb[vidx] = desc->iotlb[index];
+             switch (from) {
-+        desc->vfulltlb[vidx] = desc->fulltlb[index];
++            case PLUGIN_GEN_AFTER_TB:
-         tlb_n_used_entries_dec(env, mmu_idx);
++                if (plugin_tb->mem_helper) {
-     }
++                    gen_disable_mem_helper();
++                }
-@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
++                break;
-      * subtract here is that of the page base, and not the same as the
++
-      * vaddr we add back in io_readx()/io_writex()/get_page_addr_code().
+             case PLUGIN_GEN_AFTER_INSN:
-      */
+                 assert(insn != NULL);
--    desc->iotlb[index].addr = iotlb - vaddr_page;
+-                gen_disable_mem_helper(plugin_tb, insn);
--    desc->iotlb[index].attrs = attrs;
++                if (insn->mem_helper) {
-+    desc->fulltlb[index].xlat_section = iotlb - vaddr_page;
++                    gen_disable_mem_helper();
-+    desc->fulltlb[index].attrs = attrs;
++                }
+                 break;
-     /* Now calculate the new entry */
-     tn.addend = addend - vaddr_page;
+             case PLUGIN_GEN_FROM_TB:
@@ -XXX,XX +XXX,XX @@ static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr,
      }
  }
 -static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
 +static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full,
                           int mmu_idx, target_ulong addr, uintptr_t retaddr,
                           MMUAccessType access_type, MemOp op)
  {
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
      bool locked = false;
      MemTxResult r;
 -    section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
 +    section = iotlb_to_section(cpu, full->xlat_section, full->attrs);
      mr = section->mr;
 -    mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
 +    mr_offset = (full->xlat_section & TARGET_PAGE_MASK) + addr;
      cpu->mem_io_pc = retaddr;
      if (!cpu->can_do_io) {
          cpu_io_recompile(cpu, retaddr);
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
          qemu_mutex_lock_iothread();
          locked = true;
      }
 -    r = memory_region_dispatch_read(mr, mr_offset, &val, op, iotlbentry->attrs);
 +    r = memory_region_dispatch_read(mr, mr_offset, &val, op, full->attrs);
      if (r != MEMTX_OK) {
          hwaddr physaddr = mr_offset +
              section->offset_within_address_space -
              section->offset_within_region;
          cpu_transaction_failed(cpu, physaddr, addr, memop_size(op), access_type,
 -                               mmu_idx, iotlbentry->attrs, r, retaddr);
 +                               mmu_idx, full->attrs, r, retaddr);
      }
      if (locked) {
          qemu_mutex_unlock_iothread();
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
  }
  /*
 - * Save a potentially trashed IOTLB entry for later lookup by plugin.
 - * This is read by tlb_plugin_lookup if the iotlb entry doesn't match
 + * Save a potentially trashed CPUTLBEntryFull for later lookup by plugin.
 + * This is read by tlb_plugin_lookup if the fulltlb entry doesn't match
   * because of the side effect of io_writex changing memory layout.
   */
  static void save_iotlb_data(CPUState *cs, hwaddr addr,
@@ -XXX,XX +XXX,XX @@ static void save_iotlb_data(CPUState *cs, hwaddr addr,
  #endif
  }
 -static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
 +static void io_writex(CPUArchState *env, CPUTLBEntryFull *full,
                        int mmu_idx, uint64_t val, target_ulong addr,
                        uintptr_t retaddr, MemOp op)
  {
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
      bool locked = false;
      MemTxResult r;
 -    section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
 +    section = iotlb_to_section(cpu, full->xlat_section, full->attrs);
      mr = section->mr;
 -    mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
 +    mr_offset = (full->xlat_section & TARGET_PAGE_MASK) + addr;
      if (!cpu->can_do_io) {
          cpu_io_recompile(cpu, retaddr);
      }
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
       * The memory_region_dispatch may trigger a flush/resize
       * so for plugins we save the iotlb_data just in case.
       */
 -    save_iotlb_data(cpu, iotlbentry->addr, section, mr_offset);
 +    save_iotlb_data(cpu, full->xlat_section, section, mr_offset);
      if (!qemu_mutex_iothread_locked()) {
          qemu_mutex_lock_iothread();
          locked = true;
      }
 -    r = memory_region_dispatch_write(mr, mr_offset, val, op, iotlbentry->attrs);
 +    r = memory_region_dispatch_write(mr, mr_offset, val, op, full->attrs);
      if (r != MEMTX_OK) {
          hwaddr physaddr = mr_offset +
              section->offset_within_address_space -
              section->offset_within_region;
          cpu_transaction_failed(cpu, physaddr, addr, memop_size(op),
 -                               MMU_DATA_STORE, mmu_idx, iotlbentry->attrs, r,
 +                               MMU_DATA_STORE, mmu_idx, full->attrs, r,
                                 retaddr);
      }
      if (locked) {
@@ -XXX,XX +XXX,XX @@ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index,
              copy_tlb_helper_locked(vtlb, &tmptlb);
              qemu_spin_unlock(&env_tlb(env)->c.lock);
 -            CPUIOTLBEntry tmpio, *io = &env_tlb(env)->d[mmu_idx].iotlb[index];
 -            CPUIOTLBEntry *vio = &env_tlb(env)->d[mmu_idx].viotlb[vidx];
 -            tmpio = *io; *io = *vio; *vio = tmpio;
 +            CPUTLBEntryFull *f1 = &env_tlb(env)->d[mmu_idx].fulltlb[index];
 +            CPUTLBEntryFull *f2 = &env_tlb(env)->d[mmu_idx].vfulltlb[vidx];
 +            CPUTLBEntryFull tmpf;
 +            tmpf = *f1; *f1 = *f2; *f2 = tmpf;
              return true;
          }
      }
@@ -XXX,XX +XXX,XX @@ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index,
                   (ADDR) & TARGET_PAGE_MASK)
  static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
 -                           CPUIOTLBEntry *iotlbentry, uintptr_t retaddr)
 +                           CPUTLBEntryFull *full, uintptr_t retaddr)
  {
 -    ram_addr_t ram_addr = mem_vaddr + iotlbentry->addr;
 +    ram_addr_t ram_addr = mem_vaddr + full->xlat_section;
      trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size);
@@ -XXX,XX +XXX,XX @@ int probe_access_flags(CPUArchState *env, target_ulong addr,
      /* Handle clean RAM pages.  */
      if (unlikely(flags & TLB_NOTDIRTY)) {
          uintptr_t index = tlb_index(env, mmu_idx, addr);
 -        CPUIOTLBEntry *iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
 +        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
 -        notdirty_write(env_cpu(env), addr, 1, iotlbentry, retaddr);
 +        notdirty_write(env_cpu(env), addr, 1, full, retaddr);
          flags &= ~TLB_NOTDIRTY;
      }
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
      if (unlikely(flags & (TLB_NOTDIRTY | TLB_WATCHPOINT))) {
          uintptr_t index = tlb_index(env, mmu_idx, addr);
 -        CPUIOTLBEntry *iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
 +        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
          /* Handle watchpoints.  */
          if (flags & TLB_WATCHPOINT) {
              int wp_access = (access_type == MMU_DATA_STORE
                               ? BP_MEM_WRITE : BP_MEM_READ);
              cpu_check_watchpoint(env_cpu(env), addr, size,
 -                                 iotlbentry->attrs, wp_access, retaddr);
 +                                 full->attrs, wp_access, retaddr);
          }
          /* Handle clean RAM pages.  */
          if (flags & TLB_NOTDIRTY) {
 -            notdirty_write(env_cpu(env), addr, 1, iotlbentry, retaddr);
 +            notdirty_write(env_cpu(env), addr, 1, full, retaddr);
          }
      }
@@ -XXX,XX +XXX,XX @@ tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, target_ulong addr,
   * should have just filled the TLB. The one corner case is io_writex
   * which can cause TLB flushes and potential resizing of the TLBs
   * losing the information we need. In those cases we need to recover
 - * data from a copy of the iotlbentry. As long as this always occurs
 + * data from a copy of the CPUTLBEntryFull. As long as this always occurs
   * from the same thread (which a mem callback will be) this is safe.
   */
@@ -XXX,XX +XXX,XX @@ bool tlb_plugin_lookup(CPUState *cpu, target_ulong addr, int mmu_idx,
      if (likely(tlb_hit(tlb_addr, addr))) {
          /* We must have an iotlb entry for MMIO */
          if (tlb_addr & TLB_MMIO) {
 -            CPUIOTLBEntry *iotlbentry;
 -            iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
 +            CPUTLBEntryFull *full;
 +            full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
              data->is_io = true;
 -            data->v.io.section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
 -            data->v.io.offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
 +            data->v.io.section =
 +                iotlb_to_section(cpu, full->xlat_section, full->attrs);
 +            data->v.io.offset = (full->xlat_section & TARGET_PAGE_MASK) + addr;
          } else {
              data->is_io = false;
              data->v.ram.hostaddr = (void *)((uintptr_t)addr + tlbe->addend);
@@ -XXX,XX +XXX,XX @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
      if (unlikely(tlb_addr & TLB_NOTDIRTY)) {
          notdirty_write(env_cpu(env), addr, size,
 -                       &env_tlb(env)->d[mmu_idx].iotlb[index], retaddr);
 +                       &env_tlb(env)->d[mmu_idx].fulltlb[index], retaddr);
      }
      return hostaddr;
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, MemOpIdx oi,
      /* Handle anything that isn't just a straight memory access.  */
      if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
 -        CPUIOTLBEntry *iotlbentry;
 +        CPUTLBEntryFull *full;
          bool need_swap;
          /* For anything that is unaligned, recurse through full_load.  */
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, MemOpIdx oi,
              goto do_unaligned_access;
          }
 -        iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
 +        full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
          /* Handle watchpoints.  */
          if (unlikely(tlb_addr & TLB_WATCHPOINT)) {
              /* On watchpoint hit, this will longjmp out.  */
              cpu_check_watchpoint(env_cpu(env), addr, size,
 -                                 iotlbentry->attrs, BP_MEM_READ, retaddr);
 +                                 full->attrs, BP_MEM_READ, retaddr);
          }
          need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
          /* Handle I/O access.  */
          if (likely(tlb_addr & TLB_MMIO)) {
 -            return io_readx(env, iotlbentry, mmu_idx, addr, retaddr,
 +            return io_readx(env, full, mmu_idx, addr, retaddr,
                              access_type, op ^ (need_swap * MO_BSWAP));
          }
@@ -XXX,XX +XXX,XX @@ store_helper_unaligned(CPUArchState *env, target_ulong addr, uint64_t val,
       */
      if (unlikely(tlb_addr & TLB_WATCHPOINT)) {
          cpu_check_watchpoint(env_cpu(env), addr, size - size2,
 -                             env_tlb(env)->d[mmu_idx].iotlb[index].attrs,
 +                             env_tlb(env)->d[mmu_idx].fulltlb[index].attrs,
                               BP_MEM_WRITE, retaddr);
      }
      if (unlikely(tlb_addr2 & TLB_WATCHPOINT)) {
          cpu_check_watchpoint(env_cpu(env), page2, size2,
 -                             env_tlb(env)->d[mmu_idx].iotlb[index2].attrs,
 +                             env_tlb(env)->d[mmu_idx].fulltlb[index2].attrs,
                               BP_MEM_WRITE, retaddr);
      }
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
      /* Handle anything that isn't just a straight memory access.  */
      if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
 -        CPUIOTLBEntry *iotlbentry;
 +        CPUTLBEntryFull *full;
          bool need_swap;
          /* For anything that is unaligned, recurse through byte stores.  */
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
              goto do_unaligned_access;
          }
 -        iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
 +        full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
          /* Handle watchpoints.  */
          if (unlikely(tlb_addr & TLB_WATCHPOINT)) {
              /* On watchpoint hit, this will longjmp out.  */
              cpu_check_watchpoint(env_cpu(env), addr, size,
 -                                 iotlbentry->attrs, BP_MEM_WRITE, retaddr);
 +                                 full->attrs, BP_MEM_WRITE, retaddr);
          }
          need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
          /* Handle I/O access.  */
          if (tlb_addr & TLB_MMIO) {
 -            io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr,
 +            io_writex(env, full, mmu_idx, val, addr, retaddr,
                        op ^ (need_swap * MO_BSWAP));
              return;
          }
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
          /* Handle clean RAM pages.  */
          if (tlb_addr & TLB_NOTDIRTY) {
 -            notdirty_write(env_cpu(env), addr, size, iotlbentry, retaddr);
 +            notdirty_write(env_cpu(env), addr, size, full, retaddr);
          }
          haddr = (void *)((uintptr_t)addr + entry->addend);
 diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/arm/mte_helper.c
 +++ b/target/arm/mte_helper.c
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
      return tags + index;
  #else
      uintptr_t index;
 -    CPUIOTLBEntry *iotlbentry;
 +    CPUTLBEntryFull *full;
      int in_page, flags;
      ram_addr_t ptr_ra;
      hwaddr ptr_paddr, tag_paddr, xlat;
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
      assert(!(flags & TLB_INVALID_MASK));
      /*
 -     * Find the iotlbentry for ptr.  This *must* be present in the TLB
 +     * Find the CPUTLBEntryFull for ptr.  This *must* be present in the TLB
       * because we just found the mapping.
       * TODO: Perhaps there should be a cputlb helper that returns a
       * matching tlb entry + iotlb entry.
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
          g_assert(tlb_hit(comparator, ptr));
      }
  # endif
 -    iotlbentry = &env_tlb(env)->d[ptr_mmu_idx].iotlb[index];
 +    full = &env_tlb(env)->d[ptr_mmu_idx].fulltlb[index];
      /* If the virtual page MemAttr != Tagged, access unchecked. */
 -    if (!arm_tlb_mte_tagged(&iotlbentry->attrs)) {
 +    if (!arm_tlb_mte_tagged(&full->attrs)) {
          return NULL;
      }
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
          int wp = ptr_access == MMU_DATA_LOAD ? BP_MEM_READ : BP_MEM_WRITE;
          assert(ra != 0);
          cpu_check_watchpoint(env_cpu(env), ptr, ptr_size,
 -                             iotlbentry->attrs, wp, ra);
 +                             full->attrs, wp, ra);
      }
      /*
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
      tag_paddr = ptr_paddr >> (LOG2_TAG_GRANULE + 1);
      /* Look up the address in tag space. */
 -    tag_asi = iotlbentry->attrs.secure ? ARMASIdx_TagS : ARMASIdx_TagNS;
 +    tag_asi = full->attrs.secure ? ARMASIdx_TagS : ARMASIdx_TagNS;
      tag_as = cpu_get_address_space(env_cpu(env), tag_asi);
      mr = address_space_translate(tag_as, tag_paddr, &xlat, NULL,
                                   tag_access == MMU_DATA_STORE,
 -                                 iotlbentry->attrs);
 +                                 full->attrs);
      /*
       * Note that @mr will never be NULL.  If there is nothing in the address
 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/arm/sve_helper.c
 +++ b/target/arm/sve_helper.c
@@ -XXX,XX +XXX,XX @@ bool sve_probe_page(SVEHostPage *info, bool nofault, CPUARMState *env,
          g_assert(tlb_hit(comparator, addr));
  # endif
 -        CPUIOTLBEntry *iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
 -        info->attrs = iotlbentry->attrs;
 +        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
 +        info->attrs = full->attrs;
      }
  #endif
 diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/arm/translate-a64.c
 +++ b/target/arm/translate-a64.c
@@ -XXX,XX +XXX,XX @@ static bool is_guarded_page(CPUARMState *env, DisasContext *s)
       * table entry even for that case.
       */
      return (tlb_hit(entry->addr_code, addr) &&
 -            arm_tlb_bti_gp(&env_tlb(env)->d[mmu_idx].iotlb[index].attrs));
 +            arm_tlb_bti_gp(&env_tlb(env)->d[mmu_idx].fulltlb[index].attrs));
  #endif
  }
 --
 .34.1

-[PULL 19/20] tcg/ppc: Optimize 26-bit jumps
+[PULL 09/20] plugins: Use emit_before_op for PLUGIN_GEN_FROM_INSN
-From: Leandro Lupori <leandro.lupori@eldorado.org.br>
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 PowerPC64 processors handle direct branches better than indirect
 ones, resulting in less stalled cycles and branch misses.
 However, PPC's tb_target_set_jmp_target() was only using direct
 branches for 16-bit jumps, while PowerPC64's unconditional branch
 instructions are able to handle displacements of up to 26 bits.
 To take advantage of this, now jumps whose displacements fit in
 between 17 and 26 bits are also converted to direct branches.
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Signed-off-by: Leandro Lupori <leandro.lupori@eldorado.org.br>
 [rth: Expanded some commentary.]
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- tcg/ppc/tcg-target.c.inc | 119 +++++++++++++++++++++++++++++----------
+ include/qemu/plugin.h  |   1 -
-file changed, 88 insertions(+), 31 deletions(-)
+ accel/tcg/plugin-gen.c | 286 ++++++++++-------------------------------
  plugins/api.c          |   8 +-
 files changed, 67 insertions(+), 228 deletions(-)
-diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc
+diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
 index XXXXXXX..XXXXXXX 100644
---- a/tcg/ppc/tcg-target.c.inc
+--- a/include/qemu/plugin.h
-+++ b/tcg/ppc/tcg-target.c.inc
++++ b/include/qemu/plugin.h
-@@ -XXX,XX +XXX,XX @@ static void tcg_out_mb(TCGContext *s, TCGArg a0)
+@@ -XXX,XX +XXX,XX @@ enum plugin_dyn_cb_type {
-     tcg_out32(s, insn);
- }
+ enum plugin_dyn_cb_subtype {
+     PLUGIN_CB_REGULAR,
-+static inline uint64_t make_pair(tcg_insn_unit i1, tcg_insn_unit i2)
+-    PLUGIN_CB_REGULAR_R,
-+{
+     PLUGIN_CB_INLINE,
-+    if (HOST_BIG_ENDIAN) {
+     PLUGIN_N_CB_SUBTYPES,
-+        return (uint64_t)i1 << 32 | i2;
+ };
-+    }
+diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
-+    return (uint64_t)i2 << 32 | i1;
+index XXXXXXX..XXXXXXX 100644
-+}
+--- a/accel/tcg/plugin-gen.c
-+
++++ b/accel/tcg/plugin-gen.c
-+static inline void ppc64_replace2(uintptr_t rx, uintptr_t rw,
+@@ -XXX,XX +XXX,XX @@ void HELPER(plugin_vcpu_mem_cb)(unsigned int vcpu_index,
-+                                  tcg_insn_unit i0, tcg_insn_unit i1)
+                                 void *userdata)
-+{
+ { }
-+#if TCG_TARGET_REG_BITS == 64
-+    qatomic_set((uint64_t *)rw, make_pair(i0, i1));
+-static void gen_empty_udata_cb(void (*gen_helper)(TCGv_i32, TCGv_ptr))
-+    flush_idcache_range(rx, rw, 8);
+-{
-+#else
+-    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
-+    qemu_build_not_reached();
+-    TCGv_ptr udata = tcg_temp_ebb_new_ptr();
-+#endif
+-
-+}
+-    tcg_gen_movi_ptr(udata, 0);
-+
+-    tcg_gen_ld_i32(cpu_index, tcg_env,
-+static inline void ppc64_replace4(uintptr_t rx, uintptr_t rw,
+-                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
-+                                  tcg_insn_unit i0, tcg_insn_unit i1,
+-    gen_helper(cpu_index, udata);
-+                                  tcg_insn_unit i2, tcg_insn_unit i3)
+-
-+{
+-    tcg_temp_free_ptr(udata);
-+    uint64_t p[2];
+-    tcg_temp_free_i32(cpu_index);
-+
+-}
-+    p[!HOST_BIG_ENDIAN] = make_pair(i0, i1);
+-
-+    p[HOST_BIG_ENDIAN] = make_pair(i2, i3);
+-static void gen_empty_udata_cb_no_wg(void)
 -{
 -    gen_empty_udata_cb(gen_helper_plugin_vcpu_udata_cb_no_wg);
 -}
 -
 -static void gen_empty_udata_cb_no_rwg(void)
 -{
 -    gen_empty_udata_cb(gen_helper_plugin_vcpu_udata_cb_no_rwg);
 -}
 -
  /*
   * For now we only support addi_i64.
   * When we support more ops, we can generate one empty inline cb for each.
@@ -XXX,XX +XXX,XX @@ static void gen_empty_mem_cb(TCGv_i64 addr, uint32_t info)
      tcg_temp_free_i32(cpu_index);
  }
 -/*
 - * Share the same function for enable/disable. When enabling, the NULL
 - * pointer will be overwritten later.
 - */
 -static void gen_empty_mem_helper(void)
 -{
 -    TCGv_ptr ptr = tcg_temp_ebb_new_ptr();
 -
 -    tcg_gen_movi_ptr(ptr, 0);
 -    tcg_gen_st_ptr(ptr, tcg_env, offsetof(CPUState, plugin_mem_cbs) -
 -                                 offsetof(ArchCPU, env));
 -    tcg_temp_free_ptr(ptr);
 -}
 -
  static void gen_plugin_cb_start(enum plugin_gen_from from,
                                  enum plugin_gen_cb type, unsigned wr)
  {
      tcg_gen_plugin_cb_start(from, type, wr);
  }
 -static void gen_wrapped(enum plugin_gen_from from,
 -                        enum plugin_gen_cb type, void (*func)(void))
 -{
 -    gen_plugin_cb_start(from, type, 0);
 -    func();
 -    tcg_gen_plugin_cb_end();
 -}
 -
  static void plugin_gen_empty_callback(enum plugin_gen_from from)
  {
      switch (from) {
      case PLUGIN_GEN_AFTER_INSN:
      case PLUGIN_GEN_FROM_TB:
 -        tcg_gen_plugin_cb(from);
 -        break;
      case PLUGIN_GEN_FROM_INSN:
 -        /*
 -         * Note: plugin_gen_inject() relies on ENABLE_MEM_HELPER being
 -         * the first callback of an instruction
 -         */
 -        gen_wrapped(from, PLUGIN_GEN_ENABLE_MEM_HELPER,
 -                    gen_empty_mem_helper);
 -        gen_wrapped(from, PLUGIN_GEN_CB_UDATA, gen_empty_udata_cb_no_rwg);
 -        gen_wrapped(from, PLUGIN_GEN_CB_UDATA_R, gen_empty_udata_cb_no_wg);
 -        gen_wrapped(from, PLUGIN_GEN_CB_INLINE, gen_empty_inline_cb);
 +        tcg_gen_plugin_cb(from);
          break;
      default:
          g_assert_not_reached();
@@ -XXX,XX +XXX,XX @@ static TCGOp *copy_mul_i32(TCGOp **begin_op, TCGOp *op, uint32_t v)
      return op;
  }
 -static TCGOp *copy_st_ptr(TCGOp **begin_op, TCGOp *op)
 -{
 -    if (UINTPTR_MAX == UINT32_MAX) {
 -        /* st_i32 */
 -        op = copy_op(begin_op, op, INDEX_op_st_i32);
 -    } else {
 -        /* st_i64 */
 -        op = copy_st_i64(begin_op, op);
 -    }
 -    return op;
 -}
 -
  static TCGOp *copy_call(TCGOp **begin_op, TCGOp *op, void *func, int *cb_idx)
  {
      TCGOp *old_op;
@@ -XXX,XX +XXX,XX @@ static TCGOp *copy_call(TCGOp **begin_op, TCGOp *op, void *func, int *cb_idx)
      return op;
  }
 -/*
 - * When we append/replace ops here we are sensitive to changing patterns of
 - * TCGOps generated by the tcg_gen_FOO calls when we generated the
 - * empty callbacks. This will assert very quickly in a debug build as
 - * we assert the ops we are replacing are the correct ones.
 - */
 -static TCGOp *append_udata_cb(const struct qemu_plugin_dyn_cb *cb,
 -                              TCGOp *begin_op, TCGOp *op, int *cb_idx)
 -{
 -    /* const_ptr */
 -    op = copy_const_ptr(&begin_op, op, cb->userp);
 -
 -    /* copy the ld_i32, but note that we only have to copy it once */
 -    if (*cb_idx == -1) {
 -        op = copy_op(&begin_op, op, INDEX_op_ld_i32);
 -    } else {
 -        begin_op = QTAILQ_NEXT(begin_op, link);
 -        tcg_debug_assert(begin_op && begin_op->opc == INDEX_op_ld_i32);
 -    }
 -
 -    /* call */
 -    op = copy_call(&begin_op, op, cb->regular.f.vcpu_udata, cb_idx);
 -
 -    return op;
 -}
 -
  static TCGOp *append_inline_cb(const struct qemu_plugin_dyn_cb *cb,
                                 TCGOp *begin_op, TCGOp *op,
                                 int *unused)
@@ -XXX,XX +XXX,XX @@ typedef TCGOp *(*inject_fn)(const struct qemu_plugin_dyn_cb *cb,
                              TCGOp *begin_op, TCGOp *op, int *intp);
  typedef bool (*op_ok_fn)(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb);
 -static bool op_ok(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb)
 -{
 -    return true;
 -}
 -
  static bool op_rw(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb)
  {
      int w;
@@ -XXX,XX +XXX,XX @@ static void inject_cb_type(const GArray *cbs, TCGOp *begin_op,
      rm_ops_range(begin_op, end_op);
  }
 -static void
 -inject_udata_cb(const GArray *cbs, TCGOp *begin_op)
 -{
 -    inject_cb_type(cbs, begin_op, append_udata_cb, op_ok);
 -}
 -
  static void
  inject_inline_cb(const GArray *cbs, TCGOp *begin_op, op_ok_fn ok)
  {
@@ -XXX,XX +XXX,XX @@ inject_mem_cb(const GArray *cbs, TCGOp *begin_op)
      inject_cb_type(cbs, begin_op, append_mem_cb, op_rw);
  }
 -/* we could change the ops in place, but we can reuse more code by copying */
 -static void inject_mem_helper(TCGOp *begin_op, GArray *arr)
 -{
 -    TCGOp *orig_op = begin_op;
 -    TCGOp *end_op;
 -    TCGOp *op;
 -
 -    end_op = find_op(begin_op, INDEX_op_plugin_cb_end);
 -    tcg_debug_assert(end_op);
 -
 -    /* const ptr */
 -    op = copy_const_ptr(&begin_op, end_op, arr);
 -
 -    /* st_ptr */
 -    op = copy_st_ptr(&begin_op, op);
 -
 -    rm_ops_range(orig_op, end_op);
 -}
 -
 -/*
 - * Tracking memory accesses performed from helpers requires extra work.
 - * If an instruction is emulated with helpers, we do two things:
 - * (1) copy the CB descriptors, and keep track of it so that they can be
 - * freed later on, and (2) point CPUState.plugin_mem_cbs to the descriptors, so
 - * that we can read them at run-time (i.e. when the helper executes).
 - * This run-time access is performed from qemu_plugin_vcpu_mem_cb.
 - *
 - * Note that plugin_gen_disable_mem_helpers undoes (2). Since it
 - * is possible that the code we generate after the instruction is
 - * dead, we also add checks before generating tb_exit etc.
 - */
 -static void inject_mem_enable_helper(struct qemu_plugin_tb *ptb,
 -                                     struct qemu_plugin_insn *plugin_insn,
 -                                     TCGOp *begin_op)
 -{
 -    GArray *cbs[2];
 -    GArray *arr;
 -    size_t n_cbs, i;
 -
 -    cbs[0] = plugin_insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
 -    cbs[1] = plugin_insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
 -
 -    n_cbs = 0;
 -    for (i = 0; i < ARRAY_SIZE(cbs); i++) {
 -        n_cbs += cbs[i]->len;
 -    }
 -
 -    plugin_insn->mem_helper = plugin_insn->calls_helpers && n_cbs;
 -    if (likely(!plugin_insn->mem_helper)) {
 -        rm_ops(begin_op);
 -        return;
 -    }
 -    ptb->mem_helper = true;
 -
 -    arr = g_array_sized_new(false, false,
 -                            sizeof(struct qemu_plugin_dyn_cb), n_cbs);
 -
 -    for (i = 0; i < ARRAY_SIZE(cbs); i++) {
 -        g_array_append_vals(arr, cbs[i]->data, cbs[i]->len);
 -    }
 -
 -    qemu_plugin_add_dyn_cb_arr(arr);
 -    inject_mem_helper(begin_op, arr);
 -}
 -
  /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
  void plugin_gen_disable_mem_helpers(void)
  {
@@ -XXX,XX +XXX,XX @@ void plugin_gen_disable_mem_helpers(void)
      }
  }
 -static void plugin_gen_insn_udata(const struct qemu_plugin_tb *ptb,
 -                                  TCGOp *begin_op, int insn_idx)
 -{
 -    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
 -
 -    inject_udata_cb(insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR], begin_op);
 -}
 -
 -static void plugin_gen_insn_udata_r(const struct qemu_plugin_tb *ptb,
 -                                    TCGOp *begin_op, int insn_idx)
 -{
 -    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
 -
 -    inject_udata_cb(insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR_R], begin_op);
 -}
 -
 -static void plugin_gen_insn_inline(const struct qemu_plugin_tb *ptb,
 -                                   TCGOp *begin_op, int insn_idx)
 -{
 -    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
 -    inject_inline_cb(insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_INLINE],
 -                     begin_op, op_ok);
 -}
 -
  static void plugin_gen_mem_regular(const struct qemu_plugin_tb *ptb,
                                     TCGOp *begin_op, int insn_idx)
  {
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_mem_inline(const struct qemu_plugin_tb *ptb,
      inject_inline_cb(cbs, begin_op, op_rw);
  }
 -static void plugin_gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
 -                                         TCGOp *begin_op, int insn_idx)
 +static void gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
 +                                  struct qemu_plugin_insn *insn)
  {
 -    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
 -    inject_mem_enable_helper(ptb, insn, begin_op);
 +    GArray *cbs[2];
 +    GArray *arr;
 +    size_t n_cbs;
 +
 +    /*
-+     * There's no convenient way to get the compiler to allocate a pair
++     * Tracking memory accesses performed from helpers requires extra work.
-+     * of registers at an even index, so copy into r6/r7 and clobber.
++     * If an instruction is emulated with helpers, we do two things:
 +     * (1) copy the CB descriptors, and keep track of it so that they can be
 +     * freed later on, and (2) point CPUState.plugin_mem_cbs to the
 +     * descriptors, so that we can read them at run-time
 +     * (i.e. when the helper executes).
 +     * This run-time access is performed from qemu_plugin_vcpu_mem_cb.
 +     *
 +     * Note that plugin_gen_disable_mem_helpers undoes (2). Since it
 +     * is possible that the code we generate after the instruction is
 +     * dead, we also add checks before generating tb_exit etc.
 +     */
-+    asm("mr  %%r6, %1\n\t"
++    if (!insn->calls_helpers) {
 +        "mr  %%r7, %2\n\t"
 +        "stq %%r6, %0"
 +        : "=Q"(*(__int128 *)rw) : "r"(p[0]), "r"(p[1]) : "r6", "r7");
 +    flush_idcache_range(rx, rw, 16);
 +}
 +
  void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx,
                                uintptr_t jmp_rw, uintptr_t addr)
  {
 -    if (TCG_TARGET_REG_BITS == 64) {
 -        tcg_insn_unit i1, i2;
 -        intptr_t tb_diff = addr - tc_ptr;
 -        intptr_t br_diff = addr - (jmp_rx + 4);
 -        uint64_t pair;
 +    tcg_insn_unit i0, i1, i2, i3;
 +    intptr_t tb_diff = addr - tc_ptr;
 +    intptr_t br_diff = addr - (jmp_rx + 4);
 +    intptr_t lo, hi;
 -        /* This does not exercise the range of the branch, but we do
 -           still need to be able to load the new value of TCG_REG_TB.
 -           But this does still happen quite often.  */
 -        if (tb_diff == (int16_t)tb_diff) {
 -            i1 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, tb_diff);
 -            i2 = B | (br_diff & 0x3fffffc);
 -        } else {
 -            intptr_t lo = (int16_t)tb_diff;
 -            intptr_t hi = (int32_t)(tb_diff - lo);
 -            assert(tb_diff == hi + lo);
 -            i1 = ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, hi >> 16);
 -            i2 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, lo);
 -        }
 -#if HOST_BIG_ENDIAN
 -        pair = (uint64_t)i1 << 32 | i2;
 -#else
 -        pair = (uint64_t)i2 << 32 | i1;
 -#endif
 -
 -        /* As per the enclosing if, this is ppc64.  Avoid the _Static_assert
 -           within qatomic_set that would fail to build a ppc32 host.  */
 -        qatomic_set__nocheck((uint64_t *)jmp_rw, pair);
 -        flush_idcache_range(jmp_rx, jmp_rw, 8);
 -    } else {
 +    if (TCG_TARGET_REG_BITS == 32) {
          intptr_t diff = addr - jmp_rx;
          tcg_debug_assert(in_range_b(diff));
          qatomic_set((uint32_t *)jmp_rw, B | (diff & 0x3fffffc));
          flush_idcache_range(jmp_rx, jmp_rw, 4);
 +        return;
      }
 +
 +    /*
 +     * For 16-bit displacements, we can use a single add + branch.
 +     * This happens quite often.
 +     */
 +    if (tb_diff == (int16_t)tb_diff) {
 +        i0 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, tb_diff);
 +        i1 = B | (br_diff & 0x3fffffc);
 +        ppc64_replace2(jmp_rx, jmp_rw, i0, i1);
 +        return;
 +    }
 +
-+    lo = (int16_t)tb_diff;
++    cbs[0] = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
-+    hi = (int32_t)(tb_diff - lo);
++    cbs[1] = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
-+    assert(tb_diff == hi + lo);
++    n_cbs = cbs[0]->len + cbs[1]->len;
-+    i0 = ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, hi >> 16);
++
-+    i1 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, lo);
++    if (n_cbs == 0) {
-+
++        insn->mem_helper = false;
 +    /*
 +     * Without stq from 2.07, we can only update two insns,
 +     * and those must be the ones that load the target address.
 +     */
 +    if (!have_isa_2_07) {
 +        ppc64_replace2(jmp_rx, jmp_rw, i0, i1);
 +        return;
 +    }
-+
++    insn->mem_helper = true;
-+    /*
++    ptb->mem_helper = true;
-+     * For 26-bit displacements, we can use a direct branch.
++
-+     * Otherwise we still need the indirect branch, which we
++    arr = g_array_sized_new(false, false,
-+     * must restore after a potential direct branch write.
++                            sizeof(struct qemu_plugin_dyn_cb), n_cbs);
-+     */
++    g_array_append_vals(arr, cbs[0]->data, cbs[0]->len);
-+    br_diff -= 4;
++    g_array_append_vals(arr, cbs[1]->data, cbs[1]->len);
-+    if (in_range_b(br_diff)) {
++
-+        i2 = B | (br_diff & 0x3fffffc);
++    qemu_plugin_add_dyn_cb_arr(arr);
-+        i3 = NOP;
++
-+    } else {
++    tcg_gen_st_ptr(tcg_constant_ptr((intptr_t)arr), tcg_env,
-+        i2 = MTSPR | RS(TCG_REG_TB) | CTR;
++                   offsetof(CPUState, plugin_mem_cbs) -
-+        i3 = BCCTR | BO_ALWAYS;
++                   offsetof(ArchCPU, env));
-+    }
+ }
-+    ppc64_replace4(jmp_rx, jmp_rw, i0, i1, i2, i3);
- }
+ static void gen_disable_mem_helper(void)
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
  static void tcg_out_call_int(TCGContext *s, int lk,
@@ -XXX,XX +XXX,XX @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
          if (s->tb_jmp_insn_offset) {
              /* Direct jump. */
              if (TCG_TARGET_REG_BITS == 64) {
 -                /* Ensure the next insns are 8-byte aligned. */
 -                if ((uintptr_t)s->code_ptr & 7) {
 +                /* Ensure the next insns are 8 or 16-byte aligned. */
 +                while ((uintptr_t)s->code_ptr & (have_isa_2_07 ? 15 : 7)) {
                      tcg_out32(s, NOP);
                  }
-                 s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s);
+                 break;
 +            case PLUGIN_GEN_FROM_INSN:
 +                assert(insn != NULL);
 +
 +                gen_enable_mem_helper(plugin_tb, insn);
 +
 +                cbs = insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR];
 +                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 +                    struct qemu_plugin_dyn_cb *cb =
 +                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 +                    gen_udata_cb(cb);
 +                }
 +
 +                cbs = insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_INLINE];
 +                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 +                    struct qemu_plugin_dyn_cb *cb =
 +                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 +                    gen_inline_cb(cb);
 +                }
 +                break;
 +
              default:
                  g_assert_not_reached();
              }
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
              enum plugin_gen_cb type = op->args[1];
              switch (from) {
 -            case PLUGIN_GEN_FROM_INSN:
 -            {
 -                g_assert(insn_idx >= 0);
 -
 -                switch (type) {
 -                case PLUGIN_GEN_CB_UDATA:
 -                    plugin_gen_insn_udata(plugin_tb, op, insn_idx);
 -                    break;
 -                case PLUGIN_GEN_CB_UDATA_R:
 -                    plugin_gen_insn_udata_r(plugin_tb, op, insn_idx);
 -                    break;
 -                case PLUGIN_GEN_CB_INLINE:
 -                    plugin_gen_insn_inline(plugin_tb, op, insn_idx);
 -                    break;
 -                case PLUGIN_GEN_ENABLE_MEM_HELPER:
 -                    plugin_gen_enable_mem_helper(plugin_tb, op, insn_idx);
 -                    break;
 -                default:
 -                    g_assert_not_reached();
 -                }
 -                break;
 -            }
              case PLUGIN_GEN_FROM_MEM:
              {
                  g_assert(insn_idx >= 0);
 diff --git a/plugins/api.c b/plugins/api.c
 index XXXXXXX..XXXXXXX 100644
 --- a/plugins/api.c
 +++ b/plugins/api.c
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_insn_exec_cb(struct qemu_plugin_insn *insn,
                                              void *udata)
  {
      if (!insn->mem_only) {
 -        int index = flags == QEMU_PLUGIN_CB_R_REGS ||
 -                    flags == QEMU_PLUGIN_CB_RW_REGS ?
 -                    PLUGIN_CB_REGULAR_R : PLUGIN_CB_REGULAR;
 -
 -        plugin_register_dyn_cb__udata(&insn->cbs[PLUGIN_CB_INSN][index],
 -                                      cb, flags, udata);
 +        plugin_register_dyn_cb__udata(
 +            &insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR], cb, flags, udata);
      }
  }
 --
 .34.1

-[PULL 12/20] accel/tcg: Use DisasContextBase in plugin_gen_tb_start
+[PULL 10/20] plugins: Use emit_before_op for PLUGIN_GEN_FROM_MEM
-Use the pc coming from db->pc_first rather than the TB.
+Introduce a new plugin_mem_cb op to hold the address temp
 and meminfo computed by tcg-op-ldst.c.  Because this now
 has its own opcode, we no longer need PLUGIN_GEN_FROM_MEM.
-Use the cached host_addr rather than re-computing for the
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 first page.  We still need a separate lookup for the second
 page because it won't be computed for DisasContextBase until
 the translator actually performs a read from the page.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/plugin-gen.h |  7 ++++---
+ include/exec/plugin-gen.h   |   4 -
- accel/tcg/plugin-gen.c    | 22 +++++++++++-----------
+ include/tcg/tcg-op-common.h |   1 +
- accel/tcg/translator.c    |  2 +-
+ include/tcg/tcg-opc.h       |   1 +
-files changed, 16 insertions(+), 15 deletions(-)
+ accel/tcg/plugin-gen.c      | 408 ++++--------------------------------
  tcg/tcg-op-ldst.c           |   6 +-
  tcg/tcg-op.c                |   5 +
 files changed, 54 insertions(+), 371 deletions(-)
 diff --git a/include/exec/plugin-gen.h b/include/exec/plugin-gen.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/exec/plugin-gen.h
 +++ b/include/exec/plugin-gen.h
-@@ -XXX,XX +XXX,XX @@ struct DisasContextBase;
+@@ -XXX,XX +XXX,XX @@ void plugin_gen_insn_start(CPUState *cpu, const struct DisasContextBase *db);
  #ifdef CONFIG_PLUGIN
 -bool plugin_gen_tb_start(CPUState *cpu, const TranslationBlock *tb, bool supress);
 +bool plugin_gen_tb_start(CPUState *cpu, const struct DisasContextBase *db,
 +                         bool supress);
  void plugin_gen_tb_end(CPUState *cpu);
  void plugin_gen_insn_start(CPUState *cpu, const struct DisasContextBase *db);
  void plugin_gen_insn_end(void);
-@@ -XXX,XX +XXX,XX @@ static inline void plugin_insn_append(abi_ptr pc, const void *from, size_t size)
  void plugin_gen_disable_mem_helpers(void);
 -void plugin_gen_empty_mem_callback(TCGv_i64 addr, uint32_t info);
  #else /* !CONFIG_PLUGIN */
--static inline
+@@ -XXX,XX +XXX,XX @@ static inline void plugin_gen_tb_end(CPUState *cpu, size_t num_insns)
--bool plugin_gen_tb_start(CPUState *cpu, const TranslationBlock *tb, bool supress)
+ static inline void plugin_gen_disable_mem_helpers(void)
-+static inline bool
+ { }
-+plugin_gen_tb_start(CPUState *cpu, const struct DisasContextBase *db, bool sup)
- {
+-static inline void plugin_gen_empty_mem_callback(TCGv_i64 addr, uint32_t info)
-     return false;
+-{ }
- }
+-
  #endif /* CONFIG_PLUGIN */
  #endif /* QEMU_PLUGIN_GEN_H */
 diff --git a/include/tcg/tcg-op-common.h b/include/tcg/tcg-op-common.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg-op-common.h
 +++ b/include/tcg/tcg-op-common.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_goto_tb(unsigned idx);
  void tcg_gen_lookup_and_goto_ptr(void);
  void tcg_gen_plugin_cb(unsigned from);
 +void tcg_gen_plugin_mem_cb(TCGv_i64 addr, unsigned meminfo);
  void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr);
  void tcg_gen_plugin_cb_end(void);
 diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg-opc.h
 +++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(goto_tb, 0, 0, 1, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
  DEF(goto_ptr, 0, 1, 0, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
  DEF(plugin_cb, 0, 0, 1, TCG_OPF_NOT_PRESENT)
 +DEF(plugin_mem_cb, 0, 1, 1, TCG_OPF_NOT_PRESENT)
  DEF(plugin_cb_start, 0, 0, 3, TCG_OPF_NOT_PRESENT)
  DEF(plugin_cb_end, 0, 0, 0, TCG_OPF_NOT_PRESENT)
 diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/plugin-gen.c
 +++ b/accel/tcg/plugin-gen.c
-@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(const struct qemu_plugin_tb *plugin_tb)
+@@ -XXX,XX +XXX,XX @@
-     pr_ops();
+ enum plugin_gen_from {
- }
+     PLUGIN_GEN_FROM_TB,
+     PLUGIN_GEN_FROM_INSN,
--bool plugin_gen_tb_start(CPUState *cpu, const TranslationBlock *tb, bool mem_only)
+-    PLUGIN_GEN_FROM_MEM,
-+bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
+     PLUGIN_GEN_AFTER_INSN,
-+                         bool mem_only)
+     PLUGIN_GEN_AFTER_TB,
      PLUGIN_GEN_N_FROMS,
@@ -XXX,XX +XXX,XX @@ void HELPER(plugin_vcpu_mem_cb)(unsigned int vcpu_index,
                                  void *userdata)
  { }
 -/*
 - * For now we only support addi_i64.
 - * When we support more ops, we can generate one empty inline cb for each.
 - */
 -static void gen_empty_inline_cb(void)
 -{
 -    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
 -    TCGv_ptr cpu_index_as_ptr = tcg_temp_ebb_new_ptr();
 -    TCGv_i64 val = tcg_temp_ebb_new_i64();
 -    TCGv_ptr ptr = tcg_temp_ebb_new_ptr();
 -
 -    tcg_gen_ld_i32(cpu_index, tcg_env,
 -                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
 -    /* second operand will be replaced by immediate value */
 -    tcg_gen_mul_i32(cpu_index, cpu_index, cpu_index);
 -    tcg_gen_ext_i32_ptr(cpu_index_as_ptr, cpu_index);
 -
 -    tcg_gen_movi_ptr(ptr, 0);
 -    tcg_gen_add_ptr(ptr, ptr, cpu_index_as_ptr);
 -    tcg_gen_ld_i64(val, ptr, 0);
 -    /* second operand will be replaced by immediate value */
 -    tcg_gen_add_i64(val, val, val);
 -
 -    tcg_gen_st_i64(val, ptr, 0);
 -    tcg_temp_free_ptr(ptr);
 -    tcg_temp_free_i64(val);
 -    tcg_temp_free_ptr(cpu_index_as_ptr);
 -    tcg_temp_free_i32(cpu_index);
 -}
 -
 -static void gen_empty_mem_cb(TCGv_i64 addr, uint32_t info)
 -{
 -    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
 -    TCGv_i32 meminfo = tcg_temp_ebb_new_i32();
 -    TCGv_ptr udata = tcg_temp_ebb_new_ptr();
 -
 -    tcg_gen_movi_i32(meminfo, info);
 -    tcg_gen_movi_ptr(udata, 0);
 -    tcg_gen_ld_i32(cpu_index, tcg_env,
 -                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
 -
 -    gen_helper_plugin_vcpu_mem_cb(cpu_index, meminfo, addr, udata);
 -
 -    tcg_temp_free_ptr(udata);
 -    tcg_temp_free_i32(meminfo);
 -    tcg_temp_free_i32(cpu_index);
 -}
 -
 -static void gen_plugin_cb_start(enum plugin_gen_from from,
 -                                enum plugin_gen_cb type, unsigned wr)
 -{
 -    tcg_gen_plugin_cb_start(from, type, wr);
 -}
 -
  static void plugin_gen_empty_callback(enum plugin_gen_from from)
  {
-     bool ret = false;
+     switch (from) {
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_empty_callback(enum plugin_gen_from from)
@@ -XXX,XX +XXX,XX @@ bool plugin_gen_tb_start(CPUState *cpu, const TranslationBlock *tb, bool mem_onl
          ret = true;
 -        ptb->vaddr = tb->pc;
 +        ptb->vaddr = db->pc_first;
          ptb->vaddr2 = -1;
 -        get_page_addr_code_hostp(cpu->env_ptr, tb->pc, &ptb->haddr1);
 +        ptb->haddr1 = db->host_addr[0];
          ptb->haddr2 = NULL;
          ptb->mem_only = mem_only;
@@ -XXX,XX +XXX,XX @@ void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
       * Note that we skip this when haddr1 == NULL, e.g. when we're
       * fetching instructions from a region not backed by RAM.
       */
 -    if (likely(ptb->haddr1 != NULL && ptb->vaddr2 == -1) &&
 -        unlikely((db->pc_next & TARGET_PAGE_MASK) !=
 -                 (db->pc_first & TARGET_PAGE_MASK))) {
 -        get_page_addr_code_hostp(cpu->env_ptr, db->pc_next,
 -                                 &ptb->haddr2);
 -        ptb->vaddr2 = db->pc_next;
 -    }
 -    if (likely(ptb->vaddr2 == -1)) {
 +    if (ptb->haddr1 == NULL) {
 +        pinsn->haddr = NULL;
 +    } else if (is_same_page(db, db->pc_next)) {
          pinsn->haddr = ptb->haddr1 + pinsn->vaddr - ptb->vaddr;
      } else {
 +        if (ptb->vaddr2 == -1) {
 +            ptb->vaddr2 = TARGET_PAGE_ALIGN(db->pc_first);
 +            get_page_addr_code_hostp(cpu->env_ptr, ptb->vaddr2, &ptb->haddr2);
 +        }
          pinsn->haddr = ptb->haddr2 + pinsn->vaddr - ptb->vaddr2;
      }
  }
-diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
 -void plugin_gen_empty_mem_callback(TCGv_i64 addr, uint32_t info)
 -{
 -    enum qemu_plugin_mem_rw rw = get_plugin_meminfo_rw(info);
 -
 -    gen_plugin_cb_start(PLUGIN_GEN_FROM_MEM, PLUGIN_GEN_CB_MEM, rw);
 -    gen_empty_mem_cb(addr, info);
 -    tcg_gen_plugin_cb_end();
 -
 -    gen_plugin_cb_start(PLUGIN_GEN_FROM_MEM, PLUGIN_GEN_CB_INLINE, rw);
 -    gen_empty_inline_cb();
 -    tcg_gen_plugin_cb_end();
 -}
 -
 -static TCGOp *find_op(TCGOp *op, TCGOpcode opc)
 -{
 -    while (op) {
 -        if (op->opc == opc) {
 -            return op;
 -        }
 -        op = QTAILQ_NEXT(op, link);
 -    }
 -    return NULL;
 -}
 -
 -static TCGOp *rm_ops_range(TCGOp *begin, TCGOp *end)
 -{
 -    TCGOp *ret = QTAILQ_NEXT(end, link);
 -
 -    QTAILQ_REMOVE_SEVERAL(&tcg_ctx->ops, begin, end, link);
 -    return ret;
 -}
 -
 -/* remove all ops until (and including) plugin_cb_end */
 -static TCGOp *rm_ops(TCGOp *op)
 -{
 -    TCGOp *end_op = find_op(op, INDEX_op_plugin_cb_end);
 -
 -    tcg_debug_assert(end_op);
 -    return rm_ops_range(op, end_op);
 -}
 -
 -static TCGOp *copy_op_nocheck(TCGOp **begin_op, TCGOp *op)
 -{
 -    TCGOp *old_op = QTAILQ_NEXT(*begin_op, link);
 -    unsigned nargs = old_op->nargs;
 -
 -    *begin_op = old_op;
 -    op = tcg_op_insert_after(tcg_ctx, op, old_op->opc, nargs);
 -    memcpy(op->args, old_op->args, sizeof(op->args[0]) * nargs);
 -
 -    return op;
 -}
 -
 -static TCGOp *copy_op(TCGOp **begin_op, TCGOp *op, TCGOpcode opc)
 -{
 -    op = copy_op_nocheck(begin_op, op);
 -    tcg_debug_assert((*begin_op)->opc == opc);
 -    return op;
 -}
 -
 -static TCGOp *copy_const_ptr(TCGOp **begin_op, TCGOp *op, void *ptr)
 -{
 -    if (UINTPTR_MAX == UINT32_MAX) {
 -        /* mov_i32 */
 -        op = copy_op(begin_op, op, INDEX_op_mov_i32);
 -        op->args[1] = tcgv_i32_arg(tcg_constant_i32((uintptr_t)ptr));
 -    } else {
 -        /* mov_i64 */
 -        op = copy_op(begin_op, op, INDEX_op_mov_i64);
 -        op->args[1] = tcgv_i64_arg(tcg_constant_i64((uintptr_t)ptr));
 -    }
 -    return op;
 -}
 -
 -static TCGOp *copy_ld_i32(TCGOp **begin_op, TCGOp *op)
 -{
 -    return copy_op(begin_op, op, INDEX_op_ld_i32);
 -}
 -
 -static TCGOp *copy_ext_i32_ptr(TCGOp **begin_op, TCGOp *op)
 -{
 -    if (UINTPTR_MAX == UINT32_MAX) {
 -        op = copy_op(begin_op, op, INDEX_op_mov_i32);
 -    } else {
 -        op = copy_op(begin_op, op, INDEX_op_ext_i32_i64);
 -    }
 -    return op;
 -}
 -
 -static TCGOp *copy_add_ptr(TCGOp **begin_op, TCGOp *op)
 -{
 -    if (UINTPTR_MAX == UINT32_MAX) {
 -        op = copy_op(begin_op, op, INDEX_op_add_i32);
 -    } else {
 -        op = copy_op(begin_op, op, INDEX_op_add_i64);
 -    }
 -    return op;
 -}
 -
 -static TCGOp *copy_ld_i64(TCGOp **begin_op, TCGOp *op)
 -{
 -    if (TCG_TARGET_REG_BITS == 32) {
 -        /* 2x ld_i32 */
 -        op = copy_ld_i32(begin_op, op);
 -        op = copy_ld_i32(begin_op, op);
 -    } else {
 -        /* ld_i64 */
 -        op = copy_op(begin_op, op, INDEX_op_ld_i64);
 -    }
 -    return op;
 -}
 -
 -static TCGOp *copy_st_i64(TCGOp **begin_op, TCGOp *op)
 -{
 -    if (TCG_TARGET_REG_BITS == 32) {
 -        /* 2x st_i32 */
 -        op = copy_op(begin_op, op, INDEX_op_st_i32);
 -        op = copy_op(begin_op, op, INDEX_op_st_i32);
 -    } else {
 -        /* st_i64 */
 -        op = copy_op(begin_op, op, INDEX_op_st_i64);
 -    }
 -    return op;
 -}
 -
 -static TCGOp *copy_add_i64(TCGOp **begin_op, TCGOp *op, uint64_t v)
 -{
 -    if (TCG_TARGET_REG_BITS == 32) {
 -        /* all 32-bit backends must implement add2_i32 */
 -        g_assert(TCG_TARGET_HAS_add2_i32);
 -        op = copy_op(begin_op, op, INDEX_op_add2_i32);
 -        op->args[4] = tcgv_i32_arg(tcg_constant_i32(v));
 -        op->args[5] = tcgv_i32_arg(tcg_constant_i32(v >> 32));
 -    } else {
 -        op = copy_op(begin_op, op, INDEX_op_add_i64);
 -        op->args[2] = tcgv_i64_arg(tcg_constant_i64(v));
 -    }
 -    return op;
 -}
 -
 -static TCGOp *copy_mul_i32(TCGOp **begin_op, TCGOp *op, uint32_t v)
 -{
 -    op = copy_op(begin_op, op, INDEX_op_mul_i32);
 -    op->args[2] = tcgv_i32_arg(tcg_constant_i32(v));
 -    return op;
 -}
 -
 -static TCGOp *copy_call(TCGOp **begin_op, TCGOp *op, void *func, int *cb_idx)
 -{
 -    TCGOp *old_op;
 -    int func_idx;
 -
 -    /* copy all ops until the call */
 -    do {
 -        op = copy_op_nocheck(begin_op, op);
 -    } while (op->opc != INDEX_op_call);
 -
 -    /* fill in the op call */
 -    old_op = *begin_op;
 -    TCGOP_CALLI(op) = TCGOP_CALLI(old_op);
 -    TCGOP_CALLO(op) = TCGOP_CALLO(old_op);
 -    tcg_debug_assert(op->life == 0);
 -
 -    func_idx = TCGOP_CALLO(op) + TCGOP_CALLI(op);
 -    *cb_idx = func_idx;
 -    op->args[func_idx] = (uintptr_t)func;
 -
 -    return op;
 -}
 -
 -static TCGOp *append_inline_cb(const struct qemu_plugin_dyn_cb *cb,
 -                               TCGOp *begin_op, TCGOp *op,
 -                               int *unused)
 -{
 -    char *ptr = cb->inline_insn.entry.score->data->data;
 -    size_t elem_size = g_array_get_element_size(
 -        cb->inline_insn.entry.score->data);
 -    size_t offset = cb->inline_insn.entry.offset;
 -
 -    op = copy_ld_i32(&begin_op, op);
 -    op = copy_mul_i32(&begin_op, op, elem_size);
 -    op = copy_ext_i32_ptr(&begin_op, op);
 -    op = copy_const_ptr(&begin_op, op, ptr + offset);
 -    op = copy_add_ptr(&begin_op, op);
 -    op = copy_ld_i64(&begin_op, op);
 -    op = copy_add_i64(&begin_op, op, cb->inline_insn.imm);
 -    op = copy_st_i64(&begin_op, op);
 -    return op;
 -}
 -
 -static TCGOp *append_mem_cb(const struct qemu_plugin_dyn_cb *cb,
 -                            TCGOp *begin_op, TCGOp *op, int *cb_idx)
 -{
 -    enum plugin_gen_cb type = begin_op->args[1];
 -
 -    tcg_debug_assert(type == PLUGIN_GEN_CB_MEM);
 -
 -    /* const_i32 == mov_i32 ("info", so it remains as is) */
 -    op = copy_op(&begin_op, op, INDEX_op_mov_i32);
 -
 -    /* const_ptr */
 -    op = copy_const_ptr(&begin_op, op, cb->userp);
 -
 -    /* copy the ld_i32, but note that we only have to copy it once */
 -    if (*cb_idx == -1) {
 -        op = copy_op(&begin_op, op, INDEX_op_ld_i32);
 -    } else {
 -        begin_op = QTAILQ_NEXT(begin_op, link);
 -        tcg_debug_assert(begin_op && begin_op->opc == INDEX_op_ld_i32);
 -    }
 -
 -    if (type == PLUGIN_GEN_CB_MEM) {
 -        /* call */
 -        op = copy_call(&begin_op, op, cb->regular.f.vcpu_udata, cb_idx);
 -    }
 -
 -    return op;
 -}
 -
 -typedef TCGOp *(*inject_fn)(const struct qemu_plugin_dyn_cb *cb,
 -                            TCGOp *begin_op, TCGOp *op, int *intp);
 -typedef bool (*op_ok_fn)(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb);
 -
 -static bool op_rw(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb)
 -{
 -    int w;
 -
 -    w = op->args[2];
 -    return !!(cb->rw & (w + 1));
 -}
 -
 -static void inject_cb_type(const GArray *cbs, TCGOp *begin_op,
 -                           inject_fn inject, op_ok_fn ok)
 -{
 -    TCGOp *end_op;
 -    TCGOp *op;
 -    int cb_idx = -1;
 -    int i;
 -
 -    if (!cbs || cbs->len == 0) {
 -        rm_ops(begin_op);
 -        return;
 -    }
 -
 -    end_op = find_op(begin_op, INDEX_op_plugin_cb_end);
 -    tcg_debug_assert(end_op);
 -
 -    op = end_op;
 -    for (i = 0; i < cbs->len; i++) {
 -        struct qemu_plugin_dyn_cb *cb =
 -            &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 -
 -        if (!ok(begin_op, cb)) {
 -            continue;
 -        }
 -        op = inject(cb, begin_op, op, &cb_idx);
 -    }
 -    rm_ops_range(begin_op, end_op);
 -}
 -
 -static void
 -inject_inline_cb(const GArray *cbs, TCGOp *begin_op, op_ok_fn ok)
 -{
 -    inject_cb_type(cbs, begin_op, append_inline_cb, ok);
 -}
 -
 -static void
 -inject_mem_cb(const GArray *cbs, TCGOp *begin_op)
 -{
 -    inject_cb_type(cbs, begin_op, append_mem_cb, op_rw);
 -}
 -
  /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
  void plugin_gen_disable_mem_helpers(void)
  {
@@ -XXX,XX +XXX,XX @@ void plugin_gen_disable_mem_helpers(void)
      }
  }
 -static void plugin_gen_mem_regular(const struct qemu_plugin_tb *ptb,
 -                                   TCGOp *begin_op, int insn_idx)
 -{
 -    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
 -    inject_mem_cb(insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR], begin_op);
 -}
 -
 -static void plugin_gen_mem_inline(const struct qemu_plugin_tb *ptb,
 -                                  TCGOp *begin_op, int insn_idx)
 -{
 -    const GArray *cbs;
 -    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
 -
 -    cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
 -    inject_inline_cb(cbs, begin_op, op_rw);
 -}
 -
  static void gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
                                    struct qemu_plugin_insn *insn)
  {
@@ -XXX,XX +XXX,XX @@ static void gen_inline_cb(struct qemu_plugin_dyn_cb *cb)
      tcg_temp_free_ptr(ptr);
  }
 +static void gen_mem_cb(struct qemu_plugin_dyn_cb *cb,
 +                       qemu_plugin_meminfo_t meminfo, TCGv_i64 addr)
 +{
 +    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
 +
 +    tcg_gen_ld_i32(cpu_index, tcg_env,
 +                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
 +    tcg_gen_call4(cb->regular.f.vcpu_mem, cb->regular.info, NULL,
 +                  tcgv_i32_temp(cpu_index),
 +                  tcgv_i32_temp(tcg_constant_i32(meminfo)),
 +                  tcgv_i64_temp(addr),
 +                  tcgv_ptr_temp(tcg_constant_ptr(cb->userp)));
 +    tcg_temp_free_i32(cpu_index);
 +}
 +
  /* #define DEBUG_PLUGIN_GEN_OPS */
  static void pr_ops(void)
  {
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
              break;
          }
 -        case INDEX_op_plugin_cb_start:
 +        case INDEX_op_plugin_mem_cb:
          {
 -            enum plugin_gen_from from = op->args[0];
 -            enum plugin_gen_cb type = op->args[1];
 +            TCGv_i64 addr = temp_tcgv_i64(arg_temp(op->args[0]));
 +            qemu_plugin_meminfo_t meminfo = op->args[1];
 +            struct qemu_plugin_insn *insn;
 +            const GArray *cbs;
 +            int i, n, rw;
 -            switch (from) {
 -            case PLUGIN_GEN_FROM_MEM:
 -            {
 -                g_assert(insn_idx >= 0);
 +            assert(insn_idx >= 0);
 +            insn = g_ptr_array_index(plugin_tb->insns, insn_idx);
 +            rw = qemu_plugin_mem_is_store(meminfo) ? 2 : 1;
 -                switch (type) {
 -                case PLUGIN_GEN_CB_MEM:
 -                    plugin_gen_mem_regular(plugin_tb, op, insn_idx);
 -                    break;
 -                case PLUGIN_GEN_CB_INLINE:
 -                    plugin_gen_mem_inline(plugin_tb, op, insn_idx);
 -                    break;
 -                default:
 -                    g_assert_not_reached();
 +            tcg_ctx->emit_before_op = op;
 +
 +            cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
 +            for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 +                struct qemu_plugin_dyn_cb *cb =
 +                    &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 +                if (cb->rw & rw) {
 +                    gen_mem_cb(cb, meminfo, addr);
                  }
 +            }
 -                break;
 -            }
 -            default:
 -                g_assert_not_reached();
 +            cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
 +            for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 +                struct qemu_plugin_dyn_cb *cb =
 +                    &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 +                if (cb->rw & rw) {
 +                    gen_inline_cb(cb);
 +                }
              }
 +
 +            tcg_ctx->emit_before_op = NULL;
 +            tcg_op_remove(tcg_ctx, op);
              break;
          }
 +
          default:
              /* plugins don't care about any other ops */
              break;
 diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/translator.c
+--- a/tcg/tcg-op-ldst.c
-+++ b/accel/tcg/translator.c
++++ b/tcg/tcg-op-ldst.c
-@@ -XXX,XX +XXX,XX @@ void translator_loop(CPUState *cpu, TranslationBlock *tb, int max_insns,
+@@ -XXX,XX +XXX,XX @@ plugin_gen_mem_callbacks(TCGv_i64 copy_addr, TCGTemp *orig_addr, MemOpIdx oi,
-     ops->tb_start(db, cpu);
+                 copy_addr = tcg_temp_ebb_new_i64();
-     tcg_debug_assert(db->is_jmp == DISAS_NEXT);  /* no early exit */
+                 tcg_gen_extu_i32_i64(copy_addr, temp_tcgv_i32(orig_addr));
+             }
--    plugin_enabled = plugin_gen_tb_start(cpu, tb, cflags & CF_MEMI_ONLY);
+-            plugin_gen_empty_mem_callback(copy_addr, info);
-+    plugin_enabled = plugin_gen_tb_start(cpu, db, cflags & CF_MEMI_ONLY);
++            tcg_gen_plugin_mem_cb(copy_addr, info);
+             tcg_temp_free_i64(copy_addr);
-     while (true) {
+         } else {
-         db->num_insns++;
+             if (copy_addr) {
 -                plugin_gen_empty_mem_callback(copy_addr, info);
 +                tcg_gen_plugin_mem_cb(copy_addr, info);
                  tcg_temp_free_i64(copy_addr);
              } else {
 -                plugin_gen_empty_mem_callback(temp_tcgv_i64(orig_addr), info);
 +                tcg_gen_plugin_mem_cb(temp_tcgv_i64(orig_addr), info);
              }
          }
      }
 diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg-op.c
 +++ b/tcg/tcg-op.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_plugin_cb(unsigned from)
      tcg_gen_op1(INDEX_op_plugin_cb, from);
  }
 +void tcg_gen_plugin_mem_cb(TCGv_i64 addr, unsigned meminfo)
 +{
 +    tcg_gen_op2(INDEX_op_plugin_mem_cb, tcgv_i64_arg(addr), meminfo);
 +}
 +
  void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr)
  {
      tcg_gen_op3(INDEX_op_plugin_cb_start, from, type, wr);
 --
 .34.1

-[PULL 08/20] accel/tcg: Introduce tlb_set_page_full
+[PULL 11/20] plugins: Remove plugin helpers
-Now that we have collected all of the page data into
+These placeholder helpers are no longer required.
 CPUTLBEntryFull, provide an interface to record that
 all in one go, instead of using 4 arguments.  This interface
 allows CPUTLBEntryFull to be extended without having to
 change the number of arguments.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
-Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/cpu-defs.h | 14 +++++++++++
+ accel/tcg/plugin-helpers.h         |  5 -----
- include/exec/exec-all.h | 22 ++++++++++++++++++
+ include/exec/helper-gen-common.h   |  4 ----
- accel/tcg/cputlb.c      | 51 ++++++++++++++++++++++++++---------------
+ include/exec/helper-proto-common.h |  4 ----
-files changed, 69 insertions(+), 18 deletions(-)
+ accel/tcg/plugin-gen.c             | 20 --------------------
 files changed, 33 deletions(-)
  delete mode 100644 accel/tcg/plugin-helpers.h
-diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
+diff --git a/accel/tcg/plugin-helpers.h b/accel/tcg/plugin-helpers.h
 deleted file mode 100644
 index XXXXXXX..XXXXXXX
 --- a/accel/tcg/plugin-helpers.h
 +++ /dev/null
@@ -XXX,XX +XXX,XX @@
 -#ifdef CONFIG_PLUGIN
 -DEF_HELPER_FLAGS_2(plugin_vcpu_udata_cb_no_wg, TCG_CALL_NO_WG | TCG_CALL_PLUGIN, void, i32, ptr)
 -DEF_HELPER_FLAGS_2(plugin_vcpu_udata_cb_no_rwg, TCG_CALL_NO_RWG | TCG_CALL_PLUGIN, void, i32, ptr)
 -DEF_HELPER_FLAGS_4(plugin_vcpu_mem_cb, TCG_CALL_NO_RWG | TCG_CALL_PLUGIN, void, i32, i32, i64, ptr)
 -#endif
 diff --git a/include/exec/helper-gen-common.h b/include/exec/helper-gen-common.h
 index XXXXXXX..XXXXXXX 100644
---- a/include/exec/cpu-defs.h
+--- a/include/exec/helper-gen-common.h
-+++ b/include/exec/cpu-defs.h
++++ b/include/exec/helper-gen-common.h
-@@ -XXX,XX +XXX,XX @@ typedef struct CPUTLBEntryFull {
+@@ -XXX,XX +XXX,XX @@
-      *     + the offset within the target MemoryRegion (otherwise)
+ #include "exec/helper-gen.h.inc"
-      */
+ #undef  HELPER_H
-     hwaddr xlat_section;
-+
+-#define HELPER_H "accel/tcg/plugin-helpers.h"
-+    /*
+-#include "exec/helper-gen.h.inc"
-+     * @phys_addr contains the physical address in the address space
+-#undef  HELPER_H
-+     * given by cpu_asidx_from_attrs(cpu, @attrs).
+-
-+     */
+ #endif /* HELPER_GEN_COMMON_H */
-+    hwaddr phys_addr;
+diff --git a/include/exec/helper-proto-common.h b/include/exec/helper-proto-common.h
-+
+index XXXXXXX..XXXXXXX 100644
-+    /* @attrs contains the memory transaction attributes for the page. */
+--- a/include/exec/helper-proto-common.h
-     MemTxAttrs attrs;
++++ b/include/exec/helper-proto-common.h
-+
+@@ -XXX,XX +XXX,XX @@
-+    /* @prot contains the complete protections for the page. */
+ #include "exec/helper-proto.h.inc"
-+    uint8_t prot;
+ #undef  HELPER_H
-+
-+    /* @lg_page_size contains the log2 of the page size. */
+-#define HELPER_H "accel/tcg/plugin-helpers.h"
-+    uint8_t lg_page_size;
+-#include "exec/helper-proto.h.inc"
- } CPUTLBEntryFull;
+-#undef  HELPER_H
 -
  #endif /* HELPER_PROTO_COMMON_H */
 diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/plugin-gen.c
 +++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@
  #include "exec/exec-all.h"
  #include "exec/plugin-gen.h"
  #include "exec/translator.h"
 -#include "exec/helper-proto-common.h"
 -
 -#define HELPER_H  "accel/tcg/plugin-helpers.h"
 -#include "exec/helper-info.c.inc"
 -#undef  HELPER_H
  /*
-diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
+  * plugin_cb_start TCG op args[]:
-index XXXXXXX..XXXXXXX 100644
+@@ -XXX,XX +XXX,XX @@ enum plugin_gen_cb {
---- a/include/exec/exec-all.h
+     PLUGIN_GEN_N_CBS,
-+++ b/include/exec/exec-all.h
+ };
-@@ -XXX,XX +XXX,XX @@ void tlb_flush_range_by_mmuidx_all_cpus_synced(CPUState *cpu,
-                                                uint16_t idxmap,
+-/*
-                                                unsigned bits);
+- * These helpers are stubs that get dynamically switched out for calls
+- * direct to the plugin if they are subscribed to.
-+/**
+- */
-+ * tlb_set_page_full:
+-void HELPER(plugin_vcpu_udata_cb_no_wg)(uint32_t cpu_index, void *udata)
-+ * @cpu: CPU context
+-{ }
-+ * @mmu_idx: mmu index of the tlb to modify
+-
-+ * @vaddr: virtual address of the entry to add
+-void HELPER(plugin_vcpu_udata_cb_no_rwg)(uint32_t cpu_index, void *udata)
-+ * @full: the details of the tlb entry
+-{ }
-+ *
+-
-+ * Add an entry to @cpu tlb index @mmu_idx.  All of the fields of
+-void HELPER(plugin_vcpu_mem_cb)(unsigned int vcpu_index,
-+ * @full must be filled, except for xlat_section, and constitute
+-                                qemu_plugin_meminfo_t info, uint64_t vaddr,
-+ * the complete description of the translated page.
+-                                void *userdata)
-+ *
+-{ }
-+ * This is generally called by the target tlb_fill function after
+-
-+ * having performed a successful page table walk to find the physical
+ static void plugin_gen_empty_callback(enum plugin_gen_from from)
 + * address and attributes for the translation.
 + *
 + * At most one entry for a given virtual address is permitted. Only a
 + * single TARGET_PAGE_SIZE region is mapped; @full->lg_page_size is only
 + * used by tlb_flush_page.
 + */
 +void tlb_set_page_full(CPUState *cpu, int mmu_idx, target_ulong vaddr,
 +                       CPUTLBEntryFull *full);
 +
  /**
   * tlb_set_page_with_attrs:
   * @cpu: CPU to add this TLB entry for
 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cputlb.c
 +++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tlb_add_large_page(CPUArchState *env, int mmu_idx,
      env_tlb(env)->d[mmu_idx].large_page_mask = lp_mask;
  }
 -/* Add a new TLB entry. At most one entry for a given virtual address
 +/*
 + * Add a new TLB entry. At most one entry for a given virtual address
   * is permitted. Only a single TARGET_PAGE_SIZE region is mapped, the
   * supplied size is only used by tlb_flush_page.
   *
   * Called from TCG-generated code, which is under an RCU read-side
   * critical section.
   */
 -void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
 -                             hwaddr paddr, MemTxAttrs attrs, int prot,
 -                             int mmu_idx, target_ulong size)
 +void tlb_set_page_full(CPUState *cpu, int mmu_idx,
 +                       target_ulong vaddr, CPUTLBEntryFull *full)
  {
-     CPUArchState *env = cpu->env_ptr;
+     switch (from) {
      CPUTLB *tlb = env_tlb(env);
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
      CPUTLBEntry *te, tn;
      hwaddr iotlb, xlat, sz, paddr_page;
      target_ulong vaddr_page;
 -    int asidx = cpu_asidx_from_attrs(cpu, attrs);
 -    int wp_flags;
 +    int asidx, wp_flags, prot;
      bool is_ram, is_romd;
      assert_cpu_is_self(cpu);
 -    if (size <= TARGET_PAGE_SIZE) {
 +    if (full->lg_page_size <= TARGET_PAGE_BITS) {
          sz = TARGET_PAGE_SIZE;
      } else {
 -        tlb_add_large_page(env, mmu_idx, vaddr, size);
 -        sz = size;
 +        sz = (hwaddr)1 << full->lg_page_size;
 +        tlb_add_large_page(env, mmu_idx, vaddr, sz);
      }
      vaddr_page = vaddr & TARGET_PAGE_MASK;
 -    paddr_page = paddr & TARGET_PAGE_MASK;
 +    paddr_page = full->phys_addr & TARGET_PAGE_MASK;
 +    prot = full->prot;
 +    asidx = cpu_asidx_from_attrs(cpu, full->attrs);
      section = address_space_translate_for_iotlb(cpu, asidx, paddr_page,
 -                                                &xlat, &sz, attrs, &prot);
 +                                                &xlat, &sz, full->attrs, &prot);
      assert(sz >= TARGET_PAGE_SIZE);
      tlb_debug("vaddr=" TARGET_FMT_lx " paddr=0x" TARGET_FMT_plx
                " prot=%x idx=%d\n",
 -              vaddr, paddr, prot, mmu_idx);
 +              vaddr, full->phys_addr, prot, mmu_idx);
      address = vaddr_page;
 -    if (size < TARGET_PAGE_SIZE) {
 +    if (full->lg_page_size < TARGET_PAGE_BITS) {
          /* Repeat the MMU check and TLB fill on every access.  */
          address |= TLB_INVALID_MASK;
      }
 -    if (attrs.byte_swap) {
 +    if (full->attrs.byte_swap) {
          address |= TLB_BSWAP;
      }
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
       * subtract here is that of the page base, and not the same as the
       * vaddr we add back in io_readx()/io_writex()/get_page_addr_code().
       */
 +    desc->fulltlb[index] = *full;
      desc->fulltlb[index].xlat_section = iotlb - vaddr_page;
 -    desc->fulltlb[index].attrs = attrs;
 +    desc->fulltlb[index].phys_addr = paddr_page;
 +    desc->fulltlb[index].prot = prot;
      /* Now calculate the new entry */
      tn.addend = addend - vaddr_page;
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
      qemu_spin_unlock(&tlb->c.lock);
  }
 -/* Add a new TLB entry, but without specifying the memory
 - * transaction attributes to be used.
 - */
 +void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
 +                             hwaddr paddr, MemTxAttrs attrs, int prot,
 +                             int mmu_idx, target_ulong size)
 +{
 +    CPUTLBEntryFull full = {
 +        .phys_addr = paddr,
 +        .attrs = attrs,
 +        .prot = prot,
 +        .lg_page_size = ctz64(size)
 +    };
 +
 +    assert(is_power_of_2(size));
 +    tlb_set_page_full(cpu, mmu_idx, vaddr, &full);
 +}
 +
  void tlb_set_page(CPUState *cpu, target_ulong vaddr,
                    hwaddr paddr, int prot,
                    int mmu_idx, target_ulong size)
 --
 .34.1

-[PULL 05/20] accel/tcg: Drop addr member from SavedIOTLB
+[PULL 12/20] tcg: Remove TCG_CALL_PLUGIN
-This field is only written, not read; remove it.
+Since we no longer emit plugin helpers during the initial code
 translation phase, we don't need to specially mark plugin helpers.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/hw/core/cpu.h | 1 -
+ include/tcg/tcg.h |  2 --
- accel/tcg/cputlb.c    | 7 +++----
+ plugins/core.c    | 10 ++++------
-files changed, 3 insertions(+), 5 deletions(-)
+ tcg/tcg.c         |  4 +---
 files changed, 5 insertions(+), 11 deletions(-)
-diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
+diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
 index XXXXXXX..XXXXXXX 100644
---- a/include/hw/core/cpu.h
+--- a/include/tcg/tcg.h
-+++ b/include/hw/core/cpu.h
++++ b/include/tcg/tcg.h
-@@ -XXX,XX +XXX,XX @@ struct CPUWatchpoint {
+@@ -XXX,XX +XXX,XX @@ typedef TCGv_ptr TCGv_env;
-  * the memory regions get moved around  by io_writex.
+ #define TCG_CALL_NO_SIDE_EFFECTS    0x0004
-  */
+ /* Helper is G_NORETURN.  */
- typedef struct SavedIOTLB {
+ #define TCG_CALL_NO_RETURN          0x0008
--    hwaddr addr;
+-/* Helper is part of Plugins.  */
-     MemoryRegionSection *section;
+-#define TCG_CALL_PLUGIN             0x0010
-     hwaddr mr_offset;
- } SavedIOTLB;
+ /* convenience version of most used call flags */
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
+ #define TCG_CALL_NO_RWG         TCG_CALL_NO_READ_GLOBALS
 diff --git a/plugins/core.c b/plugins/core.c
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
+--- a/plugins/core.c
-+++ b/accel/tcg/cputlb.c
++++ b/plugins/core.c
-@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full,
+@@ -XXX,XX +XXX,XX @@ void plugin_register_dyn_cb__udata(GArray **arr,
-  * This is read by tlb_plugin_lookup if the fulltlb entry doesn't match
+                                    void *udata)
   * because of the side effect of io_writex changing memory layout.
   */
 -static void save_iotlb_data(CPUState *cs, hwaddr addr,
 -                            MemoryRegionSection *section, hwaddr mr_offset)
 +static void save_iotlb_data(CPUState *cs, MemoryRegionSection *section,
 +                            hwaddr mr_offset)
  {
+     static TCGHelperInfo info[3] = {
+-        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG | TCG_CALL_PLUGIN,
+-        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG | TCG_CALL_PLUGIN,
+-        [QEMU_PLUGIN_CB_RW_REGS].flags = TCG_CALL_PLUGIN,
++        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG,
++        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG,
+         /*
+          * Match qemu_plugin_vcpu_udata_cb_t:
+          *   void (*)(uint32_t, void *)
+@@ -XXX,XX +XXX,XX @@ void plugin_register_vcpu_mem_cb(GArray **arr,
+         !__builtin_types_compatible_p(qemu_plugin_meminfo_t, int32_t));
+     static TCGHelperInfo info[3] = {
+-        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG | TCG_CALL_PLUGIN,
+-        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG | TCG_CALL_PLUGIN,
+-        [QEMU_PLUGIN_CB_RW_REGS].flags = TCG_CALL_PLUGIN,
++        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG,
++        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG,
+         /*
+          * Match qemu_plugin_vcpu_mem_cb_t:
+          *   void (*)(uint32_t, qemu_plugin_meminfo_t, uint64_t, void *)
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@ static void tcg_gen_callN(void *func, TCGHelperInfo *info,
  #ifdef CONFIG_PLUGIN
-     SavedIOTLB *saved = &cs->saved_iotlb;
+     /* Flag helpers that may affect guest state */
--    saved->addr = addr;
+-    if (tcg_ctx->plugin_insn &&
-     saved->section = section;
+-        !(info->flags & TCG_CALL_PLUGIN) &&
-     saved->mr_offset = mr_offset;
+-        !(info->flags & TCG_CALL_NO_SIDE_EFFECTS)) {
 +    if (tcg_ctx->plugin_insn && !(info->flags & TCG_CALL_NO_SIDE_EFFECTS)) {
          tcg_ctx->plugin_insn->calls_helpers = true;
      }
  #endif
-@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full,
-      * The memory_region_dispatch may trigger a flush/resize
-      * so for plugins we save the iotlb_data just in case.
-      */
--    save_iotlb_data(cpu, full->xlat_section, section, mr_offset);
-+    save_iotlb_data(cpu, section, mr_offset);
-     if (!qemu_mutex_iothread_locked()) {
-         qemu_mutex_lock_iothread();
 --
 .34.1

-[PULL 20/20] target/sh4: Fix TB_FLAG_UNALIGN
+[PULL 13/20] tcg: Remove INDEX_op_plugin_cb_{start,end}
-The value previously chosen overlaps GUSA_MASK.
+These opcodes are no longer used.
-Rename all DELAY_SLOT_* and GUSA_* defines to emphasize
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 that they are included in TB_FLAGs.  Add aliases for the
 FPSCR and SR bits that are included in TB_FLAGS, so that
 we don't accidentally reassign those bits.
 Fixes: 4da06fb3062 ("target/sh4: Implement prctl_unalign_sigbus")
 Resolves: https://gitlab.com/qemu-project/qemu/-/issues/856
 Reviewed-by: Yoshinori Sato <ysato@users.sourceforge.jp>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- target/sh4/cpu.h        | 56 +++++++++++++------------
+ include/tcg/tcg-op-common.h |  2 --
- linux-user/sh4/signal.c |  6 +--
+ include/tcg/tcg-opc.h       |  2 --
- target/sh4/cpu.c        |  6 +--
+ accel/tcg/plugin-gen.c      | 18 ------------------
- target/sh4/helper.c     |  6 +--
+ tcg/tcg-op.c                | 10 ----------
- target/sh4/translate.c  | 90 ++++++++++++++++++++++-------------------
+files changed, 32 deletions(-)
 files changed, 88 insertions(+), 76 deletions(-)
-diff --git a/target/sh4/cpu.h b/target/sh4/cpu.h
+diff --git a/include/tcg/tcg-op-common.h b/include/tcg/tcg-op-common.h
 index XXXXXXX..XXXXXXX 100644
---- a/target/sh4/cpu.h
+--- a/include/tcg/tcg-op-common.h
-+++ b/target/sh4/cpu.h
++++ b/include/tcg/tcg-op-common.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_lookup_and_goto_ptr(void);
  void tcg_gen_plugin_cb(unsigned from);
  void tcg_gen_plugin_mem_cb(TCGv_i64 addr, unsigned meminfo);
 -void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr);
 -void tcg_gen_plugin_cb_end(void);
  /* 32 bit ops */
 diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg-opc.h
 +++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(goto_ptr, 0, 1, 0, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
  DEF(plugin_cb, 0, 0, 1, TCG_OPF_NOT_PRESENT)
  DEF(plugin_mem_cb, 0, 1, 1, TCG_OPF_NOT_PRESENT)
 -DEF(plugin_cb_start, 0, 0, 3, TCG_OPF_NOT_PRESENT)
 -DEF(plugin_cb_end, 0, 0, 0, TCG_OPF_NOT_PRESENT)
  /* Replicate ld/st ops for 32 and 64-bit guest addresses. */
  DEF(qemu_ld_a32_i32, 1, 1, 1,
 diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/plugin-gen.c
 +++ b/accel/tcg/plugin-gen.c
 @@ -XXX,XX +XXX,XX @@
- #define FPSCR_RM_NEAREST       (0 << 0)
+ #include "exec/plugin-gen.h"
- #define FPSCR_RM_ZERO          (1 << 0)
+ #include "exec/translator.h"
--#define DELAY_SLOT_MASK        0x7
+-/*
--#define DELAY_SLOT             (1 << 0)
+- * plugin_cb_start TCG op args[]:
--#define DELAY_SLOT_CONDITIONAL (1 << 1)
+- * 0: enum plugin_gen_from
--#define DELAY_SLOT_RTE         (1 << 2)
+- * 1: enum plugin_gen_cb
-+#define TB_FLAG_DELAY_SLOT       (1 << 0)
+- * 2: set to 1 for mem callback that is a write, 0 otherwise.
-+#define TB_FLAG_DELAY_SLOT_COND  (1 << 1)
+- */
 +#define TB_FLAG_DELAY_SLOT_RTE   (1 << 2)
 +#define TB_FLAG_PENDING_MOVCA    (1 << 3)
 +#define TB_FLAG_GUSA_SHIFT       4                      /* [11:4] */
 +#define TB_FLAG_GUSA_EXCLUSIVE   (1 << 12)
 +#define TB_FLAG_UNALIGN          (1 << 13)
 +#define TB_FLAG_SR_FD            (1 << SR_FD)           /* 15 */
 +#define TB_FLAG_FPSCR_PR         FPSCR_PR               /* 19 */
 +#define TB_FLAG_FPSCR_SZ         FPSCR_SZ               /* 20 */
 +#define TB_FLAG_FPSCR_FR         FPSCR_FR               /* 21 */
 +#define TB_FLAG_SR_RB            (1 << SR_RB)           /* 29 */
 +#define TB_FLAG_SR_MD            (1 << SR_MD)           /* 30 */
 -#define TB_FLAG_PENDING_MOVCA  (1 << 3)
 -#define TB_FLAG_UNALIGN        (1 << 4)
 -
--#define GUSA_SHIFT             4
+ enum plugin_gen_from {
--#ifdef CONFIG_USER_ONLY
+     PLUGIN_GEN_FROM_TB,
--#define GUSA_EXCLUSIVE         (1 << 12)
+     PLUGIN_GEN_FROM_INSN,
--#define GUSA_MASK              ((0xff << GUSA_SHIFT) | GUSA_EXCLUSIVE)
+     PLUGIN_GEN_AFTER_INSN,
--#else
+     PLUGIN_GEN_AFTER_TB,
--/* Provide dummy versions of the above to allow tests against tbflags
+-    PLUGIN_GEN_N_FROMS,
--   to be elided while avoiding ifdefs.  */
+-};
 -#define GUSA_EXCLUSIVE         0
 -#define GUSA_MASK              0
 -#endif
 -
--#define TB_FLAG_ENVFLAGS_MASK  (DELAY_SLOT_MASK | GUSA_MASK)
+-enum plugin_gen_cb {
-+#define TB_FLAG_DELAY_SLOT_MASK  (TB_FLAG_DELAY_SLOT |       \
+-    PLUGIN_GEN_CB_UDATA,
-+                                  TB_FLAG_DELAY_SLOT_COND |  \
+-    PLUGIN_GEN_CB_UDATA_R,
-+                                  TB_FLAG_DELAY_SLOT_RTE)
+-    PLUGIN_GEN_CB_INLINE,
-+#define TB_FLAG_GUSA_MASK        ((0xff << TB_FLAG_GUSA_SHIFT) | \
+-    PLUGIN_GEN_CB_MEM,
-+                                  TB_FLAG_GUSA_EXCLUSIVE)
+-    PLUGIN_GEN_ENABLE_MEM_HELPER,
-+#define TB_FLAG_FPSCR_MASK       (TB_FLAG_FPSCR_PR | \
+-    PLUGIN_GEN_DISABLE_MEM_HELPER,
-+                                  TB_FLAG_FPSCR_SZ | \
+-    PLUGIN_GEN_N_CBS,
-+                                  TB_FLAG_FPSCR_FR)
+ };
-+#define TB_FLAG_SR_MASK          (TB_FLAG_SR_FD | \
-+                                  TB_FLAG_SR_RB | \
+ static void plugin_gen_empty_callback(enum plugin_gen_from from)
-+                                  TB_FLAG_SR_MD)
+diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
 +#define TB_FLAG_ENVFLAGS_MASK    (TB_FLAG_DELAY_SLOT_MASK | \
 +                                  TB_FLAG_GUSA_MASK)
  typedef struct tlb_t {
      uint32_t vpn;        /* virtual page number */
@@ -XXX,XX +XXX,XX @@ static inline int cpu_mmu_index (CPUSH4State *env, bool ifetch)
  {
      /* The instruction in a RTE delay slot is fetched in privileged
         mode, but executed in user mode.  */
 -    if (ifetch && (env->flags & DELAY_SLOT_RTE)) {
 +    if (ifetch && (env->flags & TB_FLAG_DELAY_SLOT_RTE)) {
          return 0;
      } else {
          return (env->sr & (1u << SR_MD)) == 0 ? 1 : 0;
@@ -XXX,XX +XXX,XX @@ static inline void cpu_get_tb_cpu_state(CPUSH4State *env, target_ulong *pc,
  {
      *pc = env->pc;
      /* For a gUSA region, notice the end of the region.  */
 -    *cs_base = env->flags & GUSA_MASK ? env->gregs[0] : 0;
 -    *flags = env->flags /* TB_FLAG_ENVFLAGS_MASK: bits 0-2, 4-12 */
 -            | (env->fpscr & (FPSCR_FR | FPSCR_SZ | FPSCR_PR))  /* Bits 19-21 */
 -            | (env->sr & ((1u << SR_MD) | (1u << SR_RB)))      /* Bits 29-30 */
 -            | (env->sr & (1u << SR_FD))                        /* Bit 15 */
 +    *cs_base = env->flags & TB_FLAG_GUSA_MASK ? env->gregs[0] : 0;
 +    *flags = env->flags
 +            | (env->fpscr & TB_FLAG_FPSCR_MASK)
 +            | (env->sr & TB_FLAG_SR_MASK)
              | (env->movcal_backup ? TB_FLAG_PENDING_MOVCA : 0); /* Bit 3 */
  #ifdef CONFIG_USER_ONLY
      *flags |= TB_FLAG_UNALIGN * !env_cpu(env)->prctl_unalign_sigbus;
 diff --git a/linux-user/sh4/signal.c b/linux-user/sh4/signal.c
 index XXXXXXX..XXXXXXX 100644
---- a/linux-user/sh4/signal.c
+--- a/tcg/tcg-op.c
-+++ b/linux-user/sh4/signal.c
++++ b/tcg/tcg-op.c
-@@ -XXX,XX +XXX,XX @@ static void restore_sigcontext(CPUSH4State *regs, struct target_sigcontext *sc)
+@@ -XXX,XX +XXX,XX @@ void tcg_gen_plugin_mem_cb(TCGv_i64 addr, unsigned meminfo)
-     __get_user(regs->fpul, &sc->sc_fpul);
+     tcg_gen_op2(INDEX_op_plugin_mem_cb, tcgv_i64_arg(addr), meminfo);
      regs->tra = -1;         /* disable syscall checks */
 -    regs->flags &= ~(DELAY_SLOT_MASK | GUSA_MASK);
 +    regs->flags = 0;
  }
- void setup_frame(int sig, struct target_sigaction *ka,
+-void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr)
-@@ -XXX,XX +XXX,XX @@ void setup_frame(int sig, struct target_sigaction *ka,
+-{
-     regs->gregs[5] = 0;
+-    tcg_gen_op3(INDEX_op_plugin_cb_start, from, type, wr);
-     regs->gregs[6] = frame_addr += offsetof(typeof(*frame), sc);
+-}
-     regs->pc = (unsigned long) ka->_sa_handler;
+-
--    regs->flags &= ~(DELAY_SLOT_MASK | GUSA_MASK);
+-void tcg_gen_plugin_cb_end(void)
-+    regs->flags &= ~(TB_FLAG_DELAY_SLOT_MASK | TB_FLAG_GUSA_MASK);
+-{
+-    tcg_emit_op(INDEX_op_plugin_cb_end, 0);
-     unlock_user_struct(frame, frame_addr, 1);
+-}
-     return;
+-
-@@ -XXX,XX +XXX,XX @@ void setup_rt_frame(int sig, struct target_sigaction *ka,
+ /* 32 bit ops */
-     regs->gregs[5] = frame_addr + offsetof(typeof(*frame), info);
-     regs->gregs[6] = frame_addr + offsetof(typeof(*frame), uc);
+ void tcg_gen_discard_i32(TCGv_i32 arg)
      regs->pc = (unsigned long) ka->_sa_handler;
 -    regs->flags &= ~(DELAY_SLOT_MASK | GUSA_MASK);
 +    regs->flags &= ~(TB_FLAG_DELAY_SLOT_MASK | TB_FLAG_GUSA_MASK);
      unlock_user_struct(frame, frame_addr, 1);
      return;
 diff --git a/target/sh4/cpu.c b/target/sh4/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/sh4/cpu.c
 +++ b/target/sh4/cpu.c
@@ -XXX,XX +XXX,XX @@ static void superh_cpu_synchronize_from_tb(CPUState *cs,
      SuperHCPU *cpu = SUPERH_CPU(cs);
      cpu->env.pc = tb_pc(tb);
 -    cpu->env.flags = tb->flags & TB_FLAG_ENVFLAGS_MASK;
 +    cpu->env.flags = tb->flags;
  }
  #ifndef CONFIG_USER_ONLY
@@ -XXX,XX +XXX,XX @@ static bool superh_io_recompile_replay_branch(CPUState *cs,
      SuperHCPU *cpu = SUPERH_CPU(cs);
      CPUSH4State *env = &cpu->env;
 -    if ((env->flags & ((DELAY_SLOT | DELAY_SLOT_CONDITIONAL))) != 0
 +    if ((env->flags & (TB_FLAG_DELAY_SLOT | TB_FLAG_DELAY_SLOT_COND))
          && env->pc != tb_pc(tb)) {
          env->pc -= 2;
 -        env->flags &= ~(DELAY_SLOT | DELAY_SLOT_CONDITIONAL);
 +        env->flags &= ~(TB_FLAG_DELAY_SLOT | TB_FLAG_DELAY_SLOT_COND);
          return true;
      }
      return false;
 diff --git a/target/sh4/helper.c b/target/sh4/helper.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/sh4/helper.c
 +++ b/target/sh4/helper.c
@@ -XXX,XX +XXX,XX @@ void superh_cpu_do_interrupt(CPUState *cs)
      env->sr |= (1u << SR_BL) | (1u << SR_MD) | (1u << SR_RB);
      env->lock_addr = -1;
 -    if (env->flags & DELAY_SLOT_MASK) {
 +    if (env->flags & TB_FLAG_DELAY_SLOT_MASK) {
          /* Branch instruction should be executed again before delay slot. */
      env->spc -= 2;
      /* Clear flags for exception/interrupt routine. */
 -        env->flags &= ~DELAY_SLOT_MASK;
 +        env->flags &= ~TB_FLAG_DELAY_SLOT_MASK;
      }
      if (do_exp) {
@@ -XXX,XX +XXX,XX @@ bool superh_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
          CPUSH4State *env = &cpu->env;
          /* Delay slots are indivisible, ignore interrupts */
 -        if (env->flags & DELAY_SLOT_MASK) {
 +        if (env->flags & TB_FLAG_DELAY_SLOT_MASK) {
              return false;
          } else {
              superh_cpu_do_interrupt(cs);
 diff --git a/target/sh4/translate.c b/target/sh4/translate.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/sh4/translate.c
 +++ b/target/sh4/translate.c
@@ -XXX,XX +XXX,XX @@ void superh_cpu_dump_state(CPUState *cs, FILE *f, int flags)
              i, env->gregs[i], i + 1, env->gregs[i + 1],
              i + 2, env->gregs[i + 2], i + 3, env->gregs[i + 3]);
      }
 -    if (env->flags & DELAY_SLOT) {
 +    if (env->flags & TB_FLAG_DELAY_SLOT) {
          qemu_printf("in delay slot (delayed_pc=0x%08x)\n",
              env->delayed_pc);
 -    } else if (env->flags & DELAY_SLOT_CONDITIONAL) {
 +    } else if (env->flags & TB_FLAG_DELAY_SLOT_COND) {
          qemu_printf("in conditional delay slot (delayed_pc=0x%08x)\n",
              env->delayed_pc);
 -    } else if (env->flags & DELAY_SLOT_RTE) {
 +    } else if (env->flags & TB_FLAG_DELAY_SLOT_RTE) {
          qemu_fprintf(f, "in rte delay slot (delayed_pc=0x%08x)\n",
                       env->delayed_pc);
      }
@@ -XXX,XX +XXX,XX @@ static inline void gen_save_cpu_state(DisasContext *ctx, bool save_pc)
  static inline bool use_exit_tb(DisasContext *ctx)
  {
 -    return (ctx->tbflags & GUSA_EXCLUSIVE) != 0;
 +    return (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE) != 0;
  }
  static bool use_goto_tb(DisasContext *ctx, target_ulong dest)
@@ -XXX,XX +XXX,XX @@ static void gen_conditional_jump(DisasContext *ctx, target_ulong dest,
      TCGLabel *l1 = gen_new_label();
      TCGCond cond_not_taken = jump_if_true ? TCG_COND_EQ : TCG_COND_NE;
 -    if (ctx->tbflags & GUSA_EXCLUSIVE) {
 +    if (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE) {
          /* When in an exclusive region, we must continue to the end.
             Therefore, exit the region on a taken branch, but otherwise
             fall through to the next instruction.  */
          tcg_gen_brcondi_i32(cond_not_taken, cpu_sr_t, 0, l1);
 -        tcg_gen_movi_i32(cpu_flags, ctx->envflags & ~GUSA_MASK);
 +        tcg_gen_movi_i32(cpu_flags, ctx->envflags & ~TB_FLAG_GUSA_MASK);
          /* Note that this won't actually use a goto_tb opcode because we
             disallow it in use_goto_tb, but it handles exit + singlestep.  */
          gen_goto_tb(ctx, 0, dest);
@@ -XXX,XX +XXX,XX @@ static void gen_delayed_conditional_jump(DisasContext * ctx)
      tcg_gen_mov_i32(ds, cpu_delayed_cond);
      tcg_gen_discard_i32(cpu_delayed_cond);
 -    if (ctx->tbflags & GUSA_EXCLUSIVE) {
 +    if (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE) {
          /* When in an exclusive region, we must continue to the end.
             Therefore, exit the region on a taken branch, but otherwise
             fall through to the next instruction.  */
          tcg_gen_brcondi_i32(TCG_COND_EQ, ds, 0, l1);
          /* Leave the gUSA region.  */
 -        tcg_gen_movi_i32(cpu_flags, ctx->envflags & ~GUSA_MASK);
 +        tcg_gen_movi_i32(cpu_flags, ctx->envflags & ~TB_FLAG_GUSA_MASK);
          gen_jump(ctx);
          gen_set_label(l1);
@@ -XXX,XX +XXX,XX @@ static inline void gen_store_fpr64(DisasContext *ctx, TCGv_i64 t, int reg)
  #define XHACK(x) ((((x) & 1 ) << 4) | ((x) & 0xe))
  #define CHECK_NOT_DELAY_SLOT \
 -    if (ctx->envflags & DELAY_SLOT_MASK) {  \
 -        goto do_illegal_slot;               \
 +    if (ctx->envflags & TB_FLAG_DELAY_SLOT_MASK) {  \
 +        goto do_illegal_slot;                       \
      }
  #define CHECK_PRIVILEGED \
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
      case 0x000b:        /* rts */
      CHECK_NOT_DELAY_SLOT
      tcg_gen_mov_i32(cpu_delayed_pc, cpu_pr);
 -        ctx->envflags |= DELAY_SLOT;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT;
      ctx->delayed_pc = (uint32_t) - 1;
      return;
      case 0x0028:        /* clrmac */
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
      CHECK_NOT_DELAY_SLOT
          gen_write_sr(cpu_ssr);
      tcg_gen_mov_i32(cpu_delayed_pc, cpu_spc);
 -        ctx->envflags |= DELAY_SLOT_RTE;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT_RTE;
      ctx->delayed_pc = (uint32_t) - 1;
          ctx->base.is_jmp = DISAS_STOP;
      return;
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
      return;
      case 0xe000:        /* mov #imm,Rn */
  #ifdef CONFIG_USER_ONLY
 -        /* Detect the start of a gUSA region.  If so, update envflags
 -           and end the TB.  This will allow us to see the end of the
 -           region (stored in R0) in the next TB.  */
 +        /*
 +         * Detect the start of a gUSA region (mov #-n, r15).
 +         * If so, update envflags and end the TB.  This will allow us
 +         * to see the end of the region (stored in R0) in the next TB.
 +         */
          if (B11_8 == 15 && B7_0s < 0 &&
              (tb_cflags(ctx->base.tb) & CF_PARALLEL)) {
 -            ctx->envflags = deposit32(ctx->envflags, GUSA_SHIFT, 8, B7_0s);
 +            ctx->envflags =
 +                deposit32(ctx->envflags, TB_FLAG_GUSA_SHIFT, 8, B7_0s);
              ctx->base.is_jmp = DISAS_STOP;
          }
  #endif
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
      case 0xa000:        /* bra disp */
      CHECK_NOT_DELAY_SLOT
          ctx->delayed_pc = ctx->base.pc_next + 4 + B11_0s * 2;
 -        ctx->envflags |= DELAY_SLOT;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT;
      return;
      case 0xb000:        /* bsr disp */
      CHECK_NOT_DELAY_SLOT
          tcg_gen_movi_i32(cpu_pr, ctx->base.pc_next + 4);
          ctx->delayed_pc = ctx->base.pc_next + 4 + B11_0s * 2;
 -        ctx->envflags |= DELAY_SLOT;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT;
      return;
      }
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
      CHECK_NOT_DELAY_SLOT
          tcg_gen_xori_i32(cpu_delayed_cond, cpu_sr_t, 1);
          ctx->delayed_pc = ctx->base.pc_next + 4 + B7_0s * 2;
 -        ctx->envflags |= DELAY_SLOT_CONDITIONAL;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT_COND;
      return;
      case 0x8900:        /* bt label */
      CHECK_NOT_DELAY_SLOT
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
      CHECK_NOT_DELAY_SLOT
          tcg_gen_mov_i32(cpu_delayed_cond, cpu_sr_t);
          ctx->delayed_pc = ctx->base.pc_next + 4 + B7_0s * 2;
 -        ctx->envflags |= DELAY_SLOT_CONDITIONAL;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT_COND;
      return;
      case 0x8800:        /* cmp/eq #imm,R0 */
          tcg_gen_setcondi_i32(TCG_COND_EQ, cpu_sr_t, REG(0), B7_0s);
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
      case 0x0023:        /* braf Rn */
      CHECK_NOT_DELAY_SLOT
          tcg_gen_addi_i32(cpu_delayed_pc, REG(B11_8), ctx->base.pc_next + 4);
 -        ctx->envflags |= DELAY_SLOT;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT;
      ctx->delayed_pc = (uint32_t) - 1;
      return;
      case 0x0003:        /* bsrf Rn */
      CHECK_NOT_DELAY_SLOT
          tcg_gen_movi_i32(cpu_pr, ctx->base.pc_next + 4);
      tcg_gen_add_i32(cpu_delayed_pc, REG(B11_8), cpu_pr);
 -        ctx->envflags |= DELAY_SLOT;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT;
      ctx->delayed_pc = (uint32_t) - 1;
      return;
      case 0x4015:        /* cmp/pl Rn */
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
      case 0x402b:        /* jmp @Rn */
      CHECK_NOT_DELAY_SLOT
      tcg_gen_mov_i32(cpu_delayed_pc, REG(B11_8));
 -        ctx->envflags |= DELAY_SLOT;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT;
      ctx->delayed_pc = (uint32_t) - 1;
      return;
      case 0x400b:        /* jsr @Rn */
      CHECK_NOT_DELAY_SLOT
          tcg_gen_movi_i32(cpu_pr, ctx->base.pc_next + 4);
      tcg_gen_mov_i32(cpu_delayed_pc, REG(B11_8));
 -        ctx->envflags |= DELAY_SLOT;
 +        ctx->envflags |= TB_FLAG_DELAY_SLOT;
      ctx->delayed_pc = (uint32_t) - 1;
      return;
      case 0x400e:        /* ldc Rm,SR */
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
      fflush(stderr);
  #endif
   do_illegal:
 -    if (ctx->envflags & DELAY_SLOT_MASK) {
 +    if (ctx->envflags & TB_FLAG_DELAY_SLOT_MASK) {
   do_illegal_slot:
          gen_save_cpu_state(ctx, true);
          gen_helper_raise_slot_illegal_instruction(cpu_env);
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
   do_fpu_disabled:
      gen_save_cpu_state(ctx, true);
 -    if (ctx->envflags & DELAY_SLOT_MASK) {
 +    if (ctx->envflags & TB_FLAG_DELAY_SLOT_MASK) {
          gen_helper_raise_slot_fpu_disable(cpu_env);
      } else {
          gen_helper_raise_fpu_disable(cpu_env);
@@ -XXX,XX +XXX,XX @@ static void decode_opc(DisasContext * ctx)
      _decode_opc(ctx);
 -    if (old_flags & DELAY_SLOT_MASK) {
 +    if (old_flags & TB_FLAG_DELAY_SLOT_MASK) {
          /* go out of the delay slot */
 -        ctx->envflags &= ~DELAY_SLOT_MASK;
 +        ctx->envflags &= ~TB_FLAG_DELAY_SLOT_MASK;
          /* When in an exclusive region, we must continue to the end
             for conditional branches.  */
 -        if (ctx->tbflags & GUSA_EXCLUSIVE
 -            && old_flags & DELAY_SLOT_CONDITIONAL) {
 +        if (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE
 +            && old_flags & TB_FLAG_DELAY_SLOT_COND) {
              gen_delayed_conditional_jump(ctx);
              return;
          }
          /* Otherwise this is probably an invalid gUSA region.
             Drop the GUSA bits so the next TB doesn't see them.  */
 -        ctx->envflags &= ~GUSA_MASK;
 +        ctx->envflags &= ~TB_FLAG_GUSA_MASK;
          tcg_gen_movi_i32(cpu_flags, ctx->envflags);
 -        if (old_flags & DELAY_SLOT_CONDITIONAL) {
 +        if (old_flags & TB_FLAG_DELAY_SLOT_COND) {
          gen_delayed_conditional_jump(ctx);
          } else {
              gen_jump(ctx);
@@ -XXX,XX +XXX,XX @@ static void decode_gusa(DisasContext *ctx, CPUSH4State *env)
      }
      /* The entire region has been translated.  */
 -    ctx->envflags &= ~GUSA_MASK;
 +    ctx->envflags &= ~TB_FLAG_GUSA_MASK;
      ctx->base.pc_next = pc_end;
      ctx->base.num_insns += max_insns - 1;
      return;
@@ -XXX,XX +XXX,XX @@ static void decode_gusa(DisasContext *ctx, CPUSH4State *env)
      /* Restart with the EXCLUSIVE bit set, within a TB run via
         cpu_exec_step_atomic holding the exclusive lock.  */
 -    ctx->envflags |= GUSA_EXCLUSIVE;
 +    ctx->envflags |= TB_FLAG_GUSA_EXCLUSIVE;
      gen_save_cpu_state(ctx, false);
      gen_helper_exclusive(cpu_env);
      ctx->base.is_jmp = DISAS_NORETURN;
@@ -XXX,XX +XXX,XX @@ static void sh4_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs)
                    (tbflags & (1 << SR_RB))) * 0x10;
      ctx->fbank = tbflags & FPSCR_FR ? 0x10 : 0;
 -    if (tbflags & GUSA_MASK) {
 +#ifdef CONFIG_USER_ONLY
 +    if (tbflags & TB_FLAG_GUSA_MASK) {
 +        /* In gUSA exclusive region. */
          uint32_t pc = ctx->base.pc_next;
          uint32_t pc_end = ctx->base.tb->cs_base;
 -        int backup = sextract32(ctx->tbflags, GUSA_SHIFT, 8);
 +        int backup = sextract32(ctx->tbflags, TB_FLAG_GUSA_SHIFT, 8);
          int max_insns = (pc_end - pc) / 2;
          if (pc != pc_end + backup || max_insns < 2) {
              /* This is a malformed gUSA region.  Don't do anything special,
                 since the interpreter is likely to get confused.  */
 -            ctx->envflags &= ~GUSA_MASK;
 -        } else if (tbflags & GUSA_EXCLUSIVE) {
 +            ctx->envflags &= ~TB_FLAG_GUSA_MASK;
 +        } else if (tbflags & TB_FLAG_GUSA_EXCLUSIVE) {
              /* Regardless of single-stepping or the end of the page,
                 we must complete execution of the gUSA region while
                 holding the exclusive lock.  */
@@ -XXX,XX +XXX,XX @@ static void sh4_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs)
              return;
          }
      }
 +#endif
      /* Since the ISA is fixed-width, we can bound by the number
         of instructions remaining on the page.  */
@@ -XXX,XX +XXX,XX @@ static void sh4_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
      DisasContext *ctx = container_of(dcbase, DisasContext, base);
  #ifdef CONFIG_USER_ONLY
 -    if (unlikely(ctx->envflags & GUSA_MASK)
 -        && !(ctx->envflags & GUSA_EXCLUSIVE)) {
 +    if (unlikely(ctx->envflags & TB_FLAG_GUSA_MASK)
 +        && !(ctx->envflags & TB_FLAG_GUSA_EXCLUSIVE)) {
          /* We're in an gUSA region, and we have not already fallen
             back on using an exclusive region.  Attempt to parse the
             region into a single supported atomic operation.  Failure
@@ -XXX,XX +XXX,XX @@ static void sh4_tr_tb_stop(DisasContextBase *dcbase, CPUState *cs)
  {
      DisasContext *ctx = container_of(dcbase, DisasContext, base);
 -    if (ctx->tbflags & GUSA_EXCLUSIVE) {
 +    if (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE) {
          /* Ending the region of exclusivity.  Clear the bits.  */
 -        ctx->envflags &= ~GUSA_MASK;
 +        ctx->envflags &= ~TB_FLAG_GUSA_MASK;
      }
      switch (ctx->base.is_jmp) {
 --
 .34.1

-[PULL 03/20] cputlb: used cached CPUClass in our hot-paths
+[PULL 14/20] plugins: Simplify callback queues
-From: Alex Bennée <alex.bennee@linaro.org>
+We have qemu_plugin_dyn_cb.type to differentiate the various
 callback types, so we do not need to keep them in separate queues.
-Before: 35.912 s ±  0.168 s
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
   After: 35.565 s ±  0.087 s
 Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Message-Id: <20220811151413.3350684-5-alex.bennee@linaro.org>
 Signed-off-by: Cédric Le Goater <clg@kaod.org>
 Message-Id: <20220923084803.498337-5-clg@kaod.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/cputlb.c | 15 ++++++---------
+ include/qemu/plugin.h  | 35 ++++++----------
-file changed, 6 insertions(+), 9 deletions(-)
+ accel/tcg/plugin-gen.c | 90 ++++++++++++++++++++++--------------------
  plugins/api.c          | 18 +++------
 files changed, 65 insertions(+), 78 deletions(-)
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
+diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
+--- a/include/qemu/plugin.h
-+++ b/accel/tcg/cputlb.c
++++ b/include/qemu/plugin.h
-@@ -XXX,XX +XXX,XX @@ void tlb_set_page(CPUState *cpu, target_ulong vaddr,
+@@ -XXX,XX +XXX,XX @@ union qemu_plugin_cb_sig {
- static void tlb_fill(CPUState *cpu, target_ulong addr, int size,
+ };
-                      MMUAccessType access_type, int mmu_idx, uintptr_t retaddr)
- {
+ enum plugin_dyn_cb_type {
--    CPUClass *cc = CPU_GET_CLASS(cpu);
+-    PLUGIN_CB_INSN,
-     bool ok;
+-    PLUGIN_CB_MEM,
 -    PLUGIN_N_CB_TYPES,
 -};
 -
 -enum plugin_dyn_cb_subtype {
      PLUGIN_CB_REGULAR,
      PLUGIN_CB_INLINE,
 -    PLUGIN_N_CB_SUBTYPES,
  };
  /*
@@ -XXX,XX +XXX,XX @@ enum plugin_dyn_cb_subtype {
   */
  struct qemu_plugin_dyn_cb {
      void *userp;
 -    enum plugin_dyn_cb_subtype type;
 +    enum plugin_dyn_cb_type type;
      /* @rw applies to mem callbacks only (both regular and inline) */
      enum qemu_plugin_mem_rw rw;
      /* fields specific to each dyn_cb type go here */
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_insn {
      GByteArray *data;
      uint64_t vaddr;
      void *haddr;
 -    GArray *cbs[PLUGIN_N_CB_TYPES][PLUGIN_N_CB_SUBTYPES];
 +    GArray *insn_cbs;
 +    GArray *mem_cbs;
      bool calls_helpers;
      /* if set, the instruction calls helpers that might access guest memory */
@@ -XXX,XX +XXX,XX @@ static inline void qemu_plugin_insn_cleanup_fn(gpointer data)
  static inline struct qemu_plugin_insn *qemu_plugin_insn_alloc(void)
  {
 -    int i, j;
      struct qemu_plugin_insn *insn = g_new0(struct qemu_plugin_insn, 1);
 -    insn->data = g_byte_array_sized_new(4);
 -    for (i = 0; i < PLUGIN_N_CB_TYPES; i++) {
 -        for (j = 0; j < PLUGIN_N_CB_SUBTYPES; j++) {
 -            insn->cbs[i][j] = g_array_new(false, false,
 -                                          sizeof(struct qemu_plugin_dyn_cb));
 -        }
 -    }
 +    insn->data = g_byte_array_sized_new(4);
      return insn;
  }
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_tb {
      /* if set, the TB calls helpers that might access guest memory */
      bool mem_helper;
 -    GArray *cbs[PLUGIN_N_CB_SUBTYPES];
 +    GArray *cbs;
  };
  /**
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb,
                                                   uint64_t pc)
  {
      struct qemu_plugin_insn *insn;
 -    int i, j;
      if (unlikely(tb->n == tb->insns->len)) {
          struct qemu_plugin_insn *new_insn = qemu_plugin_insn_alloc();
          g_ptr_array_add(tb->insns, new_insn);
      }
 +
      insn = g_ptr_array_index(tb->insns, tb->n++);
      g_byte_array_set_size(insn->data, 0);
      insn->calls_helpers = false;
      insn->mem_helper = false;
      insn->vaddr = pc;
 -
 -    for (i = 0; i < PLUGIN_N_CB_TYPES; i++) {
 -        for (j = 0; j < PLUGIN_N_CB_SUBTYPES; j++) {
 -            g_array_set_size(insn->cbs[i][j], 0);
 -        }
 +    if (insn->insn_cbs) {
 +        g_array_set_size(insn->insn_cbs, 0);
 +    }
 +    if (insn->mem_cbs) {
 +        g_array_set_size(insn->mem_cbs, 0);
      }
      return insn;
 diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/plugin-gen.c
 +++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ void plugin_gen_disable_mem_helpers(void)
  static void gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
                                    struct qemu_plugin_insn *insn)
  {
 -    GArray *cbs[2];
      GArray *arr;
 -    size_t n_cbs;
 +    size_t len;
      /*
-      * This is not a probe, so only valid return is success; failure
+      * Tracking memory accesses performed from helpers requires extra work.
-      * should result in exception + longjmp to the cpu loop.
+@@ -XXX,XX +XXX,XX @@ static void gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
-      */
+         return;
--    ok = cc->tcg_ops->tlb_fill(cpu, addr, size,
+     }
--                               access_type, mmu_idx, false, retaddr);
-+    ok = cpu->cc->tcg_ops->tlb_fill(cpu, addr, size,
+-    cbs[0] = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
-+                                    access_type, mmu_idx, false, retaddr);
+-    cbs[1] = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
-     assert(ok);
+-    n_cbs = cbs[0]->len + cbs[1]->len;
  }
@@ -XXX,XX +XXX,XX @@ static inline void cpu_unaligned_access(CPUState *cpu, vaddr addr,
                                          MMUAccessType access_type,
                                          int mmu_idx, uintptr_t retaddr)
  {
 -    CPUClass *cc = CPU_GET_CLASS(cpu);
 -
--    cc->tcg_ops->do_unaligned_access(cpu, addr, access_type, mmu_idx, retaddr);
+-    if (n_cbs == 0) {
-+    cpu->cc->tcg_ops->do_unaligned_access(cpu, addr, access_type,
++    if (!insn->mem_cbs || !insn->mem_cbs->len) {
-+                                          mmu_idx, retaddr);
+         insn->mem_helper = false;
- }
+         return;
+     }
- static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr,
+     insn->mem_helper = true;
-@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
+     ptb->mem_helper = true;
-     if (!tlb_hit_page(tlb_addr, page_addr)) {
-         if (!victim_tlb_hit(env, mmu_idx, index, elt_ofs, page_addr)) {
++    /*
-             CPUState *cs = env_cpu(env);
++     * TODO: It seems like we should be able to use ref/unref
--            CPUClass *cc = CPU_GET_CLASS(cs);
++     * to avoid needing to actually copy this array.
++     * Alternately, perhaps we could allocate new memory adjacent
--            if (!cc->tcg_ops->tlb_fill(cs, addr, fault_size, access_type,
++     * to the TranslationBlock itself, so that we do not have to
--                                       mmu_idx, nonfault, retaddr)) {
++     * actively manage the lifetime after this.
-+            if (!cs->cc->tcg_ops->tlb_fill(cs, addr, fault_size, access_type,
++     */
-+                                           mmu_idx, nonfault, retaddr)) {
++    len = insn->mem_cbs->len;
-                 /* Non-faulting page table read failed.  */
+     arr = g_array_sized_new(false, false,
-                 *phost = NULL;
+-                            sizeof(struct qemu_plugin_dyn_cb), n_cbs);
-                 return TLB_INVALID_MASK;
+-    g_array_append_vals(arr, cbs[0]->data, cbs[0]->len);
 -    g_array_append_vals(arr, cbs[1]->data, cbs[1]->len);
 -
 +                            sizeof(struct qemu_plugin_dyn_cb), len);
 +    memcpy(arr->data, insn->mem_cbs->data,
 +           len * sizeof(struct qemu_plugin_dyn_cb));
      qemu_plugin_add_dyn_cb_arr(arr);
      tcg_gen_st_ptr(tcg_constant_ptr((intptr_t)arr), tcg_env,
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
              case PLUGIN_GEN_FROM_TB:
                  assert(insn == NULL);
 -                cbs = plugin_tb->cbs[PLUGIN_CB_REGULAR];
 +                cbs = plugin_tb->cbs;
                  for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
                      struct qemu_plugin_dyn_cb *cb =
                          &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 -                    gen_udata_cb(cb);
 -                }
 -                cbs = plugin_tb->cbs[PLUGIN_CB_INLINE];
 -                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 -                    struct qemu_plugin_dyn_cb *cb =
 -                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 -                    gen_inline_cb(cb);
 +                    switch (cb->type) {
 +                    case PLUGIN_CB_REGULAR:
 +                        gen_udata_cb(cb);
 +                        break;
 +                    case PLUGIN_CB_INLINE:
 +                        gen_inline_cb(cb);
 +                        break;
 +                    default:
 +                        g_assert_not_reached();
 +                    }
                  }
                  break;
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
                  gen_enable_mem_helper(plugin_tb, insn);
 -                cbs = insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR];
 +                cbs = insn->insn_cbs;
                  for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
                      struct qemu_plugin_dyn_cb *cb =
                          &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 -                    gen_udata_cb(cb);
 -                }
 -                cbs = insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_INLINE];
 -                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 -                    struct qemu_plugin_dyn_cb *cb =
 -                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 -                    gen_inline_cb(cb);
 +                    switch (cb->type) {
 +                    case PLUGIN_CB_REGULAR:
 +                        gen_udata_cb(cb);
 +                        break;
 +                    case PLUGIN_CB_INLINE:
 +                        gen_inline_cb(cb);
 +                        break;
 +                    default:
 +                        g_assert_not_reached();
 +                    }
                  }
                  break;
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
              tcg_ctx->emit_before_op = op;
 -            cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
 +            cbs = insn->mem_cbs;
              for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
                  struct qemu_plugin_dyn_cb *cb =
                      &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 -                if (cb->rw & rw) {
 -                    gen_mem_cb(cb, meminfo, addr);
 -                }
 -            }
 -            cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
 -            for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 -                struct qemu_plugin_dyn_cb *cb =
 -                    &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
                  if (cb->rw & rw) {
 -                    gen_inline_cb(cb);
 +                    switch (cb->type) {
 +                    case PLUGIN_CB_REGULAR:
 +                        gen_mem_cb(cb, meminfo, addr);
 +                        break;
 +                    case PLUGIN_CB_INLINE:
 +                        gen_inline_cb(cb);
 +                        break;
 +                    default:
 +                        g_assert_not_reached();
 +                    }
                  }
              }
@@ -XXX,XX +XXX,XX @@ bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
      if (test_bit(QEMU_PLUGIN_EV_VCPU_TB_TRANS, cpu->plugin_state->event_mask)) {
          struct qemu_plugin_tb *ptb = tcg_ctx->plugin_tb;
 -        int i;
          /* reset callbacks */
 -        for (i = 0; i < PLUGIN_N_CB_SUBTYPES; i++) {
 -            if (ptb->cbs[i]) {
 -                g_array_set_size(ptb->cbs[i], 0);
 -            }
 +        if (ptb->cbs) {
 +            g_array_set_size(ptb->cbs, 0);
          }
          ptb->n = 0;
 diff --git a/plugins/api.c b/plugins/api.c
 index XXXXXXX..XXXXXXX 100644
 --- a/plugins/api.c
 +++ b/plugins/api.c
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_tb_exec_cb(struct qemu_plugin_tb *tb,
                                            void *udata)
  {
      if (!tb->mem_only) {
 -        plugin_register_dyn_cb__udata(&tb->cbs[PLUGIN_CB_REGULAR],
 -                                      cb, flags, udata);
 +        plugin_register_dyn_cb__udata(&tb->cbs, cb, flags, udata);
      }
  }
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_tb_exec_inline_per_vcpu(
      uint64_t imm)
  {
      if (!tb->mem_only) {
 -        plugin_register_inline_op_on_entry(
 -            &tb->cbs[PLUGIN_CB_INLINE], 0, op, entry, imm);
 +        plugin_register_inline_op_on_entry(&tb->cbs, 0, op, entry, imm);
      }
  }
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_insn_exec_cb(struct qemu_plugin_insn *insn,
                                              void *udata)
  {
      if (!insn->mem_only) {
 -        plugin_register_dyn_cb__udata(
 -            &insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR], cb, flags, udata);
 +        plugin_register_dyn_cb__udata(&insn->insn_cbs, cb, flags, udata);
      }
  }
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_insn_exec_inline_per_vcpu(
      uint64_t imm)
  {
      if (!insn->mem_only) {
 -        plugin_register_inline_op_on_entry(
 -            &insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_INLINE], 0, op, entry, imm);
 +        plugin_register_inline_op_on_entry(&insn->insn_cbs, 0, op, entry, imm);
      }
  }
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_mem_cb(struct qemu_plugin_insn *insn,
                                        enum qemu_plugin_mem_rw rw,
                                        void *udata)
  {
 -    plugin_register_vcpu_mem_cb(&insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR],
 -                                cb, flags, rw, udata);
 +    plugin_register_vcpu_mem_cb(&insn->mem_cbs, cb, flags, rw, udata);
  }
  void qemu_plugin_register_vcpu_mem_inline_per_vcpu(
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_mem_inline_per_vcpu(
      qemu_plugin_u64 entry,
      uint64_t imm)
  {
 -    plugin_register_inline_op_on_entry(
 -        &insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE], rw, op, entry, imm);
 +    plugin_register_inline_op_on_entry(&insn->mem_cbs, rw, op, entry, imm);
  }
  void qemu_plugin_register_vcpu_tb_trans_cb(qemu_plugin_id_t id,
 --
 .34.1

-[PULL 02/20] hw/core/cpu-sysemu: used cached class in cpu_asidx_from_attrs
+[PULL 15/20] plugins: Introduce PLUGIN_CB_MEM_REGULAR
-From: Alex Bennée <alex.bennee@linaro.org>
+Use different enumerators for vcpu_udata and vcpu_mem callbacks.
-This is a heavily used function so lets avoid the cost of
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 CPU_GET_CLASS. On the romulus-bmc run it has a modest effect:
   Before: 36.812 s ±  0.506 s
   After:  35.912 s ±  0.168 s
 Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Message-Id: <20220811151413.3350684-4-alex.bennee@linaro.org>
 Signed-off-by: Cédric Le Goater <clg@kaod.org>
 Message-Id: <20220923084803.498337-4-clg@kaod.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- hw/core/cpu-sysemu.c | 5 ++---
+ include/qemu/plugin.h  | 1 +
-file changed, 2 insertions(+), 3 deletions(-)
+ accel/tcg/plugin-gen.c | 2 +-
  plugins/core.c         | 4 ++--
 files changed, 4 insertions(+), 3 deletions(-)
-diff --git a/hw/core/cpu-sysemu.c b/hw/core/cpu-sysemu.c
+diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
 index XXXXXXX..XXXXXXX 100644
---- a/hw/core/cpu-sysemu.c
+--- a/include/qemu/plugin.h
-+++ b/hw/core/cpu-sysemu.c
++++ b/include/qemu/plugin.h
-@@ -XXX,XX +XXX,XX @@ hwaddr cpu_get_phys_page_debug(CPUState *cpu, vaddr addr)
+@@ -XXX,XX +XXX,XX @@ union qemu_plugin_cb_sig {
- int cpu_asidx_from_attrs(CPUState *cpu, MemTxAttrs attrs)
+ enum plugin_dyn_cb_type {
- {
+     PLUGIN_CB_REGULAR,
--    CPUClass *cc = CPU_GET_CLASS(cpu);
++    PLUGIN_CB_MEM_REGULAR,
-     int ret = 0;
+     PLUGIN_CB_INLINE,
+ };
--    if (cc->sysemu_ops->asidx_from_attrs) {
--        ret = cc->sysemu_ops->asidx_from_attrs(cpu, attrs);
+diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
-+    if (cpu->cc->sysemu_ops->asidx_from_attrs) {
+index XXXXXXX..XXXXXXX 100644
-+        ret = cpu->cc->sysemu_ops->asidx_from_attrs(cpu, attrs);
+--- a/accel/tcg/plugin-gen.c
-         assert(ret < cpu->num_ases && ret >= 0);
++++ b/accel/tcg/plugin-gen.c
-     }
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
-     return ret;
                  if (cb->rw & rw) {
                      switch (cb->type) {
 -                    case PLUGIN_CB_REGULAR:
 +                    case PLUGIN_CB_MEM_REGULAR:
                          gen_mem_cb(cb, meminfo, addr);
                          break;
                      case PLUGIN_CB_INLINE:
 diff --git a/plugins/core.c b/plugins/core.c
 index XXXXXXX..XXXXXXX 100644
 --- a/plugins/core.c
 +++ b/plugins/core.c
@@ -XXX,XX +XXX,XX @@ void plugin_register_vcpu_mem_cb(GArray **arr,
      struct qemu_plugin_dyn_cb *dyn_cb = plugin_get_dyn_cb(arr);
      dyn_cb->userp = udata;
 -    dyn_cb->type = PLUGIN_CB_REGULAR;
 +    dyn_cb->type = PLUGIN_CB_MEM_REGULAR;
      dyn_cb->rw = rw;
      dyn_cb->regular.f.vcpu_mem = cb;
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_vcpu_mem_cb(CPUState *cpu, uint64_t vaddr,
                  break;
          }
          switch (cb->type) {
 -        case PLUGIN_CB_REGULAR:
 +        case PLUGIN_CB_MEM_REGULAR:
              cb->regular.f.vcpu_mem(cpu->cpu_index, make_plugin_meminfo(oi, rw),
                                     vaddr, cb->userp);
              break;
 --
 .34.1

-[PULL 17/20] accel/tcg: Introduce tb_pc and log_pc
+[PULL 16/20] plugins: Replace pr_ops with a proper debug dump flag
-The availability of tb->pc will shortly be conditional.
+The DEBUG_PLUGIN_GEN_OPS ifdef is replaced with "-d op_plugin".
-Introduce accessor functions to minimize ifdefs.
+The second pr_ops call can be obtained with "-d op".
-Pass around a known pc to places like tcg_gen_code,
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 where the caller must already have the value.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/internal.h                    |  6 ++++
+ include/qemu/log.h     |  1 +
- include/exec/exec-all.h                 |  6 ++++
+ include/tcg/tcg.h      |  1 +
- include/tcg/tcg.h                       |  2 +-
+ accel/tcg/plugin-gen.c | 67 +++++++-----------------------------------
- accel/tcg/cpu-exec.c                    | 46 ++++++++++++++-----------
+ tcg/tcg.c              | 29 +++++++++++++++++-
- accel/tcg/translate-all.c               | 37 +++++++++++---------
+ util/log.c             |  4 +++
- target/arm/cpu.c                        |  4 +--
+files changed, 45 insertions(+), 57 deletions(-)
  target/avr/cpu.c                        |  2 +-
  target/hexagon/cpu.c                    |  2 +-
  target/hppa/cpu.c                       |  4 +--
  target/i386/tcg/tcg-cpu.c               |  2 +-
  target/loongarch/cpu.c                  |  2 +-
  target/microblaze/cpu.c                 |  2 +-
  target/mips/tcg/exception.c             |  2 +-
  target/mips/tcg/sysemu/special_helper.c |  2 +-
  target/openrisc/cpu.c                   |  2 +-
  target/riscv/cpu.c                      |  4 +--
  target/rx/cpu.c                         |  2 +-
  target/sh4/cpu.c                        |  4 +--
  target/sparc/cpu.c                      |  2 +-
  target/tricore/cpu.c                    |  2 +-
  tcg/tcg.c                               |  8 ++---
 files changed, 82 insertions(+), 61 deletions(-)
-diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
+diff --git a/include/qemu/log.h b/include/qemu/log.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/internal.h
+--- a/include/qemu/log.h
-+++ b/accel/tcg/internal.h
++++ b/include/qemu/log.h
-@@ -XXX,XX +XXX,XX @@ G_NORETURN void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr);
+@@ -XXX,XX +XXX,XX @@ bool qemu_log_separate(void);
- void page_init(void);
+ #define LOG_STRACE         (1 << 19)
- void tb_htable_init(void);
+ #define LOG_PER_THREAD     (1 << 20)
+ #define CPU_LOG_TB_VPU     (1 << 21)
-+/* Return the current PC from CPU, which may be cached in TB. */
++#define LOG_TB_OP_PLUGIN   (1 << 22)
-+static inline target_ulong log_pc(CPUState *cpu, const TranslationBlock *tb)
-+{
+ /* Lock/unlock output. */
-+    return tb_pc(tb);
 +}
 +
  #endif /* ACCEL_TCG_INTERNAL_H */
 diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/exec/exec-all.h
 +++ b/include/exec/exec-all.h
@@ -XXX,XX +XXX,XX @@ struct TranslationBlock {
      uintptr_t jmp_dest[2];
  };
 +/* Hide the read to avoid ifdefs for TARGET_TB_PCREL. */
 +static inline target_ulong tb_pc(const TranslationBlock *tb)
 +{
 +    return tb->pc;
 +}
 +
  /* Hide the qatomic_read to make code a little easier on the eyes */
  static inline uint32_t tb_cflags(const TranslationBlock *tb)
  {
 diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg.h
 +++ b/include/tcg/tcg.h
-@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void);
+@@ -XXX,XX +XXX,XX @@ static inline const TCGOpcode *tcg_swap_vecop_list(const TCGOpcode *n)
  void tcg_prologue_init(TCGContext *s);
  void tcg_func_start(TCGContext *s);
 -int tcg_gen_code(TCGContext *s, TranslationBlock *tb);
 +int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start);
  void tcg_set_frame(TCGContext *s, TCGReg reg, intptr_t start, intptr_t size);
 diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cpu-exec.c
 +++ b/accel/tcg/cpu-exec.c
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
      const TranslationBlock *tb = p;
      const struct tb_desc *desc = d;
 -    if (tb->pc == desc->pc &&
 +    if (tb_pc(tb) == desc->pc &&
          tb->page_addr[0] == desc->page_addr0 &&
          tb->cs_base == desc->cs_base &&
          tb->flags == desc->flags &&
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
      return tb;
  }
--static inline void log_cpu_exec(target_ulong pc, CPUState *cpu,
+ bool tcg_can_emit_vecop_list(const TCGOpcode *, TCGType, unsigned);
--                                const TranslationBlock *tb)
++void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs);
-+static void log_cpu_exec(target_ulong pc, CPUState *cpu,
-+                         const TranslationBlock *tb)
+ #endif /* TCG_H */
 diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/plugin-gen.c
 +++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@
   */
  #include "qemu/osdep.h"
  #include "qemu/plugin.h"
 +#include "qemu/log.h"
  #include "cpu.h"
  #include "tcg/tcg.h"
  #include "tcg/tcg-temp-internal.h"
@@ -XXX,XX +XXX,XX @@ static void gen_mem_cb(struct qemu_plugin_dyn_cb *cb,
      tcg_temp_free_i32(cpu_index);
  }
 -/* #define DEBUG_PLUGIN_GEN_OPS */
 -static void pr_ops(void)
 -{
 -#ifdef DEBUG_PLUGIN_GEN_OPS
 -    TCGOp *op;
 -    int i = 0;
 -
 -    QTAILQ_FOREACH(op, &tcg_ctx->ops, link) {
 -        const char *name = "";
 -        const char *type = "";
 -
 -        if (op->opc == INDEX_op_plugin_cb_start) {
 -            switch (op->args[0]) {
 -            case PLUGIN_GEN_FROM_TB:
 -                name = "tb";
 -                break;
 -            case PLUGIN_GEN_FROM_INSN:
 -                name = "insn";
 -                break;
 -            case PLUGIN_GEN_FROM_MEM:
 -                name = "mem";
 -                break;
 -            case PLUGIN_GEN_AFTER_INSN:
 -                name = "after insn";
 -                break;
 -            default:
 -                break;
 -            }
 -            switch (op->args[1]) {
 -            case PLUGIN_GEN_CB_UDATA:
 -                type = "udata";
 -                break;
 -            case PLUGIN_GEN_CB_INLINE:
 -                type = "inline";
 -                break;
 -            case PLUGIN_GEN_CB_MEM:
 -                type = "mem";
 -                break;
 -            case PLUGIN_GEN_ENABLE_MEM_HELPER:
 -                type = "enable mem helper";
 -                break;
 -            case PLUGIN_GEN_DISABLE_MEM_HELPER:
 -                type = "disable mem helper";
 -                break;
 -            default:
 -                break;
 -            }
 -        }
 -        printf("op[%2i]: %s %s %s\n", i, tcg_op_defs[op->opc].name, name, type);
 -        i++;
 -    }
 -#endif
 -}
 -
  static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
  {
--    if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_CPU | CPU_LOG_EXEC))
+     TCGOp *op, *next;
--        && qemu_log_in_addr_range(pc)) {
+     int insn_idx = -1;
--
-+    if (qemu_log_in_addr_range(pc)) {
+-    pr_ops();
-         qemu_log_mask(CPU_LOG_EXEC,
++    if (unlikely(qemu_loglevel_mask(LOG_TB_OP_PLUGIN)
-                       "Trace %d: %p [" TARGET_FMT_lx
++                 && qemu_log_in_addr_range(plugin_tb->vaddr))) {
-                       "/" TARGET_FMT_lx "/%08x/%08x] %s\n",
++        FILE *logfile = qemu_log_trylock();
-@@ -XXX,XX +XXX,XX @@ const void *HELPER(lookup_tb_ptr)(CPUArchState *env)
++        if (logfile) {
-         return tcg_code_gen_epilogue;
++            fprintf(logfile, "OP before plugin injection:\n");
-     }
++            tcg_dump_ops(tcg_ctx, logfile, false);
++            fprintf(logfile, "\n");
--    log_cpu_exec(pc, cpu, tb);
++            qemu_log_unlock(logfile);
-+    if (qemu_loglevel_mask(CPU_LOG_TB_CPU | CPU_LOG_EXEC)) {
++        }
 +        log_cpu_exec(pc, cpu, tb);
 +    }
-     return tb->tc.ptr;
+     /*
- }
+      * While injecting code, we cannot afford to reuse any ebb temps
-@@ -XXX,XX +XXX,XX @@ cpu_tb_exec(CPUState *cpu, TranslationBlock *itb, int *tb_exit)
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
-     TranslationBlock *last_tb;
+             break;
      const void *tb_ptr = itb->tc.ptr;
 -    log_cpu_exec(itb->pc, cpu, itb);
 +    if (qemu_loglevel_mask(CPU_LOG_TB_CPU | CPU_LOG_EXEC)) {
 +        log_cpu_exec(log_pc(cpu, itb), cpu, itb);
 +    }
      qemu_thread_jit_execute();
      ret = tcg_qemu_tb_exec(env, tb_ptr);
@@ -XXX,XX +XXX,XX @@ cpu_tb_exec(CPUState *cpu, TranslationBlock *itb, int *tb_exit)
           * of the start of the TB.
           */
          CPUClass *cc = CPU_GET_CLASS(cpu);
 -        qemu_log_mask_and_addr(CPU_LOG_EXEC, last_tb->pc,
 -                               "Stopped execution of TB chain before %p ["
 -                               TARGET_FMT_lx "] %s\n",
 -                               last_tb->tc.ptr, last_tb->pc,
 -                               lookup_symbol(last_tb->pc));
 +
          if (cc->tcg_ops->synchronize_from_tb) {
              cc->tcg_ops->synchronize_from_tb(cpu, last_tb);
          } else {
              assert(cc->set_pc);
 -            cc->set_pc(cpu, last_tb->pc);
 +            cc->set_pc(cpu, tb_pc(last_tb));
 +        }
 +        if (qemu_loglevel_mask(CPU_LOG_EXEC)) {
 +            target_ulong pc = log_pc(cpu, last_tb);
 +            if (qemu_log_in_addr_range(pc)) {
 +                qemu_log("Stopped execution of TB chain before %p ["
 +                         TARGET_FMT_lx "] %s\n",
 +                         last_tb->tc.ptr, pc, lookup_symbol(pc));
 +            }
          }
      }
+-    pr_ops();
@@ -XXX,XX +XXX,XX @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
      qemu_spin_unlock(&tb_next->jmp_lock);
 -    qemu_log_mask_and_addr(CPU_LOG_EXEC, tb->pc,
 -                           "Linking TBs %p [" TARGET_FMT_lx
 -                           "] index %d -> %p [" TARGET_FMT_lx "]\n",
 -                           tb->tc.ptr, tb->pc, n,
 -                           tb_next->tc.ptr, tb_next->pc);
 +    qemu_log_mask(CPU_LOG_EXEC, "Linking TBs %p index %d -> %p\n",
 +                  tb->tc.ptr, n, tb_next->tc.ptr);
      return;
   out_unlock_next:
@@ -XXX,XX +XXX,XX @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
  }
- static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
+ bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
 +                                    target_ulong pc,
                                      TranslationBlock **last_tb, int *tb_exit)
  {
      int32_t insns_left;
 -    trace_exec_tb(tb, tb->pc);
 +    trace_exec_tb(tb, pc);
      tb = cpu_tb_exec(cpu, tb, tb_exit);
      if (*tb_exit != TB_EXIT_REQUESTED) {
          *last_tb = tb;
@@ -XXX,XX +XXX,XX @@ int cpu_exec(CPUState *cpu)
                  tb_add_jump(last_tb, tb_exit, tb);
              }
 -            cpu_loop_exec_tb(cpu, tb, &last_tb, &tb_exit);
 +            cpu_loop_exec_tb(cpu, tb, pc, &last_tb, &tb_exit);
              /* Try to align the host and virtual clocks
                 if the guest is in advance */
 diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/translate-all.c
 +++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
          for (j = 0; j < TARGET_INSN_START_WORDS; ++j) {
              if (i == 0) {
 -                prev = (j == 0 ? tb->pc : 0);
 +                prev = (j == 0 ? tb_pc(tb) : 0);
              } else {
                  prev = tcg_ctx->gen_insn_data[i - 1][j];
              }
@@ -XXX,XX +XXX,XX @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
  static int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
                                       uintptr_t searched_pc, bool reset_icount)
  {
 -    target_ulong data[TARGET_INSN_START_WORDS] = { tb->pc };
 +    target_ulong data[TARGET_INSN_START_WORDS] = { tb_pc(tb) };
      uintptr_t host_pc = (uintptr_t)tb->tc.ptr;
      CPUArchState *env = cpu->env_ptr;
      const uint8_t *p = tb->tc.ptr + tb->tc.size;
@@ -XXX,XX +XXX,XX @@ static bool tb_cmp(const void *ap, const void *bp)
      const TranslationBlock *a = ap;
      const TranslationBlock *b = bp;
 -    return a->pc == b->pc &&
 +    return tb_pc(a) == tb_pc(b) &&
          a->cs_base == b->cs_base &&
          a->flags == b->flags &&
          (tb_cflags(a) & ~CF_INVALID) == (tb_cflags(b) & ~CF_INVALID) &&
@@ -XXX,XX +XXX,XX @@ static void do_tb_invalidate_check(void *p, uint32_t hash, void *userp)
      TranslationBlock *tb = p;
      target_ulong addr = *(target_ulong *)userp;
 -    if (!(addr + TARGET_PAGE_SIZE <= tb->pc || addr >= tb->pc + tb->size)) {
 +    if (!(addr + TARGET_PAGE_SIZE <= tb_pc(tb) ||
 +          addr >= tb_pc(tb) + tb->size)) {
          printf("ERROR invalidate: address=" TARGET_FMT_lx
 -               " PC=%08lx size=%04x\n", addr, (long)tb->pc, tb->size);
 +               " PC=%08lx size=%04x\n", addr, (long)tb_pc(tb), tb->size);
      }
  }
@@ -XXX,XX +XXX,XX @@ static void do_tb_page_check(void *p, uint32_t hash, void *userp)
      TranslationBlock *tb = p;
      int flags1, flags2;
 -    flags1 = page_get_flags(tb->pc);
 -    flags2 = page_get_flags(tb->pc + tb->size - 1);
 +    flags1 = page_get_flags(tb_pc(tb));
 +    flags2 = page_get_flags(tb_pc(tb) + tb->size - 1);
      if ((flags1 & PAGE_WRITE) || (flags2 & PAGE_WRITE)) {
          printf("ERROR page flags: PC=%08lx size=%04x f1=%x f2=%x\n",
 -               (long)tb->pc, tb->size, flags1, flags2);
 +               (long)tb_pc(tb), tb->size, flags1, flags2);
      }
  }
@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
      /* remove the TB from the hash list */
      phys_pc = tb->page_addr[0];
 -    h = tb_hash_func(phys_pc, tb->pc, tb->flags, orig_cflags,
 +    h = tb_hash_func(phys_pc, tb_pc(tb), tb->flags, orig_cflags,
                       tb->trace_vcpu_dstate);
      if (!qht_remove(&tb_ctx.htable, tb, h)) {
          return;
@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
      }
      /* add in the hash table */
 -    h = tb_hash_func(phys_pc, tb->pc, tb->flags, tb->cflags,
 +    h = tb_hash_func(phys_pc, tb_pc(tb), tb->flags, tb->cflags,
                       tb->trace_vcpu_dstate);
      qht_insert(&tb_ctx.htable, tb, h, &existing_tb);
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
      tcg_ctx->cpu = NULL;
      max_insns = tb->icount;
 -    trace_translate_block(tb, tb->pc, tb->tc.ptr);
 +    trace_translate_block(tb, pc, tb->tc.ptr);
      /* generate machine code */
      tb->jmp_reset_offset[0] = TB_JMP_RESET_OFFSET_INVALID;
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
      ti = profile_getclock();
  #endif
 -    gen_code_size = tcg_gen_code(tcg_ctx, tb);
 +    gen_code_size = tcg_gen_code(tcg_ctx, tb, pc);
      if (unlikely(gen_code_size < 0)) {
   error_return:
          switch (gen_code_size) {
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
  #ifdef DEBUG_DISAS
      if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM) &&
 -        qemu_log_in_addr_range(tb->pc)) {
 +        qemu_log_in_addr_range(pc)) {
          FILE *logfile = qemu_log_trylock();
          if (logfile) {
              int code_size, data_size;
@@ -XXX,XX +XXX,XX @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
       */
      cpu->cflags_next_tb = curr_cflags(cpu) | CF_MEMI_ONLY | CF_LAST_IO | n;
 -    qemu_log_mask_and_addr(CPU_LOG_EXEC, tb->pc,
 -                           "cpu_io_recompile: rewound execution of TB to "
 -                           TARGET_FMT_lx "\n", tb->pc);
 +    if (qemu_loglevel_mask(CPU_LOG_EXEC)) {
 +        target_ulong pc = log_pc(cpu, tb);
 +        if (qemu_log_in_addr_range(pc)) {
 +            qemu_log("cpu_io_recompile: rewound execution of TB to "
 +                     TARGET_FMT_lx "\n", pc);
 +        }
 +    }
      cpu_loop_exit_noexc(cpu);
  }
 diff --git a/target/arm/cpu.c b/target/arm/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/arm/cpu.c
 +++ b/target/arm/cpu.c
@@ -XXX,XX +XXX,XX @@ void arm_cpu_synchronize_from_tb(CPUState *cs,
       * never possible for an AArch64 TB to chain to an AArch32 TB.
       */
      if (is_a64(env)) {
 -        env->pc = tb->pc;
 +        env->pc = tb_pc(tb);
      } else {
 -        env->regs[15] = tb->pc;
 +        env->regs[15] = tb_pc(tb);
      }
  }
  #endif /* CONFIG_TCG */
 diff --git a/target/avr/cpu.c b/target/avr/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/avr/cpu.c
 +++ b/target/avr/cpu.c
@@ -XXX,XX +XXX,XX @@ static void avr_cpu_synchronize_from_tb(CPUState *cs,
      AVRCPU *cpu = AVR_CPU(cs);
      CPUAVRState *env = &cpu->env;
 -    env->pc_w = tb->pc / 2; /* internally PC points to words */
 +    env->pc_w = tb_pc(tb) / 2; /* internally PC points to words */
  }
  static void avr_cpu_reset(DeviceState *ds)
 diff --git a/target/hexagon/cpu.c b/target/hexagon/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/hexagon/cpu.c
 +++ b/target/hexagon/cpu.c
@@ -XXX,XX +XXX,XX @@ static void hexagon_cpu_synchronize_from_tb(CPUState *cs,
  {
      HexagonCPU *cpu = HEXAGON_CPU(cs);
      CPUHexagonState *env = &cpu->env;
 -    env->gpr[HEX_REG_PC] = tb->pc;
 +    env->gpr[HEX_REG_PC] = tb_pc(tb);
  }
  static bool hexagon_cpu_has_work(CPUState *cs)
 diff --git a/target/hppa/cpu.c b/target/hppa/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/hppa/cpu.c
 +++ b/target/hppa/cpu.c
@@ -XXX,XX +XXX,XX @@ static void hppa_cpu_synchronize_from_tb(CPUState *cs,
      HPPACPU *cpu = HPPA_CPU(cs);
  #ifdef CONFIG_USER_ONLY
 -    cpu->env.iaoq_f = tb->pc;
 +    cpu->env.iaoq_f = tb_pc(tb);
      cpu->env.iaoq_b = tb->cs_base;
  #else
      /* Recover the IAOQ values from the GVA + PRIV.  */
@@ -XXX,XX +XXX,XX @@ static void hppa_cpu_synchronize_from_tb(CPUState *cs,
      int32_t diff = cs_base;
      cpu->env.iasq_f = iasq_f;
 -    cpu->env.iaoq_f = (tb->pc & ~iasq_f) + priv;
 +    cpu->env.iaoq_f = (tb_pc(tb) & ~iasq_f) + priv;
      if (diff) {
          cpu->env.iaoq_b = cpu->env.iaoq_f + diff;
      }
 diff --git a/target/i386/tcg/tcg-cpu.c b/target/i386/tcg/tcg-cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/i386/tcg/tcg-cpu.c
 +++ b/target/i386/tcg/tcg-cpu.c
@@ -XXX,XX +XXX,XX @@ static void x86_cpu_synchronize_from_tb(CPUState *cs,
  {
      X86CPU *cpu = X86_CPU(cs);
 -    cpu->env.eip = tb->pc - tb->cs_base;
 +    cpu->env.eip = tb_pc(tb) - tb->cs_base;
  }
  #ifndef CONFIG_USER_ONLY
 diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/loongarch/cpu.c
 +++ b/target/loongarch/cpu.c
@@ -XXX,XX +XXX,XX @@ static void loongarch_cpu_synchronize_from_tb(CPUState *cs,
      LoongArchCPU *cpu = LOONGARCH_CPU(cs);
      CPULoongArchState *env = &cpu->env;
 -    env->pc = tb->pc;
 +    env->pc = tb_pc(tb);
  }
  #endif /* CONFIG_TCG */
 diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/microblaze/cpu.c
 +++ b/target/microblaze/cpu.c
@@ -XXX,XX +XXX,XX @@ static void mb_cpu_synchronize_from_tb(CPUState *cs,
  {
      MicroBlazeCPU *cpu = MICROBLAZE_CPU(cs);
 -    cpu->env.pc = tb->pc;
 +    cpu->env.pc = tb_pc(tb);
      cpu->env.iflags = tb->flags & IFLAGS_TB_MASK;
  }
 diff --git a/target/mips/tcg/exception.c b/target/mips/tcg/exception.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/mips/tcg/exception.c
 +++ b/target/mips/tcg/exception.c
@@ -XXX,XX +XXX,XX @@ void mips_cpu_synchronize_from_tb(CPUState *cs, const TranslationBlock *tb)
      MIPSCPU *cpu = MIPS_CPU(cs);
      CPUMIPSState *env = &cpu->env;
 -    env->active_tc.PC = tb->pc;
 +    env->active_tc.PC = tb_pc(tb);
      env->hflags &= ~MIPS_HFLAG_BMASK;
      env->hflags |= tb->flags & MIPS_HFLAG_BMASK;
  }
 diff --git a/target/mips/tcg/sysemu/special_helper.c b/target/mips/tcg/sysemu/special_helper.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/mips/tcg/sysemu/special_helper.c
 +++ b/target/mips/tcg/sysemu/special_helper.c
@@ -XXX,XX +XXX,XX @@ bool mips_io_recompile_replay_branch(CPUState *cs, const TranslationBlock *tb)
      CPUMIPSState *env = &cpu->env;
      if ((env->hflags & MIPS_HFLAG_BMASK) != 0
 -        && env->active_tc.PC != tb->pc) {
 +        && env->active_tc.PC != tb_pc(tb)) {
          env->active_tc.PC -= (env->hflags & MIPS_HFLAG_B16 ? 2 : 4);
          env->hflags &= ~MIPS_HFLAG_BMASK;
          return true;
 diff --git a/target/openrisc/cpu.c b/target/openrisc/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/openrisc/cpu.c
 +++ b/target/openrisc/cpu.c
@@ -XXX,XX +XXX,XX @@ static void openrisc_cpu_synchronize_from_tb(CPUState *cs,
  {
      OpenRISCCPU *cpu = OPENRISC_CPU(cs);
 -    cpu->env.pc = tb->pc;
 +    cpu->env.pc = tb_pc(tb);
  }
 diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/riscv/cpu.c
 +++ b/target/riscv/cpu.c
@@ -XXX,XX +XXX,XX @@ static void riscv_cpu_synchronize_from_tb(CPUState *cs,
      RISCVMXL xl = FIELD_EX32(tb->flags, TB_FLAGS, XL);
      if (xl == MXL_RV32) {
 -        env->pc = (int32_t)tb->pc;
 +        env->pc = (int32_t)tb_pc(tb);
      } else {
 -        env->pc = tb->pc;
 +        env->pc = tb_pc(tb);
      }
  }
 diff --git a/target/rx/cpu.c b/target/rx/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/rx/cpu.c
 +++ b/target/rx/cpu.c
@@ -XXX,XX +XXX,XX @@ static void rx_cpu_synchronize_from_tb(CPUState *cs,
  {
      RXCPU *cpu = RX_CPU(cs);
 -    cpu->env.pc = tb->pc;
 +    cpu->env.pc = tb_pc(tb);
  }
  static bool rx_cpu_has_work(CPUState *cs)
 diff --git a/target/sh4/cpu.c b/target/sh4/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/sh4/cpu.c
 +++ b/target/sh4/cpu.c
@@ -XXX,XX +XXX,XX @@ static void superh_cpu_synchronize_from_tb(CPUState *cs,
  {
      SuperHCPU *cpu = SUPERH_CPU(cs);
 -    cpu->env.pc = tb->pc;
 +    cpu->env.pc = tb_pc(tb);
      cpu->env.flags = tb->flags & TB_FLAG_ENVFLAGS_MASK;
  }
@@ -XXX,XX +XXX,XX @@ static bool superh_io_recompile_replay_branch(CPUState *cs,
      CPUSH4State *env = &cpu->env;
      if ((env->flags & ((DELAY_SLOT | DELAY_SLOT_CONDITIONAL))) != 0
 -        && env->pc != tb->pc) {
 +        && env->pc != tb_pc(tb)) {
          env->pc -= 2;
          env->flags &= ~(DELAY_SLOT | DELAY_SLOT_CONDITIONAL);
          return true;
 diff --git a/target/sparc/cpu.c b/target/sparc/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/sparc/cpu.c
 +++ b/target/sparc/cpu.c
@@ -XXX,XX +XXX,XX @@ static void sparc_cpu_synchronize_from_tb(CPUState *cs,
  {
      SPARCCPU *cpu = SPARC_CPU(cs);
 -    cpu->env.pc = tb->pc;
 +    cpu->env.pc = tb_pc(tb);
      cpu->env.npc = tb->cs_base;
  }
 diff --git a/target/tricore/cpu.c b/target/tricore/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/tricore/cpu.c
 +++ b/target/tricore/cpu.c
@@ -XXX,XX +XXX,XX @@ static void tricore_cpu_synchronize_from_tb(CPUState *cs,
      TriCoreCPU *cpu = TRICORE_CPU(cs);
      CPUTriCoreState *env = &cpu->env;
 -    env->PC = tb->pc;
 +    env->PC = tb_pc(tb);
  }
  static void tricore_cpu_reset(DeviceState *dev)
 diff --git a/tcg/tcg.c b/tcg/tcg.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg.c
 +++ b/tcg/tcg.c
-@@ -XXX,XX +XXX,XX @@ int64_t tcg_cpu_exec_time(void)
+@@ -XXX,XX +XXX,XX @@ static const char bswap_flag_name[][6] = {
- #endif
+     [TCG_BSWAP_IZ | TCG_BSWAP_OS] = "iz,os",
+ };
--int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
++#ifdef CONFIG_PLUGIN
-+int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start)
++static const char * const plugin_from_name[] = {
 +    "from-tb",
 +    "from-insn",
 +    "after-insn",
 +    "after-tb",
 +};
 +#endif
 +
  static inline bool tcg_regset_single(TCGRegSet d)
  {
- #ifdef CONFIG_PROFILER
+     return (d & (d - 1)) == 0;
-     TCGProfile *prof = &s->prof;
+@@ -XXX,XX +XXX,XX @@ static inline TCGReg tcg_regset_first(TCGRegSet d)
-@@ -XXX,XX +XXX,XX @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
+ #define ne_fprintf(...) \
+     ({ int ret_ = fprintf(__VA_ARGS__); ret_ >= 0 ? ret_ : 0; })
- #ifdef DEBUG_DISAS
-     if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_OP)
+-static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs)
--                 && qemu_log_in_addr_range(tb->pc))) {
++void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs)
-+                 && qemu_log_in_addr_range(pc_start))) {
+ {
-         FILE *logfile = qemu_log_trylock();
+     char buf[128];
-         if (logfile) {
+     TCGOp *op;
-             fprintf(logfile, "OP:\n");
+@@ -XXX,XX +XXX,XX @@ static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs)
-@@ -XXX,XX +XXX,XX @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
+                     i = k = 1;
-     if (s->nb_indirects > 0) {
+                 }
- #ifdef DEBUG_DISAS
+                 break;
-         if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_OP_IND)
++#ifdef CONFIG_PLUGIN
--                     && qemu_log_in_addr_range(tb->pc))) {
++            case INDEX_op_plugin_cb:
-+                     && qemu_log_in_addr_range(pc_start))) {
++                {
-             FILE *logfile = qemu_log_trylock();
++                    TCGArg from = op->args[k++];
-             if (logfile) {
++                    const char *name = NULL;
-                 fprintf(logfile, "OP before indirect lowering:\n");
++
-@@ -XXX,XX +XXX,XX @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
++                    if (from < ARRAY_SIZE(plugin_from_name)) {
++                        name = plugin_from_name[from];
- #ifdef DEBUG_DISAS
++                    }
-     if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_OP_OPT)
++                    if (name) {
--                 && qemu_log_in_addr_range(tb->pc))) {
++                        col += ne_fprintf(f, "%s", name);
-+                 && qemu_log_in_addr_range(pc_start))) {
++                    } else {
-         FILE *logfile = qemu_log_trylock();
++                        col += ne_fprintf(f, "$0x%" TCG_PRIlx, from);
-         if (logfile) {
++                    }
-             fprintf(logfile, "OP after optimization and liveness analysis:\n");
++                    i = 1;
 +                }
 +                break;
 +#endif
              default:
                  i = 0;
                  break;
 diff --git a/util/log.c b/util/log.c
 index XXXXXXX..XXXXXXX 100644
 --- a/util/log.c
 +++ b/util/log.c
@@ -XXX,XX +XXX,XX @@ const QEMULogItem qemu_log_items[] = {
        "show micro ops after optimization" },
      { CPU_LOG_TB_OP_IND, "op_ind",
        "show micro ops before indirect lowering" },
 +#ifdef CONFIG_PLUGIN
 +    { LOG_TB_OP_PLUGIN, "op_plugin",
 +      "show micro ops before plugin injection" },
 +#endif
      { CPU_LOG_INT, "int",
        "show interrupts/exceptions in short format" },
      { CPU_LOG_EXEC, "exec",
 --
 .34.1

-[PULL 18/20] accel/tcg: Introduce TARGET_TB_PCREL
+[PULL 17/20] plugins: Split out common cb expanders
-Prepare for targets to be able to produce TBs that can
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 run in more than one virtual context.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/internal.h      |  4 +++
+ accel/tcg/plugin-gen.c | 84 +++++++++++++++++++++---------------------
- accel/tcg/tb-jmp-cache.h  | 41 +++++++++++++++++++++++++
+file changed, 41 insertions(+), 43 deletions(-)
  include/exec/cpu-defs.h   |  3 ++
  include/exec/exec-all.h   | 32 ++++++++++++++++++--
  accel/tcg/cpu-exec.c      | 16 ++++++----
  accel/tcg/translate-all.c | 64 ++++++++++++++++++++++++++-------------
 files changed, 131 insertions(+), 29 deletions(-)
-diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
+diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/internal.h
+--- a/accel/tcg/plugin-gen.c
-+++ b/accel/tcg/internal.h
++++ b/accel/tcg/plugin-gen.c
-@@ -XXX,XX +XXX,XX @@ void tb_htable_init(void);
+@@ -XXX,XX +XXX,XX @@ static void gen_mem_cb(struct qemu_plugin_dyn_cb *cb,
- /* Return the current PC from CPU, which may be cached in TB. */
+     tcg_temp_free_i32(cpu_index);
  static inline target_ulong log_pc(CPUState *cpu, const TranslationBlock *tb)
  {
 +#if TARGET_TB_PCREL
 +    return cpu->cc->get_pc(cpu);
 +#else
      return tb_pc(tb);
 +#endif
  }
- #endif /* ACCEL_TCG_INTERNAL_H */
++static void inject_cb(struct qemu_plugin_dyn_cb *cb)
-diff --git a/accel/tcg/tb-jmp-cache.h b/accel/tcg/tb-jmp-cache.h
++
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tb-jmp-cache.h
 +++ b/accel/tcg/tb-jmp-cache.h
@@ -XXX,XX +XXX,XX @@
  /*
   * Accessed in parallel; all accesses to 'tb' must be atomic.
 + * For TARGET_TB_PCREL, accesses to 'pc' must be protected by
 + * a load_acquire/store_release to 'tb'.
   */
  struct CPUJumpCache {
      struct {
          TranslationBlock *tb;
 +#if TARGET_TB_PCREL
 +        target_ulong pc;
 +#endif
      } array[TB_JMP_CACHE_SIZE];
  };
 +static inline TranslationBlock *
 +tb_jmp_cache_get_tb(CPUJumpCache *jc, uint32_t hash)
 +{
-+#if TARGET_TB_PCREL
++    switch (cb->type) {
-+    /* Use acquire to ensure current load of pc from jc. */
++    case PLUGIN_CB_REGULAR:
-+    return qatomic_load_acquire(&jc->array[hash].tb);
++        gen_udata_cb(cb);
-+#else
++        break;
-+    /* Use rcu_read to ensure current load of pc from *tb. */
++    case PLUGIN_CB_INLINE:
-+    return qatomic_rcu_read(&jc->array[hash].tb);
++        gen_inline_cb(cb);
-+#endif
++        break;
 +    default:
 +        g_assert_not_reached();
 +    }
 +}
 +
-+static inline target_ulong
++static void inject_mem_cb(struct qemu_plugin_dyn_cb *cb,
-+tb_jmp_cache_get_pc(CPUJumpCache *jc, uint32_t hash, TranslationBlock *tb)
++                          enum qemu_plugin_mem_rw rw,
 +                          qemu_plugin_meminfo_t meminfo, TCGv_i64 addr)
 +{
-+#if TARGET_TB_PCREL
++    if (cb->rw & rw) {
-+    return jc->array[hash].pc;
++        switch (cb->type) {
-+#else
++        case PLUGIN_CB_MEM_REGULAR:
-+    return tb_pc(tb);
++            gen_mem_cb(cb, meminfo, addr);
-+#endif
++            break;
-+}
++        default:
-+
++            inject_cb(cb);
-+static inline void
++            break;
 +tb_jmp_cache_set(CPUJumpCache *jc, uint32_t hash,
 +                 TranslationBlock *tb, target_ulong pc)
 +{
 +#if TARGET_TB_PCREL
 +    jc->array[hash].pc = pc;
 +    /* Use store_release on tb to ensure pc is written first. */
 +    qatomic_store_release(&jc->array[hash].tb, tb);
 +#else
 +    /* Use the pc value already stored in tb->pc. */
 +    qatomic_set(&jc->array[hash].tb, tb);
 +#endif
 +}
 +
  #endif /* ACCEL_TCG_TB_JMP_CACHE_H */
 diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/exec/cpu-defs.h
 +++ b/include/exec/cpu-defs.h
@@ -XXX,XX +XXX,XX @@
  #  error TARGET_PAGE_BITS must be defined in cpu-param.h
  # endif
  #endif
 +#ifndef TARGET_TB_PCREL
 +# define TARGET_TB_PCREL 0
 +#endif
  #define TARGET_LONG_SIZE (TARGET_LONG_BITS / 8)
 diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/exec/exec-all.h
 +++ b/include/exec/exec-all.h
@@ -XXX,XX +XXX,XX @@ struct tb_tc {
  };
  struct TranslationBlock {
 -    target_ulong pc;   /* simulated PC corresponding to this block (EIP + CS base) */
 -    target_ulong cs_base; /* CS base for this block */
 +#if !TARGET_TB_PCREL
 +    /*
 +     * Guest PC corresponding to this block.  This must be the true
 +     * virtual address.  Therefore e.g. x86 stores EIP + CS_BASE, and
 +     * targets like Arm, MIPS, HP-PA, which reuse low bits for ISA or
 +     * privilege, must store those bits elsewhere.
 +     *
 +     * If TARGET_TB_PCREL, the opcodes for the TranslationBlock are
 +     * written such that the TB is associated only with the physical
 +     * page and may be run in any virtual address context.  In this case,
 +     * PC must always be taken from ENV in a target-specific manner.
 +     * Unwind information is taken as offsets from the page, to be
 +     * deposited into the "current" PC.
 +     */
 +    target_ulong pc;
 +#endif
 +
 +    /*
 +     * Target-specific data associated with the TranslationBlock, e.g.:
 +     * x86: the original user, the Code Segment virtual base,
 +     * arm: an extension of tb->flags,
 +     * s390x: instruction data for EXECUTE,
 +     * sparc: the next pc of the instruction queue (for delay slots).
 +     */
 +    target_ulong cs_base;
 +
      uint32_t flags; /* flags defining in which context the code was generated */
      uint32_t cflags;    /* compile flags */
@@ -XXX,XX +XXX,XX @@ struct TranslationBlock {
  /* Hide the read to avoid ifdefs for TARGET_TB_PCREL. */
  static inline target_ulong tb_pc(const TranslationBlock *tb)
  {
 +#if TARGET_TB_PCREL
 +    qemu_build_not_reached();
 +#else
      return tb->pc;
 +#endif
  }
  /* Hide the qatomic_read to make code a little easier on the eyes */
 diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cpu-exec.c
 +++ b/accel/tcg/cpu-exec.c
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
      const TranslationBlock *tb = p;
      const struct tb_desc *desc = d;
 -    if (tb_pc(tb) == desc->pc &&
 +    if ((TARGET_TB_PCREL || tb_pc(tb) == desc->pc) &&
          tb->page_addr[0] == desc->page_addr0 &&
          tb->cs_base == desc->cs_base &&
          tb->flags == desc->flags &&
@@ -XXX,XX +XXX,XX @@ static TranslationBlock *tb_htable_lookup(CPUState *cpu, target_ulong pc,
          return NULL;
      }
      desc.page_addr0 = phys_pc;
 -    h = tb_hash_func(phys_pc, pc, flags, cflags, *cpu->trace_dstate);
 +    h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : pc),
 +                     flags, cflags, *cpu->trace_dstate);
      return qht_lookup_custom(&tb_ctx.htable, &desc, h, tb_lookup_cmp);
  }
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
                                            uint32_t flags, uint32_t cflags)
  {
      TranslationBlock *tb;
 +    CPUJumpCache *jc;
      uint32_t hash;
      /* we should never be trying to look up an INVALID tb */
      tcg_debug_assert(!(cflags & CF_INVALID));
      hash = tb_jmp_cache_hash_func(pc);
 -    tb = qatomic_rcu_read(&cpu->tb_jmp_cache->array[hash].tb);
 +    jc = cpu->tb_jmp_cache;
 +    tb = tb_jmp_cache_get_tb(jc, hash);
      if (likely(tb &&
 -               tb->pc == pc &&
 +               tb_jmp_cache_get_pc(jc, hash, tb) == pc &&
                 tb->cs_base == cs_base &&
                 tb->flags == flags &&
                 tb->trace_vcpu_dstate == *cpu->trace_dstate &&
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
      if (tb == NULL) {
          return NULL;
      }
 -    qatomic_set(&cpu->tb_jmp_cache->array[hash].tb, tb);
 +    tb_jmp_cache_set(jc, hash, tb, pc);
      return tb;
  }
@@ -XXX,XX +XXX,XX @@ cpu_tb_exec(CPUState *cpu, TranslationBlock *itb, int *tb_exit)
          if (cc->tcg_ops->synchronize_from_tb) {
              cc->tcg_ops->synchronize_from_tb(cpu, last_tb);
          } else {
 +            assert(!TARGET_TB_PCREL);
              assert(cc->set_pc);
              cc->set_pc(cpu, tb_pc(last_tb));
          }
@@ -XXX,XX +XXX,XX @@ int cpu_exec(CPUState *cpu)
                   * for the fast lookup
                   */
                  h = tb_jmp_cache_hash_func(pc);
 -                qatomic_set(&cpu->tb_jmp_cache->array[h].tb, tb);
 +                tb_jmp_cache_set(cpu->tb_jmp_cache, h, tb, pc);
              }
  #ifndef CONFIG_USER_ONLY
 diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/translate-all.c
 +++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
          for (j = 0; j < TARGET_INSN_START_WORDS; ++j) {
              if (i == 0) {
 -                prev = (j == 0 ? tb_pc(tb) : 0);
 +                prev = (!TARGET_TB_PCREL && j == 0 ? tb_pc(tb) : 0);
              } else {
                  prev = tcg_ctx->gen_insn_data[i - 1][j];
              }
@@ -XXX,XX +XXX,XX @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
  static int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
                                       uintptr_t searched_pc, bool reset_icount)
  {
 -    target_ulong data[TARGET_INSN_START_WORDS] = { tb_pc(tb) };
 +    target_ulong data[TARGET_INSN_START_WORDS];
      uintptr_t host_pc = (uintptr_t)tb->tc.ptr;
      CPUArchState *env = cpu->env_ptr;
      const uint8_t *p = tb->tc.ptr + tb->tc.size;
@@ -XXX,XX +XXX,XX @@ static int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
          return -1;
      }
 +    memset(data, 0, sizeof(data));
 +    if (!TARGET_TB_PCREL) {
 +        data[0] = tb_pc(tb);
 +    }
 +
      /* Reconstruct the stored insn data while looking for the point at
         which the end of the insn exceeds the searched_pc.  */
      for (i = 0; i < num_insns; ++i) {
@@ -XXX,XX +XXX,XX @@ static bool tb_cmp(const void *ap, const void *bp)
      const TranslationBlock *a = ap;
      const TranslationBlock *b = bp;
 -    return tb_pc(a) == tb_pc(b) &&
 -        a->cs_base == b->cs_base &&
 -        a->flags == b->flags &&
 -        (tb_cflags(a) & ~CF_INVALID) == (tb_cflags(b) & ~CF_INVALID) &&
 -        a->trace_vcpu_dstate == b->trace_vcpu_dstate &&
 -        a->page_addr[0] == b->page_addr[0] &&
 -        a->page_addr[1] == b->page_addr[1];
 +    return ((TARGET_TB_PCREL || tb_pc(a) == tb_pc(b)) &&
 +            a->cs_base == b->cs_base &&
 +            a->flags == b->flags &&
 +            (tb_cflags(a) & ~CF_INVALID) == (tb_cflags(b) & ~CF_INVALID) &&
 +            a->trace_vcpu_dstate == b->trace_vcpu_dstate &&
 +            a->page_addr[0] == b->page_addr[0] &&
 +            a->page_addr[1] == b->page_addr[1]);
  }
  void tb_htable_init(void)
@@ -XXX,XX +XXX,XX @@ static inline void tb_jmp_unlink(TranslationBlock *dest)
      qemu_spin_unlock(&dest->jmp_lock);
  }
 +static void tb_jmp_cache_inval_tb(TranslationBlock *tb)
 +{
 +    CPUState *cpu;
 +
 +    if (TARGET_TB_PCREL) {
 +        /* A TB may be at any virtual address */
 +        CPU_FOREACH(cpu) {
 +            tcg_flush_jmp_cache(cpu);
 +        }
 +    } else {
 +        uint32_t h = tb_jmp_cache_hash_func(tb_pc(tb));
 +
 +        CPU_FOREACH(cpu) {
 +            CPUJumpCache *jc = cpu->tb_jmp_cache;
 +
 +            if (qatomic_read(&jc->array[h].tb) == tb) {
 +                qatomic_set(&jc->array[h].tb, NULL);
 +            }
 +        }
 +    }
 +}
 +
- /*
+ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
   * In user-mode, call with mmap_lock held.
   * In !user-mode, if @rm_from_page_list is set, call with the TB's pages'
@@ -XXX,XX +XXX,XX @@ static inline void tb_jmp_unlink(TranslationBlock *dest)
   */
  static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
  {
--    CPUState *cpu;
+     TCGOp *op, *next;
-     PageDesc *p;
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
-     uint32_t h;
-     tb_page_addr_t phys_pc;
+                 cbs = plugin_tb->cbs;
-@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
+                 for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
+-                    struct qemu_plugin_dyn_cb *cb =
-     /* remove the TB from the hash list */
+-                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-     phys_pc = tb->page_addr[0];
+-
--    h = tb_hash_func(phys_pc, tb_pc(tb), tb->flags, orig_cflags,
+-                    switch (cb->type) {
--                     tb->trace_vcpu_dstate);
+-                    case PLUGIN_CB_REGULAR:
-+    h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : tb_pc(tb)),
+-                        gen_udata_cb(cb);
-+                     tb->flags, orig_cflags, tb->trace_vcpu_dstate);
+-                        break;
-     if (!qht_remove(&tb_ctx.htable, tb, h)) {
+-                    case PLUGIN_CB_INLINE:
-         return;
+-                        gen_inline_cb(cb);
-     }
+-                        break;
-@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
+-                    default:
-     }
+-                        g_assert_not_reached();
+-                    }
-     /* remove the TB from the hash list */
++                    inject_cb(
--    h = tb_jmp_cache_hash_func(tb->pc);
++                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i));
--    CPU_FOREACH(cpu) {
+                 }
--        CPUJumpCache *jc = cpu->tb_jmp_cache;
+                 break;
--        if (qatomic_read(&jc->array[h].tb) == tb) {
--            qatomic_set(&jc->array[h].tb, NULL);
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
--        }
--    }
+                 cbs = insn->insn_cbs;
-+    tb_jmp_cache_inval_tb(tb);
+                 for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
+-                    struct qemu_plugin_dyn_cb *cb =
-     /* suppress this TB from the two jump lists */
+-                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-     tb_remove_from_jmp_list(tb, 0);
+-
-@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
+-                    switch (cb->type) {
-     }
+-                    case PLUGIN_CB_REGULAR:
+-                        gen_udata_cb(cb);
-     /* add in the hash table */
+-                        break;
--    h = tb_hash_func(phys_pc, tb_pc(tb), tb->flags, tb->cflags,
+-                    case PLUGIN_CB_INLINE:
--                     tb->trace_vcpu_dstate);
+-                        gen_inline_cb(cb);
-+    h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : tb_pc(tb)),
+-                        break;
-+                     tb->flags, tb->cflags, tb->trace_vcpu_dstate);
+-                    default:
-     qht_insert(&tb_ctx.htable, tb, h, &existing_tb);
+-                        g_assert_not_reached();
+-                    }
-     /* remove TB from the page(s) if we couldn't insert it */
++                    inject_cb(
-@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
++                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i));
+                 }
-     gen_code_buf = tcg_ctx->code_gen_ptr;
+                 break;
-     tb->tc.ptr = tcg_splitwx_to_rx(gen_code_buf);
-+#if !TARGET_TB_PCREL
+@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
-     tb->pc = pc;
+         {
-+#endif
+             TCGv_i64 addr = temp_tcgv_i64(arg_temp(op->args[0]));
-     tb->cs_base = cs_base;
+             qemu_plugin_meminfo_t meminfo = op->args[1];
-     tb->flags = flags;
++            enum qemu_plugin_mem_rw rw =
-     tb->cflags = cflags;
++                (qemu_plugin_mem_is_store(meminfo)
 +                 ? QEMU_PLUGIN_MEM_W : QEMU_PLUGIN_MEM_R);
              struct qemu_plugin_insn *insn;
              const GArray *cbs;
 -            int i, n, rw;
 +            int i, n;
              assert(insn_idx >= 0);
              insn = g_ptr_array_index(plugin_tb->insns, insn_idx);
 -            rw = qemu_plugin_mem_is_store(meminfo) ? 2 : 1;
              tcg_ctx->emit_before_op = op;
              cbs = insn->mem_cbs;
              for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
 -                struct qemu_plugin_dyn_cb *cb =
 -                    &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
 -
 -                if (cb->rw & rw) {
 -                    switch (cb->type) {
 -                    case PLUGIN_CB_MEM_REGULAR:
 -                        gen_mem_cb(cb, meminfo, addr);
 -                        break;
 -                    case PLUGIN_CB_INLINE:
 -                        gen_inline_cb(cb);
 -                        break;
 -                    default:
 -                        g_assert_not_reached();
 -                    }
 -                }
 +                inject_mem_cb(&g_array_index(cbs, struct qemu_plugin_dyn_cb, i),
 +                              rw, meminfo, addr);
              }
              tcg_ctx->emit_before_op = NULL;
 --
 .34.1

-[PULL 14/20] accel/tcg: Inline tb_flush_jmp_cache
+[PULL 18/20] plugins: Merge qemu_plugin_tb_insn_get to plugin-gen.c
-This function has two users, who use it incompatibly.
+Merge qemu_plugin_insn_alloc and qemu_plugin_tb_insn_get into
-In tlb_flush_page_by_mmuidx_async_0, when flushing a
+plugin_gen_insn_start, since it is used nowhere else.
 single page, we need to flush exactly two pages.
 In tlb_flush_range_by_mmuidx_async_0, when flushing a
 range of pages, we need to flush N+1 pages.
-This avoids double-flushing of jmp cache pages in a range.
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/cputlb.c | 25 ++++++++++++++-----------
+ include/qemu/plugin.h  | 39 ---------------------------------------
-file changed, 14 insertions(+), 11 deletions(-)
+ accel/tcg/plugin-gen.c | 39 ++++++++++++++++++++++++++++++++-------
 files changed, 32 insertions(+), 46 deletions(-)
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
+diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
+--- a/include/qemu/plugin.h
-+++ b/accel/tcg/cputlb.c
++++ b/include/qemu/plugin.h
-@@ -XXX,XX +XXX,XX @@ static void tb_jmp_cache_clear_page(CPUState *cpu, target_ulong page_addr)
+@@ -XXX,XX +XXX,XX @@ static inline void qemu_plugin_insn_cleanup_fn(gpointer data)
-     }
+     g_byte_array_free(insn->data, true);
  }
--static void tb_flush_jmp_cache(CPUState *cpu, target_ulong addr)
+-static inline struct qemu_plugin_insn *qemu_plugin_insn_alloc(void)
 -{
--    /* Discard jump cache entries for any tb which might potentially
+-    struct qemu_plugin_insn *insn = g_new0(struct qemu_plugin_insn, 1);
--       overlap the flushed page.  */
+-
--    tb_jmp_cache_clear_page(cpu, addr - TARGET_PAGE_SIZE);
+-    insn->data = g_byte_array_sized_new(4);
--    tb_jmp_cache_clear_page(cpu, addr);
+-    return insn;
 -}
 -
  /* Internal context for this TranslationBlock */
  struct qemu_plugin_tb {
      GPtrArray *insns;
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_tb {
      GArray *cbs;
  };
 -/**
 - * qemu_plugin_tb_insn_get(): get next plugin record for translation.
 - * @tb: the internal tb context
 - * @pc: address of instruction
 - */
 -static inline
 -struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb,
 -                                                 uint64_t pc)
 -{
 -    struct qemu_plugin_insn *insn;
 -
 -    if (unlikely(tb->n == tb->insns->len)) {
 -        struct qemu_plugin_insn *new_insn = qemu_plugin_insn_alloc();
 -        g_ptr_array_add(tb->insns, new_insn);
 -    }
 -
 -    insn = g_ptr_array_index(tb->insns, tb->n++);
 -    g_byte_array_set_size(insn->data, 0);
 -    insn->calls_helpers = false;
 -    insn->mem_helper = false;
 -    insn->vaddr = pc;
 -    if (insn->insn_cbs) {
 -        g_array_set_size(insn->insn_cbs, 0);
 -    }
 -    if (insn->mem_cbs) {
 -        g_array_set_size(insn->mem_cbs, 0);
 -    }
 -
 -    return insn;
 -}
 -
  /**
-  * tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
+  * struct CPUPluginState - per-CPU state for plugins
-  * @desc: The CPUTLBDesc portion of the TLB
+  * @event_mask: plugin event bitmap. Modified only via async work.
-@@ -XXX,XX +XXX,XX @@ static void tlb_flush_page_by_mmuidx_async_0(CPUState *cpu,
+diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/plugin-gen.c
 +++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
  void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
  {
      struct qemu_plugin_tb *ptb = tcg_ctx->plugin_tb;
 -    struct qemu_plugin_insn *pinsn;
 +    struct qemu_plugin_insn *insn;
 +    size_t n = db->num_insns;
 +    vaddr pc;
 -    pinsn = qemu_plugin_tb_insn_get(ptb, db->pc_next);
 -    tcg_ctx->plugin_insn = pinsn;
 -    plugin_gen_empty_callback(PLUGIN_GEN_FROM_INSN);
 +    assert(n >= 1);
 +    ptb->n = n;
 +    if (n <= ptb->insns->len) {
 +        insn = g_ptr_array_index(ptb->insns, n - 1);
 +        g_byte_array_set_size(insn->data, 0);
 +    } else {
 +        assert(n - 1 == ptb->insns->len);
 +        insn = g_new0(struct qemu_plugin_insn, 1);
 +        insn->data = g_byte_array_sized_new(4);
 +        g_ptr_array_add(ptb->insns, insn);
 +    }
 +
 +    tcg_ctx->plugin_insn = insn;
 +    insn->calls_helpers = false;
 +    insn->mem_helper = false;
 +    if (insn->insn_cbs) {
 +        g_array_set_size(insn->insn_cbs, 0);
 +    }
 +    if (insn->mem_cbs) {
 +        g_array_set_size(insn->mem_cbs, 0);
 +    }
 +
 +    pc = db->pc_next;
 +    insn->vaddr = pc;
      /*
       * Detect page crossing to get the new host address.
@@ -XXX,XX +XXX,XX @@ void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
       * fetching instructions from a region not backed by RAM.
       */
      if (ptb->haddr1 == NULL) {
 -        pinsn->haddr = NULL;
 +        insn->haddr = NULL;
      } else if (is_same_page(db, db->pc_next)) {
 -        pinsn->haddr = ptb->haddr1 + pinsn->vaddr - ptb->vaddr;
 +        insn->haddr = ptb->haddr1 + pc - ptb->vaddr;
      } else {
          if (ptb->vaddr2 == -1) {
              ptb->vaddr2 = TARGET_PAGE_ALIGN(db->pc_first);
              get_page_addr_code_hostp(cpu_env(cpu), ptb->vaddr2, &ptb->haddr2);
          }
 -        pinsn->haddr = ptb->haddr2 + pinsn->vaddr - ptb->vaddr2;
 +        insn->haddr = ptb->haddr2 + pc - ptb->vaddr2;
      }
-     qemu_spin_unlock(&env_tlb(env)->c.lock);
++
++    plugin_gen_empty_callback(PLUGIN_GEN_FROM_INSN);
 -    tb_flush_jmp_cache(cpu, addr);
 +    /*
 +     * Discard jump cache entries for any tb which might potentially
 +     * overlap the flushed page, which includes the previous.
 +     */
 +    tb_jmp_cache_clear_page(cpu, addr - TARGET_PAGE_SIZE);
 +    tb_jmp_cache_clear_page(cpu, addr);
  }
- /**
+ void plugin_gen_insn_end(void)
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_by_mmuidx_async_0(CPUState *cpu,
          return;
      }
 -    for (target_ulong i = 0; i < d.len; i += TARGET_PAGE_SIZE) {
 -        tb_flush_jmp_cache(cpu, d.addr + i);
 +    /*
 +     * Discard jump cache entries for any tb which might potentially
 +     * overlap the flushed pages, which includes the previous.
 +     */
 +    d.addr -= TARGET_PAGE_SIZE;
 +    for (target_ulong i = 0, n = d.len / TARGET_PAGE_SIZE + 1; i < n; i++) {
 +        tb_jmp_cache_clear_page(cpu, d.addr);
 +        d.addr += TARGET_PAGE_SIZE;
      }
  }
 --
 .34.1

-[PULL 07/20] accel/tcg: Introduce probe_access_full
+[PULL 19/20] plugins: Inline plugin_gen_empty_callback
-Add an interface to return the CPUTLBEntryFull struct
+Each caller can use tcg_gen_plugin_cb directly.
 that goes with the lookup.  The result is not intended
 to be valid across multiple lookups, so the user must
 use the results immediately.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
-Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/exec-all.h | 15 +++++++++++++
+ accel/tcg/plugin-gen.c | 19 +++----------------
- include/qemu/typedefs.h |  1 +
+file changed, 3 insertions(+), 16 deletions(-)
  accel/tcg/cputlb.c      | 47 +++++++++++++++++++++++++----------------
 files changed, 45 insertions(+), 18 deletions(-)
-diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
+diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
---- a/include/exec/exec-all.h
+--- a/accel/tcg/plugin-gen.c
-+++ b/include/exec/exec-all.h
++++ b/accel/tcg/plugin-gen.c
-@@ -XXX,XX +XXX,XX @@ int probe_access_flags(CPUArchState *env, target_ulong addr,
+@@ -XXX,XX +XXX,XX @@ enum plugin_gen_from {
-                        MMUAccessType access_type, int mmu_idx,
+     PLUGIN_GEN_AFTER_TB,
-                        bool nonfault, void **phost, uintptr_t retaddr);
+ };
-+#ifndef CONFIG_USER_ONLY
+-static void plugin_gen_empty_callback(enum plugin_gen_from from)
-+/**
+-{
-+ * probe_access_full:
+-    switch (from) {
-+ * Like probe_access_flags, except also return into @pfull.
+-    case PLUGIN_GEN_AFTER_INSN:
-+ *
+-    case PLUGIN_GEN_FROM_TB:
-+ * The CPUTLBEntryFull structure returned via @pfull is transient
+-    case PLUGIN_GEN_FROM_INSN:
-+ * and must be consumed or copied immediately, before any further
+-        tcg_gen_plugin_cb(from);
-+ * access or changes to TLB @mmu_idx.
+-        break;
-+ */
+-    default:
-+int probe_access_full(CPUArchState *env, target_ulong addr,
+-        g_assert_not_reached();
-+                      MMUAccessType access_type, int mmu_idx,
+-    }
-+                      bool nonfault, void **phost,
+-}
-+                      CPUTLBEntryFull **pfull, uintptr_t retaddr);
+-
-+#endif
+ /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
-+
+ void plugin_gen_disable_mem_helpers(void)
  #define CODE_GEN_ALIGN           16 /* must be >= of the size of a icache line */
  /* Estimated block size for TB allocation.  */
 diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/qemu/typedefs.h
 +++ b/include/qemu/typedefs.h
@@ -XXX,XX +XXX,XX @@ typedef struct ConfidentialGuestSupport ConfidentialGuestSupport;
  typedef struct CPUAddressSpace CPUAddressSpace;
  typedef struct CPUArchState CPUArchState;
  typedef struct CPUState CPUState;
 +typedef struct CPUTLBEntryFull CPUTLBEntryFull;
  typedef struct DeviceListener DeviceListener;
  typedef struct DeviceState DeviceState;
  typedef struct DirtyBitmapSnapshot DirtyBitmapSnapshot;
 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cputlb.c
 +++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
  static int probe_access_internal(CPUArchState *env, target_ulong addr,
                                   int fault_size, MMUAccessType access_type,
                                   int mmu_idx, bool nonfault,
 -                                 void **phost, uintptr_t retaddr)
 +                                 void **phost, CPUTLBEntryFull **pfull,
 +                                 uintptr_t retaddr)
  {
-     uintptr_t index = tlb_index(env, mmu_idx, addr);
+@@ -XXX,XX +XXX,XX @@ bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
-     CPUTLBEntry *entry = tlb_entry(env, mmu_idx, addr);
+         ptb->mem_only = mem_only;
-@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
+         ptb->mem_helper = false;
-                                            mmu_idx, nonfault, retaddr)) {
-                 /* Non-faulting page table read failed.  */
+-        plugin_gen_empty_callback(PLUGIN_GEN_FROM_TB);
-                 *phost = NULL;
++        tcg_gen_plugin_cb(PLUGIN_GEN_FROM_TB);
 +                *pfull = NULL;
                  return TLB_INVALID_MASK;
              }
              /* TLB resize via tlb_fill may have moved the entry.  */
 +            index = tlb_index(env, mmu_idx, addr);
              entry = tlb_entry(env, mmu_idx, addr);
              /*
@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
      }
-     flags &= tlb_addr;
+     tcg_ctx->plugin_insn = NULL;
-+    *pfull = &env_tlb(env)->d[mmu_idx].fulltlb[index];
+@@ -XXX,XX +XXX,XX @@ void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
-+
+         insn->haddr = ptb->haddr2 + pc - ptb->vaddr2;
-     /* Fold all "mmio-like" bits into TLB_MMIO.  This is not RAM.  */
+     }
-     if (unlikely(flags & ~(TLB_WATCHPOINT | TLB_NOTDIRTY))) {
-         *phost = NULL;
+-    plugin_gen_empty_callback(PLUGIN_GEN_FROM_INSN);
-@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
++    tcg_gen_plugin_cb(PLUGIN_GEN_FROM_INSN);
      return flags;
  }
--int probe_access_flags(CPUArchState *env, target_ulong addr,
+ void plugin_gen_insn_end(void)
 -                       MMUAccessType access_type, int mmu_idx,
 -                       bool nonfault, void **phost, uintptr_t retaddr)
 +int probe_access_full(CPUArchState *env, target_ulong addr,
 +                      MMUAccessType access_type, int mmu_idx,
 +                      bool nonfault, void **phost, CPUTLBEntryFull **pfull,
 +                      uintptr_t retaddr)
  {
--    int flags;
+-    plugin_gen_empty_callback(PLUGIN_GEN_AFTER_INSN);
--
++    tcg_gen_plugin_cb(PLUGIN_GEN_AFTER_INSN);
 -    flags = probe_access_internal(env, addr, 0, access_type, mmu_idx,
 -                                  nonfault, phost, retaddr);
 +    int flags = probe_access_internal(env, addr, 0, access_type, mmu_idx,
 +                                      nonfault, phost, pfull, retaddr);
      /* Handle clean RAM pages.  */
      if (unlikely(flags & TLB_NOTDIRTY)) {
 -        uintptr_t index = tlb_index(env, mmu_idx, addr);
 -        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
 -
 -        notdirty_write(env_cpu(env), addr, 1, full, retaddr);
 +        notdirty_write(env_cpu(env), addr, 1, *pfull, retaddr);
          flags &= ~TLB_NOTDIRTY;
      }
      return flags;
  }
-+int probe_access_flags(CPUArchState *env, target_ulong addr,
+ /*
 +                       MMUAccessType access_type, int mmu_idx,
 +                       bool nonfault, void **phost, uintptr_t retaddr)
 +{
 +    CPUTLBEntryFull *full;
 +
 +    return probe_access_full(env, addr, access_type, mmu_idx,
 +                             nonfault, phost, &full, retaddr);
 +}
 +
  void *probe_access(CPUArchState *env, target_ulong addr, int size,
                     MMUAccessType access_type, int mmu_idx, uintptr_t retaddr)
  {
 +    CPUTLBEntryFull *full;
      void *host;
      int flags;
      g_assert(-(addr | TARGET_PAGE_MASK) >= size);
      flags = probe_access_internal(env, addr, size, access_type, mmu_idx,
 -                                  false, &host, retaddr);
 +                                  false, &host, &full, retaddr);
      /* Per the interface, size == 0 merely faults the access. */
      if (size == 0) {
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
      }
      if (unlikely(flags & (TLB_NOTDIRTY | TLB_WATCHPOINT))) {
 -        uintptr_t index = tlb_index(env, mmu_idx, addr);
 -        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
 -
          /* Handle watchpoints.  */
          if (flags & TLB_WATCHPOINT) {
              int wp_access = (access_type == MMU_DATA_STORE
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
  void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr,
                          MMUAccessType access_type, int mmu_idx)
  {
 +    CPUTLBEntryFull *full;
      void *host;
      int flags;
      flags = probe_access_internal(env, addr, 0, access_type,
 -                                  mmu_idx, true, &host, 0);
 +                                  mmu_idx, true, &host, &full, 0);
      /* No combination of flags are expected by the caller. */
      return flags ? NULL : host;
@@ -XXX,XX +XXX,XX @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr,
  tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, target_ulong addr,
                                          void **hostp)
  {
 +    CPUTLBEntryFull *full;
      void *p;
      (void)probe_access_internal(env, addr, 1, MMU_INST_FETCH,
 -                                cpu_mmu_index(env, true), false, &p, 0);
 +                                cpu_mmu_index(env, true), false, &p, &full, 0);
      if (p == NULL) {
          return -1;
      }
 --
 .34.1

-[PULL 01/20] cpu: cache CPUClass in CPUState for hot code paths
+[PULL 20/20] plugins: Update the documentation block for plugin-gen.c
-From: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
 The class cast checkers are quite expensive and always on (unlike the
 dynamic case who's checks are gated by CONFIG_QOM_CAST_DEBUG). To
 avoid the overhead of repeatedly checking something which should never
 change we cache the CPUClass reference for use in the hot code paths.
 Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Message-Id: <20220811151413.3350684-3-alex.bennee@linaro.org>
 Signed-off-by: Cédric Le Goater <clg@kaod.org>
 Message-Id: <20220923084803.498337-3-clg@kaod.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/hw/core/cpu.h | 9 +++++++++
+ accel/tcg/plugin-gen.c | 31 ++++---------------------------
- cpu.c                 | 9 ++++-----
+file changed, 4 insertions(+), 27 deletions(-)
 files changed, 13 insertions(+), 5 deletions(-)
-diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
+diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
 index XXXXXXX..XXXXXXX 100644
---- a/include/hw/core/cpu.h
+--- a/accel/tcg/plugin-gen.c
-+++ b/include/hw/core/cpu.h
++++ b/accel/tcg/plugin-gen.c
-@@ -XXX,XX +XXX,XX @@ typedef int (*WriteCoreDumpFunction)(const void *buf, size_t size,
+@@ -XXX,XX +XXX,XX @@
   * Injecting the desired instrumentation could be done with a second
   * translation pass that combined the instrumentation requests, but that
   * would be ugly and inefficient since we would decode the guest code twice.
 - * Instead, during TB translation we add "empty" instrumentation calls for all
 - * possible instrumentation events, and then once we collect the instrumentation
 - * requests from plugins, we either "fill in" those empty events or remove them
 - * if they have no requests.
 - *
 - * When "filling in" an event we first copy the empty callback's TCG ops. This
 - * might seem unnecessary, but it is done to support an arbitrary number
 - * of callbacks per event. Take for example a regular instruction callback.
 - * We first generate a callback to an empty helper function. Then, if two
 - * plugins register one callback each for this instruction, we make two copies
 - * of the TCG ops generated for the empty callback, substituting the function
 - * pointer that points to the empty helper function with the plugins' desired
 - * callback functions. After that we remove the empty callback's ops.
 - *
 - * Note that the location in TCGOp.args[] of the pointer to a helper function
 - * varies across different guest and host architectures. Instead of duplicating
 - * the logic that figures this out, we rely on the fact that the empty
 - * callbacks point to empty functions that are unique pointers in the program.
 - * Thus, to find the right location we just have to look for a match in
 - * TCGOp.args[]. This is the main reason why we first copy an empty callback's
 - * TCG ops and then fill them in; regardless of whether we have one or many
 - * callbacks for that event, the logic to add all of them is the same.
 - *
 - * When generating more than one callback per event, we make a small
 - * optimization to avoid generating redundant operations. For instance, for the
 - * second and all subsequent callbacks of an event, we do not need to reload the
 - * CPU's index into a TCG temp, since the first callback did it already.
 + * Instead, during TB translation we add "plugin_cb" marker opcodes
 + * for all possible instrumentation events, and then once we collect the
 + * instrumentation requests from plugins, we generate code for those markers
 + * or remove them if they have no requests.
   */
- #define CPU(obj) ((CPUState *)(obj))
+ #include "qemu/osdep.h"
+ #include "qemu/plugin.h"
 +/*
 + * The class checkers bring in CPU_GET_CLASS() which is potentially
 + * expensive given the eventual call to
 + * object_class_dynamic_cast_assert(). Because of this the CPUState
 + * has a cached value for the class in cs->cc which is set up in
 + * cpu_exec_realizefn() for use in hot code paths.
 + */
  typedef struct CPUClass CPUClass;
  DECLARE_CLASS_CHECKERS(CPUClass, CPU,
                         TYPE_CPU)
@@ -XXX,XX +XXX,XX @@ struct qemu_work_item;
  struct CPUState {
      /*< private >*/
      DeviceState parent_obj;
 +    /* cache to avoid expensive CPU_GET_CLASS */
 +    CPUClass *cc;
      /*< public >*/
      int nr_cores;
 diff --git a/cpu.c b/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/cpu.c
 +++ b/cpu.c
@@ -XXX,XX +XXX,XX @@ const VMStateDescription vmstate_cpu_common = {
  void cpu_exec_realizefn(CPUState *cpu, Error **errp)
  {
 -#ifndef CONFIG_USER_ONLY
 -    CPUClass *cc = CPU_GET_CLASS(cpu);
 -#endif
 +    /* cache the cpu class for the hotpath */
 +    cpu->cc = CPU_GET_CLASS(cpu);
      cpu_list_add(cpu);
      if (!accel_cpu_realizefn(cpu, errp)) {
@@ -XXX,XX +XXX,XX @@ void cpu_exec_realizefn(CPUState *cpu, Error **errp)
      if (qdev_get_vmsd(DEVICE(cpu)) == NULL) {
          vmstate_register(NULL, cpu->cpu_index, &vmstate_cpu_common, cpu);
      }
 -    if (cc->sysemu_ops->legacy_vmsd != NULL) {
 -        vmstate_register(NULL, cpu->cpu_index, cc->sysemu_ops->legacy_vmsd, cpu);
 +    if (cpu->cc->sysemu_ops->legacy_vmsd != NULL) {
 +        vmstate_register(NULL, cpu->cpu_index, cpu->cc->sysemu_ops->legacy_vmsd, cpu);
      }
  #endif /* CONFIG_USER_ONLY */
  }
 --
 .34.1

TCG patch queue, plus one target/sh4 patch that
Yoshinori Sato asked me to process.

The following changes since commit efbf38d73e5dcc4d5f8b98c6e7a12be1f3b91745:

Merge tag 'for-upstream' of git://repo.or.cz/qemu/kevin into staging (2022-10-03 15:06:07 -0400)

are available in the Git repository at:

https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20221004

for you to fetch changes up to ab419fd8a035a65942de4e63effcd55ccbf1a9fe:

target/sh4: Fix TB_FLAG_UNALIGN (2022-10-04 12:33:05 -0700)

----------------------------------------------------------------
Cache CPUClass for use in hot code paths.
Add CPUTLBEntryFull, probe_access_full, tlb_set_page_full.
Add generic support for TARGET_TB_PCREL.
tcg/ppc: Optimize 26-bit jumps using STQ for POWER 2.07
target/sh4: Fix TB_FLAG_UNALIGN

----------------------------------------------------------------
Alex Bennée (3):
      cpu: cache CPUClass in CPUState for hot code paths
      hw/core/cpu-sysemu: used cached class in cpu_asidx_from_attrs
      cputlb: used cached CPUClass in our hot-paths

Leandro Lupori (1):
      tcg/ppc: Optimize 26-bit jumps

Richard Henderson (16):
      accel/tcg: Rename CPUIOTLBEntry to CPUTLBEntryFull
      accel/tcg: Drop addr member from SavedIOTLB
      accel/tcg: Suppress auto-invalidate in probe_access_internal
      accel/tcg: Introduce probe_access_full
      accel/tcg: Introduce tlb_set_page_full
      include/exec: Introduce TARGET_PAGE_ENTRY_EXTRA
      accel/tcg: Remove PageDesc code_bitmap
      accel/tcg: Use bool for page_find_alloc
      accel/tcg: Use DisasContextBase in plugin_gen_tb_start
      accel/tcg: Do not align tb->page_addr[0]
      accel/tcg: Inline tb_flush_jmp_cache
      include/hw/core: Create struct CPUJumpCache
      hw/core: Add CPUClass.get_pc
      accel/tcg: Introduce tb_pc and log_pc
      accel/tcg: Introduce TARGET_TB_PCREL
      target/sh4: Fix TB_FLAG_UNALIGN

From: Alex Bennée <alex.bennee@linaro.org>

The class cast checkers are quite expensive and always on (unlike the
dynamic case who's checks are gated by CONFIG_QOM_CAST_DEBUG). To
avoid the overhead of repeatedly checking something which should never
change we cache the CPUClass reference for use in the hot code paths.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220811151413.3350684-3-alex.bennee@linaro.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220923084803.498337-3-clg@kaod.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/hw/core/cpu.h | 9 +++++++++
 cpu.c                 | 9 ++++-----
 2 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index XXXXXXX..XXXXXXX 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -XXX,XX +XXX,XX @@ typedef int (*WriteCoreDumpFunction)(const void *buf, size_t size,
  */
 #define CPU(obj) ((CPUState *)(obj))
 
+/*
+ * The class checkers bring in CPU_GET_CLASS() which is potentially
+ * expensive given the eventual call to
+ * object_class_dynamic_cast_assert(). Because of this the CPUState
+ * has a cached value for the class in cs->cc which is set up in
+ * cpu_exec_realizefn() for use in hot code paths.
+ */
 typedef struct CPUClass CPUClass;
 DECLARE_CLASS_CHECKERS(CPUClass, CPU,
                        TYPE_CPU)
@@ -XXX,XX +XXX,XX @@ struct qemu_work_item;
 struct CPUState {
     /*< private >*/
     DeviceState parent_obj;
+    /* cache to avoid expensive CPU_GET_CLASS */
+    CPUClass *cc;
     /*< public >*/
 
     int nr_cores;
diff --git a/cpu.c b/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/cpu.c
+++ b/cpu.c
@@ -XXX,XX +XXX,XX @@ const VMStateDescription vmstate_cpu_common = {
 
 void cpu_exec_realizefn(CPUState *cpu, Error **errp)
 {
-#ifndef CONFIG_USER_ONLY
-    CPUClass *cc = CPU_GET_CLASS(cpu);
-#endif
+    /* cache the cpu class for the hotpath */
+    cpu->cc = CPU_GET_CLASS(cpu);
 
     cpu_list_add(cpu);
     if (!accel_cpu_realizefn(cpu, errp)) {
@@ -XXX,XX +XXX,XX @@ void cpu_exec_realizefn(CPUState *cpu, Error **errp)
     if (qdev_get_vmsd(DEVICE(cpu)) == NULL) {
         vmstate_register(NULL, cpu->cpu_index, &vmstate_cpu_common, cpu);
     }
-    if (cc->sysemu_ops->legacy_vmsd != NULL) {
-        vmstate_register(NULL, cpu->cpu_index, cc->sysemu_ops->legacy_vmsd, cpu);
+    if (cpu->cc->sysemu_ops->legacy_vmsd != NULL) {
+        vmstate_register(NULL, cpu->cpu_index, cpu->cc->sysemu_ops->legacy_vmsd, cpu);
     }
 #endif /* CONFIG_USER_ONLY */
 }
-- 
2.34.1

From: Alex Bennée <alex.bennee@linaro.org>

This is a heavily used function so lets avoid the cost of
CPU_GET_CLASS. On the romulus-bmc run it has a modest effect:

Before: 36.812 s ±  0.506 s
  After:  35.912 s ±  0.168 s

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220811151413.3350684-4-alex.bennee@linaro.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220923084803.498337-4-clg@kaod.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 hw/core/cpu-sysemu.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/hw/core/cpu-sysemu.c b/hw/core/cpu-sysemu.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/core/cpu-sysemu.c
+++ b/hw/core/cpu-sysemu.c
@@ -XXX,XX +XXX,XX @@ hwaddr cpu_get_phys_page_debug(CPUState *cpu, vaddr addr)
 
 int cpu_asidx_from_attrs(CPUState *cpu, MemTxAttrs attrs)
 {
-    CPUClass *cc = CPU_GET_CLASS(cpu);
     int ret = 0;
 
-    if (cc->sysemu_ops->asidx_from_attrs) {
-        ret = cc->sysemu_ops->asidx_from_attrs(cpu, attrs);
+    if (cpu->cc->sysemu_ops->asidx_from_attrs) {
+        ret = cpu->cc->sysemu_ops->asidx_from_attrs(cpu, attrs);
         assert(ret < cpu->num_ases && ret >= 0);
     }
     return ret;
-- 
2.34.1

From: Alex Bennée <alex.bennee@linaro.org>

Before: 35.912 s ±  0.168 s
  After: 35.565 s ±  0.087 s

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220811151413.3350684-5-alex.bennee@linaro.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20220923084803.498337-5-clg@kaod.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/cputlb.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ void tlb_set_page(CPUState *cpu, target_ulong vaddr,
 static void tlb_fill(CPUState *cpu, target_ulong addr, int size,
                      MMUAccessType access_type, int mmu_idx, uintptr_t retaddr)
 {
-    CPUClass *cc = CPU_GET_CLASS(cpu);
     bool ok;
 
     /*
      * This is not a probe, so only valid return is success; failure
      * should result in exception + longjmp to the cpu loop.
      */
-    ok = cc->tcg_ops->tlb_fill(cpu, addr, size,
-                               access_type, mmu_idx, false, retaddr);
+    ok = cpu->cc->tcg_ops->tlb_fill(cpu, addr, size,
+                                    access_type, mmu_idx, false, retaddr);
     assert(ok);
 }
 
@@ -XXX,XX +XXX,XX @@ static inline void cpu_unaligned_access(CPUState *cpu, vaddr addr,
                                         MMUAccessType access_type,
                                         int mmu_idx, uintptr_t retaddr)
 {
-    CPUClass *cc = CPU_GET_CLASS(cpu);
-
-    cc->tcg_ops->do_unaligned_access(cpu, addr, access_type, mmu_idx, retaddr);
+    cpu->cc->tcg_ops->do_unaligned_access(cpu, addr, access_type,
+                                          mmu_idx, retaddr);
 }
 
 static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr,
@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
     if (!tlb_hit_page(tlb_addr, page_addr)) {
         if (!victim_tlb_hit(env, mmu_idx, index, elt_ofs, page_addr)) {
             CPUState *cs = env_cpu(env);
-            CPUClass *cc = CPU_GET_CLASS(cs);
 
-            if (!cc->tcg_ops->tlb_fill(cs, addr, fault_size, access_type,
-                                       mmu_idx, nonfault, retaddr)) {
+            if (!cs->cc->tcg_ops->tlb_fill(cs, addr, fault_size, access_type,
+                                           mmu_idx, nonfault, retaddr)) {
                 /* Non-faulting page table read failed.  */
                 *phost = NULL;
                 return TLB_INVALID_MASK;
-- 
2.34.1

This structure will shortly contain more than just
data for accessing MMIO.  Rename the 'addr' member
to 'xlat_section' to more clearly indicate its purpose.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-defs.h    |  22 ++++----
 accel/tcg/cputlb.c         | 102 +++++++++++++++++++------------------
 target/arm/mte_helper.c    |  14 ++---
 target/arm/sve_helper.c    |   4 +-
 target/arm/translate-a64.c |   2 +-
 5 files changed, 73 insertions(+), 71 deletions(-)

diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-defs.h
+++ b/include/exec/cpu-defs.h
@@ -XXX,XX +XXX,XX @@ typedef uint64_t target_ulong;
 #  endif
 # endif
 
+/* Minimalized TLB entry for use by TCG fast path. */
 typedef struct CPUTLBEntry {
     /* bit TARGET_LONG_BITS to TARGET_PAGE_BITS : virtual address
        bit TARGET_PAGE_BITS-1..4  : Nonzero for accesses that should not
@@ -XXX,XX +XXX,XX @@ typedef struct CPUTLBEntry {
 
 QEMU_BUILD_BUG_ON(sizeof(CPUTLBEntry) != (1 << CPU_TLB_ENTRY_BITS));
 
-/* The IOTLB is not accessed directly inline by generated TCG code,
- * so the CPUIOTLBEntry layout is not as critical as that of the
- * CPUTLBEntry. (This is also why we don't want to combine the two
- * structs into one.)
+/*
+ * The full TLB entry, which is not accessed by generated TCG code,
+ * so the layout is not as critical as that of CPUTLBEntry. This is
+ * also why we don't want to combine the two structs.
  */
-typedef struct CPUIOTLBEntry {
+typedef struct CPUTLBEntryFull {
     /*
-     * @addr contains:
+     * @xlat_section contains:
      *  - in the lower TARGET_PAGE_BITS, a physical section number
      *  - with the lower TARGET_PAGE_BITS masked off, an offset which
      *    must be added to the virtual address to obtain:
@@ -XXX,XX +XXX,XX @@ typedef struct CPUIOTLBEntry {
      *       number is PHYS_SECTION_NOTDIRTY or PHYS_SECTION_ROM)
      *     + the offset within the target MemoryRegion (otherwise)
      */
-    hwaddr addr;
+    hwaddr xlat_section;
     MemTxAttrs attrs;
-} CPUIOTLBEntry;
+} CPUTLBEntryFull;
 
 /*
  * Data elements that are per MMU mode, minus the bits accessed by
@@ -XXX,XX +XXX,XX @@ typedef struct CPUTLBDesc {
     size_t vindex;
     /* The tlb victim table, in two parts.  */
     CPUTLBEntry vtable[CPU_VTLB_SIZE];
-    CPUIOTLBEntry viotlb[CPU_VTLB_SIZE];
-    /* The iotlb.  */
-    CPUIOTLBEntry *iotlb;
+    CPUTLBEntryFull vfulltlb[CPU_VTLB_SIZE];
+    CPUTLBEntryFull *fulltlb;
 } CPUTLBDesc;
 
 /*
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
     }
 
     g_free(fast->table);
-    g_free(desc->iotlb);
+    g_free(desc->fulltlb);
 
     tlb_window_reset(desc, now, 0);
     /* desc->n_used_entries is cleared by the caller */
     fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
     fast->table = g_try_new(CPUTLBEntry, new_size);
-    desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
+    desc->fulltlb = g_try_new(CPUTLBEntryFull, new_size);
 
     /*
      * If the allocations fail, try smaller sizes. We just freed some
@@ -XXX,XX +XXX,XX @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
      * allocations to fail though, so we progressively reduce the allocation
      * size, aborting if we cannot even allocate the smallest TLB we support.
      */
-    while (fast->table == NULL || desc->iotlb == NULL) {
+    while (fast->table == NULL || desc->fulltlb == NULL) {
         if (new_size == (1 << CPU_TLB_DYN_MIN_BITS)) {
             error_report("%s: %s", __func__, strerror(errno));
             abort();
@@ -XXX,XX +XXX,XX @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast,
         fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS;
 
         g_free(fast->table);
-        g_free(desc->iotlb);
+        g_free(desc->fulltlb);
         fast->table = g_try_new(CPUTLBEntry, new_size);
-        desc->iotlb = g_try_new(CPUIOTLBEntry, new_size);
+        desc->fulltlb = g_try_new(CPUTLBEntryFull, new_size);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now)
     desc->n_used_entries = 0;
     fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS;
     fast->table = g_new(CPUTLBEntry, n_entries);
-    desc->iotlb = g_new(CPUIOTLBEntry, n_entries);
+    desc->fulltlb = g_new(CPUTLBEntryFull, n_entries);
     tlb_mmu_flush_locked(desc, fast);
 }
 
@@ -XXX,XX +XXX,XX @@ void tlb_destroy(CPUState *cpu)
         CPUTLBDescFast *fast = &env_tlb(env)->f[i];
 
         g_free(fast->table);
-        g_free(desc->iotlb);
+        g_free(desc->fulltlb);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
 
         /* Evict the old entry into the victim tlb.  */
         copy_tlb_helper_locked(tv, te);
-        desc->viotlb[vidx] = desc->iotlb[index];
+        desc->vfulltlb[vidx] = desc->fulltlb[index];
         tlb_n_used_entries_dec(env, mmu_idx);
     }
 
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
      * subtract here is that of the page base, and not the same as the
      * vaddr we add back in io_readx()/io_writex()/get_page_addr_code().
      */
-    desc->iotlb[index].addr = iotlb - vaddr_page;
-    desc->iotlb[index].attrs = attrs;
+    desc->fulltlb[index].xlat_section = iotlb - vaddr_page;
+    desc->fulltlb[index].attrs = attrs;
 
     /* Now calculate the new entry */
     tn.addend = addend - vaddr_page;
@@ -XXX,XX +XXX,XX @@ static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr,
     }
 }
 
-static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
+static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full,
                          int mmu_idx, target_ulong addr, uintptr_t retaddr,
                          MMUAccessType access_type, MemOp op)
 {
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     bool locked = false;
     MemTxResult r;
 
-    section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
+    section = iotlb_to_section(cpu, full->xlat_section, full->attrs);
     mr = section->mr;
-    mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
+    mr_offset = (full->xlat_section & TARGET_PAGE_MASK) + addr;
     cpu->mem_io_pc = retaddr;
     if (!cpu->can_do_io) {
         cpu_io_recompile(cpu, retaddr);
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
         qemu_mutex_lock_iothread();
         locked = true;
     }
-    r = memory_region_dispatch_read(mr, mr_offset, &val, op, iotlbentry->attrs);
+    r = memory_region_dispatch_read(mr, mr_offset, &val, op, full->attrs);
     if (r != MEMTX_OK) {
         hwaddr physaddr = mr_offset +
             section->offset_within_address_space -
             section->offset_within_region;
 
         cpu_transaction_failed(cpu, physaddr, addr, memop_size(op), access_type,
-                               mmu_idx, iotlbentry->attrs, r, retaddr);
+                               mmu_idx, full->attrs, r, retaddr);
     }
     if (locked) {
         qemu_mutex_unlock_iothread();
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
 }
 
 /*
- * Save a potentially trashed IOTLB entry for later lookup by plugin.
- * This is read by tlb_plugin_lookup if the iotlb entry doesn't match
+ * Save a potentially trashed CPUTLBEntryFull for later lookup by plugin.
+ * This is read by tlb_plugin_lookup if the fulltlb entry doesn't match
  * because of the side effect of io_writex changing memory layout.
  */
 static void save_iotlb_data(CPUState *cs, hwaddr addr,
@@ -XXX,XX +XXX,XX @@ static void save_iotlb_data(CPUState *cs, hwaddr addr,
 #endif
 }
 
-static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
+static void io_writex(CPUArchState *env, CPUTLBEntryFull *full,
                       int mmu_idx, uint64_t val, target_ulong addr,
                       uintptr_t retaddr, MemOp op)
 {
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     bool locked = false;
     MemTxResult r;
 
-    section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
+    section = iotlb_to_section(cpu, full->xlat_section, full->attrs);
     mr = section->mr;
-    mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
+    mr_offset = (full->xlat_section & TARGET_PAGE_MASK) + addr;
     if (!cpu->can_do_io) {
         cpu_io_recompile(cpu, retaddr);
     }
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
      * The memory_region_dispatch may trigger a flush/resize
      * so for plugins we save the iotlb_data just in case.
      */
-    save_iotlb_data(cpu, iotlbentry->addr, section, mr_offset);
+    save_iotlb_data(cpu, full->xlat_section, section, mr_offset);
 
     if (!qemu_mutex_iothread_locked()) {
         qemu_mutex_lock_iothread();
         locked = true;
     }
-    r = memory_region_dispatch_write(mr, mr_offset, val, op, iotlbentry->attrs);
+    r = memory_region_dispatch_write(mr, mr_offset, val, op, full->attrs);
     if (r != MEMTX_OK) {
         hwaddr physaddr = mr_offset +
             section->offset_within_address_space -
             section->offset_within_region;
 
         cpu_transaction_failed(cpu, physaddr, addr, memop_size(op),
-                               MMU_DATA_STORE, mmu_idx, iotlbentry->attrs, r,
+                               MMU_DATA_STORE, mmu_idx, full->attrs, r,
                                retaddr);
     }
     if (locked) {
@@ -XXX,XX +XXX,XX @@ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index,
             copy_tlb_helper_locked(vtlb, &tmptlb);
             qemu_spin_unlock(&env_tlb(env)->c.lock);
 
-            CPUIOTLBEntry tmpio, *io = &env_tlb(env)->d[mmu_idx].iotlb[index];
-            CPUIOTLBEntry *vio = &env_tlb(env)->d[mmu_idx].viotlb[vidx];
-            tmpio = *io; *io = *vio; *vio = tmpio;
+            CPUTLBEntryFull *f1 = &env_tlb(env)->d[mmu_idx].fulltlb[index];
+            CPUTLBEntryFull *f2 = &env_tlb(env)->d[mmu_idx].vfulltlb[vidx];
+            CPUTLBEntryFull tmpf;
+            tmpf = *f1; *f1 = *f2; *f2 = tmpf;
             return true;
         }
     }
@@ -XXX,XX +XXX,XX @@ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index,
                  (ADDR) & TARGET_PAGE_MASK)
 
 static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
-                           CPUIOTLBEntry *iotlbentry, uintptr_t retaddr)
+                           CPUTLBEntryFull *full, uintptr_t retaddr)
 {
-    ram_addr_t ram_addr = mem_vaddr + iotlbentry->addr;
+    ram_addr_t ram_addr = mem_vaddr + full->xlat_section;
 
     trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size);
 
@@ -XXX,XX +XXX,XX @@ int probe_access_flags(CPUArchState *env, target_ulong addr,
     /* Handle clean RAM pages.  */
     if (unlikely(flags & TLB_NOTDIRTY)) {
         uintptr_t index = tlb_index(env, mmu_idx, addr);
-        CPUIOTLBEntry *iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
+        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
 
-        notdirty_write(env_cpu(env), addr, 1, iotlbentry, retaddr);
+        notdirty_write(env_cpu(env), addr, 1, full, retaddr);
         flags &= ~TLB_NOTDIRTY;
     }
 
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
 
     if (unlikely(flags & (TLB_NOTDIRTY | TLB_WATCHPOINT))) {
         uintptr_t index = tlb_index(env, mmu_idx, addr);
-        CPUIOTLBEntry *iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
+        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
 
         /* Handle watchpoints.  */
         if (flags & TLB_WATCHPOINT) {
             int wp_access = (access_type == MMU_DATA_STORE
                              ? BP_MEM_WRITE : BP_MEM_READ);
             cpu_check_watchpoint(env_cpu(env), addr, size,
-                                 iotlbentry->attrs, wp_access, retaddr);
+                                 full->attrs, wp_access, retaddr);
         }
 
         /* Handle clean RAM pages.  */
         if (flags & TLB_NOTDIRTY) {
-            notdirty_write(env_cpu(env), addr, 1, iotlbentry, retaddr);
+            notdirty_write(env_cpu(env), addr, 1, full, retaddr);
         }
     }
 
@@ -XXX,XX +XXX,XX @@ tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, target_ulong addr,
  * should have just filled the TLB. The one corner case is io_writex
  * which can cause TLB flushes and potential resizing of the TLBs
  * losing the information we need. In those cases we need to recover
- * data from a copy of the iotlbentry. As long as this always occurs
+ * data from a copy of the CPUTLBEntryFull. As long as this always occurs
  * from the same thread (which a mem callback will be) this is safe.
  */
 
@@ -XXX,XX +XXX,XX @@ bool tlb_plugin_lookup(CPUState *cpu, target_ulong addr, int mmu_idx,
     if (likely(tlb_hit(tlb_addr, addr))) {
         /* We must have an iotlb entry for MMIO */
         if (tlb_addr & TLB_MMIO) {
-            CPUIOTLBEntry *iotlbentry;
-            iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
+            CPUTLBEntryFull *full;
+            full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
             data->is_io = true;
-            data->v.io.section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
-            data->v.io.offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
+            data->v.io.section =
+                iotlb_to_section(cpu, full->xlat_section, full->attrs);
+            data->v.io.offset = (full->xlat_section & TARGET_PAGE_MASK) + addr;
         } else {
             data->is_io = false;
             data->v.ram.hostaddr = (void *)((uintptr_t)addr + tlbe->addend);
@@ -XXX,XX +XXX,XX @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
 
     if (unlikely(tlb_addr & TLB_NOTDIRTY)) {
         notdirty_write(env_cpu(env), addr, size,
-                       &env_tlb(env)->d[mmu_idx].iotlb[index], retaddr);
+                       &env_tlb(env)->d[mmu_idx].fulltlb[index], retaddr);
     }
 
     return hostaddr;
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, MemOpIdx oi,
 
     /* Handle anything that isn't just a straight memory access.  */
     if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
-        CPUIOTLBEntry *iotlbentry;
+        CPUTLBEntryFull *full;
         bool need_swap;
 
         /* For anything that is unaligned, recurse through full_load.  */
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, MemOpIdx oi,
             goto do_unaligned_access;
         }
 
-        iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
+        full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
 
         /* Handle watchpoints.  */
         if (unlikely(tlb_addr & TLB_WATCHPOINT)) {
             /* On watchpoint hit, this will longjmp out.  */
             cpu_check_watchpoint(env_cpu(env), addr, size,
-                                 iotlbentry->attrs, BP_MEM_READ, retaddr);
+                                 full->attrs, BP_MEM_READ, retaddr);
         }
 
         need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
 
         /* Handle I/O access.  */
         if (likely(tlb_addr & TLB_MMIO)) {
-            return io_readx(env, iotlbentry, mmu_idx, addr, retaddr,
+            return io_readx(env, full, mmu_idx, addr, retaddr,
                             access_type, op ^ (need_swap * MO_BSWAP));
         }
 
@@ -XXX,XX +XXX,XX @@ store_helper_unaligned(CPUArchState *env, target_ulong addr, uint64_t val,
      */
     if (unlikely(tlb_addr & TLB_WATCHPOINT)) {
         cpu_check_watchpoint(env_cpu(env), addr, size - size2,
-                             env_tlb(env)->d[mmu_idx].iotlb[index].attrs,
+                             env_tlb(env)->d[mmu_idx].fulltlb[index].attrs,
                              BP_MEM_WRITE, retaddr);
     }
     if (unlikely(tlb_addr2 & TLB_WATCHPOINT)) {
         cpu_check_watchpoint(env_cpu(env), page2, size2,
-                             env_tlb(env)->d[mmu_idx].iotlb[index2].attrs,
+                             env_tlb(env)->d[mmu_idx].fulltlb[index2].attrs,
                              BP_MEM_WRITE, retaddr);
     }
 
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
 
     /* Handle anything that isn't just a straight memory access.  */
     if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
-        CPUIOTLBEntry *iotlbentry;
+        CPUTLBEntryFull *full;
         bool need_swap;
 
         /* For anything that is unaligned, recurse through byte stores.  */
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
             goto do_unaligned_access;
         }
 
-        iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
+        full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
 
         /* Handle watchpoints.  */
         if (unlikely(tlb_addr & TLB_WATCHPOINT)) {
             /* On watchpoint hit, this will longjmp out.  */
             cpu_check_watchpoint(env_cpu(env), addr, size,
-                                 iotlbentry->attrs, BP_MEM_WRITE, retaddr);
+                                 full->attrs, BP_MEM_WRITE, retaddr);
         }
 
         need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
 
         /* Handle I/O access.  */
         if (tlb_addr & TLB_MMIO) {
-            io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr,
+            io_writex(env, full, mmu_idx, val, addr, retaddr,
                       op ^ (need_swap * MO_BSWAP));
             return;
         }
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
 
         /* Handle clean RAM pages.  */
         if (tlb_addr & TLB_NOTDIRTY) {
-            notdirty_write(env_cpu(env), addr, size, iotlbentry, retaddr);
+            notdirty_write(env_cpu(env), addr, size, full, retaddr);
         }
 
         haddr = (void *)((uintptr_t)addr + entry->addend);
diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c
index XXXXXXX..XXXXXXX 100644
--- a/target/arm/mte_helper.c
+++ b/target/arm/mte_helper.c
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
     return tags + index;
 #else
     uintptr_t index;
-    CPUIOTLBEntry *iotlbentry;
+    CPUTLBEntryFull *full;
     int in_page, flags;
     ram_addr_t ptr_ra;
     hwaddr ptr_paddr, tag_paddr, xlat;
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
     assert(!(flags & TLB_INVALID_MASK));
 
     /*
-     * Find the iotlbentry for ptr.  This *must* be present in the TLB
+     * Find the CPUTLBEntryFull for ptr.  This *must* be present in the TLB
      * because we just found the mapping.
      * TODO: Perhaps there should be a cputlb helper that returns a
      * matching tlb entry + iotlb entry.
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
         g_assert(tlb_hit(comparator, ptr));
     }
 # endif
-    iotlbentry = &env_tlb(env)->d[ptr_mmu_idx].iotlb[index];
+    full = &env_tlb(env)->d[ptr_mmu_idx].fulltlb[index];
 
     /* If the virtual page MemAttr != Tagged, access unchecked. */
-    if (!arm_tlb_mte_tagged(&iotlbentry->attrs)) {
+    if (!arm_tlb_mte_tagged(&full->attrs)) {
         return NULL;
     }
 
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
         int wp = ptr_access == MMU_DATA_LOAD ? BP_MEM_READ : BP_MEM_WRITE;
         assert(ra != 0);
         cpu_check_watchpoint(env_cpu(env), ptr, ptr_size,
-                             iotlbentry->attrs, wp, ra);
+                             full->attrs, wp, ra);
     }
 
     /*
@@ -XXX,XX +XXX,XX @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
     tag_paddr = ptr_paddr >> (LOG2_TAG_GRANULE + 1);
 
     /* Look up the address in tag space. */
-    tag_asi = iotlbentry->attrs.secure ? ARMASIdx_TagS : ARMASIdx_TagNS;
+    tag_asi = full->attrs.secure ? ARMASIdx_TagS : ARMASIdx_TagNS;
     tag_as = cpu_get_address_space(env_cpu(env), tag_asi);
     mr = address_space_translate(tag_as, tag_paddr, &xlat, NULL,
                                  tag_access == MMU_DATA_STORE,
-                                 iotlbentry->attrs);
+                                 full->attrs);
 
     /*
      * Note that @mr will never be NULL.  If there is nothing in the address
diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index XXXXXXX..XXXXXXX 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -XXX,XX +XXX,XX @@ bool sve_probe_page(SVEHostPage *info, bool nofault, CPUARMState *env,
         g_assert(tlb_hit(comparator, addr));
 # endif
 
-        CPUIOTLBEntry *iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
-        info->attrs = iotlbentry->attrs;
+        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
+        info->attrs = full->attrs;
     }
 #endif
 
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index XXXXXXX..XXXXXXX 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -XXX,XX +XXX,XX @@ static bool is_guarded_page(CPUARMState *env, DisasContext *s)
      * table entry even for that case.
      */
     return (tlb_hit(entry->addr_code, addr) &&
-            arm_tlb_bti_gp(&env_tlb(env)->d[mmu_idx].iotlb[index].attrs));
+            arm_tlb_bti_gp(&env_tlb(env)->d[mmu_idx].fulltlb[index].attrs));
 #endif
 }
 
-- 
2.34.1

This field is only written, not read; remove it.

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index XXXXXXX..XXXXXXX 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -XXX,XX +XXX,XX @@ struct CPUWatchpoint {
  * the memory regions get moved around  by io_writex.
  */
 typedef struct SavedIOTLB {
-    hwaddr addr;
     MemoryRegionSection *section;
     hwaddr mr_offset;
 } SavedIOTLB;
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full,
  * This is read by tlb_plugin_lookup if the fulltlb entry doesn't match
  * because of the side effect of io_writex changing memory layout.
  */
-static void save_iotlb_data(CPUState *cs, hwaddr addr,
-                            MemoryRegionSection *section, hwaddr mr_offset)
+static void save_iotlb_data(CPUState *cs, MemoryRegionSection *section,
+                            hwaddr mr_offset)
 {
 #ifdef CONFIG_PLUGIN
     SavedIOTLB *saved = &cs->saved_iotlb;
-    saved->addr = addr;
     saved->section = section;
     saved->mr_offset = mr_offset;
 #endif
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full,
      * The memory_region_dispatch may trigger a flush/resize
      * so for plugins we save the iotlb_data just in case.
      */
-    save_iotlb_data(cpu, full->xlat_section, section, mr_offset);
+    save_iotlb_data(cpu, section, mr_offset);
 
     if (!qemu_mutex_iothread_locked()) {
         qemu_mutex_lock_iothread();
-- 
2.34.1

When PAGE_WRITE_INV is set when calling tlb_set_page,
we immediately set TLB_INVALID_MASK in order to force
tlb_fill to be called on the next lookup.  Here in
probe_access_internal, we have just called tlb_fill
and eliminated true misses, thus the lookup must be valid.

This allows us to remove a warning comment from s390x.
There doesn't seem to be a reason to change the code though.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/cputlb.c            | 10 +++++++++-
 target/s390x/tcg/mem_helper.c |  4 ----
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
     }
     tlb_addr = tlb_read_ofs(entry, elt_ofs);
 
+    flags = TLB_FLAGS_MASK;
     page_addr = addr & TARGET_PAGE_MASK;
     if (!tlb_hit_page(tlb_addr, page_addr)) {
         if (!victim_tlb_hit(env, mmu_idx, index, elt_ofs, page_addr)) {
@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
 
             /* TLB resize via tlb_fill may have moved the entry.  */
             entry = tlb_entry(env, mmu_idx, addr);
+
+            /*
+             * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately,
+             * to force the next access through tlb_fill.  We've just
+             * called tlb_fill, so we know that this entry *is* valid.
+             */
+            flags &= ~TLB_INVALID_MASK;
         }
         tlb_addr = tlb_read_ofs(entry, elt_ofs);
     }
-    flags = tlb_addr & TLB_FLAGS_MASK;
+    flags &= tlb_addr;
 
     /* Fold all "mmio-like" bits into TLB_MMIO.  This is not RAM.  */
     if (unlikely(flags & ~(TLB_WATCHPOINT | TLB_NOTDIRTY))) {
diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c
index XXXXXXX..XXXXXXX 100644
--- a/target/s390x/tcg/mem_helper.c
+++ b/target/s390x/tcg/mem_helper.c
@@ -XXX,XX +XXX,XX @@ static int s390_probe_access(CPUArchState *env, target_ulong addr, int size,
 #else
     int flags;
 
-    /*
-     * For !CONFIG_USER_ONLY, we cannot rely on TLB_INVALID_MASK or haddr==NULL
-     * to detect if there was an exception during tlb_fill().
-     */
     env->tlb_fill_exc = 0;
     flags = probe_access_flags(env, addr, access_type, mmu_idx, nonfault, phost,
                                ra);
-- 
2.34.1

Add an interface to return the CPUTLBEntryFull struct
that goes with the lookup.  The result is not intended
to be valid across multiple lookups, so the user must
use the results immediately.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/exec-all.h | 15 +++++++++++++
 include/qemu/typedefs.h |  1 +
 accel/tcg/cputlb.c      | 47 +++++++++++++++++++++++++----------------
 3 files changed, 45 insertions(+), 18 deletions(-)

diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -XXX,XX +XXX,XX @@ int probe_access_flags(CPUArchState *env, target_ulong addr,
                        MMUAccessType access_type, int mmu_idx,
                        bool nonfault, void **phost, uintptr_t retaddr);
 
+#ifndef CONFIG_USER_ONLY
+/**
+ * probe_access_full:
+ * Like probe_access_flags, except also return into @pfull.
+ *
+ * The CPUTLBEntryFull structure returned via @pfull is transient
+ * and must be consumed or copied immediately, before any further
+ * access or changes to TLB @mmu_idx.
+ */
+int probe_access_full(CPUArchState *env, target_ulong addr,
+                      MMUAccessType access_type, int mmu_idx,
+                      bool nonfault, void **phost,
+                      CPUTLBEntryFull **pfull, uintptr_t retaddr);
+#endif
+
 #define CODE_GEN_ALIGN           16 /* must be >= of the size of a icache line */
 
 /* Estimated block size for TB allocation.  */
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -XXX,XX +XXX,XX @@ typedef struct ConfidentialGuestSupport ConfidentialGuestSupport;
 typedef struct CPUAddressSpace CPUAddressSpace;
 typedef struct CPUArchState CPUArchState;
 typedef struct CPUState CPUState;
+typedef struct CPUTLBEntryFull CPUTLBEntryFull;
 typedef struct DeviceListener DeviceListener;
 typedef struct DeviceState DeviceState;
 typedef struct DirtyBitmapSnapshot DirtyBitmapSnapshot;
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
 static int probe_access_internal(CPUArchState *env, target_ulong addr,
                                  int fault_size, MMUAccessType access_type,
                                  int mmu_idx, bool nonfault,
-                                 void **phost, uintptr_t retaddr)
+                                 void **phost, CPUTLBEntryFull **pfull,
+                                 uintptr_t retaddr)
 {
     uintptr_t index = tlb_index(env, mmu_idx, addr);
     CPUTLBEntry *entry = tlb_entry(env, mmu_idx, addr);
@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
                                            mmu_idx, nonfault, retaddr)) {
                 /* Non-faulting page table read failed.  */
                 *phost = NULL;
+                *pfull = NULL;
                 return TLB_INVALID_MASK;
             }
 
             /* TLB resize via tlb_fill may have moved the entry.  */
+            index = tlb_index(env, mmu_idx, addr);
             entry = tlb_entry(env, mmu_idx, addr);
 
             /*
@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
     }
     flags &= tlb_addr;
 
+    *pfull = &env_tlb(env)->d[mmu_idx].fulltlb[index];
+
     /* Fold all "mmio-like" bits into TLB_MMIO.  This is not RAM.  */
     if (unlikely(flags & ~(TLB_WATCHPOINT | TLB_NOTDIRTY))) {
         *phost = NULL;
@@ -XXX,XX +XXX,XX @@ static int probe_access_internal(CPUArchState *env, target_ulong addr,
     return flags;
 }
 
-int probe_access_flags(CPUArchState *env, target_ulong addr,
-                       MMUAccessType access_type, int mmu_idx,
-                       bool nonfault, void **phost, uintptr_t retaddr)
+int probe_access_full(CPUArchState *env, target_ulong addr,
+                      MMUAccessType access_type, int mmu_idx,
+                      bool nonfault, void **phost, CPUTLBEntryFull **pfull,
+                      uintptr_t retaddr)
 {
-    int flags;
-
-    flags = probe_access_internal(env, addr, 0, access_type, mmu_idx,
-                                  nonfault, phost, retaddr);
+    int flags = probe_access_internal(env, addr, 0, access_type, mmu_idx,
+                                      nonfault, phost, pfull, retaddr);
 
     /* Handle clean RAM pages.  */
     if (unlikely(flags & TLB_NOTDIRTY)) {
-        uintptr_t index = tlb_index(env, mmu_idx, addr);
-        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
-
-        notdirty_write(env_cpu(env), addr, 1, full, retaddr);
+        notdirty_write(env_cpu(env), addr, 1, *pfull, retaddr);
         flags &= ~TLB_NOTDIRTY;
     }
 
     return flags;
 }
 
+int probe_access_flags(CPUArchState *env, target_ulong addr,
+                       MMUAccessType access_type, int mmu_idx,
+                       bool nonfault, void **phost, uintptr_t retaddr)
+{
+    CPUTLBEntryFull *full;
+
+    return probe_access_full(env, addr, access_type, mmu_idx,
+                             nonfault, phost, &full, retaddr);
+}
+
 void *probe_access(CPUArchState *env, target_ulong addr, int size,
                    MMUAccessType access_type, int mmu_idx, uintptr_t retaddr)
 {
+    CPUTLBEntryFull *full;
     void *host;
     int flags;
 
     g_assert(-(addr | TARGET_PAGE_MASK) >= size);
 
     flags = probe_access_internal(env, addr, size, access_type, mmu_idx,
-                                  false, &host, retaddr);
+                                  false, &host, &full, retaddr);
 
     /* Per the interface, size == 0 merely faults the access. */
     if (size == 0) {
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
     }
 
     if (unlikely(flags & (TLB_NOTDIRTY | TLB_WATCHPOINT))) {
-        uintptr_t index = tlb_index(env, mmu_idx, addr);
-        CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index];
-
         /* Handle watchpoints.  */
         if (flags & TLB_WATCHPOINT) {
             int wp_access = (access_type == MMU_DATA_STORE
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
 void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr,
                         MMUAccessType access_type, int mmu_idx)
 {
+    CPUTLBEntryFull *full;
     void *host;
     int flags;
 
     flags = probe_access_internal(env, addr, 0, access_type,
-                                  mmu_idx, true, &host, 0);
+                                  mmu_idx, true, &host, &full, 0);
 
     /* No combination of flags are expected by the caller. */
     return flags ? NULL : host;
@@ -XXX,XX +XXX,XX @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr,
 tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, target_ulong addr,
                                         void **hostp)
 {
+    CPUTLBEntryFull *full;
     void *p;
 
     (void)probe_access_internal(env, addr, 1, MMU_INST_FETCH,
-                                cpu_mmu_index(env, true), false, &p, 0);
+                                cpu_mmu_index(env, true), false, &p, &full, 0);
     if (p == NULL) {
         return -1;
     }
-- 
2.34.1

Now that we have collected all of the page data into
CPUTLBEntryFull, provide an interface to record that
all in one go, instead of using 4 arguments.  This interface
allows CPUTLBEntryFull to be extended without having to
change the number of arguments.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-defs.h | 14 +++++++++++
 include/exec/exec-all.h | 22 ++++++++++++++++++
 accel/tcg/cputlb.c      | 51 ++++++++++++++++++++++++++---------------
 3 files changed, 69 insertions(+), 18 deletions(-)

diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-defs.h
+++ b/include/exec/cpu-defs.h
@@ -XXX,XX +XXX,XX @@ typedef struct CPUTLBEntryFull {
      *     + the offset within the target MemoryRegion (otherwise)
      */
     hwaddr xlat_section;
+
+    /*
+     * @phys_addr contains the physical address in the address space
+     * given by cpu_asidx_from_attrs(cpu, @attrs).
+     */
+    hwaddr phys_addr;
+
+    /* @attrs contains the memory transaction attributes for the page. */
     MemTxAttrs attrs;
+
+    /* @prot contains the complete protections for the page. */
+    uint8_t prot;
+
+    /* @lg_page_size contains the log2 of the page size. */
+    uint8_t lg_page_size;
 } CPUTLBEntryFull;
 
 /*
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -XXX,XX +XXX,XX @@ void tlb_flush_range_by_mmuidx_all_cpus_synced(CPUState *cpu,
                                                uint16_t idxmap,
                                                unsigned bits);
 
+/**
+ * tlb_set_page_full:
+ * @cpu: CPU context
+ * @mmu_idx: mmu index of the tlb to modify
+ * @vaddr: virtual address of the entry to add
+ * @full: the details of the tlb entry
+ *
+ * Add an entry to @cpu tlb index @mmu_idx.  All of the fields of
+ * @full must be filled, except for xlat_section, and constitute
+ * the complete description of the translated page.
+ *
+ * This is generally called by the target tlb_fill function after
+ * having performed a successful page table walk to find the physical
+ * address and attributes for the translation.
+ *
+ * At most one entry for a given virtual address is permitted. Only a
+ * single TARGET_PAGE_SIZE region is mapped; @full->lg_page_size is only
+ * used by tlb_flush_page.
+ */
+void tlb_set_page_full(CPUState *cpu, int mmu_idx, target_ulong vaddr,
+                       CPUTLBEntryFull *full);
+
 /**
  * tlb_set_page_with_attrs:
  * @cpu: CPU to add this TLB entry for
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tlb_add_large_page(CPUArchState *env, int mmu_idx,
     env_tlb(env)->d[mmu_idx].large_page_mask = lp_mask;
 }
 
-/* Add a new TLB entry. At most one entry for a given virtual address
+/*
+ * Add a new TLB entry. At most one entry for a given virtual address
  * is permitted. Only a single TARGET_PAGE_SIZE region is mapped, the
  * supplied size is only used by tlb_flush_page.
  *
  * Called from TCG-generated code, which is under an RCU read-side
  * critical section.
  */
-void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
-                             hwaddr paddr, MemTxAttrs attrs, int prot,
-                             int mmu_idx, target_ulong size)
+void tlb_set_page_full(CPUState *cpu, int mmu_idx,
+                       target_ulong vaddr, CPUTLBEntryFull *full)
 {
     CPUArchState *env = cpu->env_ptr;
     CPUTLB *tlb = env_tlb(env);
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
     CPUTLBEntry *te, tn;
     hwaddr iotlb, xlat, sz, paddr_page;
     target_ulong vaddr_page;
-    int asidx = cpu_asidx_from_attrs(cpu, attrs);
-    int wp_flags;
+    int asidx, wp_flags, prot;
     bool is_ram, is_romd;
 
     assert_cpu_is_self(cpu);
 
-    if (size <= TARGET_PAGE_SIZE) {
+    if (full->lg_page_size <= TARGET_PAGE_BITS) {
         sz = TARGET_PAGE_SIZE;
     } else {
-        tlb_add_large_page(env, mmu_idx, vaddr, size);
-        sz = size;
+        sz = (hwaddr)1 << full->lg_page_size;
+        tlb_add_large_page(env, mmu_idx, vaddr, sz);
     }
     vaddr_page = vaddr & TARGET_PAGE_MASK;
-    paddr_page = paddr & TARGET_PAGE_MASK;
+    paddr_page = full->phys_addr & TARGET_PAGE_MASK;
 
+    prot = full->prot;
+    asidx = cpu_asidx_from_attrs(cpu, full->attrs);
     section = address_space_translate_for_iotlb(cpu, asidx, paddr_page,
-                                                &xlat, &sz, attrs, &prot);
+                                                &xlat, &sz, full->attrs, &prot);
     assert(sz >= TARGET_PAGE_SIZE);
 
     tlb_debug("vaddr=" TARGET_FMT_lx " paddr=0x" TARGET_FMT_plx
               " prot=%x idx=%d\n",
-              vaddr, paddr, prot, mmu_idx);
+              vaddr, full->phys_addr, prot, mmu_idx);
 
     address = vaddr_page;
-    if (size < TARGET_PAGE_SIZE) {
+    if (full->lg_page_size < TARGET_PAGE_BITS) {
         /* Repeat the MMU check and TLB fill on every access.  */
         address |= TLB_INVALID_MASK;
     }
-    if (attrs.byte_swap) {
+    if (full->attrs.byte_swap) {
         address |= TLB_BSWAP;
     }
 
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
      * subtract here is that of the page base, and not the same as the
      * vaddr we add back in io_readx()/io_writex()/get_page_addr_code().
      */
+    desc->fulltlb[index] = *full;
     desc->fulltlb[index].xlat_section = iotlb - vaddr_page;
-    desc->fulltlb[index].attrs = attrs;
+    desc->fulltlb[index].phys_addr = paddr_page;
+    desc->fulltlb[index].prot = prot;
 
     /* Now calculate the new entry */
     tn.addend = addend - vaddr_page;
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
     qemu_spin_unlock(&tlb->c.lock);
 }
 
-/* Add a new TLB entry, but without specifying the memory
- * transaction attributes to be used.
- */
+void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
+                             hwaddr paddr, MemTxAttrs attrs, int prot,
+                             int mmu_idx, target_ulong size)
+{
+    CPUTLBEntryFull full = {
+        .phys_addr = paddr,
+        .attrs = attrs,
+        .prot = prot,
+        .lg_page_size = ctz64(size)
+    };
+
+    assert(is_power_of_2(size));
+    tlb_set_page_full(cpu, mmu_idx, vaddr, &full);
+}
+
 void tlb_set_page(CPUState *cpu, target_ulong vaddr,
                   hwaddr paddr, int prot,
                   int mmu_idx, target_ulong size)
-- 
2.34.1

This bitmap is created and discarded immediately.
We gain nothing by its existence.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220822232338.1727934-2-richard.henderson@linaro.org>
---
 accel/tcg/translate-all.c | 78 ++-------------------------------------
 1 file changed, 4 insertions(+), 74 deletions(-)

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@
 #define assert_memory_lock() tcg_debug_assert(have_mmap_lock())
 #endif
 
-#define SMC_BITMAP_USE_THRESHOLD 10
-
 typedef struct PageDesc {
     /* list of TBs intersecting this ram page */
     uintptr_t first_tb;
-#ifdef CONFIG_SOFTMMU
-    /* in order to optimize self modifying code, we count the number
-       of lookups we do to a given page to use a bitmap */
-    unsigned long *code_bitmap;
-    unsigned int code_write_count;
-#else
+#ifdef CONFIG_USER_ONLY
     unsigned long flags;
     void *target_data;
 #endif
-#ifndef CONFIG_USER_ONLY
+#ifdef CONFIG_SOFTMMU
     QemuSpin lock;
 #endif
 } PageDesc;
@@ -XXX,XX +XXX,XX @@ void tb_htable_init(void)
     qht_init(&tb_ctx.htable, tb_cmp, CODE_GEN_HTABLE_SIZE, mode);
 }
 
-/* call with @p->lock held */
-static inline void invalidate_page_bitmap(PageDesc *p)
-{
-    assert_page_locked(p);
-#ifdef CONFIG_SOFTMMU
-    g_free(p->code_bitmap);
-    p->code_bitmap = NULL;
-    p->code_write_count = 0;
-#endif
-}
-
 /* Set to NULL all the 'first_tb' fields in all PageDescs. */
 static void page_flush_tb_1(int level, void **lp)
 {
@@ -XXX,XX +XXX,XX @@ static void page_flush_tb_1(int level, void **lp)
         for (i = 0; i < V_L2_SIZE; ++i) {
             page_lock(&pd[i]);
             pd[i].first_tb = (uintptr_t)NULL;
-            invalidate_page_bitmap(pd + i);
             page_unlock(&pd[i]);
         }
     } else {
@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
     if (rm_from_page_list) {
         p = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS);
         tb_page_remove(p, tb);
-        invalidate_page_bitmap(p);
         if (tb->page_addr[1] != -1) {
             p = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS);
             tb_page_remove(p, tb);
-            invalidate_page_bitmap(p);
         }
     }
 
@@ -XXX,XX +XXX,XX @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
     }
 }
 
-#ifdef CONFIG_SOFTMMU
-/* call with @p->lock held */
-static void build_page_bitmap(PageDesc *p)
-{
-    int n, tb_start, tb_end;
-    TranslationBlock *tb;
-
-    assert_page_locked(p);
-    p->code_bitmap = bitmap_new(TARGET_PAGE_SIZE);
-
-    PAGE_FOR_EACH_TB(p, tb, n) {
-        /* NOTE: this is subtle as a TB may span two physical pages */
-        if (n == 0) {
-            /* NOTE: tb_end may be after the end of the page, but
-               it is not a problem */
-            tb_start = tb->pc & ~TARGET_PAGE_MASK;
-            tb_end = tb_start + tb->size;
-            if (tb_end > TARGET_PAGE_SIZE) {
-                tb_end = TARGET_PAGE_SIZE;
-             }
-        } else {
-            tb_start = 0;
-            tb_end = ((tb->pc + tb->size) & ~TARGET_PAGE_MASK);
-        }
-        bitmap_set(p->code_bitmap, tb_start, tb_end - tb_start);
-    }
-}
-#endif
-
 /* add the tb in the target page and protect it if necessary
  *
  * Called with mmap_lock held for user-mode emulation.
@@ -XXX,XX +XXX,XX @@ static inline void tb_page_add(PageDesc *p, TranslationBlock *tb,
     page_already_protected = p->first_tb != (uintptr_t)NULL;
 #endif
     p->first_tb = (uintptr_t)tb | n;
-    invalidate_page_bitmap(p);
 
 #if defined(CONFIG_USER_ONLY)
     /* translator_loop() must have made all TB pages non-writable */
@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
     /* remove TB from the page(s) if we couldn't insert it */
     if (unlikely(existing_tb)) {
         tb_page_remove(p, tb);
-        invalidate_page_bitmap(p);
         if (p2) {
             tb_page_remove(p2, tb);
-            invalidate_page_bitmap(p2);
         }
         tb = existing_tb;
     }
@@ -XXX,XX +XXX,XX @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
 #if !defined(CONFIG_USER_ONLY)
     /* if no code remaining, no need to continue to use slow writes */
     if (!p->first_tb) {
-        invalidate_page_bitmap(p);
         tlb_unprotect_code(start);
     }
 #endif
@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_page_fast(struct page_collection *pages,
     }
 
     assert_page_locked(p);
-    if (!p->code_bitmap &&
-        ++p->code_write_count >= SMC_BITMAP_USE_THRESHOLD) {
-        build_page_bitmap(p);
-    }
-    if (p->code_bitmap) {
-        unsigned int nr;
-        unsigned long b;
-
-        nr = start & ~TARGET_PAGE_MASK;
-        b = p->code_bitmap[BIT_WORD(nr)] >> (nr & (BITS_PER_LONG - 1));
-        if (b & ((1 << len) - 1)) {
-            goto do_invalidate;
-        }
-    } else {
-    do_invalidate:
-        tb_invalidate_phys_page_range__locked(pages, p, start, start + len,
-                                              retaddr);
-    }
+    tb_invalidate_phys_page_range__locked(pages, p, start, start + len,
+                                          retaddr);
 }
 #else
 /* Called with mmap_lock held. If pc is not 0 then it indicates the
-- 
2.34.1

Bool is more appropriate type for the alloc parameter.

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ void page_init(void)
 #endif
 }
 
-static PageDesc *page_find_alloc(tb_page_addr_t index, int alloc)
+static PageDesc *page_find_alloc(tb_page_addr_t index, bool alloc)
 {
     PageDesc *pd;
     void **lp;
@@ -XXX,XX +XXX,XX @@ static PageDesc *page_find_alloc(tb_page_addr_t index, int alloc)
 
 static inline PageDesc *page_find(tb_page_addr_t index)
 {
-    return page_find_alloc(index, 0);
+    return page_find_alloc(index, false);
 }
 
 static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
-                           PageDesc **ret_p2, tb_page_addr_t phys2, int alloc);
+                           PageDesc **ret_p2, tb_page_addr_t phys2, bool alloc);
 
 /* In user-mode page locks aren't used; mmap_lock is enough */
 #ifdef CONFIG_USER_ONLY
@@ -XXX,XX +XXX,XX @@ static inline void page_unlock(PageDesc *pd)
 /* lock the page(s) of a TB in the correct acquisition order */
 static inline void page_lock_tb(const TranslationBlock *tb)
 {
-    page_lock_pair(NULL, tb->page_addr[0], NULL, tb->page_addr[1], 0);
+    page_lock_pair(NULL, tb->page_addr[0], NULL, tb->page_addr[1], false);
 }
 
 static inline void page_unlock_tb(const TranslationBlock *tb)
@@ -XXX,XX +XXX,XX @@ void page_collection_unlock(struct page_collection *set)
 #endif /* !CONFIG_USER_ONLY */
 
 static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
-                           PageDesc **ret_p2, tb_page_addr_t phys2, int alloc)
+                           PageDesc **ret_p2, tb_page_addr_t phys2, bool alloc)
 {
     PageDesc *p1, *p2;
     tb_page_addr_t page1;
@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
      * Note that inserting into the hash table first isn't an option, since
      * we can only insert TBs that are fully initialized.
      */
-    page_lock_pair(&p, phys_pc, &p2, phys_page2, 1);
+    page_lock_pair(&p, phys_pc, &p2, phys_page2, true);
     tb_page_add(p, tb, 0, phys_pc & TARGET_PAGE_MASK);
     if (p2) {
         tb_page_add(p2, tb, 1, phys_page2);
@@ -XXX,XX +XXX,XX @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
     for (addr = start, len = end - start;
          len != 0;
          len -= TARGET_PAGE_SIZE, addr += TARGET_PAGE_SIZE) {
-        PageDesc *p = page_find_alloc(addr >> TARGET_PAGE_BITS, 1);
+        PageDesc *p = page_find_alloc(addr >> TARGET_PAGE_BITS, true);
 
         /* If the write protection bit is set, then we invalidate
            the code inside.  */
-- 
2.34.1

Use the pc coming from db->pc_first rather than the TB.

Use the cached host_addr rather than re-computing for the
first page.  We still need a separate lookup for the second
page because it won't be computed for DisasContextBase until
the translator actually performs a read from the page.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/plugin-gen.h |  7 ++++---
 accel/tcg/plugin-gen.c    | 22 +++++++++++-----------
 accel/tcg/translator.c    |  2 +-
 3 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/include/exec/plugin-gen.h b/include/exec/plugin-gen.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/plugin-gen.h
+++ b/include/exec/plugin-gen.h
@@ -XXX,XX +XXX,XX @@ struct DisasContextBase;
 
 #ifdef CONFIG_PLUGIN
 
-bool plugin_gen_tb_start(CPUState *cpu, const TranslationBlock *tb, bool supress);
+bool plugin_gen_tb_start(CPUState *cpu, const struct DisasContextBase *db,
+                         bool supress);
 void plugin_gen_tb_end(CPUState *cpu);
 void plugin_gen_insn_start(CPUState *cpu, const struct DisasContextBase *db);
 void plugin_gen_insn_end(void);
@@ -XXX,XX +XXX,XX @@ static inline void plugin_insn_append(abi_ptr pc, const void *from, size_t size)
 
 #else /* !CONFIG_PLUGIN */
 
-static inline
-bool plugin_gen_tb_start(CPUState *cpu, const TranslationBlock *tb, bool supress)
+static inline bool
+plugin_gen_tb_start(CPUState *cpu, const struct DisasContextBase *db, bool sup)
 {
     return false;
 }
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(const struct qemu_plugin_tb *plugin_tb)
     pr_ops();
 }
 
-bool plugin_gen_tb_start(CPUState *cpu, const TranslationBlock *tb, bool mem_only)
+bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
+                         bool mem_only)
 {
     bool ret = false;
 
@@ -XXX,XX +XXX,XX @@ bool plugin_gen_tb_start(CPUState *cpu, const TranslationBlock *tb, bool mem_onl
 
         ret = true;
 
-        ptb->vaddr = tb->pc;
+        ptb->vaddr = db->pc_first;
         ptb->vaddr2 = -1;
-        get_page_addr_code_hostp(cpu->env_ptr, tb->pc, &ptb->haddr1);
+        ptb->haddr1 = db->host_addr[0];
         ptb->haddr2 = NULL;
         ptb->mem_only = mem_only;
 
@@ -XXX,XX +XXX,XX @@ void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
      * Note that we skip this when haddr1 == NULL, e.g. when we're
      * fetching instructions from a region not backed by RAM.
      */
-    if (likely(ptb->haddr1 != NULL && ptb->vaddr2 == -1) &&
-        unlikely((db->pc_next & TARGET_PAGE_MASK) !=
-                 (db->pc_first & TARGET_PAGE_MASK))) {
-        get_page_addr_code_hostp(cpu->env_ptr, db->pc_next,
-                                 &ptb->haddr2);
-        ptb->vaddr2 = db->pc_next;
-    }
-    if (likely(ptb->vaddr2 == -1)) {
+    if (ptb->haddr1 == NULL) {
+        pinsn->haddr = NULL;
+    } else if (is_same_page(db, db->pc_next)) {
         pinsn->haddr = ptb->haddr1 + pinsn->vaddr - ptb->vaddr;
     } else {
+        if (ptb->vaddr2 == -1) {
+            ptb->vaddr2 = TARGET_PAGE_ALIGN(db->pc_first);
+            get_page_addr_code_hostp(cpu->env_ptr, ptb->vaddr2, &ptb->haddr2);
+        }
         pinsn->haddr = ptb->haddr2 + pinsn->vaddr - ptb->vaddr2;
     }
 }
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -XXX,XX +XXX,XX @@ void translator_loop(CPUState *cpu, TranslationBlock *tb, int max_insns,
     ops->tb_start(db, cpu);
     tcg_debug_assert(db->is_jmp == DISAS_NEXT);  /* no early exit */
 
-    plugin_enabled = plugin_gen_tb_start(cpu, tb, cflags & CF_MEMI_ONLY);
+    plugin_enabled = plugin_gen_tb_start(cpu, db, cflags & CF_MEMI_ONLY);
 
     while (true) {
         db->num_insns++;
-- 
2.34.1

Let tb->page_addr[0] contain the address of the first byte of the
translated block, rather than the address of the page containing the
start of the translated block.  We need to recover this value anyway
at various points, and it is easier to discard a page offset when it
is not needed, which happens naturally via the existing find_page shift.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/cpu-exec.c      | 16 ++++++++--------
 accel/tcg/cputlb.c        |  3 ++-
 accel/tcg/translate-all.c |  9 +++++----
 3 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -XXX,XX +XXX,XX @@ struct tb_desc {
     target_ulong pc;
     target_ulong cs_base;
     CPUArchState *env;
-    tb_page_addr_t phys_page1;
+    tb_page_addr_t page_addr0;
     uint32_t flags;
     uint32_t cflags;
     uint32_t trace_vcpu_dstate;
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
     const struct tb_desc *desc = d;
 
     if (tb->pc == desc->pc &&
-        tb->page_addr[0] == desc->phys_page1 &&
+        tb->page_addr[0] == desc->page_addr0 &&
         tb->cs_base == desc->cs_base &&
         tb->flags == desc->flags &&
         tb->trace_vcpu_dstate == desc->trace_vcpu_dstate &&
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
         if (tb->page_addr[1] == -1) {
             return true;
         } else {
-            tb_page_addr_t phys_page2;
-            target_ulong virt_page2;
+            tb_page_addr_t phys_page1;
+            target_ulong virt_page1;
 
             /*
              * We know that the first page matched, and an otherwise valid TB
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
              * is different for the new TB.  Therefore any exception raised
              * here by the faulting lookup is not premature.
              */
-            virt_page2 = TARGET_PAGE_ALIGN(desc->pc);
-            phys_page2 = get_page_addr_code(desc->env, virt_page2);
-            if (tb->page_addr[1] == phys_page2) {
+            virt_page1 = TARGET_PAGE_ALIGN(desc->pc);
+            phys_page1 = get_page_addr_code(desc->env, virt_page1);
+            if (tb->page_addr[1] == phys_page1) {
                 return true;
             }
         }
@@ -XXX,XX +XXX,XX @@ static TranslationBlock *tb_htable_lookup(CPUState *cpu, target_ulong pc,
     if (phys_pc == -1) {
         return NULL;
     }
-    desc.phys_page1 = phys_pc & TARGET_PAGE_MASK;
+    desc.page_addr0 = phys_pc;
     h = tb_hash_func(phys_pc, pc, flags, cflags, *cpu->trace_dstate);
     return qht_lookup_custom(&tb_ctx.htable, &desc, h, tb_lookup_cmp);
 }
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ void tlb_flush_page_bits_by_mmuidx_all_cpus_synced(CPUState *src_cpu,
    can be detected */
 void tlb_protect_code(ram_addr_t ram_addr)
 {
-    cpu_physical_memory_test_and_clear_dirty(ram_addr, TARGET_PAGE_SIZE,
+    cpu_physical_memory_test_and_clear_dirty(ram_addr & TARGET_PAGE_MASK,
+                                             TARGET_PAGE_SIZE,
                                              DIRTY_MEMORY_CODE);
 }
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
     qemu_spin_unlock(&tb->jmp_lock);
 
     /* remove the TB from the hash list */
-    phys_pc = tb->page_addr[0] + (tb->pc & ~TARGET_PAGE_MASK);
+    phys_pc = tb->page_addr[0];
     h = tb_hash_func(phys_pc, tb->pc, tb->flags, orig_cflags,
                      tb->trace_vcpu_dstate);
     if (!qht_remove(&tb_ctx.htable, tb, h)) {
@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
      * we can only insert TBs that are fully initialized.
      */
     page_lock_pair(&p, phys_pc, &p2, phys_page2, true);
-    tb_page_add(p, tb, 0, phys_pc & TARGET_PAGE_MASK);
+    tb_page_add(p, tb, 0, phys_pc);
     if (p2) {
         tb_page_add(p2, tb, 1, phys_page2);
     } else {
@@ -XXX,XX +XXX,XX @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
         if (n == 0) {
             /* NOTE: tb_end may be after the end of the page, but
                it is not a problem */
-            tb_start = tb->page_addr[0] + (tb->pc & ~TARGET_PAGE_MASK);
+            tb_start = tb->page_addr[0];
             tb_end = tb_start + tb->size;
         } else {
             tb_start = tb->page_addr[1];
-            tb_end = tb_start + ((tb->pc + tb->size) & ~TARGET_PAGE_MASK);
+            tb_end = tb_start + ((tb->page_addr[0] + tb->size)
+                                 & ~TARGET_PAGE_MASK);
         }
         if (!(tb_end <= start || tb_start >= end)) {
 #ifdef TARGET_HAS_PRECISE_SMC
-- 
2.34.1

This function has two users, who use it incompatibly.
In tlb_flush_page_by_mmuidx_async_0, when flushing a
single page, we need to flush exactly two pages.
In tlb_flush_range_by_mmuidx_async_0, when flushing a
range of pages, we need to flush N+1 pages.

This avoids double-flushing of jmp cache pages in a range.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/cputlb.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tb_jmp_cache_clear_page(CPUState *cpu, target_ulong page_addr)
     }
 }
 
-static void tb_flush_jmp_cache(CPUState *cpu, target_ulong addr)
-{
-    /* Discard jump cache entries for any tb which might potentially
-       overlap the flushed page.  */
-    tb_jmp_cache_clear_page(cpu, addr - TARGET_PAGE_SIZE);
-    tb_jmp_cache_clear_page(cpu, addr);
-}
-
 /**
  * tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary
  * @desc: The CPUTLBDesc portion of the TLB
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_page_by_mmuidx_async_0(CPUState *cpu,
     }
     qemu_spin_unlock(&env_tlb(env)->c.lock);
 
-    tb_flush_jmp_cache(cpu, addr);
+    /*
+     * Discard jump cache entries for any tb which might potentially
+     * overlap the flushed page, which includes the previous.
+     */
+    tb_jmp_cache_clear_page(cpu, addr - TARGET_PAGE_SIZE);
+    tb_jmp_cache_clear_page(cpu, addr);
 }
 
 /**
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_by_mmuidx_async_0(CPUState *cpu,
         return;
     }
 
-    for (target_ulong i = 0; i < d.len; i += TARGET_PAGE_SIZE) {
-        tb_flush_jmp_cache(cpu, d.addr + i);
+    /*
+     * Discard jump cache entries for any tb which might potentially
+     * overlap the flushed pages, which includes the previous.
+     */
+    d.addr -= TARGET_PAGE_SIZE;
+    for (target_ulong i = 0, n = d.len / TARGET_PAGE_SIZE + 1; i < n; i++) {
+        tb_jmp_cache_clear_page(cpu, d.addr);
+        d.addr += TARGET_PAGE_SIZE;
     }
 }
 
-- 
2.34.1

Wrap the bare TranslationBlock pointer into a structure.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/tb-hash.h       |  1 +
 accel/tcg/tb-jmp-cache.h  | 24 ++++++++++++++++++++++++
 include/exec/cpu-common.h |  1 +
 include/hw/core/cpu.h     | 15 +--------------
 include/qemu/typedefs.h   |  1 +
 accel/stubs/tcg-stub.c    |  4 ++++
 accel/tcg/cpu-exec.c      | 10 +++++++---
 accel/tcg/cputlb.c        |  9 +++++----
 accel/tcg/translate-all.c | 28 +++++++++++++++++++++++++---
 hw/core/cpu-common.c      |  3 +--
 plugins/core.c            |  2 +-
 trace/control-target.c    |  2 +-
 12 files changed, 72 insertions(+), 28 deletions(-)
 create mode 100644 accel/tcg/tb-jmp-cache.h

diff --git a/accel/tcg/tb-hash.h b/accel/tcg/tb-hash.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tb-hash.h
+++ b/accel/tcg/tb-hash.h
@@ -XXX,XX +XXX,XX @@
 #include "exec/cpu-defs.h"
 #include "exec/exec-all.h"
 #include "qemu/xxhash.h"
+#include "tb-jmp-cache.h"
 
 #ifdef CONFIG_SOFTMMU
 
diff --git a/accel/tcg/tb-jmp-cache.h b/accel/tcg/tb-jmp-cache.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/accel/tcg/tb-jmp-cache.h
@@ -XXX,XX +XXX,XX @@
+/*
+ * The per-CPU TranslationBlock jump cache.
+ *
+ *  Copyright (c) 2003 Fabrice Bellard
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef ACCEL_TCG_TB_JMP_CACHE_H
+#define ACCEL_TCG_TB_JMP_CACHE_H
+
+#define TB_JMP_CACHE_BITS 12
+#define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS)
+
+/*
+ * Accessed in parallel; all accesses to 'tb' must be atomic.
+ */
+struct CPUJumpCache {
+    struct {
+        TranslationBlock *tb;
+    } array[TB_JMP_CACHE_SIZE];
+};
+
+#endif /* ACCEL_TCG_TB_JMP_CACHE_H */
diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -XXX,XX +XXX,XX @@ void cpu_list_unlock(void);
 unsigned int cpu_list_generation_id_get(void);
 
 void tcg_flush_softmmu_tlb(CPUState *cs);
+void tcg_flush_jmp_cache(CPUState *cs);
 
 void tcg_iommu_init_notifier_list(CPUState *cpu);
 void tcg_iommu_free_notifier_list(CPUState *cpu);
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index XXXXXXX..XXXXXXX 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -XXX,XX +XXX,XX @@ struct kvm_run;
 struct hax_vcpu_state;
 struct hvf_vcpu_state;
 
-#define TB_JMP_CACHE_BITS 12
-#define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS)
-
 /* work queue */
 
 /* The union type allows passing of 64 bit target pointers on 32 bit
@@ -XXX,XX +XXX,XX @@ struct CPUState {
     CPUArchState *env_ptr;
     IcountDecr *icount_decr_ptr;
 
-    /* Accessed in parallel; all accesses must be atomic */
-    TranslationBlock *tb_jmp_cache[TB_JMP_CACHE_SIZE];
+    CPUJumpCache *tb_jmp_cache;
 
     struct GDBRegisterState *gdb_regs;
     int gdb_num_regs;
@@ -XXX,XX +XXX,XX @@ extern CPUTailQ cpus;
 
 extern __thread CPUState *current_cpu;
 
-static inline void cpu_tb_jmp_cache_clear(CPUState *cpu)
-{
-    unsigned int i;
-
-    for (i = 0; i < TB_JMP_CACHE_SIZE; i++) {
-        qatomic_set(&cpu->tb_jmp_cache[i], NULL);
-    }
-}
-
 /**
  * qemu_tcg_mttcg_enabled:
  * Check whether we are running MultiThread TCG or not.
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -XXX,XX +XXX,XX @@ typedef struct CoMutex CoMutex;
 typedef struct ConfidentialGuestSupport ConfidentialGuestSupport;
 typedef struct CPUAddressSpace CPUAddressSpace;
 typedef struct CPUArchState CPUArchState;
+typedef struct CPUJumpCache CPUJumpCache;
 typedef struct CPUState CPUState;
 typedef struct CPUTLBEntryFull CPUTLBEntryFull;
 typedef struct DeviceListener DeviceListener;
diff --git a/accel/stubs/tcg-stub.c b/accel/stubs/tcg-stub.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/stubs/tcg-stub.c
+++ b/accel/stubs/tcg-stub.c
@@ -XXX,XX +XXX,XX @@ void tlb_set_dirty(CPUState *cpu, target_ulong vaddr)
 {
 }
 
+void tcg_flush_jmp_cache(CPUState *cpu)
+{
+}
+
 int probe_access_flags(CPUArchState *env, target_ulong addr,
                        MMUAccessType access_type, int mmu_idx,
                        bool nonfault, void **phost, uintptr_t retaddr)
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -XXX,XX +XXX,XX @@
 #include "sysemu/replay.h"
 #include "sysemu/tcg.h"
 #include "exec/helper-proto.h"
+#include "tb-jmp-cache.h"
 #include "tb-hash.h"
 #include "tb-context.h"
 #include "internal.h"
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
     tcg_debug_assert(!(cflags & CF_INVALID));
 
     hash = tb_jmp_cache_hash_func(pc);
-    tb = qatomic_rcu_read(&cpu->tb_jmp_cache[hash]);
+    tb = qatomic_rcu_read(&cpu->tb_jmp_cache->array[hash].tb);
 
     if (likely(tb &&
                tb->pc == pc &&
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
     if (tb == NULL) {
         return NULL;
     }
-    qatomic_set(&cpu->tb_jmp_cache[hash], tb);
+    qatomic_set(&cpu->tb_jmp_cache->array[hash].tb, tb);
     return tb;
 }
 
@@ -XXX,XX +XXX,XX @@ int cpu_exec(CPUState *cpu)
 
             tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
             if (tb == NULL) {
+                uint32_t h;
+
                 mmap_lock();
                 tb = tb_gen_code(cpu, pc, cs_base, flags, cflags);
                 mmap_unlock();
@@ -XXX,XX +XXX,XX @@ int cpu_exec(CPUState *cpu)
                  * We add the TB in the virtual pc hash table
                  * for the fast lookup
                  */
-                qatomic_set(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)], tb);
+                h = tb_jmp_cache_hash_func(pc);
+                qatomic_set(&cpu->tb_jmp_cache->array[h].tb, tb);
             }
 
 #ifndef CONFIG_USER_ONLY
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
 
 static void tb_jmp_cache_clear_page(CPUState *cpu, target_ulong page_addr)
 {
-    unsigned int i, i0 = tb_jmp_cache_hash_page(page_addr);
+    int i, i0 = tb_jmp_cache_hash_page(page_addr);
+    CPUJumpCache *jc = cpu->tb_jmp_cache;
 
     for (i = 0; i < TB_JMP_PAGE_SIZE; i++) {
-        qatomic_set(&cpu->tb_jmp_cache[i0 + i], NULL);
+        qatomic_set(&jc->array[i0 + i].tb, NULL);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
 
     qemu_spin_unlock(&env_tlb(env)->c.lock);
 
-    cpu_tb_jmp_cache_clear(cpu);
+    tcg_flush_jmp_cache(cpu);
 
     if (to_clean == ALL_MMUIDX_BITS) {
         qatomic_set(&env_tlb(env)->c.full_flush_count,
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_by_mmuidx_async_0(CPUState *cpu,
      * longer to clear each entry individually than it will to clear it all.
      */
     if (d.len >= (TARGET_PAGE_SIZE * TB_JMP_CACHE_SIZE)) {
-        cpu_tb_jmp_cache_clear(cpu);
+        tcg_flush_jmp_cache(cpu);
         return;
     }
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@
 #include "sysemu/tcg.h"
 #include "qapi/error.h"
 #include "hw/core/tcg-cpu-ops.h"
+#include "tb-jmp-cache.h"
 #include "tb-hash.h"
 #include "tb-context.h"
 #include "internal.h"
@@ -XXX,XX +XXX,XX @@ static void do_tb_flush(CPUState *cpu, run_on_cpu_data tb_flush_count)
     }
 
     CPU_FOREACH(cpu) {
-        cpu_tb_jmp_cache_clear(cpu);
+        tcg_flush_jmp_cache(cpu);
     }
 
     qht_reset_size(&tb_ctx.htable, CODE_GEN_HTABLE_SIZE);
@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
     /* remove the TB from the hash list */
     h = tb_jmp_cache_hash_func(tb->pc);
     CPU_FOREACH(cpu) {
-        if (qatomic_read(&cpu->tb_jmp_cache[h]) == tb) {
-            qatomic_set(&cpu->tb_jmp_cache[h], NULL);
+        CPUJumpCache *jc = cpu->tb_jmp_cache;
+        if (qatomic_read(&jc->array[h].tb) == tb) {
+            qatomic_set(&jc->array[h].tb, NULL);
         }
     }
 
@@ -XXX,XX +XXX,XX @@ int page_unprotect(target_ulong address, uintptr_t pc)
 }
 #endif /* CONFIG_USER_ONLY */
 
+/*
+ * Called by generic code at e.g. cpu reset after cpu creation,
+ * therefore we must be prepared to allocate the jump cache.
+ */
+void tcg_flush_jmp_cache(CPUState *cpu)
+{
+    CPUJumpCache *jc = cpu->tb_jmp_cache;
+
+    if (likely(jc)) {
+        for (int i = 0; i < TB_JMP_CACHE_SIZE; i++) {
+            qatomic_set(&jc->array[i].tb, NULL);
+        }
+    } else {
+        /* This should happen once during realize, and thus never race. */
+        jc = g_new0(CPUJumpCache, 1);
+        jc = qatomic_xchg(&cpu->tb_jmp_cache, jc);
+        assert(jc == NULL);
+    }
+}
+
 /* This is a wrapper for common code that can not use CONFIG_SOFTMMU */
 void tcg_flush_softmmu_tlb(CPUState *cs)
 {
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -XXX,XX +XXX,XX @@ static void cpu_common_reset(DeviceState *dev)
     cpu->cflags_next_tb = -1;
 
     if (tcg_enabled()) {
-        cpu_tb_jmp_cache_clear(cpu);
-
+        tcg_flush_jmp_cache(cpu);
         tcg_flush_softmmu_tlb(cpu);
     }
 }
diff --git a/plugins/core.c b/plugins/core.c
index XXXXXXX..XXXXXXX 100644
--- a/plugins/core.c
+++ b/plugins/core.c
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_ctx *plugin_id_to_ctx_locked(qemu_plugin_id_t id)
 static void plugin_cpu_update__async(CPUState *cpu, run_on_cpu_data data)
 {
     bitmap_copy(cpu->plugin_mask, &data.host_ulong, QEMU_PLUGIN_EV_MAX);
-    cpu_tb_jmp_cache_clear(cpu);
+    tcg_flush_jmp_cache(cpu);
 }
 
 static void plugin_cpu_update__locked(gpointer k, gpointer v, gpointer udata)
diff --git a/trace/control-target.c b/trace/control-target.c
index XXXXXXX..XXXXXXX 100644
--- a/trace/control-target.c
+++ b/trace/control-target.c
@@ -XXX,XX +XXX,XX @@ static void trace_event_synchronize_vcpu_state_dynamic(
 {
     bitmap_copy(vcpu->trace_dstate, vcpu->trace_dstate_delayed,
                 CPU_TRACE_DSTATE_MAX_EVENTS);
-    cpu_tb_jmp_cache_clear(vcpu);
+    tcg_flush_jmp_cache(vcpu);
 }
 
 void trace_event_set_vcpu_state_dynamic(CPUState *vcpu,
-- 
2.34.1

Populate this new method for all targets.  Always match
the result that would be given by cpu_get_tb_cpu_state,
as we will want these values to correspond in the logs.

Reviewed-by: Taylor Simpson <tsimpson@quicinc.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> (target/sparc)
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
Cc: Eduardo Habkost <eduardo@habkost.net> (supporter:Machine core)
Cc: Marcel Apfelbaum <marcel.apfelbaum@gmail.com> (supporter:Machine core)
Cc: "Philippe Mathieu-Daudé" <f4bug@amsat.org> (reviewer:Machine core)
Cc: Yanan Wang <wangyanan55@huawei.com> (reviewer:Machine core)
Cc: Michael Rolnik <mrolnik@gmail.com> (maintainer:AVR TCG CPUs)
Cc: "Edgar E. Iglesias" <edgar.iglesias@gmail.com> (maintainer:CRIS TCG CPUs)
Cc: Taylor Simpson <tsimpson@quicinc.com> (supporter:Hexagon TCG CPUs)
Cc: Song Gao <gaosong@loongson.cn> (maintainer:LoongArch TCG CPUs)
Cc: Xiaojuan Yang <yangxiaojuan@loongson.cn> (maintainer:LoongArch TCG CPUs)
Cc: Laurent Vivier <laurent@vivier.eu> (maintainer:M68K TCG CPUs)
Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> (reviewer:MIPS TCG CPUs)
Cc: Aleksandar Rikalo <aleksandar.rikalo@syrmia.com> (reviewer:MIPS TCG CPUs)
Cc: Chris Wulff <crwulff@gmail.com> (maintainer:NiosII TCG CPUs)
Cc: Marek Vasut <marex@denx.de> (maintainer:NiosII TCG CPUs)
Cc: Stafford Horne <shorne@gmail.com> (odd fixer:OpenRISC TCG CPUs)
Cc: Yoshinori Sato <ysato@users.sourceforge.jp> (reviewer:RENESAS RX CPUs)
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> (maintainer:SPARC TCG CPUs)
Cc: Bastian Koppelmann <kbastian@mail.uni-paderborn.de> (maintainer:TriCore TCG CPUs)
Cc: Max Filippov <jcmvbkbc@gmail.com> (maintainer:Xtensa TCG CPUs)
Cc: qemu-arm@nongnu.org (open list:ARM TCG CPUs)
Cc: qemu-ppc@nongnu.org (open list:PowerPC TCG CPUs)
Cc: qemu-riscv@nongnu.org (open list:RISC-V TCG CPUs)
Cc: qemu-s390x@nongnu.org (open list:S390 TCG CPUs)
---
 include/hw/core/cpu.h   |  3 +++
 target/alpha/cpu.c      |  9 +++++++++
 target/arm/cpu.c        | 13 +++++++++++++
 target/avr/cpu.c        |  8 ++++++++
 target/cris/cpu.c       |  8 ++++++++
 target/hexagon/cpu.c    |  8 ++++++++
 target/hppa/cpu.c       |  8 ++++++++
 target/i386/cpu.c       |  9 +++++++++
 target/loongarch/cpu.c  |  9 +++++++++
 target/m68k/cpu.c       |  8 ++++++++
 target/microblaze/cpu.c |  8 ++++++++
 target/mips/cpu.c       |  8 ++++++++
 target/nios2/cpu.c      |  9 +++++++++
 target/openrisc/cpu.c   |  8 ++++++++
 target/ppc/cpu_init.c   |  8 ++++++++
 target/riscv/cpu.c      | 13 +++++++++++++
 target/rx/cpu.c         |  8 ++++++++
 target/s390x/cpu.c      |  8 ++++++++
 target/sh4/cpu.c        |  8 ++++++++
 target/sparc/cpu.c      |  8 ++++++++
 target/tricore/cpu.c    |  9 +++++++++
 target/xtensa/cpu.c     |  8 ++++++++
 22 files changed, 186 insertions(+)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index XXXXXXX..XXXXXXX 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -XXX,XX +XXX,XX @@ struct SysemuCPUOps;
  *       If the target behaviour here is anything other than "set
  *       the PC register to the value passed in" then the target must
  *       also implement the synchronize_from_tb hook.
+ * @get_pc: Callback for getting the Program Counter register.
+ *       As above, with the semantics of the target architecture.
  * @gdb_read_register: Callback for letting GDB read a register.
  * @gdb_write_register: Callback for letting GDB write a register.
  * @gdb_adjust_breakpoint: Callback for adjusting the address of a
@@ -XXX,XX +XXX,XX @@ struct CPUClass {
     void (*dump_state)(CPUState *cpu, FILE *, int flags);
     int64_t (*get_arch_id)(CPUState *cpu);
     void (*set_pc)(CPUState *cpu, vaddr value);
+    vaddr (*get_pc)(CPUState *cpu);
     int (*gdb_read_register)(CPUState *cpu, GByteArray *buf, int reg);
     int (*gdb_write_register)(CPUState *cpu, uint8_t *buf, int reg);
     vaddr (*gdb_adjust_breakpoint)(CPUState *cpu, vaddr addr);
diff --git a/target/alpha/cpu.c b/target/alpha/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/alpha/cpu.c
+++ b/target/alpha/cpu.c
@@ -XXX,XX +XXX,XX @@ static void alpha_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc = value;
 }
 
+static vaddr alpha_cpu_get_pc(CPUState *cs)
+{
+    AlphaCPU *cpu = ALPHA_CPU(cs);
+
+    return cpu->env.pc;
+}
+
+
 static bool alpha_cpu_has_work(CPUState *cs)
 {
     /* Here we are checking to see if the CPU should wake up from HALT.
@@ -XXX,XX +XXX,XX @@ static void alpha_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = alpha_cpu_has_work;
     cc->dump_state = alpha_cpu_dump_state;
     cc->set_pc = alpha_cpu_set_pc;
+    cc->get_pc = alpha_cpu_get_pc;
     cc->gdb_read_register = alpha_cpu_gdb_read_register;
     cc->gdb_write_register = alpha_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -XXX,XX +XXX,XX @@ static void arm_cpu_set_pc(CPUState *cs, vaddr value)
     }
 }
 
+static vaddr arm_cpu_get_pc(CPUState *cs)
+{
+    ARMCPU *cpu = ARM_CPU(cs);
+    CPUARMState *env = &cpu->env;
+
+    if (is_a64(env)) {
+        return env->pc;
+    } else {
+        return env->regs[15];
+    }
+}
+
 #ifdef CONFIG_TCG
 void arm_cpu_synchronize_from_tb(CPUState *cs,
                                  const TranslationBlock *tb)
@@ -XXX,XX +XXX,XX @@ static void arm_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = arm_cpu_has_work;
     cc->dump_state = arm_cpu_dump_state;
     cc->set_pc = arm_cpu_set_pc;
+    cc->get_pc = arm_cpu_get_pc;
     cc->gdb_read_register = arm_cpu_gdb_read_register;
     cc->gdb_write_register = arm_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/avr/cpu.c b/target/avr/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/avr/cpu.c
+++ b/target/avr/cpu.c
@@ -XXX,XX +XXX,XX @@ static void avr_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc_w = value / 2; /* internally PC points to words */
 }
 
+static vaddr avr_cpu_get_pc(CPUState *cs)
+{
+    AVRCPU *cpu = AVR_CPU(cs);
+
+    return cpu->env.pc_w * 2;
+}
+
 static bool avr_cpu_has_work(CPUState *cs)
 {
     AVRCPU *cpu = AVR_CPU(cs);
@@ -XXX,XX +XXX,XX @@ static void avr_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = avr_cpu_has_work;
     cc->dump_state = avr_cpu_dump_state;
     cc->set_pc = avr_cpu_set_pc;
+    cc->get_pc = avr_cpu_get_pc;
     dc->vmsd = &vms_avr_cpu;
     cc->sysemu_ops = &avr_sysemu_ops;
     cc->disas_set_info = avr_cpu_disas_set_info;
diff --git a/target/cris/cpu.c b/target/cris/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/cris/cpu.c
+++ b/target/cris/cpu.c
@@ -XXX,XX +XXX,XX @@ static void cris_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc = value;
 }
 
+static vaddr cris_cpu_get_pc(CPUState *cs)
+{
+    CRISCPU *cpu = CRIS_CPU(cs);
+
+    return cpu->env.pc;
+}
+
 static bool cris_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_NMI);
@@ -XXX,XX +XXX,XX @@ static void cris_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = cris_cpu_has_work;
     cc->dump_state = cris_cpu_dump_state;
     cc->set_pc = cris_cpu_set_pc;
+    cc->get_pc = cris_cpu_get_pc;
     cc->gdb_read_register = cris_cpu_gdb_read_register;
     cc->gdb_write_register = cris_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/hexagon/cpu.c b/target/hexagon/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/hexagon/cpu.c
+++ b/target/hexagon/cpu.c
@@ -XXX,XX +XXX,XX @@ static void hexagon_cpu_set_pc(CPUState *cs, vaddr value)
     env->gpr[HEX_REG_PC] = value;
 }
 
+static vaddr hexagon_cpu_get_pc(CPUState *cs)
+{
+    HexagonCPU *cpu = HEXAGON_CPU(cs);
+    CPUHexagonState *env = &cpu->env;
+    return env->gpr[HEX_REG_PC];
+}
+
 static void hexagon_cpu_synchronize_from_tb(CPUState *cs,
                                             const TranslationBlock *tb)
 {
@@ -XXX,XX +XXX,XX @@ static void hexagon_cpu_class_init(ObjectClass *c, void *data)
     cc->has_work = hexagon_cpu_has_work;
     cc->dump_state = hexagon_dump_state;
     cc->set_pc = hexagon_cpu_set_pc;
+    cc->get_pc = hexagon_cpu_get_pc;
     cc->gdb_read_register = hexagon_gdb_read_register;
     cc->gdb_write_register = hexagon_gdb_write_register;
     cc->gdb_num_core_regs = TOTAL_PER_THREAD_REGS + NUM_VREGS + NUM_QREGS;
diff --git a/target/hppa/cpu.c b/target/hppa/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/hppa/cpu.c
+++ b/target/hppa/cpu.c
@@ -XXX,XX +XXX,XX @@ static void hppa_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.iaoq_b = value + 4;
 }
 
+static vaddr hppa_cpu_get_pc(CPUState *cs)
+{
+    HPPACPU *cpu = HPPA_CPU(cs);
+
+    return cpu->env.iaoq_f;
+}
+
 static void hppa_cpu_synchronize_from_tb(CPUState *cs,
                                          const TranslationBlock *tb)
 {
@@ -XXX,XX +XXX,XX @@ static void hppa_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = hppa_cpu_has_work;
     cc->dump_state = hppa_cpu_dump_state;
     cc->set_pc = hppa_cpu_set_pc;
+    cc->get_pc = hppa_cpu_get_pc;
     cc->gdb_read_register = hppa_cpu_gdb_read_register;
     cc->gdb_write_register = hppa_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -XXX,XX +XXX,XX @@ static void x86_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.eip = value;
 }
 
+static vaddr x86_cpu_get_pc(CPUState *cs)
+{
+    X86CPU *cpu = X86_CPU(cs);
+
+    /* Match cpu_get_tb_cpu_state. */
+    return cpu->env.eip + cpu->env.segs[R_CS].base;
+}
+
 int x86_cpu_pending_interrupt(CPUState *cs, int interrupt_request)
 {
     X86CPU *cpu = X86_CPU(cs);
@@ -XXX,XX +XXX,XX @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
     cc->has_work = x86_cpu_has_work;
     cc->dump_state = x86_cpu_dump_state;
     cc->set_pc = x86_cpu_set_pc;
+    cc->get_pc = x86_cpu_get_pc;
     cc->gdb_read_register = x86_cpu_gdb_read_register;
     cc->gdb_write_register = x86_cpu_gdb_write_register;
     cc->get_arch_id = x86_cpu_get_arch_id;
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -XXX,XX +XXX,XX @@ static void loongarch_cpu_set_pc(CPUState *cs, vaddr value)
     env->pc = value;
 }
 
+static vaddr loongarch_cpu_get_pc(CPUState *cs)
+{
+    LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+    CPULoongArchState *env = &cpu->env;
+
+    return env->pc;
+}
+
 #ifndef CONFIG_USER_ONLY
 #include "hw/loongarch/virt.h"
 
@@ -XXX,XX +XXX,XX @@ static void loongarch_cpu_class_init(ObjectClass *c, void *data)
     cc->has_work = loongarch_cpu_has_work;
     cc->dump_state = loongarch_cpu_dump_state;
     cc->set_pc = loongarch_cpu_set_pc;
+    cc->get_pc = loongarch_cpu_get_pc;
 #ifndef CONFIG_USER_ONLY
     dc->vmsd = &vmstate_loongarch_cpu;
     cc->sysemu_ops = &loongarch_sysemu_ops;
diff --git a/target/m68k/cpu.c b/target/m68k/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/m68k/cpu.c
+++ b/target/m68k/cpu.c
@@ -XXX,XX +XXX,XX @@ static void m68k_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc = value;
 }
 
+static vaddr m68k_cpu_get_pc(CPUState *cs)
+{
+    M68kCPU *cpu = M68K_CPU(cs);
+
+    return cpu->env.pc;
+}
+
 static bool m68k_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & CPU_INTERRUPT_HARD;
@@ -XXX,XX +XXX,XX @@ static void m68k_cpu_class_init(ObjectClass *c, void *data)
     cc->has_work = m68k_cpu_has_work;
     cc->dump_state = m68k_cpu_dump_state;
     cc->set_pc = m68k_cpu_set_pc;
+    cc->get_pc = m68k_cpu_get_pc;
     cc->gdb_read_register = m68k_cpu_gdb_read_register;
     cc->gdb_write_register = m68k_cpu_gdb_write_register;
 #if defined(CONFIG_SOFTMMU)
diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/microblaze/cpu.c
+++ b/target/microblaze/cpu.c
@@ -XXX,XX +XXX,XX @@ static void mb_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.iflags = 0;
 }
 
+static vaddr mb_cpu_get_pc(CPUState *cs)
+{
+    MicroBlazeCPU *cpu = MICROBLAZE_CPU(cs);
+
+    return cpu->env.pc;
+}
+
 static void mb_cpu_synchronize_from_tb(CPUState *cs,
                                        const TranslationBlock *tb)
 {
@@ -XXX,XX +XXX,XX @@ static void mb_cpu_class_init(ObjectClass *oc, void *data)
 
     cc->dump_state = mb_cpu_dump_state;
     cc->set_pc = mb_cpu_set_pc;
+    cc->get_pc = mb_cpu_get_pc;
     cc->gdb_read_register = mb_cpu_gdb_read_register;
     cc->gdb_write_register = mb_cpu_gdb_write_register;
 
diff --git a/target/mips/cpu.c b/target/mips/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/mips/cpu.c
+++ b/target/mips/cpu.c
@@ -XXX,XX +XXX,XX @@ static void mips_cpu_set_pc(CPUState *cs, vaddr value)
     mips_env_set_pc(&cpu->env, value);
 }
 
+static vaddr mips_cpu_get_pc(CPUState *cs)
+{
+    MIPSCPU *cpu = MIPS_CPU(cs);
+
+    return cpu->env.active_tc.PC;
+}
+
 static bool mips_cpu_has_work(CPUState *cs)
 {
     MIPSCPU *cpu = MIPS_CPU(cs);
@@ -XXX,XX +XXX,XX @@ static void mips_cpu_class_init(ObjectClass *c, void *data)
     cc->has_work = mips_cpu_has_work;
     cc->dump_state = mips_cpu_dump_state;
     cc->set_pc = mips_cpu_set_pc;
+    cc->get_pc = mips_cpu_get_pc;
     cc->gdb_read_register = mips_cpu_gdb_read_register;
     cc->gdb_write_register = mips_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/nios2/cpu.c b/target/nios2/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/nios2/cpu.c
+++ b/target/nios2/cpu.c
@@ -XXX,XX +XXX,XX @@ static void nios2_cpu_set_pc(CPUState *cs, vaddr value)
     env->pc = value;
 }
 
+static vaddr nios2_cpu_get_pc(CPUState *cs)
+{
+    Nios2CPU *cpu = NIOS2_CPU(cs);
+    CPUNios2State *env = &cpu->env;
+
+    return env->pc;
+}
+
 static bool nios2_cpu_has_work(CPUState *cs)
 {
     return cs->interrupt_request & CPU_INTERRUPT_HARD;
@@ -XXX,XX +XXX,XX @@ static void nios2_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = nios2_cpu_has_work;
     cc->dump_state = nios2_cpu_dump_state;
     cc->set_pc = nios2_cpu_set_pc;
+    cc->get_pc = nios2_cpu_get_pc;
     cc->disas_set_info = nios2_cpu_disas_set_info;
 #ifndef CONFIG_USER_ONLY
     cc->sysemu_ops = &nios2_sysemu_ops;
diff --git a/target/openrisc/cpu.c b/target/openrisc/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/openrisc/cpu.c
+++ b/target/openrisc/cpu.c
@@ -XXX,XX +XXX,XX @@ static void openrisc_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.dflag = 0;
 }
 
+static vaddr openrisc_cpu_get_pc(CPUState *cs)
+{
+    OpenRISCCPU *cpu = OPENRISC_CPU(cs);
+
+    return cpu->env.pc;
+}
+
 static void openrisc_cpu_synchronize_from_tb(CPUState *cs,
                                              const TranslationBlock *tb)
 {
@@ -XXX,XX +XXX,XX @@ static void openrisc_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = openrisc_cpu_has_work;
     cc->dump_state = openrisc_cpu_dump_state;
     cc->set_pc = openrisc_cpu_set_pc;
+    cc->get_pc = openrisc_cpu_get_pc;
     cc->gdb_read_register = openrisc_cpu_gdb_read_register;
     cc->gdb_write_register = openrisc_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index XXXXXXX..XXXXXXX 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -XXX,XX +XXX,XX @@ static void ppc_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.nip = value;
 }
 
+static vaddr ppc_cpu_get_pc(CPUState *cs)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(cs);
+
+    return cpu->env.nip;
+}
+
 static bool ppc_cpu_has_work(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -XXX,XX +XXX,XX @@ static void ppc_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = ppc_cpu_has_work;
     cc->dump_state = ppc_cpu_dump_state;
     cc->set_pc = ppc_cpu_set_pc;
+    cc->get_pc = ppc_cpu_get_pc;
     cc->gdb_read_register = ppc_cpu_gdb_read_register;
     cc->gdb_write_register = ppc_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -XXX,XX +XXX,XX @@ static void riscv_cpu_set_pc(CPUState *cs, vaddr value)
     }
 }
 
+static vaddr riscv_cpu_get_pc(CPUState *cs)
+{
+    RISCVCPU *cpu = RISCV_CPU(cs);
+    CPURISCVState *env = &cpu->env;
+
+    /* Match cpu_get_tb_cpu_state. */
+    if (env->xl == MXL_RV32) {
+        return env->pc & UINT32_MAX;
+    }
+    return env->pc;
+}
+
 static void riscv_cpu_synchronize_from_tb(CPUState *cs,
                                           const TranslationBlock *tb)
 {
@@ -XXX,XX +XXX,XX @@ static void riscv_cpu_class_init(ObjectClass *c, void *data)
     cc->has_work = riscv_cpu_has_work;
     cc->dump_state = riscv_cpu_dump_state;
     cc->set_pc = riscv_cpu_set_pc;
+    cc->get_pc = riscv_cpu_get_pc;
     cc->gdb_read_register = riscv_cpu_gdb_read_register;
     cc->gdb_write_register = riscv_cpu_gdb_write_register;
     cc->gdb_num_core_regs = 33;
diff --git a/target/rx/cpu.c b/target/rx/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/rx/cpu.c
+++ b/target/rx/cpu.c
@@ -XXX,XX +XXX,XX @@ static void rx_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc = value;
 }
 
+static vaddr rx_cpu_get_pc(CPUState *cs)
+{
+    RXCPU *cpu = RX_CPU(cs);
+
+    return cpu->env.pc;
+}
+
 static void rx_cpu_synchronize_from_tb(CPUState *cs,
                                        const TranslationBlock *tb)
 {
@@ -XXX,XX +XXX,XX @@ static void rx_cpu_class_init(ObjectClass *klass, void *data)
     cc->has_work = rx_cpu_has_work;
     cc->dump_state = rx_cpu_dump_state;
     cc->set_pc = rx_cpu_set_pc;
+    cc->get_pc = rx_cpu_get_pc;
 
 #ifndef CONFIG_USER_ONLY
     cc->sysemu_ops = &rx_sysemu_ops;
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -XXX,XX +XXX,XX @@ static void s390_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.psw.addr = value;
 }
 
+static vaddr s390_cpu_get_pc(CPUState *cs)
+{
+    S390CPU *cpu = S390_CPU(cs);
+
+    return cpu->env.psw.addr;
+}
+
 static bool s390_cpu_has_work(CPUState *cs)
 {
     S390CPU *cpu = S390_CPU(cs);
@@ -XXX,XX +XXX,XX @@ static void s390_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = s390_cpu_has_work;
     cc->dump_state = s390_cpu_dump_state;
     cc->set_pc = s390_cpu_set_pc;
+    cc->get_pc = s390_cpu_get_pc;
     cc->gdb_read_register = s390_cpu_gdb_read_register;
     cc->gdb_write_register = s390_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/sh4/cpu.c b/target/sh4/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/sh4/cpu.c
+++ b/target/sh4/cpu.c
@@ -XXX,XX +XXX,XX @@ static void superh_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc = value;
 }
 
+static vaddr superh_cpu_get_pc(CPUState *cs)
+{
+    SuperHCPU *cpu = SUPERH_CPU(cs);
+
+    return cpu->env.pc;
+}
+
 static void superh_cpu_synchronize_from_tb(CPUState *cs,
                                            const TranslationBlock *tb)
 {
@@ -XXX,XX +XXX,XX @@ static void superh_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = superh_cpu_has_work;
     cc->dump_state = superh_cpu_dump_state;
     cc->set_pc = superh_cpu_set_pc;
+    cc->get_pc = superh_cpu_get_pc;
     cc->gdb_read_register = superh_cpu_gdb_read_register;
     cc->gdb_write_register = superh_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/sparc/cpu.c b/target/sparc/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/sparc/cpu.c
+++ b/target/sparc/cpu.c
@@ -XXX,XX +XXX,XX @@ static void sparc_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.npc = value + 4;
 }
 
+static vaddr sparc_cpu_get_pc(CPUState *cs)
+{
+    SPARCCPU *cpu = SPARC_CPU(cs);
+
+    return cpu->env.pc;
+}
+
 static void sparc_cpu_synchronize_from_tb(CPUState *cs,
                                           const TranslationBlock *tb)
 {
@@ -XXX,XX +XXX,XX @@ static void sparc_cpu_class_init(ObjectClass *oc, void *data)
     cc->memory_rw_debug = sparc_cpu_memory_rw_debug;
 #endif
     cc->set_pc = sparc_cpu_set_pc;
+    cc->get_pc = sparc_cpu_get_pc;
     cc->gdb_read_register = sparc_cpu_gdb_read_register;
     cc->gdb_write_register = sparc_cpu_gdb_write_register;
 #ifndef CONFIG_USER_ONLY
diff --git a/target/tricore/cpu.c b/target/tricore/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/tricore/cpu.c
+++ b/target/tricore/cpu.c
@@ -XXX,XX +XXX,XX @@ static void tricore_cpu_set_pc(CPUState *cs, vaddr value)
     env->PC = value & ~(target_ulong)1;
 }
 
+static vaddr tricore_cpu_get_pc(CPUState *cs)
+{
+    TriCoreCPU *cpu = TRICORE_CPU(cs);
+    CPUTriCoreState *env = &cpu->env;
+
+    return env->PC;
+}
+
 static void tricore_cpu_synchronize_from_tb(CPUState *cs,
                                             const TranslationBlock *tb)
 {
@@ -XXX,XX +XXX,XX @@ static void tricore_cpu_class_init(ObjectClass *c, void *data)
 
     cc->dump_state = tricore_cpu_dump_state;
     cc->set_pc = tricore_cpu_set_pc;
+    cc->get_pc = tricore_cpu_get_pc;
     cc->sysemu_ops = &tricore_sysemu_ops;
     cc->tcg_ops = &tricore_tcg_ops;
 }
diff --git a/target/xtensa/cpu.c b/target/xtensa/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/xtensa/cpu.c
+++ b/target/xtensa/cpu.c
@@ -XXX,XX +XXX,XX @@ static void xtensa_cpu_set_pc(CPUState *cs, vaddr value)
     cpu->env.pc = value;
 }
 
+static vaddr xtensa_cpu_get_pc(CPUState *cs)
+{
+    XtensaCPU *cpu = XTENSA_CPU(cs);
+
+    return cpu->env.pc;
+}
+
 static bool xtensa_cpu_has_work(CPUState *cs)
 {
 #ifndef CONFIG_USER_ONLY
@@ -XXX,XX +XXX,XX @@ static void xtensa_cpu_class_init(ObjectClass *oc, void *data)
     cc->has_work = xtensa_cpu_has_work;
     cc->dump_state = xtensa_cpu_dump_state;
     cc->set_pc = xtensa_cpu_set_pc;
+    cc->get_pc = xtensa_cpu_get_pc;
     cc->gdb_read_register = xtensa_cpu_gdb_read_register;
     cc->gdb_write_register = xtensa_cpu_gdb_write_register;
     cc->gdb_stop_before_watchpoint = true;
-- 
2.34.1

The availability of tb->pc will shortly be conditional.
Introduce accessor functions to minimize ifdefs.

Pass around a known pc to places like tcg_gen_code,
where the caller must already have the value.

diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/internal.h
+++ b/accel/tcg/internal.h
@@ -XXX,XX +XXX,XX @@ G_NORETURN void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr);
 void page_init(void);
 void tb_htable_init(void);
 
+/* Return the current PC from CPU, which may be cached in TB. */
+static inline target_ulong log_pc(CPUState *cpu, const TranslationBlock *tb)
+{
+    return tb_pc(tb);
+}
+
 #endif /* ACCEL_TCG_INTERNAL_H */
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -XXX,XX +XXX,XX @@ struct TranslationBlock {
     uintptr_t jmp_dest[2];
 };
 
+/* Hide the read to avoid ifdefs for TARGET_TB_PCREL. */
+static inline target_ulong tb_pc(const TranslationBlock *tb)
+{
+    return tb->pc;
+}
+
 /* Hide the qatomic_read to make code a little easier on the eyes */
 static inline uint32_t tb_cflags(const TranslationBlock *tb)
 {
diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void);
 void tcg_prologue_init(TCGContext *s);
 void tcg_func_start(TCGContext *s);
 
-int tcg_gen_code(TCGContext *s, TranslationBlock *tb);
+int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start);
 
 void tcg_set_frame(TCGContext *s, TCGReg reg, intptr_t start, intptr_t size);
 
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
     const TranslationBlock *tb = p;
     const struct tb_desc *desc = d;
 
-    if (tb->pc == desc->pc &&
+    if (tb_pc(tb) == desc->pc &&
         tb->page_addr[0] == desc->page_addr0 &&
         tb->cs_base == desc->cs_base &&
         tb->flags == desc->flags &&
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
     return tb;
 }
 
-static inline void log_cpu_exec(target_ulong pc, CPUState *cpu,
-                                const TranslationBlock *tb)
+static void log_cpu_exec(target_ulong pc, CPUState *cpu,
+                         const TranslationBlock *tb)
 {
-    if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_CPU | CPU_LOG_EXEC))
-        && qemu_log_in_addr_range(pc)) {
-
+    if (qemu_log_in_addr_range(pc)) {
         qemu_log_mask(CPU_LOG_EXEC,
                       "Trace %d: %p [" TARGET_FMT_lx
                       "/" TARGET_FMT_lx "/%08x/%08x] %s\n",
@@ -XXX,XX +XXX,XX @@ const void *HELPER(lookup_tb_ptr)(CPUArchState *env)
         return tcg_code_gen_epilogue;
     }
 
-    log_cpu_exec(pc, cpu, tb);
+    if (qemu_loglevel_mask(CPU_LOG_TB_CPU | CPU_LOG_EXEC)) {
+        log_cpu_exec(pc, cpu, tb);
+    }
 
     return tb->tc.ptr;
 }
@@ -XXX,XX +XXX,XX @@ cpu_tb_exec(CPUState *cpu, TranslationBlock *itb, int *tb_exit)
     TranslationBlock *last_tb;
     const void *tb_ptr = itb->tc.ptr;
 
-    log_cpu_exec(itb->pc, cpu, itb);
+    if (qemu_loglevel_mask(CPU_LOG_TB_CPU | CPU_LOG_EXEC)) {
+        log_cpu_exec(log_pc(cpu, itb), cpu, itb);
+    }
 
     qemu_thread_jit_execute();
     ret = tcg_qemu_tb_exec(env, tb_ptr);
@@ -XXX,XX +XXX,XX @@ cpu_tb_exec(CPUState *cpu, TranslationBlock *itb, int *tb_exit)
          * of the start of the TB.
          */
         CPUClass *cc = CPU_GET_CLASS(cpu);
-        qemu_log_mask_and_addr(CPU_LOG_EXEC, last_tb->pc,
-                               "Stopped execution of TB chain before %p ["
-                               TARGET_FMT_lx "] %s\n",
-                               last_tb->tc.ptr, last_tb->pc,
-                               lookup_symbol(last_tb->pc));
+
         if (cc->tcg_ops->synchronize_from_tb) {
             cc->tcg_ops->synchronize_from_tb(cpu, last_tb);
         } else {
             assert(cc->set_pc);
-            cc->set_pc(cpu, last_tb->pc);
+            cc->set_pc(cpu, tb_pc(last_tb));
+        }
+        if (qemu_loglevel_mask(CPU_LOG_EXEC)) {
+            target_ulong pc = log_pc(cpu, last_tb);
+            if (qemu_log_in_addr_range(pc)) {
+                qemu_log("Stopped execution of TB chain before %p ["
+                         TARGET_FMT_lx "] %s\n",
+                         last_tb->tc.ptr, pc, lookup_symbol(pc));
+            }
         }
     }
 
@@ -XXX,XX +XXX,XX @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
 
     qemu_spin_unlock(&tb_next->jmp_lock);
 
-    qemu_log_mask_and_addr(CPU_LOG_EXEC, tb->pc,
-                           "Linking TBs %p [" TARGET_FMT_lx
-                           "] index %d -> %p [" TARGET_FMT_lx "]\n",
-                           tb->tc.ptr, tb->pc, n,
-                           tb_next->tc.ptr, tb_next->pc);
+    qemu_log_mask(CPU_LOG_EXEC, "Linking TBs %p index %d -> %p\n",
+                  tb->tc.ptr, n, tb_next->tc.ptr);
     return;
 
  out_unlock_next:
@@ -XXX,XX +XXX,XX @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
 }
 
 static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
+                                    target_ulong pc,
                                     TranslationBlock **last_tb, int *tb_exit)
 {
     int32_t insns_left;
 
-    trace_exec_tb(tb, tb->pc);
+    trace_exec_tb(tb, pc);
     tb = cpu_tb_exec(cpu, tb, tb_exit);
     if (*tb_exit != TB_EXIT_REQUESTED) {
         *last_tb = tb;
@@ -XXX,XX +XXX,XX @@ int cpu_exec(CPUState *cpu)
                 tb_add_jump(last_tb, tb_exit, tb);
             }
 
-            cpu_loop_exec_tb(cpu, tb, &last_tb, &tb_exit);
+            cpu_loop_exec_tb(cpu, tb, pc, &last_tb, &tb_exit);
 
             /* Try to align the host and virtual clocks
                if the guest is in advance */
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
 
         for (j = 0; j < TARGET_INSN_START_WORDS; ++j) {
             if (i == 0) {
-                prev = (j == 0 ? tb->pc : 0);
+                prev = (j == 0 ? tb_pc(tb) : 0);
             } else {
                 prev = tcg_ctx->gen_insn_data[i - 1][j];
             }
@@ -XXX,XX +XXX,XX @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
 static int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
                                      uintptr_t searched_pc, bool reset_icount)
 {
-    target_ulong data[TARGET_INSN_START_WORDS] = { tb->pc };
+    target_ulong data[TARGET_INSN_START_WORDS] = { tb_pc(tb) };
     uintptr_t host_pc = (uintptr_t)tb->tc.ptr;
     CPUArchState *env = cpu->env_ptr;
     const uint8_t *p = tb->tc.ptr + tb->tc.size;
@@ -XXX,XX +XXX,XX @@ static bool tb_cmp(const void *ap, const void *bp)
     const TranslationBlock *a = ap;
     const TranslationBlock *b = bp;
 
-    return a->pc == b->pc &&
+    return tb_pc(a) == tb_pc(b) &&
         a->cs_base == b->cs_base &&
         a->flags == b->flags &&
         (tb_cflags(a) & ~CF_INVALID) == (tb_cflags(b) & ~CF_INVALID) &&
@@ -XXX,XX +XXX,XX @@ static void do_tb_invalidate_check(void *p, uint32_t hash, void *userp)
     TranslationBlock *tb = p;
     target_ulong addr = *(target_ulong *)userp;
 
-    if (!(addr + TARGET_PAGE_SIZE <= tb->pc || addr >= tb->pc + tb->size)) {
+    if (!(addr + TARGET_PAGE_SIZE <= tb_pc(tb) ||
+          addr >= tb_pc(tb) + tb->size)) {
         printf("ERROR invalidate: address=" TARGET_FMT_lx
-               " PC=%08lx size=%04x\n", addr, (long)tb->pc, tb->size);
+               " PC=%08lx size=%04x\n", addr, (long)tb_pc(tb), tb->size);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ static void do_tb_page_check(void *p, uint32_t hash, void *userp)
     TranslationBlock *tb = p;
     int flags1, flags2;
 
-    flags1 = page_get_flags(tb->pc);
-    flags2 = page_get_flags(tb->pc + tb->size - 1);
+    flags1 = page_get_flags(tb_pc(tb));
+    flags2 = page_get_flags(tb_pc(tb) + tb->size - 1);
     if ((flags1 & PAGE_WRITE) || (flags2 & PAGE_WRITE)) {
         printf("ERROR page flags: PC=%08lx size=%04x f1=%x f2=%x\n",
-               (long)tb->pc, tb->size, flags1, flags2);
+               (long)tb_pc(tb), tb->size, flags1, flags2);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
 
     /* remove the TB from the hash list */
     phys_pc = tb->page_addr[0];
-    h = tb_hash_func(phys_pc, tb->pc, tb->flags, orig_cflags,
+    h = tb_hash_func(phys_pc, tb_pc(tb), tb->flags, orig_cflags,
                      tb->trace_vcpu_dstate);
     if (!qht_remove(&tb_ctx.htable, tb, h)) {
         return;
@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
     }
 
     /* add in the hash table */
-    h = tb_hash_func(phys_pc, tb->pc, tb->flags, tb->cflags,
+    h = tb_hash_func(phys_pc, tb_pc(tb), tb->flags, tb->cflags,
                      tb->trace_vcpu_dstate);
     qht_insert(&tb_ctx.htable, tb, h, &existing_tb);
 
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
     tcg_ctx->cpu = NULL;
     max_insns = tb->icount;
 
-    trace_translate_block(tb, tb->pc, tb->tc.ptr);
+    trace_translate_block(tb, pc, tb->tc.ptr);
 
     /* generate machine code */
     tb->jmp_reset_offset[0] = TB_JMP_RESET_OFFSET_INVALID;
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
     ti = profile_getclock();
 #endif
 
-    gen_code_size = tcg_gen_code(tcg_ctx, tb);
+    gen_code_size = tcg_gen_code(tcg_ctx, tb, pc);
     if (unlikely(gen_code_size < 0)) {
  error_return:
         switch (gen_code_size) {
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 
 #ifdef DEBUG_DISAS
     if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM) &&
-        qemu_log_in_addr_range(tb->pc)) {
+        qemu_log_in_addr_range(pc)) {
         FILE *logfile = qemu_log_trylock();
         if (logfile) {
             int code_size, data_size;
@@ -XXX,XX +XXX,XX @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
      */
     cpu->cflags_next_tb = curr_cflags(cpu) | CF_MEMI_ONLY | CF_LAST_IO | n;
 
-    qemu_log_mask_and_addr(CPU_LOG_EXEC, tb->pc,
-                           "cpu_io_recompile: rewound execution of TB to "
-                           TARGET_FMT_lx "\n", tb->pc);
+    if (qemu_loglevel_mask(CPU_LOG_EXEC)) {
+        target_ulong pc = log_pc(cpu, tb);
+        if (qemu_log_in_addr_range(pc)) {
+            qemu_log("cpu_io_recompile: rewound execution of TB to "
+                     TARGET_FMT_lx "\n", pc);
+        }
+    }
 
     cpu_loop_exit_noexc(cpu);
 }
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -XXX,XX +XXX,XX @@ void arm_cpu_synchronize_from_tb(CPUState *cs,
      * never possible for an AArch64 TB to chain to an AArch32 TB.
      */
     if (is_a64(env)) {
-        env->pc = tb->pc;
+        env->pc = tb_pc(tb);
     } else {
-        env->regs[15] = tb->pc;
+        env->regs[15] = tb_pc(tb);
     }
 }
 #endif /* CONFIG_TCG */
diff --git a/target/avr/cpu.c b/target/avr/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/avr/cpu.c
+++ b/target/avr/cpu.c
@@ -XXX,XX +XXX,XX @@ static void avr_cpu_synchronize_from_tb(CPUState *cs,
     AVRCPU *cpu = AVR_CPU(cs);
     CPUAVRState *env = &cpu->env;
 
-    env->pc_w = tb->pc / 2; /* internally PC points to words */
+    env->pc_w = tb_pc(tb) / 2; /* internally PC points to words */
 }
 
 static void avr_cpu_reset(DeviceState *ds)
diff --git a/target/hexagon/cpu.c b/target/hexagon/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/hexagon/cpu.c
+++ b/target/hexagon/cpu.c
@@ -XXX,XX +XXX,XX @@ static void hexagon_cpu_synchronize_from_tb(CPUState *cs,
 {
     HexagonCPU *cpu = HEXAGON_CPU(cs);
     CPUHexagonState *env = &cpu->env;
-    env->gpr[HEX_REG_PC] = tb->pc;
+    env->gpr[HEX_REG_PC] = tb_pc(tb);
 }
 
 static bool hexagon_cpu_has_work(CPUState *cs)
diff --git a/target/hppa/cpu.c b/target/hppa/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/hppa/cpu.c
+++ b/target/hppa/cpu.c
@@ -XXX,XX +XXX,XX @@ static void hppa_cpu_synchronize_from_tb(CPUState *cs,
     HPPACPU *cpu = HPPA_CPU(cs);
 
 #ifdef CONFIG_USER_ONLY
-    cpu->env.iaoq_f = tb->pc;
+    cpu->env.iaoq_f = tb_pc(tb);
     cpu->env.iaoq_b = tb->cs_base;
 #else
     /* Recover the IAOQ values from the GVA + PRIV.  */
@@ -XXX,XX +XXX,XX @@ static void hppa_cpu_synchronize_from_tb(CPUState *cs,
     int32_t diff = cs_base;
 
     cpu->env.iasq_f = iasq_f;
-    cpu->env.iaoq_f = (tb->pc & ~iasq_f) + priv;
+    cpu->env.iaoq_f = (tb_pc(tb) & ~iasq_f) + priv;
     if (diff) {
         cpu->env.iaoq_b = cpu->env.iaoq_f + diff;
     }
diff --git a/target/i386/tcg/tcg-cpu.c b/target/i386/tcg/tcg-cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/i386/tcg/tcg-cpu.c
+++ b/target/i386/tcg/tcg-cpu.c
@@ -XXX,XX +XXX,XX @@ static void x86_cpu_synchronize_from_tb(CPUState *cs,
 {
     X86CPU *cpu = X86_CPU(cs);
 
-    cpu->env.eip = tb->pc - tb->cs_base;
+    cpu->env.eip = tb_pc(tb) - tb->cs_base;
 }
 
 #ifndef CONFIG_USER_ONLY
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -XXX,XX +XXX,XX @@ static void loongarch_cpu_synchronize_from_tb(CPUState *cs,
     LoongArchCPU *cpu = LOONGARCH_CPU(cs);
     CPULoongArchState *env = &cpu->env;
 
-    env->pc = tb->pc;
+    env->pc = tb_pc(tb);
 }
 #endif /* CONFIG_TCG */
 
diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/microblaze/cpu.c
+++ b/target/microblaze/cpu.c
@@ -XXX,XX +XXX,XX @@ static void mb_cpu_synchronize_from_tb(CPUState *cs,
 {
     MicroBlazeCPU *cpu = MICROBLAZE_CPU(cs);
 
-    cpu->env.pc = tb->pc;
+    cpu->env.pc = tb_pc(tb);
     cpu->env.iflags = tb->flags & IFLAGS_TB_MASK;
 }
 
diff --git a/target/mips/tcg/exception.c b/target/mips/tcg/exception.c
index XXXXXXX..XXXXXXX 100644
--- a/target/mips/tcg/exception.c
+++ b/target/mips/tcg/exception.c
@@ -XXX,XX +XXX,XX @@ void mips_cpu_synchronize_from_tb(CPUState *cs, const TranslationBlock *tb)
     MIPSCPU *cpu = MIPS_CPU(cs);
     CPUMIPSState *env = &cpu->env;
 
-    env->active_tc.PC = tb->pc;
+    env->active_tc.PC = tb_pc(tb);
     env->hflags &= ~MIPS_HFLAG_BMASK;
     env->hflags |= tb->flags & MIPS_HFLAG_BMASK;
 }
diff --git a/target/mips/tcg/sysemu/special_helper.c b/target/mips/tcg/sysemu/special_helper.c
index XXXXXXX..XXXXXXX 100644
--- a/target/mips/tcg/sysemu/special_helper.c
+++ b/target/mips/tcg/sysemu/special_helper.c
@@ -XXX,XX +XXX,XX @@ bool mips_io_recompile_replay_branch(CPUState *cs, const TranslationBlock *tb)
     CPUMIPSState *env = &cpu->env;
 
     if ((env->hflags & MIPS_HFLAG_BMASK) != 0
-        && env->active_tc.PC != tb->pc) {
+        && env->active_tc.PC != tb_pc(tb)) {
         env->active_tc.PC -= (env->hflags & MIPS_HFLAG_B16 ? 2 : 4);
         env->hflags &= ~MIPS_HFLAG_BMASK;
         return true;
diff --git a/target/openrisc/cpu.c b/target/openrisc/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/openrisc/cpu.c
+++ b/target/openrisc/cpu.c
@@ -XXX,XX +XXX,XX @@ static void openrisc_cpu_synchronize_from_tb(CPUState *cs,
 {
     OpenRISCCPU *cpu = OPENRISC_CPU(cs);
 
-    cpu->env.pc = tb->pc;
+    cpu->env.pc = tb_pc(tb);
 }
 
 
diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -XXX,XX +XXX,XX @@ static void riscv_cpu_synchronize_from_tb(CPUState *cs,
     RISCVMXL xl = FIELD_EX32(tb->flags, TB_FLAGS, XL);
 
     if (xl == MXL_RV32) {
-        env->pc = (int32_t)tb->pc;
+        env->pc = (int32_t)tb_pc(tb);
     } else {
-        env->pc = tb->pc;
+        env->pc = tb_pc(tb);
     }
 }
 
diff --git a/target/rx/cpu.c b/target/rx/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/rx/cpu.c
+++ b/target/rx/cpu.c
@@ -XXX,XX +XXX,XX @@ static void rx_cpu_synchronize_from_tb(CPUState *cs,
 {
     RXCPU *cpu = RX_CPU(cs);
 
-    cpu->env.pc = tb->pc;
+    cpu->env.pc = tb_pc(tb);
 }
 
 static bool rx_cpu_has_work(CPUState *cs)
diff --git a/target/sh4/cpu.c b/target/sh4/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/sh4/cpu.c
+++ b/target/sh4/cpu.c
@@ -XXX,XX +XXX,XX @@ static void superh_cpu_synchronize_from_tb(CPUState *cs,
 {
     SuperHCPU *cpu = SUPERH_CPU(cs);
 
-    cpu->env.pc = tb->pc;
+    cpu->env.pc = tb_pc(tb);
     cpu->env.flags = tb->flags & TB_FLAG_ENVFLAGS_MASK;
 }
 
@@ -XXX,XX +XXX,XX @@ static bool superh_io_recompile_replay_branch(CPUState *cs,
     CPUSH4State *env = &cpu->env;
 
     if ((env->flags & ((DELAY_SLOT | DELAY_SLOT_CONDITIONAL))) != 0
-        && env->pc != tb->pc) {
+        && env->pc != tb_pc(tb)) {
         env->pc -= 2;
         env->flags &= ~(DELAY_SLOT | DELAY_SLOT_CONDITIONAL);
         return true;
diff --git a/target/sparc/cpu.c b/target/sparc/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/sparc/cpu.c
+++ b/target/sparc/cpu.c
@@ -XXX,XX +XXX,XX @@ static void sparc_cpu_synchronize_from_tb(CPUState *cs,
 {
     SPARCCPU *cpu = SPARC_CPU(cs);
 
-    cpu->env.pc = tb->pc;
+    cpu->env.pc = tb_pc(tb);
     cpu->env.npc = tb->cs_base;
 }
 
diff --git a/target/tricore/cpu.c b/target/tricore/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/tricore/cpu.c
+++ b/target/tricore/cpu.c
@@ -XXX,XX +XXX,XX @@ static void tricore_cpu_synchronize_from_tb(CPUState *cs,
     TriCoreCPU *cpu = TRICORE_CPU(cs);
     CPUTriCoreState *env = &cpu->env;
 
-    env->PC = tb->pc;
+    env->PC = tb_pc(tb);
 }
 
 static void tricore_cpu_reset(DeviceState *dev)
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ int64_t tcg_cpu_exec_time(void)
 #endif
 
 
-int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
+int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start)
 {
 #ifdef CONFIG_PROFILER
     TCGProfile *prof = &s->prof;
@@ -XXX,XX +XXX,XX @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
 
 #ifdef DEBUG_DISAS
     if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_OP)
-                 && qemu_log_in_addr_range(tb->pc))) {
+                 && qemu_log_in_addr_range(pc_start))) {
         FILE *logfile = qemu_log_trylock();
         if (logfile) {
             fprintf(logfile, "OP:\n");
@@ -XXX,XX +XXX,XX @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
     if (s->nb_indirects > 0) {
 #ifdef DEBUG_DISAS
         if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_OP_IND)
-                     && qemu_log_in_addr_range(tb->pc))) {
+                     && qemu_log_in_addr_range(pc_start))) {
             FILE *logfile = qemu_log_trylock();
             if (logfile) {
                 fprintf(logfile, "OP before indirect lowering:\n");
@@ -XXX,XX +XXX,XX @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
 
 #ifdef DEBUG_DISAS
     if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_OP_OPT)
-                 && qemu_log_in_addr_range(tb->pc))) {
+                 && qemu_log_in_addr_range(pc_start))) {
         FILE *logfile = qemu_log_trylock();
         if (logfile) {
             fprintf(logfile, "OP after optimization and liveness analysis:\n");
-- 
2.34.1

Prepare for targets to be able to produce TBs that can
run in more than one virtual context.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/internal.h      |  4 +++
 accel/tcg/tb-jmp-cache.h  | 41 +++++++++++++++++++++++++
 include/exec/cpu-defs.h   |  3 ++
 include/exec/exec-all.h   | 32 ++++++++++++++++++--
 accel/tcg/cpu-exec.c      | 16 ++++++----
 accel/tcg/translate-all.c | 64 ++++++++++++++++++++++++++-------------
 6 files changed, 131 insertions(+), 29 deletions(-)

diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/internal.h
+++ b/accel/tcg/internal.h
@@ -XXX,XX +XXX,XX @@ void tb_htable_init(void);
 /* Return the current PC from CPU, which may be cached in TB. */
 static inline target_ulong log_pc(CPUState *cpu, const TranslationBlock *tb)
 {
+#if TARGET_TB_PCREL
+    return cpu->cc->get_pc(cpu);
+#else
     return tb_pc(tb);
+#endif
 }
 
 #endif /* ACCEL_TCG_INTERNAL_H */
diff --git a/accel/tcg/tb-jmp-cache.h b/accel/tcg/tb-jmp-cache.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tb-jmp-cache.h
+++ b/accel/tcg/tb-jmp-cache.h
@@ -XXX,XX +XXX,XX @@
 
 /*
  * Accessed in parallel; all accesses to 'tb' must be atomic.
+ * For TARGET_TB_PCREL, accesses to 'pc' must be protected by
+ * a load_acquire/store_release to 'tb'.
  */
 struct CPUJumpCache {
     struct {
         TranslationBlock *tb;
+#if TARGET_TB_PCREL
+        target_ulong pc;
+#endif
     } array[TB_JMP_CACHE_SIZE];
 };
 
+static inline TranslationBlock *
+tb_jmp_cache_get_tb(CPUJumpCache *jc, uint32_t hash)
+{
+#if TARGET_TB_PCREL
+    /* Use acquire to ensure current load of pc from jc. */
+    return qatomic_load_acquire(&jc->array[hash].tb);
+#else
+    /* Use rcu_read to ensure current load of pc from *tb. */
+    return qatomic_rcu_read(&jc->array[hash].tb);
+#endif
+}
+
+static inline target_ulong
+tb_jmp_cache_get_pc(CPUJumpCache *jc, uint32_t hash, TranslationBlock *tb)
+{
+#if TARGET_TB_PCREL
+    return jc->array[hash].pc;
+#else
+    return tb_pc(tb);
+#endif
+}
+
+static inline void
+tb_jmp_cache_set(CPUJumpCache *jc, uint32_t hash,
+                 TranslationBlock *tb, target_ulong pc)
+{
+#if TARGET_TB_PCREL
+    jc->array[hash].pc = pc;
+    /* Use store_release on tb to ensure pc is written first. */
+    qatomic_store_release(&jc->array[hash].tb, tb);
+#else
+    /* Use the pc value already stored in tb->pc. */
+    qatomic_set(&jc->array[hash].tb, tb);
+#endif
+}
+
 #endif /* ACCEL_TCG_TB_JMP_CACHE_H */
diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-defs.h
+++ b/include/exec/cpu-defs.h
@@ -XXX,XX +XXX,XX @@
 #  error TARGET_PAGE_BITS must be defined in cpu-param.h
 # endif
 #endif
+#ifndef TARGET_TB_PCREL
+# define TARGET_TB_PCREL 0
+#endif
 
 #define TARGET_LONG_SIZE (TARGET_LONG_BITS / 8)
 
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -XXX,XX +XXX,XX @@ struct tb_tc {
 };
 
 struct TranslationBlock {
-    target_ulong pc;   /* simulated PC corresponding to this block (EIP + CS base) */
-    target_ulong cs_base; /* CS base for this block */
+#if !TARGET_TB_PCREL
+    /*
+     * Guest PC corresponding to this block.  This must be the true
+     * virtual address.  Therefore e.g. x86 stores EIP + CS_BASE, and
+     * targets like Arm, MIPS, HP-PA, which reuse low bits for ISA or
+     * privilege, must store those bits elsewhere.
+     *
+     * If TARGET_TB_PCREL, the opcodes for the TranslationBlock are
+     * written such that the TB is associated only with the physical
+     * page and may be run in any virtual address context.  In this case,
+     * PC must always be taken from ENV in a target-specific manner.
+     * Unwind information is taken as offsets from the page, to be
+     * deposited into the "current" PC.
+     */
+    target_ulong pc;
+#endif
+
+    /*
+     * Target-specific data associated with the TranslationBlock, e.g.:
+     * x86: the original user, the Code Segment virtual base,
+     * arm: an extension of tb->flags,
+     * s390x: instruction data for EXECUTE,
+     * sparc: the next pc of the instruction queue (for delay slots).
+     */
+    target_ulong cs_base;
+
     uint32_t flags; /* flags defining in which context the code was generated */
     uint32_t cflags;    /* compile flags */
 
@@ -XXX,XX +XXX,XX @@ struct TranslationBlock {
 /* Hide the read to avoid ifdefs for TARGET_TB_PCREL. */
 static inline target_ulong tb_pc(const TranslationBlock *tb)
 {
+#if TARGET_TB_PCREL
+    qemu_build_not_reached();
+#else
     return tb->pc;
+#endif
 }
 
 /* Hide the qatomic_read to make code a little easier on the eyes */
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -XXX,XX +XXX,XX @@ static bool tb_lookup_cmp(const void *p, const void *d)
     const TranslationBlock *tb = p;
     const struct tb_desc *desc = d;
 
-    if (tb_pc(tb) == desc->pc &&
+    if ((TARGET_TB_PCREL || tb_pc(tb) == desc->pc) &&
         tb->page_addr[0] == desc->page_addr0 &&
         tb->cs_base == desc->cs_base &&
         tb->flags == desc->flags &&
@@ -XXX,XX +XXX,XX @@ static TranslationBlock *tb_htable_lookup(CPUState *cpu, target_ulong pc,
         return NULL;
     }
     desc.page_addr0 = phys_pc;
-    h = tb_hash_func(phys_pc, pc, flags, cflags, *cpu->trace_dstate);
+    h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : pc),
+                     flags, cflags, *cpu->trace_dstate);
     return qht_lookup_custom(&tb_ctx.htable, &desc, h, tb_lookup_cmp);
 }
 
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
                                           uint32_t flags, uint32_t cflags)
 {
     TranslationBlock *tb;
+    CPUJumpCache *jc;
     uint32_t hash;
 
     /* we should never be trying to look up an INVALID tb */
     tcg_debug_assert(!(cflags & CF_INVALID));
 
     hash = tb_jmp_cache_hash_func(pc);
-    tb = qatomic_rcu_read(&cpu->tb_jmp_cache->array[hash].tb);
+    jc = cpu->tb_jmp_cache;
+    tb = tb_jmp_cache_get_tb(jc, hash);
 
     if (likely(tb &&
-               tb->pc == pc &&
+               tb_jmp_cache_get_pc(jc, hash, tb) == pc &&
                tb->cs_base == cs_base &&
                tb->flags == flags &&
                tb->trace_vcpu_dstate == *cpu->trace_dstate &&
@@ -XXX,XX +XXX,XX @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
     if (tb == NULL) {
         return NULL;
     }
-    qatomic_set(&cpu->tb_jmp_cache->array[hash].tb, tb);
+    tb_jmp_cache_set(jc, hash, tb, pc);
     return tb;
 }
 
@@ -XXX,XX +XXX,XX @@ cpu_tb_exec(CPUState *cpu, TranslationBlock *itb, int *tb_exit)
         if (cc->tcg_ops->synchronize_from_tb) {
             cc->tcg_ops->synchronize_from_tb(cpu, last_tb);
         } else {
+            assert(!TARGET_TB_PCREL);
             assert(cc->set_pc);
             cc->set_pc(cpu, tb_pc(last_tb));
         }
@@ -XXX,XX +XXX,XX @@ int cpu_exec(CPUState *cpu)
                  * for the fast lookup
                  */
                 h = tb_jmp_cache_hash_func(pc);
-                qatomic_set(&cpu->tb_jmp_cache->array[h].tb, tb);
+                tb_jmp_cache_set(cpu->tb_jmp_cache, h, tb, pc);
             }
 
 #ifndef CONFIG_USER_ONLY
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
 
         for (j = 0; j < TARGET_INSN_START_WORDS; ++j) {
             if (i == 0) {
-                prev = (j == 0 ? tb_pc(tb) : 0);
+                prev = (!TARGET_TB_PCREL && j == 0 ? tb_pc(tb) : 0);
             } else {
                 prev = tcg_ctx->gen_insn_data[i - 1][j];
             }
@@ -XXX,XX +XXX,XX @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
 static int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
                                      uintptr_t searched_pc, bool reset_icount)
 {
-    target_ulong data[TARGET_INSN_START_WORDS] = { tb_pc(tb) };
+    target_ulong data[TARGET_INSN_START_WORDS];
     uintptr_t host_pc = (uintptr_t)tb->tc.ptr;
     CPUArchState *env = cpu->env_ptr;
     const uint8_t *p = tb->tc.ptr + tb->tc.size;
@@ -XXX,XX +XXX,XX @@ static int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
         return -1;
     }
 
+    memset(data, 0, sizeof(data));
+    if (!TARGET_TB_PCREL) {
+        data[0] = tb_pc(tb);
+    }
+
     /* Reconstruct the stored insn data while looking for the point at
        which the end of the insn exceeds the searched_pc.  */
     for (i = 0; i < num_insns; ++i) {
@@ -XXX,XX +XXX,XX @@ static bool tb_cmp(const void *ap, const void *bp)
     const TranslationBlock *a = ap;
     const TranslationBlock *b = bp;
 
-    return tb_pc(a) == tb_pc(b) &&
-        a->cs_base == b->cs_base &&
-        a->flags == b->flags &&
-        (tb_cflags(a) & ~CF_INVALID) == (tb_cflags(b) & ~CF_INVALID) &&
-        a->trace_vcpu_dstate == b->trace_vcpu_dstate &&
-        a->page_addr[0] == b->page_addr[0] &&
-        a->page_addr[1] == b->page_addr[1];
+    return ((TARGET_TB_PCREL || tb_pc(a) == tb_pc(b)) &&
+            a->cs_base == b->cs_base &&
+            a->flags == b->flags &&
+            (tb_cflags(a) & ~CF_INVALID) == (tb_cflags(b) & ~CF_INVALID) &&
+            a->trace_vcpu_dstate == b->trace_vcpu_dstate &&
+            a->page_addr[0] == b->page_addr[0] &&
+            a->page_addr[1] == b->page_addr[1]);
 }
 
 void tb_htable_init(void)
@@ -XXX,XX +XXX,XX @@ static inline void tb_jmp_unlink(TranslationBlock *dest)
     qemu_spin_unlock(&dest->jmp_lock);
 }
 
+static void tb_jmp_cache_inval_tb(TranslationBlock *tb)
+{
+    CPUState *cpu;
+
+    if (TARGET_TB_PCREL) {
+        /* A TB may be at any virtual address */
+        CPU_FOREACH(cpu) {
+            tcg_flush_jmp_cache(cpu);
+        }
+    } else {
+        uint32_t h = tb_jmp_cache_hash_func(tb_pc(tb));
+
+        CPU_FOREACH(cpu) {
+            CPUJumpCache *jc = cpu->tb_jmp_cache;
+
+            if (qatomic_read(&jc->array[h].tb) == tb) {
+                qatomic_set(&jc->array[h].tb, NULL);
+            }
+        }
+    }
+}
+
 /*
  * In user-mode, call with mmap_lock held.
  * In !user-mode, if @rm_from_page_list is set, call with the TB's pages'
@@ -XXX,XX +XXX,XX @@ static inline void tb_jmp_unlink(TranslationBlock *dest)
  */
 static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
 {
-    CPUState *cpu;
     PageDesc *p;
     uint32_t h;
     tb_page_addr_t phys_pc;
@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
 
     /* remove the TB from the hash list */
     phys_pc = tb->page_addr[0];
-    h = tb_hash_func(phys_pc, tb_pc(tb), tb->flags, orig_cflags,
-                     tb->trace_vcpu_dstate);
+    h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : tb_pc(tb)),
+                     tb->flags, orig_cflags, tb->trace_vcpu_dstate);
     if (!qht_remove(&tb_ctx.htable, tb, h)) {
         return;
     }
@@ -XXX,XX +XXX,XX @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
     }
 
     /* remove the TB from the hash list */
-    h = tb_jmp_cache_hash_func(tb->pc);
-    CPU_FOREACH(cpu) {
-        CPUJumpCache *jc = cpu->tb_jmp_cache;
-        if (qatomic_read(&jc->array[h].tb) == tb) {
-            qatomic_set(&jc->array[h].tb, NULL);
-        }
-    }
+    tb_jmp_cache_inval_tb(tb);
 
     /* suppress this TB from the two jump lists */
     tb_remove_from_jmp_list(tb, 0);
@@ -XXX,XX +XXX,XX @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
     }
 
     /* add in the hash table */
-    h = tb_hash_func(phys_pc, tb_pc(tb), tb->flags, tb->cflags,
-                     tb->trace_vcpu_dstate);
+    h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : tb_pc(tb)),
+                     tb->flags, tb->cflags, tb->trace_vcpu_dstate);
     qht_insert(&tb_ctx.htable, tb, h, &existing_tb);
 
     /* remove TB from the page(s) if we couldn't insert it */
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 
     gen_code_buf = tcg_ctx->code_gen_ptr;
     tb->tc.ptr = tcg_splitwx_to_rx(gen_code_buf);
+#if !TARGET_TB_PCREL
     tb->pc = pc;
+#endif
     tb->cs_base = cs_base;
     tb->flags = flags;
     tb->cflags = cflags;
-- 
2.34.1

From: Leandro Lupori <leandro.lupori@eldorado.org.br>

PowerPC64 processors handle direct branches better than indirect
ones, resulting in less stalled cycles and branch misses.

However, PPC's tb_target_set_jmp_target() was only using direct
branches for 16-bit jumps, while PowerPC64's unconditional branch
instructions are able to handle displacements of up to 26 bits.
To take advantage of this, now jumps whose displacements fit in
between 17 and 26 bits are also converted to direct branches.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Leandro Lupori <leandro.lupori@eldorado.org.br>
[rth: Expanded some commentary.]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/ppc/tcg-target.c.inc | 119 +++++++++++++++++++++++++++++----------
 1 file changed, 88 insertions(+), 31 deletions(-)

diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc
index XXXXXXX..XXXXXXX 100644
--- a/tcg/ppc/tcg-target.c.inc
+++ b/tcg/ppc/tcg-target.c.inc
@@ -XXX,XX +XXX,XX @@ static void tcg_out_mb(TCGContext *s, TCGArg a0)
     tcg_out32(s, insn);
 }
 
+static inline uint64_t make_pair(tcg_insn_unit i1, tcg_insn_unit i2)
+{
+    if (HOST_BIG_ENDIAN) {
+        return (uint64_t)i1 << 32 | i2;
+    }
+    return (uint64_t)i2 << 32 | i1;
+}
+
+static inline void ppc64_replace2(uintptr_t rx, uintptr_t rw,
+                                  tcg_insn_unit i0, tcg_insn_unit i1)
+{
+#if TCG_TARGET_REG_BITS == 64
+    qatomic_set((uint64_t *)rw, make_pair(i0, i1));
+    flush_idcache_range(rx, rw, 8);
+#else
+    qemu_build_not_reached();
+#endif
+}
+
+static inline void ppc64_replace4(uintptr_t rx, uintptr_t rw,
+                                  tcg_insn_unit i0, tcg_insn_unit i1,
+                                  tcg_insn_unit i2, tcg_insn_unit i3)
+{
+    uint64_t p[2];
+
+    p[!HOST_BIG_ENDIAN] = make_pair(i0, i1);
+    p[HOST_BIG_ENDIAN] = make_pair(i2, i3);
+
+    /*
+     * There's no convenient way to get the compiler to allocate a pair
+     * of registers at an even index, so copy into r6/r7 and clobber.
+     */
+    asm("mr  %%r6, %1\n\t"
+        "mr  %%r7, %2\n\t"
+        "stq %%r6, %0"
+        : "=Q"(*(__int128 *)rw) : "r"(p[0]), "r"(p[1]) : "r6", "r7");
+    flush_idcache_range(rx, rw, 16);
+}
+
 void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx,
                               uintptr_t jmp_rw, uintptr_t addr)
 {
-    if (TCG_TARGET_REG_BITS == 64) {
-        tcg_insn_unit i1, i2;
-        intptr_t tb_diff = addr - tc_ptr;
-        intptr_t br_diff = addr - (jmp_rx + 4);
-        uint64_t pair;
+    tcg_insn_unit i0, i1, i2, i3;
+    intptr_t tb_diff = addr - tc_ptr;
+    intptr_t br_diff = addr - (jmp_rx + 4);
+    intptr_t lo, hi;
 
-        /* This does not exercise the range of the branch, but we do
-           still need to be able to load the new value of TCG_REG_TB.
-           But this does still happen quite often.  */
-        if (tb_diff == (int16_t)tb_diff) {
-            i1 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, tb_diff);
-            i2 = B | (br_diff & 0x3fffffc);
-        } else {
-            intptr_t lo = (int16_t)tb_diff;
-            intptr_t hi = (int32_t)(tb_diff - lo);
-            assert(tb_diff == hi + lo);
-            i1 = ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, hi >> 16);
-            i2 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, lo);
-        }
-#if HOST_BIG_ENDIAN
-        pair = (uint64_t)i1 << 32 | i2;
-#else
-        pair = (uint64_t)i2 << 32 | i1;
-#endif
-
-        /* As per the enclosing if, this is ppc64.  Avoid the _Static_assert
-           within qatomic_set that would fail to build a ppc32 host.  */
-        qatomic_set__nocheck((uint64_t *)jmp_rw, pair);
-        flush_idcache_range(jmp_rx, jmp_rw, 8);
-    } else {
+    if (TCG_TARGET_REG_BITS == 32) {
         intptr_t diff = addr - jmp_rx;
         tcg_debug_assert(in_range_b(diff));
         qatomic_set((uint32_t *)jmp_rw, B | (diff & 0x3fffffc));
         flush_idcache_range(jmp_rx, jmp_rw, 4);
+        return;
     }
+
+    /*
+     * For 16-bit displacements, we can use a single add + branch.
+     * This happens quite often.
+     */
+    if (tb_diff == (int16_t)tb_diff) {
+        i0 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, tb_diff);
+        i1 = B | (br_diff & 0x3fffffc);
+        ppc64_replace2(jmp_rx, jmp_rw, i0, i1);
+        return;
+    }
+
+    lo = (int16_t)tb_diff;
+    hi = (int32_t)(tb_diff - lo);
+    assert(tb_diff == hi + lo);
+    i0 = ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, hi >> 16);
+    i1 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, lo);
+
+    /*
+     * Without stq from 2.07, we can only update two insns,
+     * and those must be the ones that load the target address.
+     */
+    if (!have_isa_2_07) {
+        ppc64_replace2(jmp_rx, jmp_rw, i0, i1);
+        return;
+    }
+
+    /*
+     * For 26-bit displacements, we can use a direct branch.
+     * Otherwise we still need the indirect branch, which we
+     * must restore after a potential direct branch write.
+     */
+    br_diff -= 4;
+    if (in_range_b(br_diff)) {
+        i2 = B | (br_diff & 0x3fffffc);
+        i3 = NOP;
+    } else {
+        i2 = MTSPR | RS(TCG_REG_TB) | CTR;
+        i3 = BCCTR | BO_ALWAYS;
+    }
+    ppc64_replace4(jmp_rx, jmp_rw, i0, i1, i2, i3);
 }
 
 static void tcg_out_call_int(TCGContext *s, int lk,
@@ -XXX,XX +XXX,XX @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
         if (s->tb_jmp_insn_offset) {
             /* Direct jump. */
             if (TCG_TARGET_REG_BITS == 64) {
-                /* Ensure the next insns are 8-byte aligned. */
-                if ((uintptr_t)s->code_ptr & 7) {
+                /* Ensure the next insns are 8 or 16-byte aligned. */
+                while ((uintptr_t)s->code_ptr & (have_isa_2_07 ? 15 : 7)) {
                     tcg_out32(s, NOP);
                 }
                 s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s);
-- 
2.34.1

The value previously chosen overlaps GUSA_MASK.

Rename all DELAY_SLOT_* and GUSA_* defines to emphasize
that they are included in TB_FLAGs.  Add aliases for the
FPSCR and SR bits that are included in TB_FLAGS, so that
we don't accidentally reassign those bits.

Fixes: 4da06fb3062 ("target/sh4: Implement prctl_unalign_sigbus")
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/856
Reviewed-by: Yoshinori Sato <ysato@users.sourceforge.jp>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/sh4/cpu.h        | 56 +++++++++++++------------
 linux-user/sh4/signal.c |  6 +--
 target/sh4/cpu.c        |  6 +--
 target/sh4/helper.c     |  6 +--
 target/sh4/translate.c  | 90 ++++++++++++++++++++++-------------------
 5 files changed, 88 insertions(+), 76 deletions(-)

diff --git a/target/sh4/cpu.h b/target/sh4/cpu.h
index XXXXXXX..XXXXXXX 100644
--- a/target/sh4/cpu.h
+++ b/target/sh4/cpu.h
@@ -XXX,XX +XXX,XX @@
 #define FPSCR_RM_NEAREST       (0 << 0)
 #define FPSCR_RM_ZERO          (1 << 0)
 
-#define DELAY_SLOT_MASK        0x7
-#define DELAY_SLOT             (1 << 0)
-#define DELAY_SLOT_CONDITIONAL (1 << 1)
-#define DELAY_SLOT_RTE         (1 << 2)
+#define TB_FLAG_DELAY_SLOT       (1 << 0)
+#define TB_FLAG_DELAY_SLOT_COND  (1 << 1)
+#define TB_FLAG_DELAY_SLOT_RTE   (1 << 2)
+#define TB_FLAG_PENDING_MOVCA    (1 << 3)
+#define TB_FLAG_GUSA_SHIFT       4                      /* [11:4] */
+#define TB_FLAG_GUSA_EXCLUSIVE   (1 << 12)
+#define TB_FLAG_UNALIGN          (1 << 13)
+#define TB_FLAG_SR_FD            (1 << SR_FD)           /* 15 */
+#define TB_FLAG_FPSCR_PR         FPSCR_PR               /* 19 */
+#define TB_FLAG_FPSCR_SZ         FPSCR_SZ               /* 20 */
+#define TB_FLAG_FPSCR_FR         FPSCR_FR               /* 21 */
+#define TB_FLAG_SR_RB            (1 << SR_RB)           /* 29 */
+#define TB_FLAG_SR_MD            (1 << SR_MD)           /* 30 */
 
-#define TB_FLAG_PENDING_MOVCA  (1 << 3)
-#define TB_FLAG_UNALIGN        (1 << 4)
-
-#define GUSA_SHIFT             4
-#ifdef CONFIG_USER_ONLY
-#define GUSA_EXCLUSIVE         (1 << 12)
-#define GUSA_MASK              ((0xff << GUSA_SHIFT) | GUSA_EXCLUSIVE)
-#else
-/* Provide dummy versions of the above to allow tests against tbflags
-   to be elided while avoiding ifdefs.  */
-#define GUSA_EXCLUSIVE         0
-#define GUSA_MASK              0
-#endif
-
-#define TB_FLAG_ENVFLAGS_MASK  (DELAY_SLOT_MASK | GUSA_MASK)
+#define TB_FLAG_DELAY_SLOT_MASK  (TB_FLAG_DELAY_SLOT |       \
+                                  TB_FLAG_DELAY_SLOT_COND |  \
+                                  TB_FLAG_DELAY_SLOT_RTE)
+#define TB_FLAG_GUSA_MASK        ((0xff << TB_FLAG_GUSA_SHIFT) | \
+                                  TB_FLAG_GUSA_EXCLUSIVE)
+#define TB_FLAG_FPSCR_MASK       (TB_FLAG_FPSCR_PR | \
+                                  TB_FLAG_FPSCR_SZ | \
+                                  TB_FLAG_FPSCR_FR)
+#define TB_FLAG_SR_MASK          (TB_FLAG_SR_FD | \
+                                  TB_FLAG_SR_RB | \
+                                  TB_FLAG_SR_MD)
+#define TB_FLAG_ENVFLAGS_MASK    (TB_FLAG_DELAY_SLOT_MASK | \
+                                  TB_FLAG_GUSA_MASK)
 
 typedef struct tlb_t {
     uint32_t vpn;		/* virtual page number */
@@ -XXX,XX +XXX,XX @@ static inline int cpu_mmu_index (CPUSH4State *env, bool ifetch)
 {
     /* The instruction in a RTE delay slot is fetched in privileged
        mode, but executed in user mode.  */
-    if (ifetch && (env->flags & DELAY_SLOT_RTE)) {
+    if (ifetch && (env->flags & TB_FLAG_DELAY_SLOT_RTE)) {
         return 0;
     } else {
         return (env->sr & (1u << SR_MD)) == 0 ? 1 : 0;
@@ -XXX,XX +XXX,XX @@ static inline void cpu_get_tb_cpu_state(CPUSH4State *env, target_ulong *pc,
 {
     *pc = env->pc;
     /* For a gUSA region, notice the end of the region.  */
-    *cs_base = env->flags & GUSA_MASK ? env->gregs[0] : 0;
-    *flags = env->flags /* TB_FLAG_ENVFLAGS_MASK: bits 0-2, 4-12 */
-            | (env->fpscr & (FPSCR_FR | FPSCR_SZ | FPSCR_PR))  /* Bits 19-21 */
-            | (env->sr & ((1u << SR_MD) | (1u << SR_RB)))      /* Bits 29-30 */
-            | (env->sr & (1u << SR_FD))                        /* Bit 15 */
+    *cs_base = env->flags & TB_FLAG_GUSA_MASK ? env->gregs[0] : 0;
+    *flags = env->flags
+            | (env->fpscr & TB_FLAG_FPSCR_MASK)
+            | (env->sr & TB_FLAG_SR_MASK)
             | (env->movcal_backup ? TB_FLAG_PENDING_MOVCA : 0); /* Bit 3 */
 #ifdef CONFIG_USER_ONLY
     *flags |= TB_FLAG_UNALIGN * !env_cpu(env)->prctl_unalign_sigbus;
diff --git a/linux-user/sh4/signal.c b/linux-user/sh4/signal.c
index XXXXXXX..XXXXXXX 100644
--- a/linux-user/sh4/signal.c
+++ b/linux-user/sh4/signal.c
@@ -XXX,XX +XXX,XX @@ static void restore_sigcontext(CPUSH4State *regs, struct target_sigcontext *sc)
     __get_user(regs->fpul, &sc->sc_fpul);
 
     regs->tra = -1;         /* disable syscall checks */
-    regs->flags &= ~(DELAY_SLOT_MASK | GUSA_MASK);
+    regs->flags = 0;
 }
 
 void setup_frame(int sig, struct target_sigaction *ka,
@@ -XXX,XX +XXX,XX @@ void setup_frame(int sig, struct target_sigaction *ka,
     regs->gregs[5] = 0;
     regs->gregs[6] = frame_addr += offsetof(typeof(*frame), sc);
     regs->pc = (unsigned long) ka->_sa_handler;
-    regs->flags &= ~(DELAY_SLOT_MASK | GUSA_MASK);
+    regs->flags &= ~(TB_FLAG_DELAY_SLOT_MASK | TB_FLAG_GUSA_MASK);
 
     unlock_user_struct(frame, frame_addr, 1);
     return;
@@ -XXX,XX +XXX,XX @@ void setup_rt_frame(int sig, struct target_sigaction *ka,
     regs->gregs[5] = frame_addr + offsetof(typeof(*frame), info);
     regs->gregs[6] = frame_addr + offsetof(typeof(*frame), uc);
     regs->pc = (unsigned long) ka->_sa_handler;
-    regs->flags &= ~(DELAY_SLOT_MASK | GUSA_MASK);
+    regs->flags &= ~(TB_FLAG_DELAY_SLOT_MASK | TB_FLAG_GUSA_MASK);
 
     unlock_user_struct(frame, frame_addr, 1);
     return;
diff --git a/target/sh4/cpu.c b/target/sh4/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/target/sh4/cpu.c
+++ b/target/sh4/cpu.c
@@ -XXX,XX +XXX,XX @@ static void superh_cpu_synchronize_from_tb(CPUState *cs,
     SuperHCPU *cpu = SUPERH_CPU(cs);
 
     cpu->env.pc = tb_pc(tb);
-    cpu->env.flags = tb->flags & TB_FLAG_ENVFLAGS_MASK;
+    cpu->env.flags = tb->flags;
 }
 
 #ifndef CONFIG_USER_ONLY
@@ -XXX,XX +XXX,XX @@ static bool superh_io_recompile_replay_branch(CPUState *cs,
     SuperHCPU *cpu = SUPERH_CPU(cs);
     CPUSH4State *env = &cpu->env;
 
-    if ((env->flags & ((DELAY_SLOT | DELAY_SLOT_CONDITIONAL))) != 0
+    if ((env->flags & (TB_FLAG_DELAY_SLOT | TB_FLAG_DELAY_SLOT_COND))
         && env->pc != tb_pc(tb)) {
         env->pc -= 2;
-        env->flags &= ~(DELAY_SLOT | DELAY_SLOT_CONDITIONAL);
+        env->flags &= ~(TB_FLAG_DELAY_SLOT | TB_FLAG_DELAY_SLOT_COND);
         return true;
     }
     return false;
diff --git a/target/sh4/helper.c b/target/sh4/helper.c
index XXXXXXX..XXXXXXX 100644
--- a/target/sh4/helper.c
+++ b/target/sh4/helper.c
@@ -XXX,XX +XXX,XX @@ void superh_cpu_do_interrupt(CPUState *cs)
     env->sr |= (1u << SR_BL) | (1u << SR_MD) | (1u << SR_RB);
     env->lock_addr = -1;
 
-    if (env->flags & DELAY_SLOT_MASK) {
+    if (env->flags & TB_FLAG_DELAY_SLOT_MASK) {
         /* Branch instruction should be executed again before delay slot. */
 	env->spc -= 2;
 	/* Clear flags for exception/interrupt routine. */
-        env->flags &= ~DELAY_SLOT_MASK;
+        env->flags &= ~TB_FLAG_DELAY_SLOT_MASK;
     }
 
     if (do_exp) {
@@ -XXX,XX +XXX,XX @@ bool superh_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
         CPUSH4State *env = &cpu->env;
 
         /* Delay slots are indivisible, ignore interrupts */
-        if (env->flags & DELAY_SLOT_MASK) {
+        if (env->flags & TB_FLAG_DELAY_SLOT_MASK) {
             return false;
         } else {
             superh_cpu_do_interrupt(cs);
diff --git a/target/sh4/translate.c b/target/sh4/translate.c
index XXXXXXX..XXXXXXX 100644
--- a/target/sh4/translate.c
+++ b/target/sh4/translate.c
@@ -XXX,XX +XXX,XX @@ void superh_cpu_dump_state(CPUState *cs, FILE *f, int flags)
 		    i, env->gregs[i], i + 1, env->gregs[i + 1],
 		    i + 2, env->gregs[i + 2], i + 3, env->gregs[i + 3]);
     }
-    if (env->flags & DELAY_SLOT) {
+    if (env->flags & TB_FLAG_DELAY_SLOT) {
         qemu_printf("in delay slot (delayed_pc=0x%08x)\n",
 		    env->delayed_pc);
-    } else if (env->flags & DELAY_SLOT_CONDITIONAL) {
+    } else if (env->flags & TB_FLAG_DELAY_SLOT_COND) {
         qemu_printf("in conditional delay slot (delayed_pc=0x%08x)\n",
 		    env->delayed_pc);
-    } else if (env->flags & DELAY_SLOT_RTE) {
+    } else if (env->flags & TB_FLAG_DELAY_SLOT_RTE) {
         qemu_fprintf(f, "in rte delay slot (delayed_pc=0x%08x)\n",
                      env->delayed_pc);
     }
@@ -XXX,XX +XXX,XX @@ static inline void gen_save_cpu_state(DisasContext *ctx, bool save_pc)
 
 static inline bool use_exit_tb(DisasContext *ctx)
 {
-    return (ctx->tbflags & GUSA_EXCLUSIVE) != 0;
+    return (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE) != 0;
 }
 
 static bool use_goto_tb(DisasContext *ctx, target_ulong dest)
@@ -XXX,XX +XXX,XX @@ static void gen_conditional_jump(DisasContext *ctx, target_ulong dest,
     TCGLabel *l1 = gen_new_label();
     TCGCond cond_not_taken = jump_if_true ? TCG_COND_EQ : TCG_COND_NE;
 
-    if (ctx->tbflags & GUSA_EXCLUSIVE) {
+    if (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE) {
         /* When in an exclusive region, we must continue to the end.
            Therefore, exit the region on a taken branch, but otherwise
            fall through to the next instruction.  */
         tcg_gen_brcondi_i32(cond_not_taken, cpu_sr_t, 0, l1);
-        tcg_gen_movi_i32(cpu_flags, ctx->envflags & ~GUSA_MASK);
+        tcg_gen_movi_i32(cpu_flags, ctx->envflags & ~TB_FLAG_GUSA_MASK);
         /* Note that this won't actually use a goto_tb opcode because we
            disallow it in use_goto_tb, but it handles exit + singlestep.  */
         gen_goto_tb(ctx, 0, dest);
@@ -XXX,XX +XXX,XX @@ static void gen_delayed_conditional_jump(DisasContext * ctx)
     tcg_gen_mov_i32(ds, cpu_delayed_cond);
     tcg_gen_discard_i32(cpu_delayed_cond);
 
-    if (ctx->tbflags & GUSA_EXCLUSIVE) {
+    if (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE) {
         /* When in an exclusive region, we must continue to the end.
            Therefore, exit the region on a taken branch, but otherwise
            fall through to the next instruction.  */
         tcg_gen_brcondi_i32(TCG_COND_EQ, ds, 0, l1);
 
         /* Leave the gUSA region.  */
-        tcg_gen_movi_i32(cpu_flags, ctx->envflags & ~GUSA_MASK);
+        tcg_gen_movi_i32(cpu_flags, ctx->envflags & ~TB_FLAG_GUSA_MASK);
         gen_jump(ctx);
 
         gen_set_label(l1);
@@ -XXX,XX +XXX,XX @@ static inline void gen_store_fpr64(DisasContext *ctx, TCGv_i64 t, int reg)
 #define XHACK(x) ((((x) & 1 ) << 4) | ((x) & 0xe))
 
 #define CHECK_NOT_DELAY_SLOT \
-    if (ctx->envflags & DELAY_SLOT_MASK) {  \
-        goto do_illegal_slot;               \
+    if (ctx->envflags & TB_FLAG_DELAY_SLOT_MASK) {  \
+        goto do_illegal_slot;                       \
     }
 
 #define CHECK_PRIVILEGED \
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
     case 0x000b:		/* rts */
 	CHECK_NOT_DELAY_SLOT
 	tcg_gen_mov_i32(cpu_delayed_pc, cpu_pr);
-        ctx->envflags |= DELAY_SLOT;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT;
 	ctx->delayed_pc = (uint32_t) - 1;
 	return;
     case 0x0028:		/* clrmac */
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
 	CHECK_NOT_DELAY_SLOT
         gen_write_sr(cpu_ssr);
 	tcg_gen_mov_i32(cpu_delayed_pc, cpu_spc);
-        ctx->envflags |= DELAY_SLOT_RTE;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT_RTE;
 	ctx->delayed_pc = (uint32_t) - 1;
         ctx->base.is_jmp = DISAS_STOP;
 	return;
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
 	return;
     case 0xe000:		/* mov #imm,Rn */
 #ifdef CONFIG_USER_ONLY
-        /* Detect the start of a gUSA region.  If so, update envflags
-           and end the TB.  This will allow us to see the end of the
-           region (stored in R0) in the next TB.  */
+        /*
+         * Detect the start of a gUSA region (mov #-n, r15).
+         * If so, update envflags and end the TB.  This will allow us
+         * to see the end of the region (stored in R0) in the next TB.
+         */
         if (B11_8 == 15 && B7_0s < 0 &&
             (tb_cflags(ctx->base.tb) & CF_PARALLEL)) {
-            ctx->envflags = deposit32(ctx->envflags, GUSA_SHIFT, 8, B7_0s);
+            ctx->envflags =
+                deposit32(ctx->envflags, TB_FLAG_GUSA_SHIFT, 8, B7_0s);
             ctx->base.is_jmp = DISAS_STOP;
         }
 #endif
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
     case 0xa000:		/* bra disp */
 	CHECK_NOT_DELAY_SLOT
         ctx->delayed_pc = ctx->base.pc_next + 4 + B11_0s * 2;
-        ctx->envflags |= DELAY_SLOT;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT;
 	return;
     case 0xb000:		/* bsr disp */
 	CHECK_NOT_DELAY_SLOT
         tcg_gen_movi_i32(cpu_pr, ctx->base.pc_next + 4);
         ctx->delayed_pc = ctx->base.pc_next + 4 + B11_0s * 2;
-        ctx->envflags |= DELAY_SLOT;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT;
 	return;
     }
 
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
 	CHECK_NOT_DELAY_SLOT
         tcg_gen_xori_i32(cpu_delayed_cond, cpu_sr_t, 1);
         ctx->delayed_pc = ctx->base.pc_next + 4 + B7_0s * 2;
-        ctx->envflags |= DELAY_SLOT_CONDITIONAL;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT_COND;
 	return;
     case 0x8900:		/* bt label */
 	CHECK_NOT_DELAY_SLOT
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
 	CHECK_NOT_DELAY_SLOT
         tcg_gen_mov_i32(cpu_delayed_cond, cpu_sr_t);
         ctx->delayed_pc = ctx->base.pc_next + 4 + B7_0s * 2;
-        ctx->envflags |= DELAY_SLOT_CONDITIONAL;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT_COND;
 	return;
     case 0x8800:		/* cmp/eq #imm,R0 */
         tcg_gen_setcondi_i32(TCG_COND_EQ, cpu_sr_t, REG(0), B7_0s);
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
     case 0x0023:		/* braf Rn */
 	CHECK_NOT_DELAY_SLOT
         tcg_gen_addi_i32(cpu_delayed_pc, REG(B11_8), ctx->base.pc_next + 4);
-        ctx->envflags |= DELAY_SLOT;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT;
 	ctx->delayed_pc = (uint32_t) - 1;
 	return;
     case 0x0003:		/* bsrf Rn */
 	CHECK_NOT_DELAY_SLOT
         tcg_gen_movi_i32(cpu_pr, ctx->base.pc_next + 4);
 	tcg_gen_add_i32(cpu_delayed_pc, REG(B11_8), cpu_pr);
-        ctx->envflags |= DELAY_SLOT;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT;
 	ctx->delayed_pc = (uint32_t) - 1;
 	return;
     case 0x4015:		/* cmp/pl Rn */
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
     case 0x402b:		/* jmp @Rn */
 	CHECK_NOT_DELAY_SLOT
 	tcg_gen_mov_i32(cpu_delayed_pc, REG(B11_8));
-        ctx->envflags |= DELAY_SLOT;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT;
 	ctx->delayed_pc = (uint32_t) - 1;
 	return;
     case 0x400b:		/* jsr @Rn */
 	CHECK_NOT_DELAY_SLOT
         tcg_gen_movi_i32(cpu_pr, ctx->base.pc_next + 4);
 	tcg_gen_mov_i32(cpu_delayed_pc, REG(B11_8));
-        ctx->envflags |= DELAY_SLOT;
+        ctx->envflags |= TB_FLAG_DELAY_SLOT;
 	ctx->delayed_pc = (uint32_t) - 1;
 	return;
     case 0x400e:		/* ldc Rm,SR */
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
     fflush(stderr);
 #endif
  do_illegal:
-    if (ctx->envflags & DELAY_SLOT_MASK) {
+    if (ctx->envflags & TB_FLAG_DELAY_SLOT_MASK) {
  do_illegal_slot:
         gen_save_cpu_state(ctx, true);
         gen_helper_raise_slot_illegal_instruction(cpu_env);
@@ -XXX,XX +XXX,XX @@ static void _decode_opc(DisasContext * ctx)
 
  do_fpu_disabled:
     gen_save_cpu_state(ctx, true);
-    if (ctx->envflags & DELAY_SLOT_MASK) {
+    if (ctx->envflags & TB_FLAG_DELAY_SLOT_MASK) {
         gen_helper_raise_slot_fpu_disable(cpu_env);
     } else {
         gen_helper_raise_fpu_disable(cpu_env);
@@ -XXX,XX +XXX,XX @@ static void decode_opc(DisasContext * ctx)
 
     _decode_opc(ctx);
 
-    if (old_flags & DELAY_SLOT_MASK) {
+    if (old_flags & TB_FLAG_DELAY_SLOT_MASK) {
         /* go out of the delay slot */
-        ctx->envflags &= ~DELAY_SLOT_MASK;
+        ctx->envflags &= ~TB_FLAG_DELAY_SLOT_MASK;
 
         /* When in an exclusive region, we must continue to the end
            for conditional branches.  */
-        if (ctx->tbflags & GUSA_EXCLUSIVE
-            && old_flags & DELAY_SLOT_CONDITIONAL) {
+        if (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE
+            && old_flags & TB_FLAG_DELAY_SLOT_COND) {
             gen_delayed_conditional_jump(ctx);
             return;
         }
         /* Otherwise this is probably an invalid gUSA region.
            Drop the GUSA bits so the next TB doesn't see them.  */
-        ctx->envflags &= ~GUSA_MASK;
+        ctx->envflags &= ~TB_FLAG_GUSA_MASK;
 
         tcg_gen_movi_i32(cpu_flags, ctx->envflags);
-        if (old_flags & DELAY_SLOT_CONDITIONAL) {
+        if (old_flags & TB_FLAG_DELAY_SLOT_COND) {
 	    gen_delayed_conditional_jump(ctx);
         } else {
             gen_jump(ctx);
@@ -XXX,XX +XXX,XX @@ static void decode_gusa(DisasContext *ctx, CPUSH4State *env)
     }
 
     /* The entire region has been translated.  */
-    ctx->envflags &= ~GUSA_MASK;
+    ctx->envflags &= ~TB_FLAG_GUSA_MASK;
     ctx->base.pc_next = pc_end;
     ctx->base.num_insns += max_insns - 1;
     return;
@@ -XXX,XX +XXX,XX @@ static void decode_gusa(DisasContext *ctx, CPUSH4State *env)
 
     /* Restart with the EXCLUSIVE bit set, within a TB run via
        cpu_exec_step_atomic holding the exclusive lock.  */
-    ctx->envflags |= GUSA_EXCLUSIVE;
+    ctx->envflags |= TB_FLAG_GUSA_EXCLUSIVE;
     gen_save_cpu_state(ctx, false);
     gen_helper_exclusive(cpu_env);
     ctx->base.is_jmp = DISAS_NORETURN;
@@ -XXX,XX +XXX,XX @@ static void sh4_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs)
                   (tbflags & (1 << SR_RB))) * 0x10;
     ctx->fbank = tbflags & FPSCR_FR ? 0x10 : 0;
 
-    if (tbflags & GUSA_MASK) {
+#ifdef CONFIG_USER_ONLY
+    if (tbflags & TB_FLAG_GUSA_MASK) {
+        /* In gUSA exclusive region. */
         uint32_t pc = ctx->base.pc_next;
         uint32_t pc_end = ctx->base.tb->cs_base;
-        int backup = sextract32(ctx->tbflags, GUSA_SHIFT, 8);
+        int backup = sextract32(ctx->tbflags, TB_FLAG_GUSA_SHIFT, 8);
         int max_insns = (pc_end - pc) / 2;
 
         if (pc != pc_end + backup || max_insns < 2) {
             /* This is a malformed gUSA region.  Don't do anything special,
                since the interpreter is likely to get confused.  */
-            ctx->envflags &= ~GUSA_MASK;
-        } else if (tbflags & GUSA_EXCLUSIVE) {
+            ctx->envflags &= ~TB_FLAG_GUSA_MASK;
+        } else if (tbflags & TB_FLAG_GUSA_EXCLUSIVE) {
             /* Regardless of single-stepping or the end of the page,
                we must complete execution of the gUSA region while
                holding the exclusive lock.  */
@@ -XXX,XX +XXX,XX @@ static void sh4_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs)
             return;
         }
     }
+#endif
 
     /* Since the ISA is fixed-width, we can bound by the number
        of instructions remaining on the page.  */
@@ -XXX,XX +XXX,XX @@ static void sh4_tr_translate_insn(DisasContextBase *dcbase, CPUState *cs)
     DisasContext *ctx = container_of(dcbase, DisasContext, base);
 
 #ifdef CONFIG_USER_ONLY
-    if (unlikely(ctx->envflags & GUSA_MASK)
-        && !(ctx->envflags & GUSA_EXCLUSIVE)) {
+    if (unlikely(ctx->envflags & TB_FLAG_GUSA_MASK)
+        && !(ctx->envflags & TB_FLAG_GUSA_EXCLUSIVE)) {
         /* We're in an gUSA region, and we have not already fallen
            back on using an exclusive region.  Attempt to parse the
            region into a single supported atomic operation.  Failure
@@ -XXX,XX +XXX,XX @@ static void sh4_tr_tb_stop(DisasContextBase *dcbase, CPUState *cs)
 {
     DisasContext *ctx = container_of(dcbase, DisasContext, base);
 
-    if (ctx->tbflags & GUSA_EXCLUSIVE) {
+    if (ctx->tbflags & TB_FLAG_GUSA_EXCLUSIVE) {
         /* Ending the region of exclusivity.  Clear the bits.  */
-        ctx->envflags &= ~GUSA_MASK;
+        ctx->envflags &= ~TB_FLAG_GUSA_MASK;
     }
 
     switch (ctx->base.is_jmp) {
-- 
2.34.1

The following changes since commit 9c6c079bc6723da8061ccfb44361d67b1dd785dd:

Merge tag 'pull-target-arm-20240430' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2024-04-30 09:58:54 -0700)

are available in the Git repository at:

https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20240501

for you to fetch changes up to 917d7f8d948d706e275c9f33169b9dd0149ded1e:

plugins: Update the documentation block for plugin-gen.c (2024-04-30 16:12:05 -0700)

----------------------------------------------------------------
plugins: Rewrite plugin tcg expansion

----------------------------------------------------------------
Richard Henderson (20):
      tcg: Make tcg/helper-info.h self-contained
      tcg: Pass function pointer to tcg_gen_call*
      plugins: Zero new qemu_plugin_dyn_cb entries
      plugins: Move function pointer in qemu_plugin_dyn_cb
      plugins: Create TCGHelperInfo for all out-of-line callbacks
      plugins: Use emit_before_op for PLUGIN_GEN_AFTER_INSN
      plugins: Use emit_before_op for PLUGIN_GEN_FROM_TB
      plugins: Add PLUGIN_GEN_AFTER_TB
      plugins: Use emit_before_op for PLUGIN_GEN_FROM_INSN
      plugins: Use emit_before_op for PLUGIN_GEN_FROM_MEM
      plugins: Remove plugin helpers
      tcg: Remove TCG_CALL_PLUGIN
      tcg: Remove INDEX_op_plugin_cb_{start,end}
      plugins: Simplify callback queues
      plugins: Introduce PLUGIN_CB_MEM_REGULAR
      plugins: Replace pr_ops with a proper debug dump flag
      plugins: Split out common cb expanders
      plugins: Merge qemu_plugin_tb_insn_get to plugin-gen.c
      plugins: Inline plugin_gen_empty_callback
      plugins: Update the documentation block for plugin-gen.c

Move MAX_CALL_IARGS from tcg.h and include for
the define of TCG_TARGET_REG_BITS.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/helper-info.h | 3 +++
 include/tcg/tcg.h         | 2 --
 tcg/tci.c                 | 1 +
 3 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/include/tcg/helper-info.h b/include/tcg/helper-info.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/helper-info.h
+++ b/include/tcg/helper-info.h
@@ -XXX,XX +XXX,XX @@
 #ifdef CONFIG_TCG_INTERPRETER
 #include <ffi.h>
 #endif
+#include "tcg-target-reg-bits.h"
+
+#define MAX_CALL_IARGS  7
 
 /*
  * Describe the calling convention of a given argument type.
diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@
 /* XXX: make safe guess about sizes */
 #define MAX_OP_PER_INSTR 266
 
-#define MAX_CALL_IARGS  7
-
 #define CPU_TEMP_BUF_NLONGS 128
 #define TCG_STATIC_FRAME_SIZE  (CPU_TEMP_BUF_NLONGS * sizeof(long))
 
diff --git a/tcg/tci.c b/tcg/tci.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tci.c
+++ b/tcg/tci.c
@@ -XXX,XX +XXX,XX @@
 
 #include "qemu/osdep.h"
 #include "tcg/tcg.h"
+#include "tcg/helper-info.h"
 #include "tcg/tcg-ldst.h"
 #include <ffi.h>
 
-- 
2.34.1

For normal helpers, read the function pointer from the
structure earlier.  For plugins, this will allow the
function pointer to come from elsewhere.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg.h             | 21 +++++++++-------
 include/exec/helper-gen.h.inc | 24 ++++++++++++-------
 tcg/tcg.c                     | 45 +++++++++++++++++++----------------
 3 files changed, 52 insertions(+), 38 deletions(-)

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ typedef struct TCGTargetOpDef {
 
 bool tcg_op_supported(TCGOpcode op);
 
-void tcg_gen_call0(TCGHelperInfo *, TCGTemp *ret);
-void tcg_gen_call1(TCGHelperInfo *, TCGTemp *ret, TCGTemp *);
-void tcg_gen_call2(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *);
-void tcg_gen_call3(TCGHelperInfo *, TCGTemp *ret, TCGTemp *,
+void tcg_gen_call0(void *func, TCGHelperInfo *, TCGTemp *ret);
+void tcg_gen_call1(void *func, TCGHelperInfo *, TCGTemp *ret, TCGTemp *);
+void tcg_gen_call2(void *func, TCGHelperInfo *, TCGTemp *ret,
                    TCGTemp *, TCGTemp *);
-void tcg_gen_call4(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *,
-                   TCGTemp *, TCGTemp *);
-void tcg_gen_call5(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *,
+void tcg_gen_call3(void *func, TCGHelperInfo *, TCGTemp *ret,
                    TCGTemp *, TCGTemp *, TCGTemp *);
-void tcg_gen_call6(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *,
+void tcg_gen_call4(void *func, TCGHelperInfo *, TCGTemp *ret,
                    TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *);
-void tcg_gen_call7(TCGHelperInfo *, TCGTemp *ret, TCGTemp *, TCGTemp *,
+void tcg_gen_call5(void *func, TCGHelperInfo *, TCGTemp *ret,
                    TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *);
+void tcg_gen_call6(void *func, TCGHelperInfo *, TCGTemp *ret,
+                   TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *,
+                   TCGTemp *, TCGTemp *);
+void tcg_gen_call7(void *func, TCGHelperInfo *, TCGTemp *ret,
+                   TCGTemp *, TCGTemp *, TCGTemp *, TCGTemp *,
+                   TCGTemp *, TCGTemp *, TCGTemp *);
 
 TCGOp *tcg_emit_op(TCGOpcode opc, unsigned nargs);
 void tcg_op_remove(TCGContext *s, TCGOp *op);
diff --git a/include/exec/helper-gen.h.inc b/include/exec/helper-gen.h.inc
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/helper-gen.h.inc
+++ b/include/exec/helper-gen.h.inc
@@ -XXX,XX +XXX,XX @@
 extern TCGHelperInfo glue(helper_info_, name);                          \
 static inline void glue(gen_helper_, name)(dh_retvar_decl0(ret))        \
 {                                                                       \
-    tcg_gen_call0(&glue(helper_info_, name), dh_retvar(ret));           \
+    tcg_gen_call0(glue(helper_info_,name).func,                         \
+                  &glue(helper_info_,name), dh_retvar(ret));            \
 }
 
 #define DEF_HELPER_FLAGS_1(name, flags, ret, t1)                        \
@@ -XXX,XX +XXX,XX @@ extern TCGHelperInfo glue(helper_info_, name);                          \
 static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
     dh_arg_decl(t1, 1))                                                 \
 {                                                                       \
-    tcg_gen_call1(&glue(helper_info_, name), dh_retvar(ret),            \
+    tcg_gen_call1(glue(helper_info_,name).func,                         \
+                  &glue(helper_info_,name), dh_retvar(ret),             \
                   dh_arg(t1, 1));                                       \
 }
 
@@ -XXX,XX +XXX,XX @@ extern TCGHelperInfo glue(helper_info_, name);                          \
 static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
     dh_arg_decl(t1, 1), dh_arg_decl(t2, 2))                             \
 {                                                                       \
-    tcg_gen_call2(&glue(helper_info_, name), dh_retvar(ret),            \
+    tcg_gen_call2(glue(helper_info_,name).func,                         \
+                  &glue(helper_info_,name), dh_retvar(ret),             \
                   dh_arg(t1, 1), dh_arg(t2, 2));                        \
 }
 
@@ -XXX,XX +XXX,XX @@ extern TCGHelperInfo glue(helper_info_, name);                          \
 static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
     dh_arg_decl(t1, 1), dh_arg_decl(t2, 2), dh_arg_decl(t3, 3))         \
 {                                                                       \
-    tcg_gen_call3(&glue(helper_info_, name), dh_retvar(ret),            \
+    tcg_gen_call3(glue(helper_info_,name).func,                         \
+                  &glue(helper_info_,name), dh_retvar(ret),             \
                   dh_arg(t1, 1), dh_arg(t2, 2), dh_arg(t3, 3));         \
 }
 
@@ -XXX,XX +XXX,XX @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
     dh_arg_decl(t1, 1), dh_arg_decl(t2, 2),                             \
     dh_arg_decl(t3, 3), dh_arg_decl(t4, 4))                             \
 {                                                                       \
-    tcg_gen_call4(&glue(helper_info_, name), dh_retvar(ret),            \
+    tcg_gen_call4(glue(helper_info_,name).func,                         \
+                  &glue(helper_info_,name), dh_retvar(ret),             \
                   dh_arg(t1, 1), dh_arg(t2, 2),                         \
                   dh_arg(t3, 3), dh_arg(t4, 4));                        \
 }
@@ -XXX,XX +XXX,XX @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
     dh_arg_decl(t1, 1), dh_arg_decl(t2, 2), dh_arg_decl(t3, 3),         \
     dh_arg_decl(t4, 4), dh_arg_decl(t5, 5))                             \
 {                                                                       \
-    tcg_gen_call5(&glue(helper_info_, name), dh_retvar(ret),            \
+    tcg_gen_call5(glue(helper_info_,name).func,                         \
+                  &glue(helper_info_,name), dh_retvar(ret),             \
                   dh_arg(t1, 1), dh_arg(t2, 2), dh_arg(t3, 3),          \
                   dh_arg(t4, 4), dh_arg(t5, 5));                        \
 }
@@ -XXX,XX +XXX,XX @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
     dh_arg_decl(t1, 1), dh_arg_decl(t2, 2), dh_arg_decl(t3, 3),         \
     dh_arg_decl(t4, 4), dh_arg_decl(t5, 5), dh_arg_decl(t6, 6))         \
 {                                                                       \
-    tcg_gen_call6(&glue(helper_info_, name), dh_retvar(ret),            \
+    tcg_gen_call6(glue(helper_info_,name).func,                         \
+                  &glue(helper_info_,name), dh_retvar(ret),             \
                   dh_arg(t1, 1), dh_arg(t2, 2), dh_arg(t3, 3),          \
                   dh_arg(t4, 4), dh_arg(t5, 5), dh_arg(t6, 6));         \
 }
@@ -XXX,XX +XXX,XX @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret)          \
     dh_arg_decl(t4, 4), dh_arg_decl(t5, 5), dh_arg_decl(t6, 6),         \
     dh_arg_decl(t7, 7))                                                 \
 {                                                                       \
-    tcg_gen_call7(&glue(helper_info_, name), dh_retvar(ret),            \
+    tcg_gen_call7(glue(helper_info_,name).func,                         \
+                  &glue(helper_info_,name), dh_retvar(ret),             \
                   dh_arg(t1, 1), dh_arg(t2, 2), dh_arg(t3, 3),          \
                   dh_arg(t4, 4), dh_arg(t5, 5), dh_arg(t6, 6),          \
                   dh_arg(t7, 7));                                       \
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ bool tcg_op_supported(TCGOpcode op)
 
 static TCGOp *tcg_op_alloc(TCGOpcode opc, unsigned nargs);
 
-static void tcg_gen_callN(TCGHelperInfo *info, TCGTemp *ret, TCGTemp **args)
+static void tcg_gen_callN(void *func, TCGHelperInfo *info,
+                          TCGTemp *ret, TCGTemp **args)
 {
     TCGv_i64 extend_free[MAX_CALL_IARGS];
     int n_extend = 0;
@@ -XXX,XX +XXX,XX @@ static void tcg_gen_callN(TCGHelperInfo *info, TCGTemp *ret, TCGTemp **args)
             g_assert_not_reached();
         }
     }
-    op->args[pi++] = (uintptr_t)info->func;
+    op->args[pi++] = (uintptr_t)func;
     op->args[pi++] = (uintptr_t)info;
     tcg_debug_assert(pi == total_args);
 
@@ -XXX,XX +XXX,XX @@ static void tcg_gen_callN(TCGHelperInfo *info, TCGTemp *ret, TCGTemp **args)
     }
 }
 
-void tcg_gen_call0(TCGHelperInfo *info, TCGTemp *ret)
+void tcg_gen_call0(void *func, TCGHelperInfo *info, TCGTemp *ret)
 {
-    tcg_gen_callN(info, ret, NULL);
+    tcg_gen_callN(func, info, ret, NULL);
 }
 
-void tcg_gen_call1(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1)
+void tcg_gen_call1(void *func, TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1)
 {
-    tcg_gen_callN(info, ret, &t1);
+    tcg_gen_callN(func, info, ret, &t1);
 }
 
-void tcg_gen_call2(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1, TCGTemp *t2)
+void tcg_gen_call2(void *func, TCGHelperInfo *info, TCGTemp *ret,
+                   TCGTemp *t1, TCGTemp *t2)
 {
     TCGTemp *args[2] = { t1, t2 };
-    tcg_gen_callN(info, ret, args);
+    tcg_gen_callN(func, info, ret, args);
 }
 
-void tcg_gen_call3(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
-                   TCGTemp *t2, TCGTemp *t3)
+void tcg_gen_call3(void *func, TCGHelperInfo *info, TCGTemp *ret,
+                   TCGTemp *t1, TCGTemp *t2, TCGTemp *t3)
 {
     TCGTemp *args[3] = { t1, t2, t3 };
-    tcg_gen_callN(info, ret, args);
+    tcg_gen_callN(func, info, ret, args);
 }
 
-void tcg_gen_call4(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
-                   TCGTemp *t2, TCGTemp *t3, TCGTemp *t4)
+void tcg_gen_call4(void *func, TCGHelperInfo *info, TCGTemp *ret,
+                   TCGTemp *t1, TCGTemp *t2, TCGTemp *t3, TCGTemp *t4)
 {
     TCGTemp *args[4] = { t1, t2, t3, t4 };
-    tcg_gen_callN(info, ret, args);
+    tcg_gen_callN(func, info, ret, args);
 }
 
-void tcg_gen_call5(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
+void tcg_gen_call5(void *func, TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
                    TCGTemp *t2, TCGTemp *t3, TCGTemp *t4, TCGTemp *t5)
 {
     TCGTemp *args[5] = { t1, t2, t3, t4, t5 };
-    tcg_gen_callN(info, ret, args);
+    tcg_gen_callN(func, info, ret, args);
 }
 
-void tcg_gen_call6(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1, TCGTemp *t2,
-                   TCGTemp *t3, TCGTemp *t4, TCGTemp *t5, TCGTemp *t6)
+void tcg_gen_call6(void *func, TCGHelperInfo *info, TCGTemp *ret,
+                   TCGTemp *t1, TCGTemp *t2, TCGTemp *t3,
+                   TCGTemp *t4, TCGTemp *t5, TCGTemp *t6)
 {
     TCGTemp *args[6] = { t1, t2, t3, t4, t5, t6 };
-    tcg_gen_callN(info, ret, args);
+    tcg_gen_callN(func, info, ret, args);
 }
 
-void tcg_gen_call7(TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
+void tcg_gen_call7(void *func, TCGHelperInfo *info, TCGTemp *ret, TCGTemp *t1,
                    TCGTemp *t2, TCGTemp *t3, TCGTemp *t4,
                    TCGTemp *t5, TCGTemp *t6, TCGTemp *t7)
 {
     TCGTemp *args[7] = { t1, t2, t3, t4, t5, t6, t7 };
-    tcg_gen_callN(info, ret, args);
+    tcg_gen_callN(func, info, ret, args);
 }
 
 static void tcg_reg_alloc_start(TCGContext *s)
-- 
2.34.1

The out-of-line function pointer is mutually exclusive
with inline expansion, so move it into the union.
Wrap the pointer in a structure named 'regular' to match
PLUGIN_CB_REGULAR.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/plugin.h  | 4 +++-
 accel/tcg/plugin-gen.c | 4 ++--
 plugins/core.c         | 8 ++++----
 3 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -XXX,XX +XXX,XX @@ enum plugin_dyn_cb_subtype {
  * instance of a callback to be called upon the execution of a particular TB.
  */
 struct qemu_plugin_dyn_cb {
-    union qemu_plugin_cb_sig f;
     void *userp;
     enum plugin_dyn_cb_subtype type;
     /* @rw applies to mem callbacks only (both regular and inline) */
     enum qemu_plugin_mem_rw rw;
     /* fields specific to each dyn_cb type go here */
     union {
+        struct {
+            union qemu_plugin_cb_sig f;
+        } regular;
         struct {
             qemu_plugin_u64 entry;
             enum qemu_plugin_op op;
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ static TCGOp *append_udata_cb(const struct qemu_plugin_dyn_cb *cb,
     }
 
     /* call */
-    op = copy_call(&begin_op, op, cb->f.vcpu_udata, cb_idx);
+    op = copy_call(&begin_op, op, cb->regular.f.vcpu_udata, cb_idx);
 
     return op;
 }
@@ -XXX,XX +XXX,XX @@ static TCGOp *append_mem_cb(const struct qemu_plugin_dyn_cb *cb,
 
     if (type == PLUGIN_GEN_CB_MEM) {
         /* call */
-        op = copy_call(&begin_op, op, cb->f.vcpu_udata, cb_idx);
+        op = copy_call(&begin_op, op, cb->regular.f.vcpu_udata, cb_idx);
     }
 
     return op;
diff --git a/plugins/core.c b/plugins/core.c
index XXXXXXX..XXXXXXX 100644
--- a/plugins/core.c
+++ b/plugins/core.c
@@ -XXX,XX +XXX,XX @@ void plugin_register_dyn_cb__udata(GArray **arr,
 
     dyn_cb->userp = udata;
     /* Note flags are discarded as unused. */
-    dyn_cb->f.vcpu_udata = cb;
+    dyn_cb->regular.f.vcpu_udata = cb;
     dyn_cb->type = PLUGIN_CB_REGULAR;
 }
 
@@ -XXX,XX +XXX,XX @@ void plugin_register_vcpu_mem_cb(GArray **arr,
     /* Note flags are discarded as unused. */
     dyn_cb->type = PLUGIN_CB_REGULAR;
     dyn_cb->rw = rw;
-    dyn_cb->f.generic = cb;
+    dyn_cb->regular.f.vcpu_mem = cb;
 }
 
 /*
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_vcpu_mem_cb(CPUState *cpu, uint64_t vaddr,
         }
         switch (cb->type) {
         case PLUGIN_CB_REGULAR:
-            cb->f.vcpu_mem(cpu->cpu_index, make_plugin_meminfo(oi, rw),
-                           vaddr, cb->userp);
+            cb->regular.f.vcpu_mem(cpu->cpu_index, make_plugin_meminfo(oi, rw),
+                                   vaddr, cb->userp);
             break;
         case PLUGIN_CB_INLINE:
             exec_inline_op(cb, cpu->cpu_index);
-- 
2.34.1

TCGHelperInfo includes the ABI for every function call.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/plugin.h |  1 +
 plugins/core.c        | 51 ++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 46 insertions(+), 6 deletions(-)

diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_dyn_cb {
     union {
         struct {
             union qemu_plugin_cb_sig f;
+            TCGHelperInfo *info;
         } regular;
         struct {
             qemu_plugin_u64 entry;
diff --git a/plugins/core.c b/plugins/core.c
index XXXXXXX..XXXXXXX 100644
--- a/plugins/core.c
+++ b/plugins/core.c
@@ -XXX,XX +XXX,XX @@ void plugin_register_dyn_cb__udata(GArray **arr,
                                    enum qemu_plugin_cb_flags flags,
                                    void *udata)
 {
-    struct qemu_plugin_dyn_cb *dyn_cb = plugin_get_dyn_cb(arr);
+    static TCGHelperInfo info[3] = {
+        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG | TCG_CALL_PLUGIN,
+        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG | TCG_CALL_PLUGIN,
+        [QEMU_PLUGIN_CB_RW_REGS].flags = TCG_CALL_PLUGIN,
+        /*
+         * Match qemu_plugin_vcpu_udata_cb_t:
+         *   void (*)(uint32_t, void *)
+         */
+        [0 ... 2].typemask = (dh_typemask(void, 0) |
+                              dh_typemask(i32, 1) |
+                              dh_typemask(ptr, 2))
+    };
 
+    struct qemu_plugin_dyn_cb *dyn_cb = plugin_get_dyn_cb(arr);
     dyn_cb->userp = udata;
-    /* Note flags are discarded as unused. */
-    dyn_cb->regular.f.vcpu_udata = cb;
     dyn_cb->type = PLUGIN_CB_REGULAR;
+    dyn_cb->regular.f.vcpu_udata = cb;
+
+    assert((unsigned)flags < ARRAY_SIZE(info));
+    dyn_cb->regular.info = &info[flags];
 }
 
 void plugin_register_vcpu_mem_cb(GArray **arr,
@@ -XXX,XX +XXX,XX @@ void plugin_register_vcpu_mem_cb(GArray **arr,
                                  enum qemu_plugin_mem_rw rw,
                                  void *udata)
 {
-    struct qemu_plugin_dyn_cb *dyn_cb;
+    /*
+     * Expect that the underlying type for enum qemu_plugin_meminfo_t
+     * is either int32_t or uint32_t, aka int or unsigned int.
+     */
+    QEMU_BUILD_BUG_ON(
+        !__builtin_types_compatible_p(qemu_plugin_meminfo_t, uint32_t) &&
+        !__builtin_types_compatible_p(qemu_plugin_meminfo_t, int32_t));
 
-    dyn_cb = plugin_get_dyn_cb(arr);
+    static TCGHelperInfo info[3] = {
+        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG | TCG_CALL_PLUGIN,
+        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG | TCG_CALL_PLUGIN,
+        [QEMU_PLUGIN_CB_RW_REGS].flags = TCG_CALL_PLUGIN,
+        /*
+         * Match qemu_plugin_vcpu_mem_cb_t:
+         *   void (*)(uint32_t, qemu_plugin_meminfo_t, uint64_t, void *)
+         */
+        [0 ... 2].typemask =
+            (dh_typemask(void, 0) |
+             dh_typemask(i32, 1) |
+             (__builtin_types_compatible_p(qemu_plugin_meminfo_t, uint32_t)
+              ? dh_typemask(i32, 2) : dh_typemask(s32, 2)) |
+             dh_typemask(i64, 3) |
+             dh_typemask(ptr, 4))
+    };
+
+    struct qemu_plugin_dyn_cb *dyn_cb = plugin_get_dyn_cb(arr);
     dyn_cb->userp = udata;
-    /* Note flags are discarded as unused. */
     dyn_cb->type = PLUGIN_CB_REGULAR;
     dyn_cb->rw = rw;
     dyn_cb->regular.f.vcpu_mem = cb;
+
+    assert((unsigned)flags < ARRAY_SIZE(info));
+    dyn_cb->regular.info = &info[flags];
 }
 
 /*
-- 
2.34.1

Introduce a new plugin_cb op and migrate one operation.
By using emit_before_op, we do not need to emit opcodes
early and modify them later -- we can simply emit the
final set of opcodes once.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg-op-common.h |  1 +
 include/tcg/tcg-opc.h       |  1 +
 accel/tcg/plugin-gen.c      | 74 +++++++++++++++++++++----------------
 tcg/tcg-op.c                |  5 +++
 4 files changed, 50 insertions(+), 31 deletions(-)

diff --git a/include/tcg/tcg-op-common.h b/include/tcg/tcg-op-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-op-common.h
+++ b/include/tcg/tcg-op-common.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_goto_tb(unsigned idx);
  */
 void tcg_gen_lookup_and_goto_ptr(void);
 
+void tcg_gen_plugin_cb(unsigned from);
 void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr);
 void tcg_gen_plugin_cb_end(void);
 
diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-opc.h
+++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(exit_tb, 0, 0, 1, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
 DEF(goto_tb, 0, 0, 1, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
 DEF(goto_ptr, 0, 1, 0, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
 
+DEF(plugin_cb, 0, 0, 1, TCG_OPF_NOT_PRESENT)
 DEF(plugin_cb_start, 0, 0, 3, TCG_OPF_NOT_PRESENT)
 DEF(plugin_cb_end, 0, 0, 0, TCG_OPF_NOT_PRESENT)
 
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_empty_callback(enum plugin_gen_from from)
 {
     switch (from) {
     case PLUGIN_GEN_AFTER_INSN:
-        gen_wrapped(from, PLUGIN_GEN_DISABLE_MEM_HELPER,
-                    gen_empty_mem_helper);
+        tcg_gen_plugin_cb(from);
         break;
     case PLUGIN_GEN_FROM_INSN:
         /*
@@ -XXX,XX +XXX,XX @@ static void inject_mem_enable_helper(struct qemu_plugin_tb *ptb,
     inject_mem_helper(begin_op, arr);
 }
 
-static void inject_mem_disable_helper(struct qemu_plugin_insn *plugin_insn,
-                                      TCGOp *begin_op)
-{
-    if (likely(!plugin_insn->mem_helper)) {
-        rm_ops(begin_op);
-        return;
-    }
-    inject_mem_helper(begin_op, NULL);
-}
-
 /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
 void plugin_gen_disable_mem_helpers(void)
 {
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
     inject_mem_enable_helper(ptb, insn, begin_op);
 }
 
-static void plugin_gen_disable_mem_helper(struct qemu_plugin_tb *ptb,
-                                          TCGOp *begin_op, int insn_idx)
+static void gen_disable_mem_helper(struct qemu_plugin_tb *ptb,
+                                   struct qemu_plugin_insn *insn)
 {
-    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
-    inject_mem_disable_helper(insn, begin_op);
+    if (insn->mem_helper) {
+        tcg_gen_st_ptr(tcg_constant_ptr(0), tcg_env,
+                       offsetof(CPUState, plugin_mem_cbs) -
+                       offsetof(ArchCPU, env));
+    }
 }
 
 /* #define DEBUG_PLUGIN_GEN_OPS */
@@ -XXX,XX +XXX,XX @@ static void pr_ops(void)
 
 static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
 {
-    TCGOp *op;
+    TCGOp *op, *next;
     int insn_idx = -1;
 
     pr_ops();
 
-    QTAILQ_FOREACH(op, &tcg_ctx->ops, link) {
+    /*
+     * While injecting code, we cannot afford to reuse any ebb temps
+     * that might be live within the existing opcode stream.
+     * The simplest solution is to release them all and create new.
+     */
+    memset(tcg_ctx->free_temps, 0, sizeof(tcg_ctx->free_temps));
+
+    QTAILQ_FOREACH_SAFE(op, &tcg_ctx->ops, link, next) {
         switch (op->opc) {
         case INDEX_op_insn_start:
             insn_idx++;
             break;
+
+        case INDEX_op_plugin_cb:
+        {
+            enum plugin_gen_from from = op->args[0];
+            struct qemu_plugin_insn *insn = NULL;
+
+            if (insn_idx >= 0) {
+                insn = g_ptr_array_index(plugin_tb->insns, insn_idx);
+            }
+
+            tcg_ctx->emit_before_op = op;
+
+            switch (from) {
+            case PLUGIN_GEN_AFTER_INSN:
+                assert(insn != NULL);
+                gen_disable_mem_helper(plugin_tb, insn);
+                break;
+            default:
+                g_assert_not_reached();
+            }
+
+            tcg_ctx->emit_before_op = NULL;
+            tcg_op_remove(tcg_ctx, op);
+            break;
+        }
+
         case INDEX_op_plugin_cb_start:
         {
             enum plugin_gen_from from = op->args[0];
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
 
                 break;
             }
-            case PLUGIN_GEN_AFTER_INSN:
-            {
-                g_assert(insn_idx >= 0);
-
-                switch (type) {
-                case PLUGIN_GEN_DISABLE_MEM_HELPER:
-                    plugin_gen_disable_mem_helper(plugin_tb, op, insn_idx);
-                    break;
-                default:
-                    g_assert_not_reached();
-                }
-                break;
-            }
             default:
                 g_assert_not_reached();
             }
diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op.c
+++ b/tcg/tcg-op.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_mb(TCGBar mb_type)
     }
 }
 
+void tcg_gen_plugin_cb(unsigned from)
+{
+    tcg_gen_op1(INDEX_op_plugin_cb, from);
+}
+
 void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr)
 {
     tcg_gen_op3(INDEX_op_plugin_cb_start, from, type, wr);
-- 
2.34.1

By having the qemu_plugin_cb_flags be recorded in the TCGHelperInfo,
we no longer need to distinguish PLUGIN_CB_REGULAR from
PLUGIN_CB_REGULAR_R, so place all TB callbacks in the same queue.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/plugin-gen.c | 96 +++++++++++++++++++++++++-----------------
 plugins/api.c          |  6 +--
 2 files changed, 58 insertions(+), 44 deletions(-)

diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_empty_callback(enum plugin_gen_from from)
 {
     switch (from) {
     case PLUGIN_GEN_AFTER_INSN:
+    case PLUGIN_GEN_FROM_TB:
         tcg_gen_plugin_cb(from);
         break;
     case PLUGIN_GEN_FROM_INSN:
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_empty_callback(enum plugin_gen_from from)
          */
         gen_wrapped(from, PLUGIN_GEN_ENABLE_MEM_HELPER,
                     gen_empty_mem_helper);
-        /* fall through */
-    case PLUGIN_GEN_FROM_TB:
         gen_wrapped(from, PLUGIN_GEN_CB_UDATA, gen_empty_udata_cb_no_rwg);
         gen_wrapped(from, PLUGIN_GEN_CB_UDATA_R, gen_empty_udata_cb_no_wg);
         gen_wrapped(from, PLUGIN_GEN_CB_INLINE, gen_empty_inline_cb);
@@ -XXX,XX +XXX,XX @@ void plugin_gen_disable_mem_helpers(void)
                    offsetof(CPUState, plugin_mem_cbs) - offsetof(ArchCPU, env));
 }
 
-static void plugin_gen_tb_udata(const struct qemu_plugin_tb *ptb,
-                                TCGOp *begin_op)
-{
-    inject_udata_cb(ptb->cbs[PLUGIN_CB_REGULAR], begin_op);
-}
-
-static void plugin_gen_tb_udata_r(const struct qemu_plugin_tb *ptb,
-                                  TCGOp *begin_op)
-{
-    inject_udata_cb(ptb->cbs[PLUGIN_CB_REGULAR_R], begin_op);
-}
-
-static void plugin_gen_tb_inline(const struct qemu_plugin_tb *ptb,
-                                 TCGOp *begin_op)
-{
-    inject_inline_cb(ptb->cbs[PLUGIN_CB_INLINE], begin_op, op_ok);
-}
-
 static void plugin_gen_insn_udata(const struct qemu_plugin_tb *ptb,
                                   TCGOp *begin_op, int insn_idx)
 {
@@ -XXX,XX +XXX,XX @@ static void gen_disable_mem_helper(struct qemu_plugin_tb *ptb,
     }
 }
 
+static void gen_udata_cb(struct qemu_plugin_dyn_cb *cb)
+{
+    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
+
+    tcg_gen_ld_i32(cpu_index, tcg_env,
+                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
+    tcg_gen_call2(cb->regular.f.vcpu_udata, cb->regular.info, NULL,
+                  tcgv_i32_temp(cpu_index),
+                  tcgv_ptr_temp(tcg_constant_ptr(cb->userp)));
+    tcg_temp_free_i32(cpu_index);
+}
+
+static void gen_inline_cb(struct qemu_plugin_dyn_cb *cb)
+{
+    GArray *arr = cb->inline_insn.entry.score->data;
+    size_t offset = cb->inline_insn.entry.offset;
+    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
+    TCGv_i64 val = tcg_temp_ebb_new_i64();
+    TCGv_ptr ptr = tcg_temp_ebb_new_ptr();
+
+    tcg_gen_ld_i32(cpu_index, tcg_env,
+                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
+    tcg_gen_muli_i32(cpu_index, cpu_index, g_array_get_element_size(arr));
+    tcg_gen_ext_i32_ptr(ptr, cpu_index);
+    tcg_temp_free_i32(cpu_index);
+
+    tcg_gen_addi_ptr(ptr, ptr, (intptr_t)arr->data);
+    tcg_gen_ld_i64(val, ptr, offset);
+    tcg_gen_addi_i64(val, val, cb->inline_insn.imm);
+    tcg_gen_st_i64(val, ptr, offset);
+
+    tcg_temp_free_i64(val);
+    tcg_temp_free_ptr(ptr);
+}
+
 /* #define DEBUG_PLUGIN_GEN_OPS */
 static void pr_ops(void)
 {
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
         {
             enum plugin_gen_from from = op->args[0];
             struct qemu_plugin_insn *insn = NULL;
+            const GArray *cbs;
+            int i, n;
 
             if (insn_idx >= 0) {
                 insn = g_ptr_array_index(plugin_tb->insns, insn_idx);
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
                 assert(insn != NULL);
                 gen_disable_mem_helper(plugin_tb, insn);
                 break;
+
+            case PLUGIN_GEN_FROM_TB:
+                assert(insn == NULL);
+
+                cbs = plugin_tb->cbs[PLUGIN_CB_REGULAR];
+                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
+                    struct qemu_plugin_dyn_cb *cb =
+                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
+                    gen_udata_cb(cb);
+                }
+
+                cbs = plugin_tb->cbs[PLUGIN_CB_INLINE];
+                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
+                    struct qemu_plugin_dyn_cb *cb =
+                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
+                    gen_inline_cb(cb);
+                }
+                break;
+
             default:
                 g_assert_not_reached();
             }
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
             enum plugin_gen_cb type = op->args[1];
 
             switch (from) {
-            case PLUGIN_GEN_FROM_TB:
-            {
-                g_assert(insn_idx == -1);
-
-                switch (type) {
-                case PLUGIN_GEN_CB_UDATA:
-                    plugin_gen_tb_udata(plugin_tb, op);
-                    break;
-                case PLUGIN_GEN_CB_UDATA_R:
-                    plugin_gen_tb_udata_r(plugin_tb, op);
-                    break;
-                case PLUGIN_GEN_CB_INLINE:
-                    plugin_gen_tb_inline(plugin_tb, op);
-                    break;
-                default:
-                    g_assert_not_reached();
-                }
-                break;
-            }
             case PLUGIN_GEN_FROM_INSN:
             {
                 g_assert(insn_idx >= 0);
diff --git a/plugins/api.c b/plugins/api.c
index XXXXXXX..XXXXXXX 100644
--- a/plugins/api.c
+++ b/plugins/api.c
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_tb_exec_cb(struct qemu_plugin_tb *tb,
                                           void *udata)
 {
     if (!tb->mem_only) {
-        int index = flags == QEMU_PLUGIN_CB_R_REGS ||
-                    flags == QEMU_PLUGIN_CB_RW_REGS ?
-                    PLUGIN_CB_REGULAR_R : PLUGIN_CB_REGULAR;
-
-        plugin_register_dyn_cb__udata(&tb->cbs[index],
+        plugin_register_dyn_cb__udata(&tb->cbs[PLUGIN_CB_REGULAR],
                                       cb, flags, udata);
     }
 }
-- 
2.34.1

Delay test of plugin_tb->mem_helper until the inject pass.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/plugin-gen.c | 37 ++++++++++++++++---------------------
 1 file changed, 16 insertions(+), 21 deletions(-)

diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ enum plugin_gen_from {
     PLUGIN_GEN_FROM_INSN,
     PLUGIN_GEN_FROM_MEM,
     PLUGIN_GEN_AFTER_INSN,
+    PLUGIN_GEN_AFTER_TB,
     PLUGIN_GEN_N_FROMS,
 };
 
@@ -XXX,XX +XXX,XX @@ static void inject_mem_enable_helper(struct qemu_plugin_tb *ptb,
 /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
 void plugin_gen_disable_mem_helpers(void)
 {
-    /*
-     * We could emit the clearing unconditionally and be done. However, this can
-     * be wasteful if for instance plugins don't track memory accesses, or if
-     * most TBs don't use helpers. Instead, emit the clearing iff the TB calls
-     * helpers that might access guest memory.
-     *
-     * Note: we do not reset plugin_tb->mem_helper here; a TB might have several
-     * exit points, and we want to emit the clearing from all of them.
-     */
-    if (!tcg_ctx->plugin_tb->mem_helper) {
-        return;
+    if (tcg_ctx->plugin_insn) {
+        tcg_gen_plugin_cb(PLUGIN_GEN_AFTER_TB);
     }
-    tcg_gen_st_ptr(tcg_constant_ptr(NULL), tcg_env,
-                   offsetof(CPUState, plugin_mem_cbs) - offsetof(ArchCPU, env));
 }
 
 static void plugin_gen_insn_udata(const struct qemu_plugin_tb *ptb,
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
     inject_mem_enable_helper(ptb, insn, begin_op);
 }
 
-static void gen_disable_mem_helper(struct qemu_plugin_tb *ptb,
-                                   struct qemu_plugin_insn *insn)
+static void gen_disable_mem_helper(void)
 {
-    if (insn->mem_helper) {
-        tcg_gen_st_ptr(tcg_constant_ptr(0), tcg_env,
-                       offsetof(CPUState, plugin_mem_cbs) -
-                       offsetof(ArchCPU, env));
-    }
+    tcg_gen_st_ptr(tcg_constant_ptr(0), tcg_env,
+                   offsetof(CPUState, plugin_mem_cbs) -
+                   offsetof(ArchCPU, env));
 }
 
 static void gen_udata_cb(struct qemu_plugin_dyn_cb *cb)
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
             tcg_ctx->emit_before_op = op;
 
             switch (from) {
+            case PLUGIN_GEN_AFTER_TB:
+                if (plugin_tb->mem_helper) {
+                    gen_disable_mem_helper();
+                }
+                break;
+
             case PLUGIN_GEN_AFTER_INSN:
                 assert(insn != NULL);
-                gen_disable_mem_helper(plugin_tb, insn);
+                if (insn->mem_helper) {
+                    gen_disable_mem_helper();
+                }
                 break;
 
             case PLUGIN_GEN_FROM_TB:
-- 
2.34.1

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/plugin.h  |   1 -
 accel/tcg/plugin-gen.c | 286 ++++++++++-------------------------------
 plugins/api.c          |   8 +-
 3 files changed, 67 insertions(+), 228 deletions(-)

diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -XXX,XX +XXX,XX @@ enum plugin_dyn_cb_type {
 
 enum plugin_dyn_cb_subtype {
     PLUGIN_CB_REGULAR,
-    PLUGIN_CB_REGULAR_R,
     PLUGIN_CB_INLINE,
     PLUGIN_N_CB_SUBTYPES,
 };
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ void HELPER(plugin_vcpu_mem_cb)(unsigned int vcpu_index,
                                 void *userdata)
 { }
 
-static void gen_empty_udata_cb(void (*gen_helper)(TCGv_i32, TCGv_ptr))
-{
-    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
-    TCGv_ptr udata = tcg_temp_ebb_new_ptr();
-
-    tcg_gen_movi_ptr(udata, 0);
-    tcg_gen_ld_i32(cpu_index, tcg_env,
-                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
-    gen_helper(cpu_index, udata);
-
-    tcg_temp_free_ptr(udata);
-    tcg_temp_free_i32(cpu_index);
-}
-
-static void gen_empty_udata_cb_no_wg(void)
-{
-    gen_empty_udata_cb(gen_helper_plugin_vcpu_udata_cb_no_wg);
-}
-
-static void gen_empty_udata_cb_no_rwg(void)
-{
-    gen_empty_udata_cb(gen_helper_plugin_vcpu_udata_cb_no_rwg);
-}
-
 /*
  * For now we only support addi_i64.
  * When we support more ops, we can generate one empty inline cb for each.
@@ -XXX,XX +XXX,XX @@ static void gen_empty_mem_cb(TCGv_i64 addr, uint32_t info)
     tcg_temp_free_i32(cpu_index);
 }
 
-/*
- * Share the same function for enable/disable. When enabling, the NULL
- * pointer will be overwritten later.
- */
-static void gen_empty_mem_helper(void)
-{
-    TCGv_ptr ptr = tcg_temp_ebb_new_ptr();
-
-    tcg_gen_movi_ptr(ptr, 0);
-    tcg_gen_st_ptr(ptr, tcg_env, offsetof(CPUState, plugin_mem_cbs) -
-                                 offsetof(ArchCPU, env));
-    tcg_temp_free_ptr(ptr);
-}
-
 static void gen_plugin_cb_start(enum plugin_gen_from from,
                                 enum plugin_gen_cb type, unsigned wr)
 {
     tcg_gen_plugin_cb_start(from, type, wr);
 }
 
-static void gen_wrapped(enum plugin_gen_from from,
-                        enum plugin_gen_cb type, void (*func)(void))
-{
-    gen_plugin_cb_start(from, type, 0);
-    func();
-    tcg_gen_plugin_cb_end();
-}
-
 static void plugin_gen_empty_callback(enum plugin_gen_from from)
 {
     switch (from) {
     case PLUGIN_GEN_AFTER_INSN:
     case PLUGIN_GEN_FROM_TB:
-        tcg_gen_plugin_cb(from);
-        break;
     case PLUGIN_GEN_FROM_INSN:
-        /*
-         * Note: plugin_gen_inject() relies on ENABLE_MEM_HELPER being
-         * the first callback of an instruction
-         */
-        gen_wrapped(from, PLUGIN_GEN_ENABLE_MEM_HELPER,
-                    gen_empty_mem_helper);
-        gen_wrapped(from, PLUGIN_GEN_CB_UDATA, gen_empty_udata_cb_no_rwg);
-        gen_wrapped(from, PLUGIN_GEN_CB_UDATA_R, gen_empty_udata_cb_no_wg);
-        gen_wrapped(from, PLUGIN_GEN_CB_INLINE, gen_empty_inline_cb);
+        tcg_gen_plugin_cb(from);
         break;
     default:
         g_assert_not_reached();
@@ -XXX,XX +XXX,XX @@ static TCGOp *copy_mul_i32(TCGOp **begin_op, TCGOp *op, uint32_t v)
     return op;
 }
 
-static TCGOp *copy_st_ptr(TCGOp **begin_op, TCGOp *op)
-{
-    if (UINTPTR_MAX == UINT32_MAX) {
-        /* st_i32 */
-        op = copy_op(begin_op, op, INDEX_op_st_i32);
-    } else {
-        /* st_i64 */
-        op = copy_st_i64(begin_op, op);
-    }
-    return op;
-}
-
 static TCGOp *copy_call(TCGOp **begin_op, TCGOp *op, void *func, int *cb_idx)
 {
     TCGOp *old_op;
@@ -XXX,XX +XXX,XX @@ static TCGOp *copy_call(TCGOp **begin_op, TCGOp *op, void *func, int *cb_idx)
     return op;
 }
 
-/*
- * When we append/replace ops here we are sensitive to changing patterns of
- * TCGOps generated by the tcg_gen_FOO calls when we generated the
- * empty callbacks. This will assert very quickly in a debug build as
- * we assert the ops we are replacing are the correct ones.
- */
-static TCGOp *append_udata_cb(const struct qemu_plugin_dyn_cb *cb,
-                              TCGOp *begin_op, TCGOp *op, int *cb_idx)
-{
-    /* const_ptr */
-    op = copy_const_ptr(&begin_op, op, cb->userp);
-
-    /* copy the ld_i32, but note that we only have to copy it once */
-    if (*cb_idx == -1) {
-        op = copy_op(&begin_op, op, INDEX_op_ld_i32);
-    } else {
-        begin_op = QTAILQ_NEXT(begin_op, link);
-        tcg_debug_assert(begin_op && begin_op->opc == INDEX_op_ld_i32);
-    }
-
-    /* call */
-    op = copy_call(&begin_op, op, cb->regular.f.vcpu_udata, cb_idx);
-
-    return op;
-}
-
 static TCGOp *append_inline_cb(const struct qemu_plugin_dyn_cb *cb,
                                TCGOp *begin_op, TCGOp *op,
                                int *unused)
@@ -XXX,XX +XXX,XX @@ typedef TCGOp *(*inject_fn)(const struct qemu_plugin_dyn_cb *cb,
                             TCGOp *begin_op, TCGOp *op, int *intp);
 typedef bool (*op_ok_fn)(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb);
 
-static bool op_ok(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb)
-{
-    return true;
-}
-
 static bool op_rw(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb)
 {
     int w;
@@ -XXX,XX +XXX,XX @@ static void inject_cb_type(const GArray *cbs, TCGOp *begin_op,
     rm_ops_range(begin_op, end_op);
 }
 
-static void
-inject_udata_cb(const GArray *cbs, TCGOp *begin_op)
-{
-    inject_cb_type(cbs, begin_op, append_udata_cb, op_ok);
-}
-
 static void
 inject_inline_cb(const GArray *cbs, TCGOp *begin_op, op_ok_fn ok)
 {
@@ -XXX,XX +XXX,XX @@ inject_mem_cb(const GArray *cbs, TCGOp *begin_op)
     inject_cb_type(cbs, begin_op, append_mem_cb, op_rw);
 }
 
-/* we could change the ops in place, but we can reuse more code by copying */
-static void inject_mem_helper(TCGOp *begin_op, GArray *arr)
-{
-    TCGOp *orig_op = begin_op;
-    TCGOp *end_op;
-    TCGOp *op;
-
-    end_op = find_op(begin_op, INDEX_op_plugin_cb_end);
-    tcg_debug_assert(end_op);
-
-    /* const ptr */
-    op = copy_const_ptr(&begin_op, end_op, arr);
-
-    /* st_ptr */
-    op = copy_st_ptr(&begin_op, op);
-
-    rm_ops_range(orig_op, end_op);
-}
-
-/*
- * Tracking memory accesses performed from helpers requires extra work.
- * If an instruction is emulated with helpers, we do two things:
- * (1) copy the CB descriptors, and keep track of it so that they can be
- * freed later on, and (2) point CPUState.plugin_mem_cbs to the descriptors, so
- * that we can read them at run-time (i.e. when the helper executes).
- * This run-time access is performed from qemu_plugin_vcpu_mem_cb.
- *
- * Note that plugin_gen_disable_mem_helpers undoes (2). Since it
- * is possible that the code we generate after the instruction is
- * dead, we also add checks before generating tb_exit etc.
- */
-static void inject_mem_enable_helper(struct qemu_plugin_tb *ptb,
-                                     struct qemu_plugin_insn *plugin_insn,
-                                     TCGOp *begin_op)
-{
-    GArray *cbs[2];
-    GArray *arr;
-    size_t n_cbs, i;
-
-    cbs[0] = plugin_insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
-    cbs[1] = plugin_insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
-
-    n_cbs = 0;
-    for (i = 0; i < ARRAY_SIZE(cbs); i++) {
-        n_cbs += cbs[i]->len;
-    }
-
-    plugin_insn->mem_helper = plugin_insn->calls_helpers && n_cbs;
-    if (likely(!plugin_insn->mem_helper)) {
-        rm_ops(begin_op);
-        return;
-    }
-    ptb->mem_helper = true;
-
-    arr = g_array_sized_new(false, false,
-                            sizeof(struct qemu_plugin_dyn_cb), n_cbs);
-
-    for (i = 0; i < ARRAY_SIZE(cbs); i++) {
-        g_array_append_vals(arr, cbs[i]->data, cbs[i]->len);
-    }
-
-    qemu_plugin_add_dyn_cb_arr(arr);
-    inject_mem_helper(begin_op, arr);
-}
-
 /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
 void plugin_gen_disable_mem_helpers(void)
 {
@@ -XXX,XX +XXX,XX @@ void plugin_gen_disable_mem_helpers(void)
     }
 }
 
-static void plugin_gen_insn_udata(const struct qemu_plugin_tb *ptb,
-                                  TCGOp *begin_op, int insn_idx)
-{
-    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
-
-    inject_udata_cb(insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR], begin_op);
-}
-
-static void plugin_gen_insn_udata_r(const struct qemu_plugin_tb *ptb,
-                                    TCGOp *begin_op, int insn_idx)
-{
-    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
-
-    inject_udata_cb(insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR_R], begin_op);
-}
-
-static void plugin_gen_insn_inline(const struct qemu_plugin_tb *ptb,
-                                   TCGOp *begin_op, int insn_idx)
-{
-    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
-    inject_inline_cb(insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_INLINE],
-                     begin_op, op_ok);
-}
-
 static void plugin_gen_mem_regular(const struct qemu_plugin_tb *ptb,
                                    TCGOp *begin_op, int insn_idx)
 {
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_mem_inline(const struct qemu_plugin_tb *ptb,
     inject_inline_cb(cbs, begin_op, op_rw);
 }
 
-static void plugin_gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
-                                         TCGOp *begin_op, int insn_idx)
+static void gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
+                                  struct qemu_plugin_insn *insn)
 {
-    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
-    inject_mem_enable_helper(ptb, insn, begin_op);
+    GArray *cbs[2];
+    GArray *arr;
+    size_t n_cbs;
+
+    /*
+     * Tracking memory accesses performed from helpers requires extra work.
+     * If an instruction is emulated with helpers, we do two things:
+     * (1) copy the CB descriptors, and keep track of it so that they can be
+     * freed later on, and (2) point CPUState.plugin_mem_cbs to the
+     * descriptors, so that we can read them at run-time
+     * (i.e. when the helper executes).
+     * This run-time access is performed from qemu_plugin_vcpu_mem_cb.
+     *
+     * Note that plugin_gen_disable_mem_helpers undoes (2). Since it
+     * is possible that the code we generate after the instruction is
+     * dead, we also add checks before generating tb_exit etc.
+     */
+    if (!insn->calls_helpers) {
+        return;
+    }
+
+    cbs[0] = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
+    cbs[1] = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
+    n_cbs = cbs[0]->len + cbs[1]->len;
+
+    if (n_cbs == 0) {
+        insn->mem_helper = false;
+        return;
+    }
+    insn->mem_helper = true;
+    ptb->mem_helper = true;
+
+    arr = g_array_sized_new(false, false,
+                            sizeof(struct qemu_plugin_dyn_cb), n_cbs);
+    g_array_append_vals(arr, cbs[0]->data, cbs[0]->len);
+    g_array_append_vals(arr, cbs[1]->data, cbs[1]->len);
+
+    qemu_plugin_add_dyn_cb_arr(arr);
+
+    tcg_gen_st_ptr(tcg_constant_ptr((intptr_t)arr), tcg_env,
+                   offsetof(CPUState, plugin_mem_cbs) -
+                   offsetof(ArchCPU, env));
 }
 
 static void gen_disable_mem_helper(void)
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
                 }
                 break;
 
+            case PLUGIN_GEN_FROM_INSN:
+                assert(insn != NULL);
+
+                gen_enable_mem_helper(plugin_tb, insn);
+
+                cbs = insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR];
+                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
+                    struct qemu_plugin_dyn_cb *cb =
+                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
+                    gen_udata_cb(cb);
+                }
+
+                cbs = insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_INLINE];
+                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
+                    struct qemu_plugin_dyn_cb *cb =
+                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
+                    gen_inline_cb(cb);
+                }
+                break;
+
             default:
                 g_assert_not_reached();
             }
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
             enum plugin_gen_cb type = op->args[1];
 
             switch (from) {
-            case PLUGIN_GEN_FROM_INSN:
-            {
-                g_assert(insn_idx >= 0);
-
-                switch (type) {
-                case PLUGIN_GEN_CB_UDATA:
-                    plugin_gen_insn_udata(plugin_tb, op, insn_idx);
-                    break;
-                case PLUGIN_GEN_CB_UDATA_R:
-                    plugin_gen_insn_udata_r(plugin_tb, op, insn_idx);
-                    break;
-                case PLUGIN_GEN_CB_INLINE:
-                    plugin_gen_insn_inline(plugin_tb, op, insn_idx);
-                    break;
-                case PLUGIN_GEN_ENABLE_MEM_HELPER:
-                    plugin_gen_enable_mem_helper(plugin_tb, op, insn_idx);
-                    break;
-                default:
-                    g_assert_not_reached();
-                }
-                break;
-            }
             case PLUGIN_GEN_FROM_MEM:
             {
                 g_assert(insn_idx >= 0);
diff --git a/plugins/api.c b/plugins/api.c
index XXXXXXX..XXXXXXX 100644
--- a/plugins/api.c
+++ b/plugins/api.c
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_insn_exec_cb(struct qemu_plugin_insn *insn,
                                             void *udata)
 {
     if (!insn->mem_only) {
-        int index = flags == QEMU_PLUGIN_CB_R_REGS ||
-                    flags == QEMU_PLUGIN_CB_RW_REGS ?
-                    PLUGIN_CB_REGULAR_R : PLUGIN_CB_REGULAR;
-
-        plugin_register_dyn_cb__udata(&insn->cbs[PLUGIN_CB_INSN][index],
-                                      cb, flags, udata);
+        plugin_register_dyn_cb__udata(
+            &insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR], cb, flags, udata);
     }
 }
 
-- 
2.34.1

Introduce a new plugin_mem_cb op to hold the address temp
and meminfo computed by tcg-op-ldst.c.  Because this now
has its own opcode, we no longer need PLUGIN_GEN_FROM_MEM.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/plugin-gen.h   |   4 -
 include/tcg/tcg-op-common.h |   1 +
 include/tcg/tcg-opc.h       |   1 +
 accel/tcg/plugin-gen.c      | 408 ++++--------------------------------
 tcg/tcg-op-ldst.c           |   6 +-
 tcg/tcg-op.c                |   5 +
 6 files changed, 54 insertions(+), 371 deletions(-)

diff --git a/include/exec/plugin-gen.h b/include/exec/plugin-gen.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/plugin-gen.h
+++ b/include/exec/plugin-gen.h
@@ -XXX,XX +XXX,XX @@ void plugin_gen_insn_start(CPUState *cpu, const struct DisasContextBase *db);
 void plugin_gen_insn_end(void);
 
 void plugin_gen_disable_mem_helpers(void);
-void plugin_gen_empty_mem_callback(TCGv_i64 addr, uint32_t info);
 
 #else /* !CONFIG_PLUGIN */
 
@@ -XXX,XX +XXX,XX @@ static inline void plugin_gen_tb_end(CPUState *cpu, size_t num_insns)
 static inline void plugin_gen_disable_mem_helpers(void)
 { }
 
-static inline void plugin_gen_empty_mem_callback(TCGv_i64 addr, uint32_t info)
-{ }
-
 #endif /* CONFIG_PLUGIN */
 
 #endif /* QEMU_PLUGIN_GEN_H */
diff --git a/include/tcg/tcg-op-common.h b/include/tcg/tcg-op-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-op-common.h
+++ b/include/tcg/tcg-op-common.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_goto_tb(unsigned idx);
 void tcg_gen_lookup_and_goto_ptr(void);
 
 void tcg_gen_plugin_cb(unsigned from);
+void tcg_gen_plugin_mem_cb(TCGv_i64 addr, unsigned meminfo);
 void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr);
 void tcg_gen_plugin_cb_end(void);
 
diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-opc.h
+++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(goto_tb, 0, 0, 1, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
 DEF(goto_ptr, 0, 1, 0, TCG_OPF_BB_EXIT | TCG_OPF_BB_END)
 
 DEF(plugin_cb, 0, 0, 1, TCG_OPF_NOT_PRESENT)
+DEF(plugin_mem_cb, 0, 1, 1, TCG_OPF_NOT_PRESENT)
 DEF(plugin_cb_start, 0, 0, 3, TCG_OPF_NOT_PRESENT)
 DEF(plugin_cb_end, 0, 0, 0, TCG_OPF_NOT_PRESENT)
 
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@
 enum plugin_gen_from {
     PLUGIN_GEN_FROM_TB,
     PLUGIN_GEN_FROM_INSN,
-    PLUGIN_GEN_FROM_MEM,
     PLUGIN_GEN_AFTER_INSN,
     PLUGIN_GEN_AFTER_TB,
     PLUGIN_GEN_N_FROMS,
@@ -XXX,XX +XXX,XX @@ void HELPER(plugin_vcpu_mem_cb)(unsigned int vcpu_index,
                                 void *userdata)
 { }
 
-/*
- * For now we only support addi_i64.
- * When we support more ops, we can generate one empty inline cb for each.
- */
-static void gen_empty_inline_cb(void)
-{
-    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
-    TCGv_ptr cpu_index_as_ptr = tcg_temp_ebb_new_ptr();
-    TCGv_i64 val = tcg_temp_ebb_new_i64();
-    TCGv_ptr ptr = tcg_temp_ebb_new_ptr();
-
-    tcg_gen_ld_i32(cpu_index, tcg_env,
-                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
-    /* second operand will be replaced by immediate value */
-    tcg_gen_mul_i32(cpu_index, cpu_index, cpu_index);
-    tcg_gen_ext_i32_ptr(cpu_index_as_ptr, cpu_index);
-
-    tcg_gen_movi_ptr(ptr, 0);
-    tcg_gen_add_ptr(ptr, ptr, cpu_index_as_ptr);
-    tcg_gen_ld_i64(val, ptr, 0);
-    /* second operand will be replaced by immediate value */
-    tcg_gen_add_i64(val, val, val);
-
-    tcg_gen_st_i64(val, ptr, 0);
-    tcg_temp_free_ptr(ptr);
-    tcg_temp_free_i64(val);
-    tcg_temp_free_ptr(cpu_index_as_ptr);
-    tcg_temp_free_i32(cpu_index);
-}
-
-static void gen_empty_mem_cb(TCGv_i64 addr, uint32_t info)
-{
-    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
-    TCGv_i32 meminfo = tcg_temp_ebb_new_i32();
-    TCGv_ptr udata = tcg_temp_ebb_new_ptr();
-
-    tcg_gen_movi_i32(meminfo, info);
-    tcg_gen_movi_ptr(udata, 0);
-    tcg_gen_ld_i32(cpu_index, tcg_env,
-                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
-
-    gen_helper_plugin_vcpu_mem_cb(cpu_index, meminfo, addr, udata);
-
-    tcg_temp_free_ptr(udata);
-    tcg_temp_free_i32(meminfo);
-    tcg_temp_free_i32(cpu_index);
-}
-
-static void gen_plugin_cb_start(enum plugin_gen_from from,
-                                enum plugin_gen_cb type, unsigned wr)
-{
-    tcg_gen_plugin_cb_start(from, type, wr);
-}
-
 static void plugin_gen_empty_callback(enum plugin_gen_from from)
 {
     switch (from) {
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_empty_callback(enum plugin_gen_from from)
     }
 }
 
-void plugin_gen_empty_mem_callback(TCGv_i64 addr, uint32_t info)
-{
-    enum qemu_plugin_mem_rw rw = get_plugin_meminfo_rw(info);
-
-    gen_plugin_cb_start(PLUGIN_GEN_FROM_MEM, PLUGIN_GEN_CB_MEM, rw);
-    gen_empty_mem_cb(addr, info);
-    tcg_gen_plugin_cb_end();
-
-    gen_plugin_cb_start(PLUGIN_GEN_FROM_MEM, PLUGIN_GEN_CB_INLINE, rw);
-    gen_empty_inline_cb();
-    tcg_gen_plugin_cb_end();
-}
-
-static TCGOp *find_op(TCGOp *op, TCGOpcode opc)
-{
-    while (op) {
-        if (op->opc == opc) {
-            return op;
-        }
-        op = QTAILQ_NEXT(op, link);
-    }
-    return NULL;
-}
-
-static TCGOp *rm_ops_range(TCGOp *begin, TCGOp *end)
-{
-    TCGOp *ret = QTAILQ_NEXT(end, link);
-
-    QTAILQ_REMOVE_SEVERAL(&tcg_ctx->ops, begin, end, link);
-    return ret;
-}
-
-/* remove all ops until (and including) plugin_cb_end */
-static TCGOp *rm_ops(TCGOp *op)
-{
-    TCGOp *end_op = find_op(op, INDEX_op_plugin_cb_end);
-
-    tcg_debug_assert(end_op);
-    return rm_ops_range(op, end_op);
-}
-
-static TCGOp *copy_op_nocheck(TCGOp **begin_op, TCGOp *op)
-{
-    TCGOp *old_op = QTAILQ_NEXT(*begin_op, link);
-    unsigned nargs = old_op->nargs;
-
-    *begin_op = old_op;
-    op = tcg_op_insert_after(tcg_ctx, op, old_op->opc, nargs);
-    memcpy(op->args, old_op->args, sizeof(op->args[0]) * nargs);
-
-    return op;
-}
-
-static TCGOp *copy_op(TCGOp **begin_op, TCGOp *op, TCGOpcode opc)
-{
-    op = copy_op_nocheck(begin_op, op);
-    tcg_debug_assert((*begin_op)->opc == opc);
-    return op;
-}
-
-static TCGOp *copy_const_ptr(TCGOp **begin_op, TCGOp *op, void *ptr)
-{
-    if (UINTPTR_MAX == UINT32_MAX) {
-        /* mov_i32 */
-        op = copy_op(begin_op, op, INDEX_op_mov_i32);
-        op->args[1] = tcgv_i32_arg(tcg_constant_i32((uintptr_t)ptr));
-    } else {
-        /* mov_i64 */
-        op = copy_op(begin_op, op, INDEX_op_mov_i64);
-        op->args[1] = tcgv_i64_arg(tcg_constant_i64((uintptr_t)ptr));
-    }
-    return op;
-}
-
-static TCGOp *copy_ld_i32(TCGOp **begin_op, TCGOp *op)
-{
-    return copy_op(begin_op, op, INDEX_op_ld_i32);
-}
-
-static TCGOp *copy_ext_i32_ptr(TCGOp **begin_op, TCGOp *op)
-{
-    if (UINTPTR_MAX == UINT32_MAX) {
-        op = copy_op(begin_op, op, INDEX_op_mov_i32);
-    } else {
-        op = copy_op(begin_op, op, INDEX_op_ext_i32_i64);
-    }
-    return op;
-}
-
-static TCGOp *copy_add_ptr(TCGOp **begin_op, TCGOp *op)
-{
-    if (UINTPTR_MAX == UINT32_MAX) {
-        op = copy_op(begin_op, op, INDEX_op_add_i32);
-    } else {
-        op = copy_op(begin_op, op, INDEX_op_add_i64);
-    }
-    return op;
-}
-
-static TCGOp *copy_ld_i64(TCGOp **begin_op, TCGOp *op)
-{
-    if (TCG_TARGET_REG_BITS == 32) {
-        /* 2x ld_i32 */
-        op = copy_ld_i32(begin_op, op);
-        op = copy_ld_i32(begin_op, op);
-    } else {
-        /* ld_i64 */
-        op = copy_op(begin_op, op, INDEX_op_ld_i64);
-    }
-    return op;
-}
-
-static TCGOp *copy_st_i64(TCGOp **begin_op, TCGOp *op)
-{
-    if (TCG_TARGET_REG_BITS == 32) {
-        /* 2x st_i32 */
-        op = copy_op(begin_op, op, INDEX_op_st_i32);
-        op = copy_op(begin_op, op, INDEX_op_st_i32);
-    } else {
-        /* st_i64 */
-        op = copy_op(begin_op, op, INDEX_op_st_i64);
-    }
-    return op;
-}
-
-static TCGOp *copy_add_i64(TCGOp **begin_op, TCGOp *op, uint64_t v)
-{
-    if (TCG_TARGET_REG_BITS == 32) {
-        /* all 32-bit backends must implement add2_i32 */
-        g_assert(TCG_TARGET_HAS_add2_i32);
-        op = copy_op(begin_op, op, INDEX_op_add2_i32);
-        op->args[4] = tcgv_i32_arg(tcg_constant_i32(v));
-        op->args[5] = tcgv_i32_arg(tcg_constant_i32(v >> 32));
-    } else {
-        op = copy_op(begin_op, op, INDEX_op_add_i64);
-        op->args[2] = tcgv_i64_arg(tcg_constant_i64(v));
-    }
-    return op;
-}
-
-static TCGOp *copy_mul_i32(TCGOp **begin_op, TCGOp *op, uint32_t v)
-{
-    op = copy_op(begin_op, op, INDEX_op_mul_i32);
-    op->args[2] = tcgv_i32_arg(tcg_constant_i32(v));
-    return op;
-}
-
-static TCGOp *copy_call(TCGOp **begin_op, TCGOp *op, void *func, int *cb_idx)
-{
-    TCGOp *old_op;
-    int func_idx;
-
-    /* copy all ops until the call */
-    do {
-        op = copy_op_nocheck(begin_op, op);
-    } while (op->opc != INDEX_op_call);
-
-    /* fill in the op call */
-    old_op = *begin_op;
-    TCGOP_CALLI(op) = TCGOP_CALLI(old_op);
-    TCGOP_CALLO(op) = TCGOP_CALLO(old_op);
-    tcg_debug_assert(op->life == 0);
-
-    func_idx = TCGOP_CALLO(op) + TCGOP_CALLI(op);
-    *cb_idx = func_idx;
-    op->args[func_idx] = (uintptr_t)func;
-
-    return op;
-}
-
-static TCGOp *append_inline_cb(const struct qemu_plugin_dyn_cb *cb,
-                               TCGOp *begin_op, TCGOp *op,
-                               int *unused)
-{
-    char *ptr = cb->inline_insn.entry.score->data->data;
-    size_t elem_size = g_array_get_element_size(
-        cb->inline_insn.entry.score->data);
-    size_t offset = cb->inline_insn.entry.offset;
-
-    op = copy_ld_i32(&begin_op, op);
-    op = copy_mul_i32(&begin_op, op, elem_size);
-    op = copy_ext_i32_ptr(&begin_op, op);
-    op = copy_const_ptr(&begin_op, op, ptr + offset);
-    op = copy_add_ptr(&begin_op, op);
-    op = copy_ld_i64(&begin_op, op);
-    op = copy_add_i64(&begin_op, op, cb->inline_insn.imm);
-    op = copy_st_i64(&begin_op, op);
-    return op;
-}
-
-static TCGOp *append_mem_cb(const struct qemu_plugin_dyn_cb *cb,
-                            TCGOp *begin_op, TCGOp *op, int *cb_idx)
-{
-    enum plugin_gen_cb type = begin_op->args[1];
-
-    tcg_debug_assert(type == PLUGIN_GEN_CB_MEM);
-
-    /* const_i32 == mov_i32 ("info", so it remains as is) */
-    op = copy_op(&begin_op, op, INDEX_op_mov_i32);
-
-    /* const_ptr */
-    op = copy_const_ptr(&begin_op, op, cb->userp);
-
-    /* copy the ld_i32, but note that we only have to copy it once */
-    if (*cb_idx == -1) {
-        op = copy_op(&begin_op, op, INDEX_op_ld_i32);
-    } else {
-        begin_op = QTAILQ_NEXT(begin_op, link);
-        tcg_debug_assert(begin_op && begin_op->opc == INDEX_op_ld_i32);
-    }
-
-    if (type == PLUGIN_GEN_CB_MEM) {
-        /* call */
-        op = copy_call(&begin_op, op, cb->regular.f.vcpu_udata, cb_idx);
-    }
-
-    return op;
-}
-
-typedef TCGOp *(*inject_fn)(const struct qemu_plugin_dyn_cb *cb,
-                            TCGOp *begin_op, TCGOp *op, int *intp);
-typedef bool (*op_ok_fn)(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb);
-
-static bool op_rw(const TCGOp *op, const struct qemu_plugin_dyn_cb *cb)
-{
-    int w;
-
-    w = op->args[2];
-    return !!(cb->rw & (w + 1));
-}
-
-static void inject_cb_type(const GArray *cbs, TCGOp *begin_op,
-                           inject_fn inject, op_ok_fn ok)
-{
-    TCGOp *end_op;
-    TCGOp *op;
-    int cb_idx = -1;
-    int i;
-
-    if (!cbs || cbs->len == 0) {
-        rm_ops(begin_op);
-        return;
-    }
-
-    end_op = find_op(begin_op, INDEX_op_plugin_cb_end);
-    tcg_debug_assert(end_op);
-
-    op = end_op;
-    for (i = 0; i < cbs->len; i++) {
-        struct qemu_plugin_dyn_cb *cb =
-            &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-
-        if (!ok(begin_op, cb)) {
-            continue;
-        }
-        op = inject(cb, begin_op, op, &cb_idx);
-    }
-    rm_ops_range(begin_op, end_op);
-}
-
-static void
-inject_inline_cb(const GArray *cbs, TCGOp *begin_op, op_ok_fn ok)
-{
-    inject_cb_type(cbs, begin_op, append_inline_cb, ok);
-}
-
-static void
-inject_mem_cb(const GArray *cbs, TCGOp *begin_op)
-{
-    inject_cb_type(cbs, begin_op, append_mem_cb, op_rw);
-}
-
 /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
 void plugin_gen_disable_mem_helpers(void)
 {
@@ -XXX,XX +XXX,XX @@ void plugin_gen_disable_mem_helpers(void)
     }
 }
 
-static void plugin_gen_mem_regular(const struct qemu_plugin_tb *ptb,
-                                   TCGOp *begin_op, int insn_idx)
-{
-    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
-    inject_mem_cb(insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR], begin_op);
-}
-
-static void plugin_gen_mem_inline(const struct qemu_plugin_tb *ptb,
-                                  TCGOp *begin_op, int insn_idx)
-{
-    const GArray *cbs;
-    struct qemu_plugin_insn *insn = g_ptr_array_index(ptb->insns, insn_idx);
-
-    cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
-    inject_inline_cb(cbs, begin_op, op_rw);
-}
-
 static void gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
                                   struct qemu_plugin_insn *insn)
 {
@@ -XXX,XX +XXX,XX @@ static void gen_inline_cb(struct qemu_plugin_dyn_cb *cb)
     tcg_temp_free_ptr(ptr);
 }
 
+static void gen_mem_cb(struct qemu_plugin_dyn_cb *cb,
+                       qemu_plugin_meminfo_t meminfo, TCGv_i64 addr)
+{
+    TCGv_i32 cpu_index = tcg_temp_ebb_new_i32();
+
+    tcg_gen_ld_i32(cpu_index, tcg_env,
+                   -offsetof(ArchCPU, env) + offsetof(CPUState, cpu_index));
+    tcg_gen_call4(cb->regular.f.vcpu_mem, cb->regular.info, NULL,
+                  tcgv_i32_temp(cpu_index),
+                  tcgv_i32_temp(tcg_constant_i32(meminfo)),
+                  tcgv_i64_temp(addr),
+                  tcgv_ptr_temp(tcg_constant_ptr(cb->userp)));
+    tcg_temp_free_i32(cpu_index);
+}
+
 /* #define DEBUG_PLUGIN_GEN_OPS */
 static void pr_ops(void)
 {
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
             break;
         }
 
-        case INDEX_op_plugin_cb_start:
+        case INDEX_op_plugin_mem_cb:
         {
-            enum plugin_gen_from from = op->args[0];
-            enum plugin_gen_cb type = op->args[1];
+            TCGv_i64 addr = temp_tcgv_i64(arg_temp(op->args[0]));
+            qemu_plugin_meminfo_t meminfo = op->args[1];
+            struct qemu_plugin_insn *insn;
+            const GArray *cbs;
+            int i, n, rw;
 
-            switch (from) {
-            case PLUGIN_GEN_FROM_MEM:
-            {
-                g_assert(insn_idx >= 0);
+            assert(insn_idx >= 0);
+            insn = g_ptr_array_index(plugin_tb->insns, insn_idx);
+            rw = qemu_plugin_mem_is_store(meminfo) ? 2 : 1;
 
-                switch (type) {
-                case PLUGIN_GEN_CB_MEM:
-                    plugin_gen_mem_regular(plugin_tb, op, insn_idx);
-                    break;
-                case PLUGIN_GEN_CB_INLINE:
-                    plugin_gen_mem_inline(plugin_tb, op, insn_idx);
-                    break;
-                default:
-                    g_assert_not_reached();
+            tcg_ctx->emit_before_op = op;
+
+            cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
+            for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
+                struct qemu_plugin_dyn_cb *cb =
+                    &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
+                if (cb->rw & rw) {
+                    gen_mem_cb(cb, meminfo, addr);
                 }
+            }
 
-                break;
-            }
-            default:
-                g_assert_not_reached();
+            cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
+            for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
+                struct qemu_plugin_dyn_cb *cb =
+                    &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
+                if (cb->rw & rw) {
+                    gen_inline_cb(cb);
+                }
             }
+
+            tcg_ctx->emit_before_op = NULL;
+            tcg_op_remove(tcg_ctx, op);
             break;
         }
+
         default:
             /* plugins don't care about any other ops */
             break;
diff --git a/tcg/tcg-op-ldst.c b/tcg/tcg-op-ldst.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op-ldst.c
+++ b/tcg/tcg-op-ldst.c
@@ -XXX,XX +XXX,XX @@ plugin_gen_mem_callbacks(TCGv_i64 copy_addr, TCGTemp *orig_addr, MemOpIdx oi,
                 copy_addr = tcg_temp_ebb_new_i64();
                 tcg_gen_extu_i32_i64(copy_addr, temp_tcgv_i32(orig_addr));
             }
-            plugin_gen_empty_mem_callback(copy_addr, info);
+            tcg_gen_plugin_mem_cb(copy_addr, info);
             tcg_temp_free_i64(copy_addr);
         } else {
             if (copy_addr) {
-                plugin_gen_empty_mem_callback(copy_addr, info);
+                tcg_gen_plugin_mem_cb(copy_addr, info);
                 tcg_temp_free_i64(copy_addr);
             } else {
-                plugin_gen_empty_mem_callback(temp_tcgv_i64(orig_addr), info);
+                tcg_gen_plugin_mem_cb(temp_tcgv_i64(orig_addr), info);
             }
         }
     }
diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op.c
+++ b/tcg/tcg-op.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_plugin_cb(unsigned from)
     tcg_gen_op1(INDEX_op_plugin_cb, from);
 }
 
+void tcg_gen_plugin_mem_cb(TCGv_i64 addr, unsigned meminfo)
+{
+    tcg_gen_op2(INDEX_op_plugin_mem_cb, tcgv_i64_arg(addr), meminfo);
+}
+
 void tcg_gen_plugin_cb_start(unsigned from, unsigned type, unsigned wr)
 {
     tcg_gen_op3(INDEX_op_plugin_cb_start, from, type, wr);
-- 
2.34.1

These placeholder helpers are no longer required.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/plugin-helpers.h         |  5 -----
 include/exec/helper-gen-common.h   |  4 ----
 include/exec/helper-proto-common.h |  4 ----
 accel/tcg/plugin-gen.c             | 20 --------------------
 4 files changed, 33 deletions(-)
 delete mode 100644 accel/tcg/plugin-helpers.h

diff --git a/accel/tcg/plugin-helpers.h b/accel/tcg/plugin-helpers.h
deleted file mode 100644
index XXXXXXX..XXXXXXX
--- a/accel/tcg/plugin-helpers.h
+++ /dev/null
@@ -XXX,XX +XXX,XX @@
-#ifdef CONFIG_PLUGIN
-DEF_HELPER_FLAGS_2(plugin_vcpu_udata_cb_no_wg, TCG_CALL_NO_WG | TCG_CALL_PLUGIN, void, i32, ptr)
-DEF_HELPER_FLAGS_2(plugin_vcpu_udata_cb_no_rwg, TCG_CALL_NO_RWG | TCG_CALL_PLUGIN, void, i32, ptr)
-DEF_HELPER_FLAGS_4(plugin_vcpu_mem_cb, TCG_CALL_NO_RWG | TCG_CALL_PLUGIN, void, i32, i32, i64, ptr)
-#endif
diff --git a/include/exec/helper-gen-common.h b/include/exec/helper-gen-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/helper-gen-common.h
+++ b/include/exec/helper-gen-common.h
@@ -XXX,XX +XXX,XX @@
 #include "exec/helper-gen.h.inc"
 #undef  HELPER_H
 
-#define HELPER_H "accel/tcg/plugin-helpers.h"
-#include "exec/helper-gen.h.inc"
-#undef  HELPER_H
-
 #endif /* HELPER_GEN_COMMON_H */
diff --git a/include/exec/helper-proto-common.h b/include/exec/helper-proto-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/helper-proto-common.h
+++ b/include/exec/helper-proto-common.h
@@ -XXX,XX +XXX,XX @@
 #include "exec/helper-proto.h.inc"
 #undef  HELPER_H
 
-#define HELPER_H "accel/tcg/plugin-helpers.h"
-#include "exec/helper-proto.h.inc"
-#undef  HELPER_H
-
 #endif /* HELPER_PROTO_COMMON_H */
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@
 #include "exec/exec-all.h"
 #include "exec/plugin-gen.h"
 #include "exec/translator.h"
-#include "exec/helper-proto-common.h"
-
-#define HELPER_H  "accel/tcg/plugin-helpers.h"
-#include "exec/helper-info.c.inc"
-#undef  HELPER_H
 
 /*
  * plugin_cb_start TCG op args[]:
@@ -XXX,XX +XXX,XX @@ enum plugin_gen_cb {
     PLUGIN_GEN_N_CBS,
 };
 
-/*
- * These helpers are stubs that get dynamically switched out for calls
- * direct to the plugin if they are subscribed to.
- */
-void HELPER(plugin_vcpu_udata_cb_no_wg)(uint32_t cpu_index, void *udata)
-{ }
-
-void HELPER(plugin_vcpu_udata_cb_no_rwg)(uint32_t cpu_index, void *udata)
-{ }
-
-void HELPER(plugin_vcpu_mem_cb)(unsigned int vcpu_index,
-                                qemu_plugin_meminfo_t info, uint64_t vaddr,
-                                void *userdata)
-{ }
-
 static void plugin_gen_empty_callback(enum plugin_gen_from from)
 {
     switch (from) {
-- 
2.34.1

Since we no longer emit plugin helpers during the initial code
translation phase, we don't need to specially mark plugin helpers.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg.h |  2 --
 plugins/core.c    | 10 ++++------
 tcg/tcg.c         |  4 +---
 3 files changed, 5 insertions(+), 11 deletions(-)

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ typedef TCGv_ptr TCGv_env;
 #define TCG_CALL_NO_SIDE_EFFECTS    0x0004
 /* Helper is G_NORETURN.  */
 #define TCG_CALL_NO_RETURN          0x0008
-/* Helper is part of Plugins.  */
-#define TCG_CALL_PLUGIN             0x0010
 
 /* convenience version of most used call flags */
 #define TCG_CALL_NO_RWG         TCG_CALL_NO_READ_GLOBALS
diff --git a/plugins/core.c b/plugins/core.c
index XXXXXXX..XXXXXXX 100644
--- a/plugins/core.c
+++ b/plugins/core.c
@@ -XXX,XX +XXX,XX @@ void plugin_register_dyn_cb__udata(GArray **arr,
                                    void *udata)
 {
     static TCGHelperInfo info[3] = {
-        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG | TCG_CALL_PLUGIN,
-        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG | TCG_CALL_PLUGIN,
-        [QEMU_PLUGIN_CB_RW_REGS].flags = TCG_CALL_PLUGIN,
+        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG,
+        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG,
         /*
          * Match qemu_plugin_vcpu_udata_cb_t:
          *   void (*)(uint32_t, void *)
@@ -XXX,XX +XXX,XX @@ void plugin_register_vcpu_mem_cb(GArray **arr,
         !__builtin_types_compatible_p(qemu_plugin_meminfo_t, int32_t));
 
     static TCGHelperInfo info[3] = {
-        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG | TCG_CALL_PLUGIN,
-        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG | TCG_CALL_PLUGIN,
-        [QEMU_PLUGIN_CB_RW_REGS].flags = TCG_CALL_PLUGIN,
+        [QEMU_PLUGIN_CB_NO_REGS].flags = TCG_CALL_NO_RWG,
+        [QEMU_PLUGIN_CB_R_REGS].flags = TCG_CALL_NO_WG,
         /*
          * Match qemu_plugin_vcpu_mem_cb_t:
          *   void (*)(uint32_t, qemu_plugin_meminfo_t, uint64_t, void *)
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static void tcg_gen_callN(void *func, TCGHelperInfo *info,
 
 #ifdef CONFIG_PLUGIN
     /* Flag helpers that may affect guest state */
-    if (tcg_ctx->plugin_insn &&
-        !(info->flags & TCG_CALL_PLUGIN) &&
-        !(info->flags & TCG_CALL_NO_SIDE_EFFECTS)) {
+    if (tcg_ctx->plugin_insn && !(info->flags & TCG_CALL_NO_SIDE_EFFECTS)) {
         tcg_ctx->plugin_insn->calls_helpers = true;
     }
 #endif
-- 
2.34.1

These opcodes are no longer used.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg-op-common.h |  2 --
 include/tcg/tcg-opc.h       |  2 --
 accel/tcg/plugin-gen.c      | 18 ------------------
 tcg/tcg-op.c                | 10 ----------
 4 files changed, 32 deletions(-)

We have qemu_plugin_dyn_cb.type to differentiate the various
callback types, so we do not need to keep them in separate queues.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/plugin.h  | 35 ++++++----------
 accel/tcg/plugin-gen.c | 90 ++++++++++++++++++++++--------------------
 plugins/api.c          | 18 +++------
 3 files changed, 65 insertions(+), 78 deletions(-)

diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -XXX,XX +XXX,XX @@ union qemu_plugin_cb_sig {
 };
 
 enum plugin_dyn_cb_type {
-    PLUGIN_CB_INSN,
-    PLUGIN_CB_MEM,
-    PLUGIN_N_CB_TYPES,
-};
-
-enum plugin_dyn_cb_subtype {
     PLUGIN_CB_REGULAR,
     PLUGIN_CB_INLINE,
-    PLUGIN_N_CB_SUBTYPES,
 };
 
 /*
@@ -XXX,XX +XXX,XX @@ enum plugin_dyn_cb_subtype {
  */
 struct qemu_plugin_dyn_cb {
     void *userp;
-    enum plugin_dyn_cb_subtype type;
+    enum plugin_dyn_cb_type type;
     /* @rw applies to mem callbacks only (both regular and inline) */
     enum qemu_plugin_mem_rw rw;
     /* fields specific to each dyn_cb type go here */
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_insn {
     GByteArray *data;
     uint64_t vaddr;
     void *haddr;
-    GArray *cbs[PLUGIN_N_CB_TYPES][PLUGIN_N_CB_SUBTYPES];
+    GArray *insn_cbs;
+    GArray *mem_cbs;
     bool calls_helpers;
 
     /* if set, the instruction calls helpers that might access guest memory */
@@ -XXX,XX +XXX,XX @@ static inline void qemu_plugin_insn_cleanup_fn(gpointer data)
 
 static inline struct qemu_plugin_insn *qemu_plugin_insn_alloc(void)
 {
-    int i, j;
     struct qemu_plugin_insn *insn = g_new0(struct qemu_plugin_insn, 1);
-    insn->data = g_byte_array_sized_new(4);
 
-    for (i = 0; i < PLUGIN_N_CB_TYPES; i++) {
-        for (j = 0; j < PLUGIN_N_CB_SUBTYPES; j++) {
-            insn->cbs[i][j] = g_array_new(false, false,
-                                          sizeof(struct qemu_plugin_dyn_cb));
-        }
-    }
+    insn->data = g_byte_array_sized_new(4);
     return insn;
 }
 
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_tb {
     /* if set, the TB calls helpers that might access guest memory */
     bool mem_helper;
 
-    GArray *cbs[PLUGIN_N_CB_SUBTYPES];
+    GArray *cbs;
 };
 
 /**
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb,
                                                  uint64_t pc)
 {
     struct qemu_plugin_insn *insn;
-    int i, j;
 
     if (unlikely(tb->n == tb->insns->len)) {
         struct qemu_plugin_insn *new_insn = qemu_plugin_insn_alloc();
         g_ptr_array_add(tb->insns, new_insn);
     }
+
     insn = g_ptr_array_index(tb->insns, tb->n++);
     g_byte_array_set_size(insn->data, 0);
     insn->calls_helpers = false;
     insn->mem_helper = false;
     insn->vaddr = pc;
-
-    for (i = 0; i < PLUGIN_N_CB_TYPES; i++) {
-        for (j = 0; j < PLUGIN_N_CB_SUBTYPES; j++) {
-            g_array_set_size(insn->cbs[i][j], 0);
-        }
+    if (insn->insn_cbs) {
+        g_array_set_size(insn->insn_cbs, 0);
+    }
+    if (insn->mem_cbs) {
+        g_array_set_size(insn->mem_cbs, 0);
     }
 
     return insn;
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ void plugin_gen_disable_mem_helpers(void)
 static void gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
                                   struct qemu_plugin_insn *insn)
 {
-    GArray *cbs[2];
     GArray *arr;
-    size_t n_cbs;
+    size_t len;
 
     /*
      * Tracking memory accesses performed from helpers requires extra work.
@@ -XXX,XX +XXX,XX @@ static void gen_enable_mem_helper(struct qemu_plugin_tb *ptb,
         return;
     }
 
-    cbs[0] = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
-    cbs[1] = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
-    n_cbs = cbs[0]->len + cbs[1]->len;
-
-    if (n_cbs == 0) {
+    if (!insn->mem_cbs || !insn->mem_cbs->len) {
         insn->mem_helper = false;
         return;
     }
     insn->mem_helper = true;
     ptb->mem_helper = true;
 
+    /*
+     * TODO: It seems like we should be able to use ref/unref
+     * to avoid needing to actually copy this array.
+     * Alternately, perhaps we could allocate new memory adjacent
+     * to the TranslationBlock itself, so that we do not have to
+     * actively manage the lifetime after this.
+     */
+    len = insn->mem_cbs->len;
     arr = g_array_sized_new(false, false,
-                            sizeof(struct qemu_plugin_dyn_cb), n_cbs);
-    g_array_append_vals(arr, cbs[0]->data, cbs[0]->len);
-    g_array_append_vals(arr, cbs[1]->data, cbs[1]->len);
-
+                            sizeof(struct qemu_plugin_dyn_cb), len);
+    memcpy(arr->data, insn->mem_cbs->data,
+           len * sizeof(struct qemu_plugin_dyn_cb));
     qemu_plugin_add_dyn_cb_arr(arr);
 
     tcg_gen_st_ptr(tcg_constant_ptr((intptr_t)arr), tcg_env,
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
             case PLUGIN_GEN_FROM_TB:
                 assert(insn == NULL);
 
-                cbs = plugin_tb->cbs[PLUGIN_CB_REGULAR];
+                cbs = plugin_tb->cbs;
                 for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
                     struct qemu_plugin_dyn_cb *cb =
                         &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-                    gen_udata_cb(cb);
-                }
 
-                cbs = plugin_tb->cbs[PLUGIN_CB_INLINE];
-                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
-                    struct qemu_plugin_dyn_cb *cb =
-                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-                    gen_inline_cb(cb);
+                    switch (cb->type) {
+                    case PLUGIN_CB_REGULAR:
+                        gen_udata_cb(cb);
+                        break;
+                    case PLUGIN_CB_INLINE:
+                        gen_inline_cb(cb);
+                        break;
+                    default:
+                        g_assert_not_reached();
+                    }
                 }
                 break;
 
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
 
                 gen_enable_mem_helper(plugin_tb, insn);
 
-                cbs = insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR];
+                cbs = insn->insn_cbs;
                 for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
                     struct qemu_plugin_dyn_cb *cb =
                         &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-                    gen_udata_cb(cb);
-                }
 
-                cbs = insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_INLINE];
-                for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
-                    struct qemu_plugin_dyn_cb *cb =
-                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-                    gen_inline_cb(cb);
+                    switch (cb->type) {
+                    case PLUGIN_CB_REGULAR:
+                        gen_udata_cb(cb);
+                        break;
+                    case PLUGIN_CB_INLINE:
+                        gen_inline_cb(cb);
+                        break;
+                    default:
+                        g_assert_not_reached();
+                    }
                 }
                 break;
 
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
 
             tcg_ctx->emit_before_op = op;
 
-            cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR];
+            cbs = insn->mem_cbs;
             for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
                 struct qemu_plugin_dyn_cb *cb =
                     &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-                if (cb->rw & rw) {
-                    gen_mem_cb(cb, meminfo, addr);
-                }
-            }
 
-            cbs = insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE];
-            for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
-                struct qemu_plugin_dyn_cb *cb =
-                    &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
                 if (cb->rw & rw) {
-                    gen_inline_cb(cb);
+                    switch (cb->type) {
+                    case PLUGIN_CB_REGULAR:
+                        gen_mem_cb(cb, meminfo, addr);
+                        break;
+                    case PLUGIN_CB_INLINE:
+                        gen_inline_cb(cb);
+                        break;
+                    default:
+                        g_assert_not_reached();
+                    }
                 }
             }
 
@@ -XXX,XX +XXX,XX @@ bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
 
     if (test_bit(QEMU_PLUGIN_EV_VCPU_TB_TRANS, cpu->plugin_state->event_mask)) {
         struct qemu_plugin_tb *ptb = tcg_ctx->plugin_tb;
-        int i;
 
         /* reset callbacks */
-        for (i = 0; i < PLUGIN_N_CB_SUBTYPES; i++) {
-            if (ptb->cbs[i]) {
-                g_array_set_size(ptb->cbs[i], 0);
-            }
+        if (ptb->cbs) {
+            g_array_set_size(ptb->cbs, 0);
         }
         ptb->n = 0;
 
diff --git a/plugins/api.c b/plugins/api.c
index XXXXXXX..XXXXXXX 100644
--- a/plugins/api.c
+++ b/plugins/api.c
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_tb_exec_cb(struct qemu_plugin_tb *tb,
                                           void *udata)
 {
     if (!tb->mem_only) {
-        plugin_register_dyn_cb__udata(&tb->cbs[PLUGIN_CB_REGULAR],
-                                      cb, flags, udata);
+        plugin_register_dyn_cb__udata(&tb->cbs, cb, flags, udata);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_tb_exec_inline_per_vcpu(
     uint64_t imm)
 {
     if (!tb->mem_only) {
-        plugin_register_inline_op_on_entry(
-            &tb->cbs[PLUGIN_CB_INLINE], 0, op, entry, imm);
+        plugin_register_inline_op_on_entry(&tb->cbs, 0, op, entry, imm);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_insn_exec_cb(struct qemu_plugin_insn *insn,
                                             void *udata)
 {
     if (!insn->mem_only) {
-        plugin_register_dyn_cb__udata(
-            &insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_REGULAR], cb, flags, udata);
+        plugin_register_dyn_cb__udata(&insn->insn_cbs, cb, flags, udata);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_insn_exec_inline_per_vcpu(
     uint64_t imm)
 {
     if (!insn->mem_only) {
-        plugin_register_inline_op_on_entry(
-            &insn->cbs[PLUGIN_CB_INSN][PLUGIN_CB_INLINE], 0, op, entry, imm);
+        plugin_register_inline_op_on_entry(&insn->insn_cbs, 0, op, entry, imm);
     }
 }
 
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_mem_cb(struct qemu_plugin_insn *insn,
                                       enum qemu_plugin_mem_rw rw,
                                       void *udata)
 {
-    plugin_register_vcpu_mem_cb(&insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_REGULAR],
-                                cb, flags, rw, udata);
+    plugin_register_vcpu_mem_cb(&insn->mem_cbs, cb, flags, rw, udata);
 }
 
 void qemu_plugin_register_vcpu_mem_inline_per_vcpu(
@@ -XXX,XX +XXX,XX @@ void qemu_plugin_register_vcpu_mem_inline_per_vcpu(
     qemu_plugin_u64 entry,
     uint64_t imm)
 {
-    plugin_register_inline_op_on_entry(
-        &insn->cbs[PLUGIN_CB_MEM][PLUGIN_CB_INLINE], rw, op, entry, imm);
+    plugin_register_inline_op_on_entry(&insn->mem_cbs, rw, op, entry, imm);
 }
 
 void qemu_plugin_register_vcpu_tb_trans_cb(qemu_plugin_id_t id,
-- 
2.34.1

Use different enumerators for vcpu_udata and vcpu_mem callbacks.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/plugin.h  | 1 +
 accel/tcg/plugin-gen.c | 2 +-
 plugins/core.c         | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)

The DEBUG_PLUGIN_GEN_OPS ifdef is replaced with "-d op_plugin".
The second pr_ops call can be obtained with "-d op".

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/log.h     |  1 +
 include/tcg/tcg.h      |  1 +
 accel/tcg/plugin-gen.c | 67 +++++++-----------------------------------
 tcg/tcg.c              | 29 +++++++++++++++++-
 util/log.c             |  4 +++
 5 files changed, 45 insertions(+), 57 deletions(-)

diff --git a/include/qemu/log.h b/include/qemu/log.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/log.h
+++ b/include/qemu/log.h
@@ -XXX,XX +XXX,XX @@ bool qemu_log_separate(void);
 #define LOG_STRACE         (1 << 19)
 #define LOG_PER_THREAD     (1 << 20)
 #define CPU_LOG_TB_VPU     (1 << 21)
+#define LOG_TB_OP_PLUGIN   (1 << 22)
 
 /* Lock/unlock output. */
 
diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ static inline const TCGOpcode *tcg_swap_vecop_list(const TCGOpcode *n)
 }
 
 bool tcg_can_emit_vecop_list(const TCGOpcode *, TCGType, unsigned);
+void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs);
 
 #endif /* TCG_H */
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@
  */
 #include "qemu/osdep.h"
 #include "qemu/plugin.h"
+#include "qemu/log.h"
 #include "cpu.h"
 #include "tcg/tcg.h"
 #include "tcg/tcg-temp-internal.h"
@@ -XXX,XX +XXX,XX @@ static void gen_mem_cb(struct qemu_plugin_dyn_cb *cb,
     tcg_temp_free_i32(cpu_index);
 }
 
-/* #define DEBUG_PLUGIN_GEN_OPS */
-static void pr_ops(void)
-{
-#ifdef DEBUG_PLUGIN_GEN_OPS
-    TCGOp *op;
-    int i = 0;
-
-    QTAILQ_FOREACH(op, &tcg_ctx->ops, link) {
-        const char *name = "";
-        const char *type = "";
-
-        if (op->opc == INDEX_op_plugin_cb_start) {
-            switch (op->args[0]) {
-            case PLUGIN_GEN_FROM_TB:
-                name = "tb";
-                break;
-            case PLUGIN_GEN_FROM_INSN:
-                name = "insn";
-                break;
-            case PLUGIN_GEN_FROM_MEM:
-                name = "mem";
-                break;
-            case PLUGIN_GEN_AFTER_INSN:
-                name = "after insn";
-                break;
-            default:
-                break;
-            }
-            switch (op->args[1]) {
-            case PLUGIN_GEN_CB_UDATA:
-                type = "udata";
-                break;
-            case PLUGIN_GEN_CB_INLINE:
-                type = "inline";
-                break;
-            case PLUGIN_GEN_CB_MEM:
-                type = "mem";
-                break;
-            case PLUGIN_GEN_ENABLE_MEM_HELPER:
-                type = "enable mem helper";
-                break;
-            case PLUGIN_GEN_DISABLE_MEM_HELPER:
-                type = "disable mem helper";
-                break;
-            default:
-                break;
-            }
-        }
-        printf("op[%2i]: %s %s %s\n", i, tcg_op_defs[op->opc].name, name, type);
-        i++;
-    }
-#endif
-}
-
 static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
 {
     TCGOp *op, *next;
     int insn_idx = -1;
 
-    pr_ops();
+    if (unlikely(qemu_loglevel_mask(LOG_TB_OP_PLUGIN)
+                 && qemu_log_in_addr_range(plugin_tb->vaddr))) {
+        FILE *logfile = qemu_log_trylock();
+        if (logfile) {
+            fprintf(logfile, "OP before plugin injection:\n");
+            tcg_dump_ops(tcg_ctx, logfile, false);
+            fprintf(logfile, "\n");
+            qemu_log_unlock(logfile);
+        }
+    }
 
     /*
      * While injecting code, we cannot afford to reuse any ebb temps
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
             break;
         }
     }
-    pr_ops();
 }
 
 bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static const char bswap_flag_name[][6] = {
     [TCG_BSWAP_IZ | TCG_BSWAP_OS] = "iz,os",
 };
 
+#ifdef CONFIG_PLUGIN
+static const char * const plugin_from_name[] = {
+    "from-tb",
+    "from-insn",
+    "after-insn",
+    "after-tb",
+};
+#endif
+
 static inline bool tcg_regset_single(TCGRegSet d)
 {
     return (d & (d - 1)) == 0;
@@ -XXX,XX +XXX,XX @@ static inline TCGReg tcg_regset_first(TCGRegSet d)
 #define ne_fprintf(...) \
     ({ int ret_ = fprintf(__VA_ARGS__); ret_ >= 0 ? ret_ : 0; })
 
-static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs)
+void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs)
 {
     char buf[128];
     TCGOp *op;
@@ -XXX,XX +XXX,XX @@ static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs)
                     i = k = 1;
                 }
                 break;
+#ifdef CONFIG_PLUGIN
+            case INDEX_op_plugin_cb:
+                {
+                    TCGArg from = op->args[k++];
+                    const char *name = NULL;
+
+                    if (from < ARRAY_SIZE(plugin_from_name)) {
+                        name = plugin_from_name[from];
+                    }
+                    if (name) {
+                        col += ne_fprintf(f, "%s", name);
+                    } else {
+                        col += ne_fprintf(f, "$0x%" TCG_PRIlx, from);
+                    }
+                    i = 1;
+                }
+                break;
+#endif
             default:
                 i = 0;
                 break;
diff --git a/util/log.c b/util/log.c
index XXXXXXX..XXXXXXX 100644
--- a/util/log.c
+++ b/util/log.c
@@ -XXX,XX +XXX,XX @@ const QEMULogItem qemu_log_items[] = {
       "show micro ops after optimization" },
     { CPU_LOG_TB_OP_IND, "op_ind",
       "show micro ops before indirect lowering" },
+#ifdef CONFIG_PLUGIN
+    { LOG_TB_OP_PLUGIN, "op_plugin",
+      "show micro ops before plugin injection" },
+#endif
     { CPU_LOG_INT, "int",
       "show interrupts/exceptions in short format" },
     { CPU_LOG_EXEC, "exec",
-- 
2.34.1

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/plugin-gen.c | 84 +++++++++++++++++++++---------------------
 1 file changed, 41 insertions(+), 43 deletions(-)

diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ static void gen_mem_cb(struct qemu_plugin_dyn_cb *cb,
     tcg_temp_free_i32(cpu_index);
 }
 
+static void inject_cb(struct qemu_plugin_dyn_cb *cb)
+
+{
+    switch (cb->type) {
+    case PLUGIN_CB_REGULAR:
+        gen_udata_cb(cb);
+        break;
+    case PLUGIN_CB_INLINE:
+        gen_inline_cb(cb);
+        break;
+    default:
+        g_assert_not_reached();
+    }
+}
+
+static void inject_mem_cb(struct qemu_plugin_dyn_cb *cb,
+                          enum qemu_plugin_mem_rw rw,
+                          qemu_plugin_meminfo_t meminfo, TCGv_i64 addr)
+{
+    if (cb->rw & rw) {
+        switch (cb->type) {
+        case PLUGIN_CB_MEM_REGULAR:
+            gen_mem_cb(cb, meminfo, addr);
+            break;
+        default:
+            inject_cb(cb);
+            break;
+        }
+    }
+}
+
 static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
 {
     TCGOp *op, *next;
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
 
                 cbs = plugin_tb->cbs;
                 for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
-                    struct qemu_plugin_dyn_cb *cb =
-                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-
-                    switch (cb->type) {
-                    case PLUGIN_CB_REGULAR:
-                        gen_udata_cb(cb);
-                        break;
-                    case PLUGIN_CB_INLINE:
-                        gen_inline_cb(cb);
-                        break;
-                    default:
-                        g_assert_not_reached();
-                    }
+                    inject_cb(
+                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i));
                 }
                 break;
 
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
 
                 cbs = insn->insn_cbs;
                 for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
-                    struct qemu_plugin_dyn_cb *cb =
-                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-
-                    switch (cb->type) {
-                    case PLUGIN_CB_REGULAR:
-                        gen_udata_cb(cb);
-                        break;
-                    case PLUGIN_CB_INLINE:
-                        gen_inline_cb(cb);
-                        break;
-                    default:
-                        g_assert_not_reached();
-                    }
+                    inject_cb(
+                        &g_array_index(cbs, struct qemu_plugin_dyn_cb, i));
                 }
                 break;
 
@@ -XXX,XX +XXX,XX @@ static void plugin_gen_inject(struct qemu_plugin_tb *plugin_tb)
         {
             TCGv_i64 addr = temp_tcgv_i64(arg_temp(op->args[0]));
             qemu_plugin_meminfo_t meminfo = op->args[1];
+            enum qemu_plugin_mem_rw rw =
+                (qemu_plugin_mem_is_store(meminfo)
+                 ? QEMU_PLUGIN_MEM_W : QEMU_PLUGIN_MEM_R);
             struct qemu_plugin_insn *insn;
             const GArray *cbs;
-            int i, n, rw;
+            int i, n;
 
             assert(insn_idx >= 0);
             insn = g_ptr_array_index(plugin_tb->insns, insn_idx);
-            rw = qemu_plugin_mem_is_store(meminfo) ? 2 : 1;
 
             tcg_ctx->emit_before_op = op;
 
             cbs = insn->mem_cbs;
             for (i = 0, n = (cbs ? cbs->len : 0); i < n; i++) {
-                struct qemu_plugin_dyn_cb *cb =
-                    &g_array_index(cbs, struct qemu_plugin_dyn_cb, i);
-
-                if (cb->rw & rw) {
-                    switch (cb->type) {
-                    case PLUGIN_CB_MEM_REGULAR:
-                        gen_mem_cb(cb, meminfo, addr);
-                        break;
-                    case PLUGIN_CB_INLINE:
-                        gen_inline_cb(cb);
-                        break;
-                    default:
-                        g_assert_not_reached();
-                    }
-                }
+                inject_mem_cb(&g_array_index(cbs, struct qemu_plugin_dyn_cb, i),
+                              rw, meminfo, addr);
             }
 
             tcg_ctx->emit_before_op = NULL;
-- 
2.34.1

Merge qemu_plugin_insn_alloc and qemu_plugin_tb_insn_get into
plugin_gen_insn_start, since it is used nowhere else.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/plugin.h  | 39 ---------------------------------------
 accel/tcg/plugin-gen.c | 39 ++++++++++++++++++++++++++++++++-------
 2 files changed, 32 insertions(+), 46 deletions(-)

diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -XXX,XX +XXX,XX @@ static inline void qemu_plugin_insn_cleanup_fn(gpointer data)
     g_byte_array_free(insn->data, true);
 }
 
-static inline struct qemu_plugin_insn *qemu_plugin_insn_alloc(void)
-{
-    struct qemu_plugin_insn *insn = g_new0(struct qemu_plugin_insn, 1);
-
-    insn->data = g_byte_array_sized_new(4);
-    return insn;
-}
-
 /* Internal context for this TranslationBlock */
 struct qemu_plugin_tb {
     GPtrArray *insns;
@@ -XXX,XX +XXX,XX @@ struct qemu_plugin_tb {
     GArray *cbs;
 };
 
-/**
- * qemu_plugin_tb_insn_get(): get next plugin record for translation.
- * @tb: the internal tb context
- * @pc: address of instruction
- */
-static inline
-struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb,
-                                                 uint64_t pc)
-{
-    struct qemu_plugin_insn *insn;
-
-    if (unlikely(tb->n == tb->insns->len)) {
-        struct qemu_plugin_insn *new_insn = qemu_plugin_insn_alloc();
-        g_ptr_array_add(tb->insns, new_insn);
-    }
-
-    insn = g_ptr_array_index(tb->insns, tb->n++);
-    g_byte_array_set_size(insn->data, 0);
-    insn->calls_helpers = false;
-    insn->mem_helper = false;
-    insn->vaddr = pc;
-    if (insn->insn_cbs) {
-        g_array_set_size(insn->insn_cbs, 0);
-    }
-    if (insn->mem_cbs) {
-        g_array_set_size(insn->mem_cbs, 0);
-    }
-
-    return insn;
-}
-
 /**
  * struct CPUPluginState - per-CPU state for plugins
  * @event_mask: plugin event bitmap. Modified only via async work.
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
 void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
 {
     struct qemu_plugin_tb *ptb = tcg_ctx->plugin_tb;
-    struct qemu_plugin_insn *pinsn;
+    struct qemu_plugin_insn *insn;
+    size_t n = db->num_insns;
+    vaddr pc;
 
-    pinsn = qemu_plugin_tb_insn_get(ptb, db->pc_next);
-    tcg_ctx->plugin_insn = pinsn;
-    plugin_gen_empty_callback(PLUGIN_GEN_FROM_INSN);
+    assert(n >= 1);
+    ptb->n = n;
+    if (n <= ptb->insns->len) {
+        insn = g_ptr_array_index(ptb->insns, n - 1);
+        g_byte_array_set_size(insn->data, 0);
+    } else {
+        assert(n - 1 == ptb->insns->len);
+        insn = g_new0(struct qemu_plugin_insn, 1);
+        insn->data = g_byte_array_sized_new(4);
+        g_ptr_array_add(ptb->insns, insn);
+    }
+
+    tcg_ctx->plugin_insn = insn;
+    insn->calls_helpers = false;
+    insn->mem_helper = false;
+    if (insn->insn_cbs) {
+        g_array_set_size(insn->insn_cbs, 0);
+    }
+    if (insn->mem_cbs) {
+        g_array_set_size(insn->mem_cbs, 0);
+    }
+
+    pc = db->pc_next;
+    insn->vaddr = pc;
 
     /*
      * Detect page crossing to get the new host address.
@@ -XXX,XX +XXX,XX @@ void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
      * fetching instructions from a region not backed by RAM.
      */
     if (ptb->haddr1 == NULL) {
-        pinsn->haddr = NULL;
+        insn->haddr = NULL;
     } else if (is_same_page(db, db->pc_next)) {
-        pinsn->haddr = ptb->haddr1 + pinsn->vaddr - ptb->vaddr;
+        insn->haddr = ptb->haddr1 + pc - ptb->vaddr;
     } else {
         if (ptb->vaddr2 == -1) {
             ptb->vaddr2 = TARGET_PAGE_ALIGN(db->pc_first);
             get_page_addr_code_hostp(cpu_env(cpu), ptb->vaddr2, &ptb->haddr2);
         }
-        pinsn->haddr = ptb->haddr2 + pinsn->vaddr - ptb->vaddr2;
+        insn->haddr = ptb->haddr2 + pc - ptb->vaddr2;
     }
+
+    plugin_gen_empty_callback(PLUGIN_GEN_FROM_INSN);
 }
 
 void plugin_gen_insn_end(void)
-- 
2.34.1

Each caller can use tcg_gen_plugin_cb directly.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/plugin-gen.c | 19 +++----------------
 1 file changed, 3 insertions(+), 16 deletions(-)

diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@ enum plugin_gen_from {
     PLUGIN_GEN_AFTER_TB,
 };
 
-static void plugin_gen_empty_callback(enum plugin_gen_from from)
-{
-    switch (from) {
-    case PLUGIN_GEN_AFTER_INSN:
-    case PLUGIN_GEN_FROM_TB:
-    case PLUGIN_GEN_FROM_INSN:
-        tcg_gen_plugin_cb(from);
-        break;
-    default:
-        g_assert_not_reached();
-    }
-}
-
 /* called before finishing a TB with exit_tb, goto_tb or goto_ptr */
 void plugin_gen_disable_mem_helpers(void)
 {
@@ -XXX,XX +XXX,XX @@ bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
         ptb->mem_only = mem_only;
         ptb->mem_helper = false;
 
-        plugin_gen_empty_callback(PLUGIN_GEN_FROM_TB);
+        tcg_gen_plugin_cb(PLUGIN_GEN_FROM_TB);
     }
 
     tcg_ctx->plugin_insn = NULL;
@@ -XXX,XX +XXX,XX @@ void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
         insn->haddr = ptb->haddr2 + pc - ptb->vaddr2;
     }
 
-    plugin_gen_empty_callback(PLUGIN_GEN_FROM_INSN);
+    tcg_gen_plugin_cb(PLUGIN_GEN_FROM_INSN);
 }
 
 void plugin_gen_insn_end(void)
 {
-    plugin_gen_empty_callback(PLUGIN_GEN_AFTER_INSN);
+    tcg_gen_plugin_cb(PLUGIN_GEN_AFTER_INSN);
 }
 
 /*
-- 
2.34.1

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/plugin-gen.c | 31 ++++---------------------------
 1 file changed, 4 insertions(+), 27 deletions(-)

diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -XXX,XX +XXX,XX @@
  * Injecting the desired instrumentation could be done with a second
  * translation pass that combined the instrumentation requests, but that
  * would be ugly and inefficient since we would decode the guest code twice.
- * Instead, during TB translation we add "empty" instrumentation calls for all
- * possible instrumentation events, and then once we collect the instrumentation
- * requests from plugins, we either "fill in" those empty events or remove them
- * if they have no requests.
- *
- * When "filling in" an event we first copy the empty callback's TCG ops. This
- * might seem unnecessary, but it is done to support an arbitrary number
- * of callbacks per event. Take for example a regular instruction callback.
- * We first generate a callback to an empty helper function. Then, if two
- * plugins register one callback each for this instruction, we make two copies
- * of the TCG ops generated for the empty callback, substituting the function
- * pointer that points to the empty helper function with the plugins' desired
- * callback functions. After that we remove the empty callback's ops.
- *
- * Note that the location in TCGOp.args[] of the pointer to a helper function
- * varies across different guest and host architectures. Instead of duplicating
- * the logic that figures this out, we rely on the fact that the empty
- * callbacks point to empty functions that are unique pointers in the program.
- * Thus, to find the right location we just have to look for a match in
- * TCGOp.args[]. This is the main reason why we first copy an empty callback's
- * TCG ops and then fill them in; regardless of whether we have one or many
- * callbacks for that event, the logic to add all of them is the same.
- *
- * When generating more than one callback per event, we make a small
- * optimization to avoid generating redundant operations. For instance, for the
- * second and all subsequent callbacks of an event, we do not need to reload the
- * CPU's index into a TCG temp, since the first callback did it already.
+ * Instead, during TB translation we add "plugin_cb" marker opcodes
+ * for all possible instrumentation events, and then once we collect the
+ * instrumentation requests from plugins, we generate code for those markers
+ * or remove them if they have no requests.
  */
 #include "qemu/osdep.h"
 #include "qemu/plugin.h"
-- 
2.34.1