Series comparison

-[PULL 0/3] tcg patch queue
+[PULL 00/34] tcg patch queue
-The following changes since commit 2ecfc0657afa5d29a373271b342f704a1a3c6737:
+This is mostly my code_gen_buffer cleanup, plus a few other random
 changes thrown in.  Including a fix for a recent float32_exp2 bug.
-  Merge remote-tracking branch 'remotes/armbru/tags/pull-misc-2020-12-10' into staging (2020-12-10 17:01:05 +0000)
 r~
 The following changes since commit 894fc4fd670aaf04a67dc7507739f914ff4bacf2:
   Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2021-06-11 09:21:48 +0100)
 are available in the Git repository at:
-  https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20201210
+  https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20210611
-for you to fetch changes up to 9e2658d62ebc23efe7df43fc0e306f129510d874:
+for you to fetch changes up to 60afaddc208d34f6dc86dd974f6e02724fba6eb6:
-  accel/tcg: rename tcg-cpus functions to match module name (2020-12-10 17:44:10 -0600)
+  docs/devel: Explain in more detail the TB chaining mechanisms (2021-06-11 09:41:25 -0700)
 ----------------------------------------------------------------
-Split CpusAccel for tcg variants
+Clean up code_gen_buffer allocation.
 Add tcg_remove_ops_after.
 Fix tcg_constant_* documentation.
 Improve TB chaining documentation.
 Fix float32_exp2.
 ----------------------------------------------------------------
-Claudio Fontana (3):
+Jose R. Ziviani (1):
-      accel/tcg: split CpusAccel into three TCG variants
+      tcg/arm: Fix tcg_out_op function signature
       accel/tcg: split tcg_start_vcpu_thread
       accel/tcg: rename tcg-cpus functions to match module name
- accel/tcg/tcg-cpus-icount.h |  17 ++
+Luis Pires (1):
- accel/tcg/tcg-cpus-rr.h     |  21 ++
+      docs/devel: Explain in more detail the TB chaining mechanisms
  accel/tcg/tcg-cpus.h        |  12 +-
  accel/tcg/tcg-all.c         |  13 +-
  accel/tcg/tcg-cpus-icount.c | 147 +++++++++++++
  accel/tcg/tcg-cpus-mttcg.c  | 140 ++++++++++++
  accel/tcg/tcg-cpus-rr.c     | 305 ++++++++++++++++++++++++++
  accel/tcg/tcg-cpus.c        | 506 +-------------------------------------------
  softmmu/icount.c            |   2 +-
  accel/tcg/meson.build       |   9 +-
 files changed, 670 insertions(+), 502 deletions(-)
  create mode 100644 accel/tcg/tcg-cpus-icount.h
  create mode 100644 accel/tcg/tcg-cpus-rr.h
  create mode 100644 accel/tcg/tcg-cpus-icount.c
  create mode 100644 accel/tcg/tcg-cpus-mttcg.c
  create mode 100644 accel/tcg/tcg-cpus-rr.c
+Richard Henderson (32):
+      meson: Split out tcg/meson.build
+      meson: Split out fpu/meson.build
+      tcg: Re-order tcg_region_init vs tcg_prologue_init
+      tcg: Remove error return from tcg_region_initial_alloc__locked
+      tcg: Split out tcg_region_initial_alloc
+      tcg: Split out tcg_region_prologue_set
+      tcg: Split out region.c
+      accel/tcg: Inline cpu_gen_init
+      accel/tcg: Move alloc_code_gen_buffer to tcg/region.c
+      accel/tcg: Rename tcg_init to tcg_init_machine
+      tcg: Create tcg_init
+      accel/tcg: Merge tcg_exec_init into tcg_init_machine
+      accel/tcg: Use MiB in tcg_init_machine
+      accel/tcg: Pass down max_cpus to tcg_init
+      tcg: Introduce tcg_max_ctxs
+      tcg: Move MAX_CODE_GEN_BUFFER_SIZE to tcg-target.h
+      tcg: Replace region.end with region.total_size
+      tcg: Rename region.start to region.after_prologue
+      tcg: Tidy tcg_n_regions
+      tcg: Tidy split_cross_256mb
+      tcg: Move in_code_gen_buffer and tests to region.c
+      tcg: Allocate code_gen_buffer into struct tcg_region_state
+      tcg: Return the map protection from alloc_code_gen_buffer
+      tcg: Sink qemu_madvise call to common code
+      util/osdep: Add qemu_mprotect_rw
+      tcg: Round the tb_size default from qemu_get_host_physmem
+      tcg: Merge buffer protection and guard page protection
+      tcg: When allocating for !splitwx, begin with PROT_NONE
+      tcg: Move tcg_init_ctx and tcg_ctx from accel/tcg/
+      tcg: Introduce tcg_remove_ops_after
+      tcg: Fix documentation for tcg_constant_* vs tcg_temp_free_*
+      softfloat: Fix tp init in float32_exp2
+ docs/devel/tcg.rst        | 101 ++++-
+ meson.build               |  12 +-
+ accel/tcg/internal.h      |   2 +
+ include/qemu/osdep.h      |   1 +
+ include/sysemu/tcg.h      |   2 -
+ include/tcg/tcg.h         |  28 +-
+ tcg/aarch64/tcg-target.h  |   1 +
+ tcg/arm/tcg-target.h      |   1 +
+ tcg/i386/tcg-target.h     |   2 +
+ tcg/mips/tcg-target.h     |   6 +
+ tcg/ppc/tcg-target.h      |   2 +
+ tcg/riscv/tcg-target.h    |   1 +
+ tcg/s390/tcg-target.h     |   3 +
+ tcg/sparc/tcg-target.h    |   1 +
+ tcg/tcg-internal.h        |  40 ++
+ tcg/tci/tcg-target.h      |   1 +
+ accel/tcg/tcg-all.c       |  32 +-
+ accel/tcg/translate-all.c | 439 +-------------------
+ bsd-user/main.c           |   3 +-
+ fpu/softfloat.c           |   2 +-
+ linux-user/main.c         |   1 -
+ tcg/region.c              | 999 ++++++++++++++++++++++++++++++++++++++++++++++
+ tcg/tcg.c                 | 649 +++---------------------------
+ util/osdep.c              |   9 +
+ tcg/arm/tcg-target.c.inc  |   3 +-
+ fpu/meson.build           |   1 +
+ tcg/meson.build           |  14 +
+files changed, 1266 insertions(+), 1090 deletions(-)
+ create mode 100644 tcg/tcg-internal.h
+ create mode 100644 tcg/region.c
+ create mode 100644 fpu/meson.build
+ create mode 100644 tcg/meson.build

-New patch
+[PULL 01/34] meson: Split out tcg/meson.build
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ meson.build     |  8 +-------
+ tcg/meson.build | 13 +++++++++++++
+files changed, 14 insertions(+), 7 deletions(-)
+ create mode 100644 tcg/meson.build
+diff --git a/meson.build b/meson.build
+index XXXXXXX..XXXXXXX 100644
+--- a/meson.build
++++ b/meson.build
+@@ -XXX,XX +XXX,XX @@ common_ss.add(capstone)
+ specific_ss.add(files('cpu.c', 'disas.c', 'gdbstub.c'), capstone)
+ specific_ss.add(when: 'CONFIG_TCG', if_true: files(
+   'fpu/softfloat.c',
+-  'tcg/optimize.c',
+-  'tcg/tcg-common.c',
+-  'tcg/tcg-op-gvec.c',
+-  'tcg/tcg-op-vec.c',
+-  'tcg/tcg-op.c',
+-  'tcg/tcg.c',
+ ))
+-specific_ss.add(when: 'CONFIG_TCG_INTERPRETER', if_true: files('tcg/tci.c'))
+ # Work around a gcc bug/misfeature wherein constant propagation looks
+ # through an alias:
+@@ -XXX,XX +XXX,XX @@ subdir('net')
+ subdir('replay')
+ subdir('semihosting')
+ subdir('hw')
++subdir('tcg')
+ subdir('accel')
+ subdir('plugins')
+ subdir('bsd-user')
+diff --git a/tcg/meson.build b/tcg/meson.build
+new file mode 100644
+index XXXXXXX..XXXXXXX
+--- /dev/null
++++ b/tcg/meson.build
+@@ -XXX,XX +XXX,XX @@
++tcg_ss = ss.source_set()
++
++tcg_ss.add(files(
++  'optimize.c',
++  'tcg.c',
++  'tcg-common.c',
++  'tcg-op.c',
++  'tcg-op-gvec.c',
++  'tcg-op-vec.c',
++))
++tcg_ss.add(when: 'CONFIG_TCG_INTERPRETER', if_true: files('tci.c'))
++
++specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
+--
+.25.1

-New patch
+[PULL 02/34] meson: Split out fpu/meson.build
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ meson.build     | 4 +---
+ fpu/meson.build | 1 +
+files changed, 2 insertions(+), 3 deletions(-)
+ create mode 100644 fpu/meson.build
+diff --git a/meson.build b/meson.build
+index XXXXXXX..XXXXXXX 100644
+--- a/meson.build
++++ b/meson.build
+@@ -XXX,XX +XXX,XX @@ subdir('softmmu')
+ common_ss.add(capstone)
+ specific_ss.add(files('cpu.c', 'disas.c', 'gdbstub.c'), capstone)
+-specific_ss.add(when: 'CONFIG_TCG', if_true: files(
+-  'fpu/softfloat.c',
+-))
+ # Work around a gcc bug/misfeature wherein constant propagation looks
+ # through an alias:
+@@ -XXX,XX +XXX,XX @@ subdir('replay')
+ subdir('semihosting')
+ subdir('hw')
+ subdir('tcg')
++subdir('fpu')
+ subdir('accel')
+ subdir('plugins')
+ subdir('bsd-user')
+diff --git a/fpu/meson.build b/fpu/meson.build
+new file mode 100644
+index XXXXXXX..XXXXXXX
+--- /dev/null
++++ b/fpu/meson.build
+@@ -0,0 +1 @@
++specific_ss.add(when: 'CONFIG_TCG', if_true: files('softfloat.c'))
+--
+.25.1

-New patch
+[PULL 03/34] tcg: Re-order tcg_region_init vs tcg_prologue_init
+Instead of delaying tcg_region_init until after tcg_prologue_init
+is complete, do tcg_region_init first and let tcg_prologue_init
+shrink the first region by the size of the generated prologue.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ accel/tcg/tcg-all.c       | 11 ---------
+ accel/tcg/translate-all.c |  3 +++
+ bsd-user/main.c           |  1 -
+ linux-user/main.c         |  1 -
+ tcg/tcg.c                 | 52 ++++++++++++++-------------------------
+files changed, 22 insertions(+), 46 deletions(-)
+diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
+index XXXXXXX..XXXXXXX 100644
+--- a/accel/tcg/tcg-all.c
++++ b/accel/tcg/tcg-all.c
+@@ -XXX,XX +XXX,XX @@ static int tcg_init(MachineState *ms)
+     tcg_exec_init(s->tb_size * 1024 * 1024, s->splitwx_enabled);
+     mttcg_enabled = s->mttcg_enabled;
+-
+-    /*
+-     * Initialize TCG regions only for softmmu.
+-     *
+-     * This needs to be done later for user mode, because the prologue
+-     * generation needs to be delayed so that GUEST_BASE is already set.
+-     */
+-#ifndef CONFIG_USER_ONLY
+-    tcg_region_init();
+-#endif /* !CONFIG_USER_ONLY */
+-
+     return 0;
+ }
+diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
+index XXXXXXX..XXXXXXX 100644
+--- a/accel/tcg/translate-all.c
++++ b/accel/tcg/translate-all.c
+@@ -XXX,XX +XXX,XX @@ void tcg_exec_init(unsigned long tb_size, int splitwx)
+                                splitwx, &error_fatal);
+     assert(ok);
++    /* TODO: allocating regions is hand-in-glove with code_gen_buffer. */
++    tcg_region_init();
++
+ #if defined(CONFIG_SOFTMMU)
+     /* There's no guest base to take into account, so go ahead and
+        initialize the prologue now.  */
+diff --git a/bsd-user/main.c b/bsd-user/main.c
+index XXXXXXX..XXXXXXX 100644
+--- a/bsd-user/main.c
++++ b/bsd-user/main.c
+@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
+      * the real value of GUEST_BASE into account.
+      */
+     tcg_prologue_init(tcg_ctx);
+-    tcg_region_init();
+     /* build Task State */
+     memset(ts, 0, sizeof(TaskState));
+diff --git a/linux-user/main.c b/linux-user/main.c
+index XXXXXXX..XXXXXXX 100644
+--- a/linux-user/main.c
++++ b/linux-user/main.c
+@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv, char **envp)
+        generating the prologue until now so that the prologue can take
+        the real value of GUEST_BASE into account.  */
+     tcg_prologue_init(tcg_ctx);
+-    tcg_region_init();
+     target_cpu_copy_regs(env, regs);
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@ TranslationBlock *tcg_tb_alloc(TCGContext *s)
+ void tcg_prologue_init(TCGContext *s)
+ {
+-    size_t prologue_size, total_size;
+-    void *buf0, *buf1;
++    size_t prologue_size;
+     /* Put the prologue at the beginning of code_gen_buffer.  */
+-    buf0 = s->code_gen_buffer;
+-    total_size = s->code_gen_buffer_size;
+-    s->code_ptr = buf0;
+-    s->code_buf = buf0;
++    tcg_region_assign(s, 0);
++    s->code_ptr = s->code_gen_ptr;
++    s->code_buf = s->code_gen_ptr;
+     s->data_gen_ptr = NULL;
+-    /*
+-     * The region trees are not yet configured, but tcg_splitwx_to_rx
+-     * needs the bounds for an assert.
+-     */
+-    region.start = buf0;
+-    region.end = buf0 + total_size;
+-
+ #ifndef CONFIG_TCG_INTERPRETER
+-    tcg_qemu_tb_exec = (tcg_prologue_fn *)tcg_splitwx_to_rx(buf0);
++    tcg_qemu_tb_exec = (tcg_prologue_fn *)tcg_splitwx_to_rx(s->code_ptr);
+ #endif
+-    /* Compute a high-water mark, at which we voluntarily flush the buffer
+-       and start over.  The size here is arbitrary, significantly larger
+-       than we expect the code generation for any one opcode to require.  */
+-    s->code_gen_highwater = s->code_gen_buffer + (total_size - TCG_HIGHWATER);
+-
+ #ifdef TCG_TARGET_NEED_POOL_LABELS
+     s->pool_labels = NULL;
+ #endif
+@@ -XXX,XX +XXX,XX @@ void tcg_prologue_init(TCGContext *s)
+     }
+ #endif
+-    buf1 = s->code_ptr;
++    prologue_size = tcg_current_code_size(s);
++
+ #ifndef CONFIG_TCG_INTERPRETER
+-    flush_idcache_range((uintptr_t)tcg_splitwx_to_rx(buf0), (uintptr_t)buf0,
+-                        tcg_ptr_byte_diff(buf1, buf0));
++    flush_idcache_range((uintptr_t)tcg_splitwx_to_rx(s->code_buf),
++                        (uintptr_t)s->code_buf, prologue_size);
+ #endif
+-    /* Deduct the prologue from the buffer.  */
+-    prologue_size = tcg_current_code_size(s);
+-    s->code_gen_ptr = buf1;
+-    s->code_gen_buffer = buf1;
+-    s->code_buf = buf1;
+-    total_size -= prologue_size;
+-    s->code_gen_buffer_size = total_size;
++    /* Deduct the prologue from the first region.  */
++    region.start = s->code_ptr;
+-    tcg_register_jit(tcg_splitwx_to_rx(s->code_gen_buffer), total_size);
++    /* Recompute boundaries of the first region. */
++    tcg_region_assign(s, 0);
++
++    tcg_register_jit(tcg_splitwx_to_rx(region.start),
++                     region.end - region.start);
+ #ifdef DEBUG_DISAS
+     if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM)) {
+         FILE *logfile = qemu_log_lock();
+         qemu_log("PROLOGUE: [size=%zu]\n", prologue_size);
+         if (s->data_gen_ptr) {
+-            size_t code_size = s->data_gen_ptr - buf0;
++            size_t code_size = s->data_gen_ptr - s->code_gen_ptr;
+             size_t data_size = prologue_size - code_size;
+             size_t i;
+-            log_disas(buf0, code_size);
++            log_disas(s->code_gen_ptr, code_size);
+             for (i = 0; i < data_size; i += sizeof(tcg_target_ulong)) {
+                 if (sizeof(tcg_target_ulong) == 8) {
+@@ -XXX,XX +XXX,XX @@ void tcg_prologue_init(TCGContext *s)
+                 }
+             }
+         } else {
+-            log_disas(buf0, prologue_size);
++            log_disas(s->code_gen_ptr, prologue_size);
+         }
+         qemu_log("\n");
+         qemu_log_flush();
+--
+.25.1

-New patch
+[PULL 04/34] tcg: Remove error return from tcg_region_initial_alloc__locked
+All callers immediately assert on error, so move the assert
+into the function itself.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/tcg.c | 19 ++++++-------------
+file changed, 6 insertions(+), 13 deletions(-)
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@ static bool tcg_region_alloc(TCGContext *s)
+  * Perform a context's first region allocation.
+  * This function does _not_ increment region.agg_size_full.
+  */
+-static inline bool tcg_region_initial_alloc__locked(TCGContext *s)
++static void tcg_region_initial_alloc__locked(TCGContext *s)
+ {
+-    return tcg_region_alloc__locked(s);
++    bool err = tcg_region_alloc__locked(s);
++    g_assert(!err);
+ }
+ /* Call from a safe-work context */
+@@ -XXX,XX +XXX,XX @@ void tcg_region_reset_all(void)
+     for (i = 0; i < n_ctxs; i++) {
+         TCGContext *s = qatomic_read(&tcg_ctxs[i]);
+-        bool err = tcg_region_initial_alloc__locked(s);
+-
+-        g_assert(!err);
++        tcg_region_initial_alloc__locked(s);
+     }
+     qemu_mutex_unlock(&region.lock);
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(void)
+     /* In user-mode we support only one ctx, so do the initial allocation now */
+ #ifdef CONFIG_USER_ONLY
+-    {
+-        bool err = tcg_region_initial_alloc__locked(tcg_ctx);
+-
+-        g_assert(!err);
+-    }
++    tcg_region_initial_alloc__locked(tcg_ctx);
+ #endif
+ }
+@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
+     MachineState *ms = MACHINE(qdev_get_machine());
+     TCGContext *s = g_malloc(sizeof(*s));
+     unsigned int i, n;
+-    bool err;
+     *s = tcg_init_ctx;
+@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
+     tcg_ctx = s;
+     qemu_mutex_lock(&region.lock);
+-    err = tcg_region_initial_alloc__locked(tcg_ctx);
+-    g_assert(!err);
++    tcg_region_initial_alloc__locked(s);
+     qemu_mutex_unlock(&region.lock);
+ }
+ #endif /* !CONFIG_USER_ONLY */
+--
+.25.1

-New patch
+[PULL 05/34] tcg: Split out tcg_region_initial_alloc
+This has only one user, and currently needs an ifdef,
+but will make more sense after some code motion.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/tcg.c | 13 ++++++++++---
+file changed, 10 insertions(+), 3 deletions(-)
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@ static void tcg_region_initial_alloc__locked(TCGContext *s)
+     g_assert(!err);
+ }
++#ifndef CONFIG_USER_ONLY
++static void tcg_region_initial_alloc(TCGContext *s)
++{
++    qemu_mutex_lock(&region.lock);
++    tcg_region_initial_alloc__locked(s);
++    qemu_mutex_unlock(&region.lock);
++}
++#endif
++
+ /* Call from a safe-work context */
+ void tcg_region_reset_all(void)
+ {
+@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
+     }
+     tcg_ctx = s;
+-    qemu_mutex_lock(&region.lock);
+-    tcg_region_initial_alloc__locked(s);
+-    qemu_mutex_unlock(&region.lock);
++    tcg_region_initial_alloc(s);
+ }
+ #endif /* !CONFIG_USER_ONLY */
+--
+.25.1

-New patch
+[PULL 06/34] tcg: Split out tcg_region_prologue_set
+This has only one user, but will make more sense after some
+code motion.
+Always leave the tcg_init_ctx initialized to the first region,
+in preparation for tcg_prologue_init().  This also requires
+that we don't re-allocate the region for the first cpu, lest
+we hit the assertion for total number of regions allocated .
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/tcg.c | 37 ++++++++++++++++++++++---------------
+file changed, 22 insertions(+), 15 deletions(-)
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(void)
+     tcg_region_trees_init();
+-    /* In user-mode we support only one ctx, so do the initial allocation now */
+-#ifdef CONFIG_USER_ONLY
+-    tcg_region_initial_alloc__locked(tcg_ctx);
+-#endif
++    /*
++     * Leave the initial context initialized to the first region.
++     * This will be the context into which we generate the prologue.
++     * It is also the only context for CONFIG_USER_ONLY.
++     */
++    tcg_region_initial_alloc__locked(&tcg_init_ctx);
++}
++
++static void tcg_region_prologue_set(TCGContext *s)
++{
++    /* Deduct the prologue from the first region.  */
++    g_assert(region.start == s->code_gen_buffer);
++    region.start = s->code_ptr;
++
++    /* Recompute boundaries of the first region. */
++    tcg_region_assign(s, 0);
++
++    /* Register the balance of the buffer with gdb. */
++    tcg_register_jit(tcg_splitwx_to_rx(region.start),
++                     region.end - region.start);
+ }
+ #ifdef CONFIG_DEBUG_TCG
+@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
+     if (n > 0) {
+         alloc_tcg_plugin_context(s);
++        tcg_region_initial_alloc(s);
+     }
+     tcg_ctx = s;
+-    tcg_region_initial_alloc(s);
+ }
+ #endif /* !CONFIG_USER_ONLY */
+@@ -XXX,XX +XXX,XX @@ void tcg_prologue_init(TCGContext *s)
+ {
+     size_t prologue_size;
+-    /* Put the prologue at the beginning of code_gen_buffer.  */
+-    tcg_region_assign(s, 0);
+     s->code_ptr = s->code_gen_ptr;
+     s->code_buf = s->code_gen_ptr;
+     s->data_gen_ptr = NULL;
+@@ -XXX,XX +XXX,XX @@ void tcg_prologue_init(TCGContext *s)
+                         (uintptr_t)s->code_buf, prologue_size);
+ #endif
+-    /* Deduct the prologue from the first region.  */
+-    region.start = s->code_ptr;
+-
+-    /* Recompute boundaries of the first region. */
+-    tcg_region_assign(s, 0);
+-
+-    tcg_register_jit(tcg_splitwx_to_rx(region.start),
+-                     region.end - region.start);
++    tcg_region_prologue_set(s);
+ #ifdef DEBUG_DISAS
+     if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM)) {
+--
+.25.1

-[PULL 1/3] accel/tcg: split CpusAccel into three TCG variants
+[PULL 07/34] tcg: Split out region.c
-From: Claudio Fontana <cfontana@suse.de>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 split up the CpusAccel tcg_cpus into three TCG variants:
 tcg_cpus_rr (single threaded, round robin cpus)
 tcg_cpus_icount (same as rr, but with instruction counting enabled)
 tcg_cpus_mttcg (multi-threaded cpus)
 Suggested-by: Richard Henderson <richard.henderson@linaro.org>
 Signed-off-by: Claudio Fontana <cfontana@suse.de>
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Message-Id: <20201015143217.29337-2-cfontana@suse.de>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/tcg-cpus-icount.h |  17 ++
+ tcg/tcg-internal.h |  37 +++
- accel/tcg/tcg-cpus-mttcg.h  |  21 ++
+ tcg/region.c       | 572 +++++++++++++++++++++++++++++++++++++++++++++
- accel/tcg/tcg-cpus-rr.h     |  20 ++
+ tcg/tcg.c          | 547 +------------------------------------------
- accel/tcg/tcg-cpus.h        |  13 +-
+ tcg/meson.build    |   1 +
- accel/tcg/tcg-all.c         |   8 +-
+files changed, 613 insertions(+), 544 deletions(-)
- accel/tcg/tcg-cpus-icount.c | 147 +++++++++++
+ create mode 100644 tcg/tcg-internal.h
- accel/tcg/tcg-cpus-mttcg.c  | 117 +++++++++
+ create mode 100644 tcg/region.c
  accel/tcg/tcg-cpus-rr.c     | 270 ++++++++++++++++++++
  accel/tcg/tcg-cpus.c        | 484 ++----------------------------------
  softmmu/icount.c            |   2 +-
  accel/tcg/meson.build       |   9 +-
 files changed, 646 insertions(+), 462 deletions(-)
  create mode 100644 accel/tcg/tcg-cpus-icount.h
  create mode 100644 accel/tcg/tcg-cpus-mttcg.h
  create mode 100644 accel/tcg/tcg-cpus-rr.h
  create mode 100644 accel/tcg/tcg-cpus-icount.c
  create mode 100644 accel/tcg/tcg-cpus-mttcg.c
  create mode 100644 accel/tcg/tcg-cpus-rr.c
-diff --git a/accel/tcg/tcg-cpus-icount.h b/accel/tcg/tcg-cpus-icount.h
+diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
-+++ b/accel/tcg/tcg-cpus-icount.h
++++ b/tcg/tcg-internal.h
 @@ -XXX,XX +XXX,XX @@
 +/*
-+ * QEMU TCG Single Threaded vCPUs implementation using instruction counting
++ * Internal declarations for Tiny Code Generator for QEMU
 + *
-+ * Copyright 2020 SUSE LLC
++ * Copyright (c) 2008 Fabrice Bellard
 + *
 + * This work is licensed under the terms of the GNU GPL, version 2 or later.
 + * See the COPYING file in the top-level directory.
 + */
 +
 +#ifndef TCG_CPUS_ICOUNT_H
 +#define TCG_CPUS_ICOUNT_H
 +
 +void handle_icount_deadline(void);
 +void prepare_icount_for_run(CPUState *cpu);
 +void process_icount_data(CPUState *cpu);
 +
 +#endif /* TCG_CPUS_ICOUNT_H */
 diff --git a/accel/tcg/tcg-cpus-mttcg.h b/accel/tcg/tcg-cpus-mttcg.h
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/accel/tcg/tcg-cpus-mttcg.h
@@ -XXX,XX +XXX,XX @@
 +/*
 + * QEMU TCG Multi Threaded vCPUs implementation
 + *
 + * Copyright 2020 SUSE LLC
 + *
 + * This work is licensed under the terms of the GNU GPL, version 2 or later.
 + * See the COPYING file in the top-level directory.
 + */
 +
 +#ifndef TCG_CPUS_MTTCG_H
 +#define TCG_CPUS_MTTCG_H
 +
 +/*
 + * In the multi-threaded case each vCPU has its own thread. The TLS
 + * variable current_cpu can be used deep in the code to find the
 + * current CPUState for a given thread.
 + */
 +
 +void *tcg_cpu_thread_fn(void *arg);
 +
 +#endif /* TCG_CPUS_MTTCG_H */
 diff --git a/accel/tcg/tcg-cpus-rr.h b/accel/tcg/tcg-cpus-rr.h
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/accel/tcg/tcg-cpus-rr.h
@@ -XXX,XX +XXX,XX @@
 +/*
 + * QEMU TCG Single Threaded vCPUs implementation
 + *
 + * Copyright 2020 SUSE LLC
 + *
 + * This work is licensed under the terms of the GNU GPL, version 2 or later.
 + * See the COPYING file in the top-level directory.
 + */
 +
 +#ifndef TCG_CPUS_RR_H
 +#define TCG_CPUS_RR_H
 +
 +#define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
 +
 +/* Kick all RR vCPUs. */
 +void qemu_cpu_kick_rr_cpus(CPUState *unused);
 +
 +void *tcg_rr_cpu_thread_fn(void *arg);
 +
 +#endif /* TCG_CPUS_RR_H */
 diff --git a/accel/tcg/tcg-cpus.h b/accel/tcg/tcg-cpus.h
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus.h
 +++ b/accel/tcg/tcg-cpus.h
@@ -XXX,XX +XXX,XX @@
  /*
 - * Accelerator CPUS Interface
 + * QEMU TCG vCPU common functionality
 + *
 + * Functionality common to all TCG vcpu variants: mttcg, rr and icount.
   *
   * Copyright 2020 SUSE LLC
   *
@@ -XXX,XX +XXX,XX @@
  #include "sysemu/cpus.h"
 -extern const CpusAccel tcg_cpus;
 +extern const CpusAccel tcg_cpus_mttcg;
 +extern const CpusAccel tcg_cpus_icount;
 +extern const CpusAccel tcg_cpus_rr;
 +
 +void tcg_start_vcpu_thread(CPUState *cpu);
 +void qemu_tcg_destroy_vcpu(CPUState *cpu);
 +int tcg_cpu_exec(CPUState *cpu);
 +void tcg_handle_interrupt(CPUState *cpu, int mask);
  #endif /* TCG_CPUS_H */
 diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-all.c
 +++ b/accel/tcg/tcg-all.c
@@ -XXX,XX +XXX,XX @@ static int tcg_init(MachineState *ms)
      tcg_exec_init(s->tb_size * 1024 * 1024);
      mttcg_enabled = s->mttcg_enabled;
 -    cpus_register_accel(&tcg_cpus);
 +    if (mttcg_enabled) {
 +        cpus_register_accel(&tcg_cpus_mttcg);
 +    } else if (icount_enabled()) {
 +        cpus_register_accel(&tcg_cpus_icount);
 +    } else {
 +        cpus_register_accel(&tcg_cpus_rr);
 +    }
      return 0;
  }
 diff --git a/accel/tcg/tcg-cpus-icount.c b/accel/tcg/tcg-cpus-icount.c
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/accel/tcg/tcg-cpus-icount.c
@@ -XXX,XX +XXX,XX @@
 +/*
 + * QEMU TCG Single Threaded vCPUs implementation using instruction counting
 + *
 + * Copyright (c) 2003-2008 Fabrice Bellard
 + * Copyright (c) 2014 Red Hat Inc.
 + *
 + * Permission is hereby granted, free of charge, to any person obtaining a copy
 + * of this software and associated documentation files (the "Software"), to deal
 + * in the Software without restriction, including without limitation the rights
 + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 ...
 + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 + * THE SOFTWARE.
 + */
 +
-+#include "qemu/osdep.h"
++#ifndef TCG_INTERNAL_H
-+#include "qemu-common.h"
++#define TCG_INTERNAL_H 1
-+#include "sysemu/tcg.h"
++
-+#include "sysemu/replay.h"
++#define TCG_HIGHWATER 1024
-+#include "qemu/main-loop.h"
++
-+#include "qemu/guest-random.h"
++extern TCGContext **tcg_ctxs;
-+#include "exec/exec-all.h"
++extern unsigned int n_tcg_ctxs;
-+#include "hw/boards.h"
++
-+
++bool tcg_region_alloc(TCGContext *s);
-+#include "tcg-cpus.h"
++void tcg_region_initial_alloc(TCGContext *s);
-+#include "tcg-cpus-icount.h"
++void tcg_region_prologue_set(TCGContext *s);
-+#include "tcg-cpus-rr.h"
++
-+
++#endif /* TCG_INTERNAL_H */
-+static int64_t tcg_get_icount_limit(void)
+diff --git a/tcg/region.c b/tcg/region.c
 +{
 +    int64_t deadline;
 +
 +    if (replay_mode != REPLAY_MODE_PLAY) {
 +        /*
 +         * Include all the timers, because they may need an attention.
 +         * Too long CPU execution may create unnecessary delay in UI.
 +         */
 +        deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
 +                                              QEMU_TIMER_ATTR_ALL);
 +        /* Check realtime timers, because they help with input processing */
 +        deadline = qemu_soonest_timeout(deadline,
 +                qemu_clock_deadline_ns_all(QEMU_CLOCK_REALTIME,
 +                                           QEMU_TIMER_ATTR_ALL));
 +
 +        /*
 +         * Maintain prior (possibly buggy) behaviour where if no deadline
 +         * was set (as there is no QEMU_CLOCK_VIRTUAL timer) or it is more than
 +         * INT32_MAX nanoseconds ahead, we still use INT32_MAX
 +         * nanoseconds.
 +         */
 +        if ((deadline < 0) || (deadline > INT32_MAX)) {
 +            deadline = INT32_MAX;
 +        }
 +
 +        return icount_round(deadline);
 +    } else {
 +        return replay_get_instructions();
 +    }
 +}
 +
 +static void notify_aio_contexts(void)
 +{
 +    /* Wake up other AioContexts.  */
 +    qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
 +    qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL);
 +}
 +
 +void handle_icount_deadline(void)
 +{
 +    assert(qemu_in_vcpu_thread());
 +    int64_t deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
 +                                                  QEMU_TIMER_ATTR_ALL);
 +
 +    if (deadline == 0) {
 +        notify_aio_contexts();
 +    }
 +}
 +
 +void prepare_icount_for_run(CPUState *cpu)
 +{
 +    int insns_left;
 +
 +    /*
 +     * These should always be cleared by process_icount_data after
 +     * each vCPU execution. However u16.high can be raised
 +     * asynchronously by cpu_exit/cpu_interrupt/tcg_handle_interrupt
 +     */
 +    g_assert(cpu_neg(cpu)->icount_decr.u16.low == 0);
 +    g_assert(cpu->icount_extra == 0);
 +
 +    cpu->icount_budget = tcg_get_icount_limit();
 +    insns_left = MIN(0xffff, cpu->icount_budget);
 +    cpu_neg(cpu)->icount_decr.u16.low = insns_left;
 +    cpu->icount_extra = cpu->icount_budget - insns_left;
 +
 +    replay_mutex_lock();
 +
 +    if (cpu->icount_budget == 0 && replay_has_checkpoint()) {
 +        notify_aio_contexts();
 +    }
 +}
 +
 +void process_icount_data(CPUState *cpu)
 +{
 +    /* Account for executed instructions */
 +    icount_update(cpu);
 +
 +    /* Reset the counters */
 +    cpu_neg(cpu)->icount_decr.u16.low = 0;
 +    cpu->icount_extra = 0;
 +    cpu->icount_budget = 0;
 +
 +    replay_account_executed_instructions();
 +
 +    replay_mutex_unlock();
 +}
 +
 +static void icount_handle_interrupt(CPUState *cpu, int mask)
 +{
 +    int old_mask = cpu->interrupt_request;
 +
 +    tcg_handle_interrupt(cpu, mask);
 +    if (qemu_cpu_is_self(cpu) &&
 +        !cpu->can_do_io
 +        && (mask & ~old_mask) != 0) {
 +        cpu_abort(cpu, "Raised interrupt while not in I/O function");
 +    }
 +}
 +
 +const CpusAccel tcg_cpus_icount = {
 +    .create_vcpu_thread = tcg_start_vcpu_thread,
 +    .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
 +
 +    .handle_interrupt = icount_handle_interrupt,
 +    .get_virtual_clock = icount_get,
 +    .get_elapsed_ticks = icount_get,
 +};
 diff --git a/accel/tcg/tcg-cpus-mttcg.c b/accel/tcg/tcg-cpus-mttcg.c
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
-+++ b/accel/tcg/tcg-cpus-mttcg.c
++++ b/tcg/region.c
 @@ -XXX,XX +XXX,XX @@
 +/*
-+ * QEMU TCG Multi Threaded vCPUs implementation
++ * Memory region management for Tiny Code Generator for QEMU
 + *
-+ * Copyright (c) 2003-2008 Fabrice Bellard
++ * Copyright (c) 2008 Fabrice Bellard
 + * Copyright (c) 2014 Red Hat Inc.
 + *
 + * Permission is hereby granted, free of charge, to any person obtaining a copy
 + * of this software and associated documentation files (the "Software"), to deal
 + * in the Software without restriction, including without limitation the rights
 + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 ...
 + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 + * THE SOFTWARE.
 + */
 +
 +#include "qemu/osdep.h"
-+#include "qemu-common.h"
-+#include "sysemu/tcg.h"
-+#include "sysemu/replay.h"
-+#include "qemu/main-loop.h"
-+#include "qemu/guest-random.h"
 +#include "exec/exec-all.h"
++#include "tcg/tcg.h"
++#if !defined(CONFIG_USER_ONLY)
 +#include "hw/boards.h"
-+
++#endif
-+#include "tcg-cpus.h"
++#include "tcg-internal.h"
-+#include "tcg-cpus-mttcg.h"
++
 +
 +struct tcg_region_tree {
 +    QemuMutex lock;
 +    GTree *tree;
 +    /* padding to avoid false sharing is computed at run-time */
 +};
 +
 +/*
-+ * In the multi-threaded case each vCPU has its own thread. The TLS
++ * We divide code_gen_buffer into equally-sized "regions" that TCG threads
-+ * variable current_cpu can be used deep in the code to find the
++ * dynamically allocate from as demand dictates. Given appropriate region
-+ * current CPUState for a given thread.
++ * sizing, this minimizes flushes even when some TCG threads generate a lot
 + * more code than others.
 + */
-+
++struct tcg_region_state {
-+void *tcg_cpu_thread_fn(void *arg)
++    QemuMutex lock;
-+{
++
-+    CPUState *cpu = arg;
++    /* fields set at init time */
-+
++    void *start;
-+    assert(tcg_enabled());
++    void *start_aligned;
-+    g_assert(!icount_enabled());
++    void *end;
-+
++    size_t n;
-+    rcu_register_thread();
++    size_t size; /* size of one region */
-+    tcg_register_thread();
++    size_t stride; /* .size + guard size */
 +
-+    qemu_mutex_lock_iothread();
++    /* fields protected by the lock */
-+    qemu_thread_get_self(cpu->thread);
++    size_t current; /* current region index */
-+
++    size_t agg_size_full; /* aggregate size of full regions */
-+    cpu->thread_id = qemu_get_thread_id();
++};
-+    cpu->can_do_io = 1;
++
-+    current_cpu = cpu;
++static struct tcg_region_state region;
-+    cpu_thread_signal_created(cpu);
++
-+    qemu_guest_random_seed_thread_part2(cpu->random_seed);
++/*
-+
++ * This is an array of struct tcg_region_tree's, with padding.
-+    /* process any pending work */
++ * We use void * to simplify the computation of region_trees[i]; each
-+    cpu->exit_request = 1;
++ * struct is found every tree_size bytes.
-+
++ */
-+    do {
++static void *region_trees;
-+        if (cpu_can_run(cpu)) {
++static size_t tree_size;
-+            int r;
++
-+            qemu_mutex_unlock_iothread();
++/* compare a pointer @ptr and a tb_tc @s */
-+            r = tcg_cpu_exec(cpu);
++static int ptr_cmp_tb_tc(const void *ptr, const struct tb_tc *s)
-+            qemu_mutex_lock_iothread();
++{
-+            switch (r) {
++    if (ptr >= s->ptr + s->size) {
-+            case EXCP_DEBUG:
++        return 1;
-+                cpu_handle_guest_debug(cpu);
++    } else if (ptr < s->ptr) {
-+                break;
++        return -1;
-+            case EXCP_HALTED:
++    }
-+                /*
++    return 0;
-+                 * during start-up the vCPU is reset and the thread is
++}
-+                 * kicked several times. If we don't ensure we go back
++
-+                 * to sleep in the halted state we won't cleanly
++static gint tb_tc_cmp(gconstpointer ap, gconstpointer bp)
-+                 * start-up when the vCPU is enabled.
++{
-+                 *
++    const struct tb_tc *a = ap;
-+                 * cpu->halted should ensure we sleep in wait_io_event
++    const struct tb_tc *b = bp;
-+                 */
++
-+                g_assert(cpu->halted);
++    /*
-+                break;
++     * When both sizes are set, we know this isn't a lookup.
-+            case EXCP_ATOMIC:
++     * This is the most likely case: every TB must be inserted; lookups
-+                qemu_mutex_unlock_iothread();
++     * are a lot less frequent.
-+                cpu_exec_step_atomic(cpu);
++     */
-+                qemu_mutex_lock_iothread();
++    if (likely(a->size && b->size)) {
-+            default:
++        if (a->ptr > b->ptr) {
-+                /* Ignore everything else? */
++            return 1;
-+                break;
++        } else if (a->ptr < b->ptr) {
-+            }
++            return -1;
 +        }
-+
++        /* a->ptr == b->ptr should happen only on deletions */
-+        qatomic_mb_set(&cpu->exit_request, 0);
++        g_assert(a->size == b->size);
-+        qemu_wait_io_event(cpu);
++        return 0;
-+    } while (!cpu->unplug || cpu_can_run(cpu));
++    }
-+
++    /*
-+    qemu_tcg_destroy_vcpu(cpu);
++     * All lookups have either .size field set to 0.
-+    qemu_mutex_unlock_iothread();
++     * From the glib sources we see that @ap is always the lookup key. However
-+    rcu_unregister_thread();
++     * the docs provide no guarantee, so we just mark this case as likely.
-+    return NULL;
++     */
-+}
++    if (likely(a->size == 0)) {
-+
++        return ptr_cmp_tb_tc(a->ptr, b);
-+static void mttcg_kick_vcpu_thread(CPUState *cpu)
++    }
-+{
++    return ptr_cmp_tb_tc(b->ptr, a);
-+    cpu_exit(cpu);
++}
-+}
++
-+
++static void tcg_region_trees_init(void)
-+const CpusAccel tcg_cpus_mttcg = {
++{
-+    .create_vcpu_thread = tcg_start_vcpu_thread,
++    size_t i;
-+    .kick_vcpu_thread = mttcg_kick_vcpu_thread,
++
-+
++    tree_size = ROUND_UP(sizeof(struct tcg_region_tree), qemu_dcache_linesize);
-+    .handle_interrupt = tcg_handle_interrupt,
++    region_trees = qemu_memalign(qemu_dcache_linesize, region.n * tree_size);
-+};
++    for (i = 0; i < region.n; i++) {
-diff --git a/accel/tcg/tcg-cpus-rr.c b/accel/tcg/tcg-cpus-rr.c
++        struct tcg_region_tree *rt = region_trees + i * tree_size;
-new file mode 100644
++
-index XXXXXXX..XXXXXXX
++        qemu_mutex_init(&rt->lock);
---- /dev/null
++        rt->tree = g_tree_new(tb_tc_cmp);
-+++ b/accel/tcg/tcg-cpus-rr.c
++    }
 +}
 +
 +static struct tcg_region_tree *tc_ptr_to_region_tree(const void *p)
 +{
 +    size_t region_idx;
 +
 +    /*
 +     * Like tcg_splitwx_to_rw, with no assert.  The pc may come from
 +     * a signal handler over which the caller has no control.
 +     */
 +    if (!in_code_gen_buffer(p)) {
 +        p -= tcg_splitwx_diff;
 +        if (!in_code_gen_buffer(p)) {
 +            return NULL;
 +        }
 +    }
 +
 +    if (p < region.start_aligned) {
 +        region_idx = 0;
 +    } else {
 +        ptrdiff_t offset = p - region.start_aligned;
 +
 +        if (offset > region.stride * (region.n - 1)) {
 +            region_idx = region.n - 1;
 +        } else {
 +            region_idx = offset / region.stride;
 +        }
 +    }
 +    return region_trees + region_idx * tree_size;
 +}
 +
 +void tcg_tb_insert(TranslationBlock *tb)
 +{
 +    struct tcg_region_tree *rt = tc_ptr_to_region_tree(tb->tc.ptr);
 +
 +    g_assert(rt != NULL);
 +    qemu_mutex_lock(&rt->lock);
 +    g_tree_insert(rt->tree, &tb->tc, tb);
 +    qemu_mutex_unlock(&rt->lock);
 +}
 +
 +void tcg_tb_remove(TranslationBlock *tb)
 +{
 +    struct tcg_region_tree *rt = tc_ptr_to_region_tree(tb->tc.ptr);
 +
 +    g_assert(rt != NULL);
 +    qemu_mutex_lock(&rt->lock);
 +    g_tree_remove(rt->tree, &tb->tc);
 +    qemu_mutex_unlock(&rt->lock);
 +}
 +
 +/*
 + * Find the TB 'tb' such that
 + * tb->tc.ptr <= tc_ptr < tb->tc.ptr + tb->tc.size
 + * Return NULL if not found.
 + */
 +TranslationBlock *tcg_tb_lookup(uintptr_t tc_ptr)
 +{
 +    struct tcg_region_tree *rt = tc_ptr_to_region_tree((void *)tc_ptr);
 +    TranslationBlock *tb;
 +    struct tb_tc s = { .ptr = (void *)tc_ptr };
 +
 +    if (rt == NULL) {
 +        return NULL;
 +    }
 +
 +    qemu_mutex_lock(&rt->lock);
 +    tb = g_tree_lookup(rt->tree, &s);
 +    qemu_mutex_unlock(&rt->lock);
 +    return tb;
 +}
 +
 +static void tcg_region_tree_lock_all(void)
 +{
 +    size_t i;
 +
 +    for (i = 0; i < region.n; i++) {
 +        struct tcg_region_tree *rt = region_trees + i * tree_size;
 +
 +        qemu_mutex_lock(&rt->lock);
 +    }
 +}
 +
 +static void tcg_region_tree_unlock_all(void)
 +{
 +    size_t i;
 +
 +    for (i = 0; i < region.n; i++) {
 +        struct tcg_region_tree *rt = region_trees + i * tree_size;
 +
 +        qemu_mutex_unlock(&rt->lock);
 +    }
 +}
 +
 +void tcg_tb_foreach(GTraverseFunc func, gpointer user_data)
 +{
 +    size_t i;
 +
 +    tcg_region_tree_lock_all();
 +    for (i = 0; i < region.n; i++) {
 +        struct tcg_region_tree *rt = region_trees + i * tree_size;
 +
 +        g_tree_foreach(rt->tree, func, user_data);
 +    }
 +    tcg_region_tree_unlock_all();
 +}
 +
 +size_t tcg_nb_tbs(void)
 +{
 +    size_t nb_tbs = 0;
 +    size_t i;
 +
 +    tcg_region_tree_lock_all();
 +    for (i = 0; i < region.n; i++) {
 +        struct tcg_region_tree *rt = region_trees + i * tree_size;
 +
 +        nb_tbs += g_tree_nnodes(rt->tree);
 +    }
 +    tcg_region_tree_unlock_all();
 +    return nb_tbs;
 +}
 +
 +static gboolean tcg_region_tree_traverse(gpointer k, gpointer v, gpointer data)
 +{
 +    TranslationBlock *tb = v;
 +
 +    tb_destroy(tb);
 +    return FALSE;
 +}
 +
 +static void tcg_region_tree_reset_all(void)
 +{
 +    size_t i;
 +
 +    tcg_region_tree_lock_all();
 +    for (i = 0; i < region.n; i++) {
 +        struct tcg_region_tree *rt = region_trees + i * tree_size;
 +
 +        g_tree_foreach(rt->tree, tcg_region_tree_traverse, NULL);
 +        /* Increment the refcount first so that destroy acts as a reset */
 +        g_tree_ref(rt->tree);
 +        g_tree_destroy(rt->tree);
 +    }
 +    tcg_region_tree_unlock_all();
 +}
 +
 +static void tcg_region_bounds(size_t curr_region, void **pstart, void **pend)
 +{
 +    void *start, *end;
 +
 +    start = region.start_aligned + curr_region * region.stride;
 +    end = start + region.size;
 +
 +    if (curr_region == 0) {
 +        start = region.start;
 +    }
 +    if (curr_region == region.n - 1) {
 +        end = region.end;
 +    }
 +
 +    *pstart = start;
 +    *pend = end;
 +}
 +
 +static void tcg_region_assign(TCGContext *s, size_t curr_region)
 +{
 +    void *start, *end;
 +
 +    tcg_region_bounds(curr_region, &start, &end);
 +
 +    s->code_gen_buffer = start;
 +    s->code_gen_ptr = start;
 +    s->code_gen_buffer_size = end - start;
 +    s->code_gen_highwater = end - TCG_HIGHWATER;
 +}
 +
 +static bool tcg_region_alloc__locked(TCGContext *s)
 +{
 +    if (region.current == region.n) {
 +        return true;
 +    }
 +    tcg_region_assign(s, region.current);
 +    region.current++;
 +    return false;
 +}
 +
 +/*
 + * Request a new region once the one in use has filled up.
 + * Returns true on error.
 + */
 +bool tcg_region_alloc(TCGContext *s)
 +{
 +    bool err;
 +    /* read the region size now; alloc__locked will overwrite it on success */
 +    size_t size_full = s->code_gen_buffer_size;
 +
 +    qemu_mutex_lock(&region.lock);
 +    err = tcg_region_alloc__locked(s);
 +    if (!err) {
 +        region.agg_size_full += size_full - TCG_HIGHWATER;
 +    }
 +    qemu_mutex_unlock(&region.lock);
 +    return err;
 +}
 +
 +/*
 + * Perform a context's first region allocation.
 + * This function does _not_ increment region.agg_size_full.
 + */
 +static void tcg_region_initial_alloc__locked(TCGContext *s)
 +{
 +    bool err = tcg_region_alloc__locked(s);
 +    g_assert(!err);
 +}
 +
 +void tcg_region_initial_alloc(TCGContext *s)
 +{
 +    qemu_mutex_lock(&region.lock);
 +    tcg_region_initial_alloc__locked(s);
 +    qemu_mutex_unlock(&region.lock);
 +}
 +
 +/* Call from a safe-work context */
 +void tcg_region_reset_all(void)
 +{
 +    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
 +    unsigned int i;
 +
 +    qemu_mutex_lock(&region.lock);
 +    region.current = 0;
 +    region.agg_size_full = 0;
 +
 +    for (i = 0; i < n_ctxs; i++) {
 +        TCGContext *s = qatomic_read(&tcg_ctxs[i]);
 +        tcg_region_initial_alloc__locked(s);
 +    }
 +    qemu_mutex_unlock(&region.lock);
 +
 +    tcg_region_tree_reset_all();
 +}
 +
 +#ifdef CONFIG_USER_ONLY
 +static size_t tcg_n_regions(void)
 +{
 +    return 1;
 +}
 +#else
 +/*
 + * It is likely that some vCPUs will translate more code than others, so we
 + * first try to set more regions than max_cpus, with those regions being of
 + * reasonable size. If that's not possible we make do by evenly dividing
 + * the code_gen_buffer among the vCPUs.
 + */
 +static size_t tcg_n_regions(void)
 +{
 +    size_t i;
 +
 +    /* Use a single region if all we have is one vCPU thread */
 +#if !defined(CONFIG_USER_ONLY)
 +    MachineState *ms = MACHINE(qdev_get_machine());
 +    unsigned int max_cpus = ms->smp.max_cpus;
 +#endif
 +    if (max_cpus == 1 || !qemu_tcg_mttcg_enabled()) {
 +        return 1;
 +    }
 +
 +    /* Try to have more regions than max_cpus, with each region being >= 2 MB */
 +    for (i = 8; i > 0; i--) {
 +        size_t regions_per_thread = i;
 +        size_t region_size;
 +
 +        region_size = tcg_init_ctx.code_gen_buffer_size;
 +        region_size /= max_cpus * regions_per_thread;
 +
 +        if (region_size >= 2 * 1024u * 1024) {
 +            return max_cpus * regions_per_thread;
 +        }
 +    }
 +    /* If we can't, then just allocate one region per vCPU thread */
 +    return max_cpus;
 +}
 +#endif
 +
 +/*
 + * Initializes region partitioning.
 + *
 + * Called at init time from the parent thread (i.e. the one calling
 + * tcg_context_init), after the target's TCG globals have been set.
 + *
 + * Region partitioning works by splitting code_gen_buffer into separate regions,
 + * and then assigning regions to TCG threads so that the threads can translate
 + * code in parallel without synchronization.
 + *
 + * In softmmu the number of TCG threads is bounded by max_cpus, so we use at
 + * least max_cpus regions in MTTCG. In !MTTCG we use a single region.
 + * Note that the TCG options from the command-line (i.e. -accel accel=tcg,[...])
 + * must have been parsed before calling this function, since it calls
 + * qemu_tcg_mttcg_enabled().
 + *
 + * In user-mode we use a single region.  Having multiple regions in user-mode
 + * is not supported, because the number of vCPU threads (recall that each thread
 + * spawned by the guest corresponds to a vCPU thread) is only bounded by the
 + * OS, and usually this number is huge (tens of thousands is not uncommon).
 + * Thus, given this large bound on the number of vCPU threads and the fact
 + * that code_gen_buffer is allocated at compile-time, we cannot guarantee
 + * that the availability of at least one region per vCPU thread.
 + *
 + * However, this user-mode limitation is unlikely to be a significant problem
 + * in practice. Multi-threaded guests share most if not all of their translated
 + * code, which makes parallel code generation less appealing than in softmmu.
 + */
 +void tcg_region_init(void)
 +{
 +    void *buf = tcg_init_ctx.code_gen_buffer;
 +    void *aligned;
 +    size_t size = tcg_init_ctx.code_gen_buffer_size;
 +    size_t page_size = qemu_real_host_page_size;
 +    size_t region_size;
 +    size_t n_regions;
 +    size_t i;
 +
 +    n_regions = tcg_n_regions();
 +
 +    /* The first region will be 'aligned - buf' bytes larger than the others */
 +    aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
 +    g_assert(aligned < tcg_init_ctx.code_gen_buffer + size);
 +    /*
 +     * Make region_size a multiple of page_size, using aligned as the start.
 +     * As a result of this we might end up with a few extra pages at the end of
 +     * the buffer; we will assign those to the last region.
 +     */
 +    region_size = (size - (aligned - buf)) / n_regions;
 +    region_size = QEMU_ALIGN_DOWN(region_size, page_size);
 +
 +    /* A region must have at least 2 pages; one code, one guard */
 +    g_assert(region_size >= 2 * page_size);
 +
 +    /* init the region struct */
 +    qemu_mutex_init(&region.lock);
 +    region.n = n_regions;
 +    region.size = region_size - page_size;
 +    region.stride = region_size;
 +    region.start = buf;
 +    region.start_aligned = aligned;
 +    /* page-align the end, since its last page will be a guard page */
 +    region.end = QEMU_ALIGN_PTR_DOWN(buf + size, page_size);
 +    /* account for that last guard page */
 +    region.end -= page_size;
 +
 +    /*
 +     * Set guard pages in the rw buffer, as that's the one into which
 +     * buffer overruns could occur.  Do not set guard pages in the rx
 +     * buffer -- let that one use hugepages throughout.
 +     */
 +    for (i = 0; i < region.n; i++) {
 +        void *start, *end;
 +
 +        tcg_region_bounds(i, &start, &end);
 +
 +        /*
 +         * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
 +         * rejects a permission change from RWX -> NONE.  Guard pages are
 +         * nice for bug detection but are not essential; ignore any failure.
 +         */
 +        (void)qemu_mprotect_none(end, page_size);
 +    }
 +
 +    tcg_region_trees_init();
 +
 +    /*
 +     * Leave the initial context initialized to the first region.
 +     * This will be the context into which we generate the prologue.
 +     * It is also the only context for CONFIG_USER_ONLY.
 +     */
 +    tcg_region_initial_alloc__locked(&tcg_init_ctx);
 +}
 +
 +void tcg_region_prologue_set(TCGContext *s)
 +{
 +    /* Deduct the prologue from the first region.  */
 +    g_assert(region.start == s->code_gen_buffer);
 +    region.start = s->code_ptr;
 +
 +    /* Recompute boundaries of the first region. */
 +    tcg_region_assign(s, 0);
 +
 +    /* Register the balance of the buffer with gdb. */
 +    tcg_register_jit(tcg_splitwx_to_rx(region.start),
 +                     region.end - region.start);
 +}
 +
 +/*
 + * Returns the size (in bytes) of all translated code (i.e. from all regions)
 + * currently in the cache.
 + * See also: tcg_code_capacity()
 + * Do not confuse with tcg_current_code_size(); that one applies to a single
 + * TCG context.
 + */
 +size_t tcg_code_size(void)
 +{
 +    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
 +    unsigned int i;
 +    size_t total;
 +
 +    qemu_mutex_lock(&region.lock);
 +    total = region.agg_size_full;
 +    for (i = 0; i < n_ctxs; i++) {
 +        const TCGContext *s = qatomic_read(&tcg_ctxs[i]);
 +        size_t size;
 +
 +        size = qatomic_read(&s->code_gen_ptr) - s->code_gen_buffer;
 +        g_assert(size <= s->code_gen_buffer_size);
 +        total += size;
 +    }
 +    qemu_mutex_unlock(&region.lock);
 +    return total;
 +}
 +
 +/*
 + * Returns the code capacity (in bytes) of the entire cache, i.e. including all
 + * regions.
 + * See also: tcg_code_size()
 + */
 +size_t tcg_code_capacity(void)
 +{
 +    size_t guard_size, capacity;
 +
 +    /* no need for synchronization; these variables are set at init time */
 +    guard_size = region.stride - region.size;
 +    capacity = region.end + guard_size - region.start;
 +    capacity -= region.n * (guard_size + TCG_HIGHWATER);
 +    return capacity;
 +}
 +
 +size_t tcg_tb_phys_invalidate_count(void)
 +{
 +    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
 +    unsigned int i;
 +    size_t total = 0;
 +
 +    for (i = 0; i < n_ctxs; i++) {
 +        const TCGContext *s = qatomic_read(&tcg_ctxs[i]);
 +
 +        total += qatomic_read(&s->tb_phys_invalidate_count);
 +    }
 +    return total;
 +}
 diff --git a/tcg/tcg.c b/tcg/tcg.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg.c
 +++ b/tcg/tcg.c
 @@ -XXX,XX +XXX,XX @@
-+/*
-+ * QEMU TCG Single Threaded vCPUs implementation
+ #include "elf.h"
-+ *
+ #include "exec/log.h"
-+ * Copyright (c) 2003-2008 Fabrice Bellard
++#include "tcg-internal.h"
-+ * Copyright (c) 2014 Red Hat Inc.
-+ *
+ /* Forward declarations for functions declared in tcg-target.c.inc and
-+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+    used here. */
-+ * of this software and associated documentation files (the "Software"), to deal
+@@ -XXX,XX +XXX,XX @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct);
-+ * in the Software without restriction, including without limitation the rights
+ static int tcg_out_ldst_finalize(TCGContext *s);
-+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ #endif
-+ * copies of the Software, and to permit persons to whom the Software is
-+ * furnished to do so, subject to the following conditions:
+-#define TCG_HIGHWATER 1024
-+ *
+-
-+ * The above copyright notice and this permission notice shall be included in
+-static TCGContext **tcg_ctxs;
-+ * all copies or substantial portions of the Software.
+-static unsigned int n_tcg_ctxs;
-+ *
++TCGContext **tcg_ctxs;
-+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
++unsigned int n_tcg_ctxs;
-+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ TCGv_env cpu_env = 0;
-+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ const void *tcg_code_gen_epilogue;
-+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ uintptr_t tcg_splitwx_diff;
-+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+@@ -XXX,XX +XXX,XX @@ uintptr_t tcg_splitwx_diff;
-+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ tcg_prologue_fn *tcg_qemu_tb_exec;
-+ * THE SOFTWARE.
+ #endif
-+ */
-+
+-struct tcg_region_tree {
-+#include "qemu/osdep.h"
+-    QemuMutex lock;
-+#include "qemu-common.h"
+-    GTree *tree;
-+#include "sysemu/tcg.h"
+-    /* padding to avoid false sharing is computed at run-time */
-+#include "sysemu/replay.h"
+-};
-+#include "qemu/main-loop.h"
+-
-+#include "qemu/guest-random.h"
+-/*
-+#include "exec/exec-all.h"
+- * We divide code_gen_buffer into equally-sized "regions" that TCG threads
-+#include "hw/boards.h"
+- * dynamically allocate from as demand dictates. Given appropriate region
-+
+- * sizing, this minimizes flushes even when some TCG threads generate a lot
-+#include "tcg-cpus.h"
+- * more code than others.
-+#include "tcg-cpus-rr.h"
+- */
-+#include "tcg-cpus-icount.h"
+-struct tcg_region_state {
-+
+-    QemuMutex lock;
-+/* Kick all RR vCPUs */
+-
-+void qemu_cpu_kick_rr_cpus(CPUState *unused)
+-    /* fields set at init time */
-+{
+-    void *start;
-+    CPUState *cpu;
+-    void *start_aligned;
-+
+-    void *end;
-+    CPU_FOREACH(cpu) {
+-    size_t n;
-+        cpu_exit(cpu);
+-    size_t size; /* size of one region */
-+    };
+-    size_t stride; /* .size + guard size */
-+}
+-
-+
+-    /* fields protected by the lock */
-+/*
+-    size_t current; /* current region index */
-+ * TCG vCPU kick timer
+-    size_t agg_size_full; /* aggregate size of full regions */
-+ *
+-};
-+ * The kick timer is responsible for moving single threaded vCPU
+-
-+ * emulation on to the next vCPU. If more than one vCPU is running a
+-static struct tcg_region_state region;
-+ * timer event with force a cpu->exit so the next vCPU can get
+-/*
-+ * scheduled.
+- * This is an array of struct tcg_region_tree's, with padding.
-+ *
+- * We use void * to simplify the computation of region_trees[i]; each
-+ * The timer is removed if all vCPUs are idle and restarted again once
+- * struct is found every tree_size bytes.
-+ * idleness is complete.
+- */
-+ */
+-static void *region_trees;
-+
+-static size_t tree_size;
-+static QEMUTimer *tcg_kick_vcpu_timer;
+ static TCGRegSet tcg_target_available_regs[TCG_TYPE_COUNT];
-+static CPUState *tcg_current_rr_cpu;
+ static TCGRegSet tcg_target_call_clobber_regs;
-+
-+#define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
+@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef constraint_sets[] = {
-+
-+static inline int64_t qemu_tcg_next_kick(void)
+ #include "tcg-target.c.inc"
-+{
-+    return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + TCG_KICK_PERIOD;
+-/* compare a pointer @ptr and a tb_tc @s */
-+}
+-static int ptr_cmp_tb_tc(const void *ptr, const struct tb_tc *s)
-+
+-{
-+/* Kick the currently round-robin scheduled vCPU to next */
+-    if (ptr >= s->ptr + s->size) {
-+static void qemu_cpu_kick_rr_next_cpu(void)
+-        return 1;
-+{
+-    } else if (ptr < s->ptr) {
-+    CPUState *cpu;
+-        return -1;
-+    do {
+-    }
-+        cpu = qatomic_mb_read(&tcg_current_rr_cpu);
+-    return 0;
-+        if (cpu) {
+-}
-+            cpu_exit(cpu);
+-
-+        }
+-static gint tb_tc_cmp(gconstpointer ap, gconstpointer bp)
-+    } while (cpu != qatomic_mb_read(&tcg_current_rr_cpu));
+-{
-+}
+-    const struct tb_tc *a = ap;
-+
+-    const struct tb_tc *b = bp;
-+static void kick_tcg_thread(void *opaque)
+-
-+{
+-    /*
-+    timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
+-     * When both sizes are set, we know this isn't a lookup.
-+    qemu_cpu_kick_rr_next_cpu();
+-     * This is the most likely case: every TB must be inserted; lookups
-+}
+-     * are a lot less frequent.
-+
+-     */
-+static void start_tcg_kick_timer(void)
+-    if (likely(a->size && b->size)) {
-+{
+-        if (a->ptr > b->ptr) {
-+    if (!tcg_kick_vcpu_timer && CPU_NEXT(first_cpu)) {
+-            return 1;
-+        tcg_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
+-        } else if (a->ptr < b->ptr) {
-+                                           kick_tcg_thread, NULL);
+-            return -1;
-+    }
+-        }
-+    if (tcg_kick_vcpu_timer && !timer_pending(tcg_kick_vcpu_timer)) {
+-        /* a->ptr == b->ptr should happen only on deletions */
-+        timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
+-        g_assert(a->size == b->size);
-+    }
+-        return 0;
-+}
+-    }
-+
+-    /*
-+static void stop_tcg_kick_timer(void)
+-     * All lookups have either .size field set to 0.
-+{
+-     * From the glib sources we see that @ap is always the lookup key. However
-+    if (tcg_kick_vcpu_timer && timer_pending(tcg_kick_vcpu_timer)) {
+-     * the docs provide no guarantee, so we just mark this case as likely.
-+        timer_del(tcg_kick_vcpu_timer);
+-     */
-+    }
+-    if (likely(a->size == 0)) {
-+}
+-        return ptr_cmp_tb_tc(a->ptr, b);
-+
+-    }
-+static void qemu_tcg_rr_wait_io_event(void)
+-    return ptr_cmp_tb_tc(b->ptr, a);
-+{
+-}
-+    CPUState *cpu;
+-
-+
+-static void tcg_region_trees_init(void)
-+    while (all_cpu_threads_idle()) {
+-{
-+        stop_tcg_kick_timer();
+-    size_t i;
-+        qemu_cond_wait_iothread(first_cpu->halt_cond);
+-
-+    }
+-    tree_size = ROUND_UP(sizeof(struct tcg_region_tree), qemu_dcache_linesize);
-+
+-    region_trees = qemu_memalign(qemu_dcache_linesize, region.n * tree_size);
-+    start_tcg_kick_timer();
+-    for (i = 0; i < region.n; i++) {
-+
+-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-+    CPU_FOREACH(cpu) {
+-
-+        qemu_wait_io_event_common(cpu);
+-        qemu_mutex_init(&rt->lock);
-+    }
+-        rt->tree = g_tree_new(tb_tc_cmp);
-+}
+-    }
-+
+-}
-+/*
+-
-+ * Destroy any remaining vCPUs which have been unplugged and have
+-static struct tcg_region_tree *tc_ptr_to_region_tree(const void *p)
-+ * finished running
+-{
-+ */
+-    size_t region_idx;
-+static void deal_with_unplugged_cpus(void)
+-
-+{
+-    /*
-+    CPUState *cpu;
+-     * Like tcg_splitwx_to_rw, with no assert.  The pc may come from
-+
+-     * a signal handler over which the caller has no control.
-+    CPU_FOREACH(cpu) {
+-     */
-+        if (cpu->unplug && !cpu_can_run(cpu)) {
+-    if (!in_code_gen_buffer(p)) {
-+            qemu_tcg_destroy_vcpu(cpu);
+-        p -= tcg_splitwx_diff;
-+            break;
+-        if (!in_code_gen_buffer(p)) {
-+        }
+-            return NULL;
-+    }
+-        }
-+}
+-    }
-+
+-
-+/*
+-    if (p < region.start_aligned) {
-+ * In the single-threaded case each vCPU is simulated in turn. If
+-        region_idx = 0;
-+ * there is more than a single vCPU we create a simple timer to kick
+-    } else {
-+ * the vCPU and ensure we don't get stuck in a tight loop in one vCPU.
+-        ptrdiff_t offset = p - region.start_aligned;
-+ * This is done explicitly rather than relying on side-effects
+-
-+ * elsewhere.
+-        if (offset > region.stride * (region.n - 1)) {
-+ */
+-            region_idx = region.n - 1;
-+
+-        } else {
-+void *tcg_rr_cpu_thread_fn(void *arg)
+-            region_idx = offset / region.stride;
-+{
+-        }
-+    CPUState *cpu = arg;
+-    }
-+
+-    return region_trees + region_idx * tree_size;
-+    assert(tcg_enabled());
+-}
-+    rcu_register_thread();
+-
-+    tcg_register_thread();
+-void tcg_tb_insert(TranslationBlock *tb)
-+
+-{
-+    qemu_mutex_lock_iothread();
+-    struct tcg_region_tree *rt = tc_ptr_to_region_tree(tb->tc.ptr);
-+    qemu_thread_get_self(cpu->thread);
+-
-+
+-    g_assert(rt != NULL);
-+    cpu->thread_id = qemu_get_thread_id();
+-    qemu_mutex_lock(&rt->lock);
-+    cpu->can_do_io = 1;
+-    g_tree_insert(rt->tree, &tb->tc, tb);
-+    cpu_thread_signal_created(cpu);
+-    qemu_mutex_unlock(&rt->lock);
-+    qemu_guest_random_seed_thread_part2(cpu->random_seed);
+-}
-+
+-
-+    /* wait for initial kick-off after machine start */
+-void tcg_tb_remove(TranslationBlock *tb)
-+    while (first_cpu->stopped) {
+-{
-+        qemu_cond_wait_iothread(first_cpu->halt_cond);
+-    struct tcg_region_tree *rt = tc_ptr_to_region_tree(tb->tc.ptr);
-+
+-
-+        /* process any pending work */
+-    g_assert(rt != NULL);
-+        CPU_FOREACH(cpu) {
+-    qemu_mutex_lock(&rt->lock);
-+            current_cpu = cpu;
+-    g_tree_remove(rt->tree, &tb->tc);
-+            qemu_wait_io_event_common(cpu);
+-    qemu_mutex_unlock(&rt->lock);
-+        }
+-}
-+    }
+-
-+
+-/*
-+    start_tcg_kick_timer();
+- * Find the TB 'tb' such that
-+
+- * tb->tc.ptr <= tc_ptr < tb->tc.ptr + tb->tc.size
-+    cpu = first_cpu;
+- * Return NULL if not found.
-+
+- */
-+    /* process any pending work */
+-TranslationBlock *tcg_tb_lookup(uintptr_t tc_ptr)
-+    cpu->exit_request = 1;
+-{
-+
+-    struct tcg_region_tree *rt = tc_ptr_to_region_tree((void *)tc_ptr);
-+    while (1) {
+-    TranslationBlock *tb;
-+        qemu_mutex_unlock_iothread();
+-    struct tb_tc s = { .ptr = (void *)tc_ptr };
-+        replay_mutex_lock();
+-
-+        qemu_mutex_lock_iothread();
+-    if (rt == NULL) {
-+
+-        return NULL;
-+        if (icount_enabled()) {
+-    }
-+            /* Account partial waits to QEMU_CLOCK_VIRTUAL.  */
+-
-+            icount_account_warp_timer();
+-    qemu_mutex_lock(&rt->lock);
-+            /*
+-    tb = g_tree_lookup(rt->tree, &s);
-+             * Run the timers here.  This is much more efficient than
+-    qemu_mutex_unlock(&rt->lock);
-+             * waking up the I/O thread and waiting for completion.
+-    return tb;
-+             */
+-}
-+            handle_icount_deadline();
+-
-+        }
+-static void tcg_region_tree_lock_all(void)
-+
+-{
-+        replay_mutex_unlock();
+-    size_t i;
-+
+-
-+        if (!cpu) {
+-    for (i = 0; i < region.n; i++) {
-+            cpu = first_cpu;
+-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-+        }
+-
-+
+-        qemu_mutex_lock(&rt->lock);
-+        while (cpu && cpu_work_list_empty(cpu) && !cpu->exit_request) {
+-    }
-+
+-}
-+            qatomic_mb_set(&tcg_current_rr_cpu, cpu);
+-
-+            current_cpu = cpu;
+-static void tcg_region_tree_unlock_all(void)
-+
+-{
-+            qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
+-    size_t i;
-+                              (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
+-
-+
+-    for (i = 0; i < region.n; i++) {
-+            if (cpu_can_run(cpu)) {
+-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-+                int r;
+-
-+
+-        qemu_mutex_unlock(&rt->lock);
-+                qemu_mutex_unlock_iothread();
+-    }
-+                if (icount_enabled()) {
+-}
-+                    prepare_icount_for_run(cpu);
+-
-+                }
+-void tcg_tb_foreach(GTraverseFunc func, gpointer user_data)
-+                r = tcg_cpu_exec(cpu);
+-{
-+                if (icount_enabled()) {
+-    size_t i;
-+                    process_icount_data(cpu);
+-
-+                }
+-    tcg_region_tree_lock_all();
-+                qemu_mutex_lock_iothread();
+-    for (i = 0; i < region.n; i++) {
-+
+-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-+                if (r == EXCP_DEBUG) {
+-
-+                    cpu_handle_guest_debug(cpu);
+-        g_tree_foreach(rt->tree, func, user_data);
-+                    break;
+-    }
-+                } else if (r == EXCP_ATOMIC) {
+-    tcg_region_tree_unlock_all();
-+                    qemu_mutex_unlock_iothread();
+-}
-+                    cpu_exec_step_atomic(cpu);
+-
-+                    qemu_mutex_lock_iothread();
+-size_t tcg_nb_tbs(void)
-+                    break;
+-{
-+                }
+-    size_t nb_tbs = 0;
-+            } else if (cpu->stop) {
+-    size_t i;
-+                if (cpu->unplug) {
+-
-+                    cpu = CPU_NEXT(cpu);
+-    tcg_region_tree_lock_all();
-+                }
+-    for (i = 0; i < region.n; i++) {
-+                break;
+-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-+            }
+-
-+
+-        nb_tbs += g_tree_nnodes(rt->tree);
-+            cpu = CPU_NEXT(cpu);
+-    }
-+        } /* while (cpu && !cpu->exit_request).. */
+-    tcg_region_tree_unlock_all();
-+
+-    return nb_tbs;
-+        /* Does not need qatomic_mb_set because a spurious wakeup is okay.  */
+-}
-+        qatomic_set(&tcg_current_rr_cpu, NULL);
+-
-+
+-static gboolean tcg_region_tree_traverse(gpointer k, gpointer v, gpointer data)
-+        if (cpu && cpu->exit_request) {
+-{
-+            qatomic_mb_set(&cpu->exit_request, 0);
+-    TranslationBlock *tb = v;
-+        }
+-
-+
+-    tb_destroy(tb);
-+        if (icount_enabled() && all_cpu_threads_idle()) {
+-    return FALSE;
-+            /*
+-}
-+             * When all cpus are sleeping (e.g in WFI), to avoid a deadlock
+-
-+             * in the main_loop, wake it up in order to start the warp timer.
+-static void tcg_region_tree_reset_all(void)
-+             */
+-{
-+            qemu_notify_event();
+-    size_t i;
-+        }
+-
-+
+-    tcg_region_tree_lock_all();
-+        qemu_tcg_rr_wait_io_event();
+-    for (i = 0; i < region.n; i++) {
-+        deal_with_unplugged_cpus();
+-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-+    }
+-
-+
+-        g_tree_foreach(rt->tree, tcg_region_tree_traverse, NULL);
-+    rcu_unregister_thread();
+-        /* Increment the refcount first so that destroy acts as a reset */
-+    return NULL;
+-        g_tree_ref(rt->tree);
-+}
+-        g_tree_destroy(rt->tree);
-+
+-    }
-+const CpusAccel tcg_cpus_rr = {
+-    tcg_region_tree_unlock_all();
-+    .create_vcpu_thread = tcg_start_vcpu_thread,
+-}
-+    .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
+-
-+
+-static void tcg_region_bounds(size_t curr_region, void **pstart, void **pend)
-+    .handle_interrupt = tcg_handle_interrupt,
+-{
-+};
+-    void *start, *end;
-diff --git a/accel/tcg/tcg-cpus.c b/accel/tcg/tcg-cpus.c
+-
 -    start = region.start_aligned + curr_region * region.stride;
 -    end = start + region.size;
 -
 -    if (curr_region == 0) {
 -        start = region.start;
 -    }
 -    if (curr_region == region.n - 1) {
 -        end = region.end;
 -    }
 -
 -    *pstart = start;
 -    *pend = end;
 -}
 -
 -static void tcg_region_assign(TCGContext *s, size_t curr_region)
 -{
 -    void *start, *end;
 -
 -    tcg_region_bounds(curr_region, &start, &end);
 -
 -    s->code_gen_buffer = start;
 -    s->code_gen_ptr = start;
 -    s->code_gen_buffer_size = end - start;
 -    s->code_gen_highwater = end - TCG_HIGHWATER;
 -}
 -
 -static bool tcg_region_alloc__locked(TCGContext *s)
 -{
 -    if (region.current == region.n) {
 -        return true;
 -    }
 -    tcg_region_assign(s, region.current);
 -    region.current++;
 -    return false;
 -}
 -
 -/*
 - * Request a new region once the one in use has filled up.
 - * Returns true on error.
 - */
 -static bool tcg_region_alloc(TCGContext *s)
 -{
 -    bool err;
 -    /* read the region size now; alloc__locked will overwrite it on success */
 -    size_t size_full = s->code_gen_buffer_size;
 -
 -    qemu_mutex_lock(&region.lock);
 -    err = tcg_region_alloc__locked(s);
 -    if (!err) {
 -        region.agg_size_full += size_full - TCG_HIGHWATER;
 -    }
 -    qemu_mutex_unlock(&region.lock);
 -    return err;
 -}
 -
 -/*
 - * Perform a context's first region allocation.
 - * This function does _not_ increment region.agg_size_full.
 - */
 -static void tcg_region_initial_alloc__locked(TCGContext *s)
 -{
 -    bool err = tcg_region_alloc__locked(s);
 -    g_assert(!err);
 -}
 -
 -#ifndef CONFIG_USER_ONLY
 -static void tcg_region_initial_alloc(TCGContext *s)
 -{
 -    qemu_mutex_lock(&region.lock);
 -    tcg_region_initial_alloc__locked(s);
 -    qemu_mutex_unlock(&region.lock);
 -}
 -#endif
 -
 -/* Call from a safe-work context */
 -void tcg_region_reset_all(void)
 -{
 -    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
 -    unsigned int i;
 -
 -    qemu_mutex_lock(&region.lock);
 -    region.current = 0;
 -    region.agg_size_full = 0;
 -
 -    for (i = 0; i < n_ctxs; i++) {
 -        TCGContext *s = qatomic_read(&tcg_ctxs[i]);
 -        tcg_region_initial_alloc__locked(s);
 -    }
 -    qemu_mutex_unlock(&region.lock);
 -
 -    tcg_region_tree_reset_all();
 -}
 -
 -#ifdef CONFIG_USER_ONLY
 -static size_t tcg_n_regions(void)
 -{
 -    return 1;
 -}
 -#else
 -/*
 - * It is likely that some vCPUs will translate more code than others, so we
 - * first try to set more regions than max_cpus, with those regions being of
 - * reasonable size. If that's not possible we make do by evenly dividing
 - * the code_gen_buffer among the vCPUs.
 - */
 -static size_t tcg_n_regions(void)
 -{
 -    size_t i;
 -
 -    /* Use a single region if all we have is one vCPU thread */
 -#if !defined(CONFIG_USER_ONLY)
 -    MachineState *ms = MACHINE(qdev_get_machine());
 -    unsigned int max_cpus = ms->smp.max_cpus;
 -#endif
 -    if (max_cpus == 1 || !qemu_tcg_mttcg_enabled()) {
 -        return 1;
 -    }
 -
 -    /* Try to have more regions than max_cpus, with each region being >= 2 MB */
 -    for (i = 8; i > 0; i--) {
 -        size_t regions_per_thread = i;
 -        size_t region_size;
 -
 -        region_size = tcg_init_ctx.code_gen_buffer_size;
 -        region_size /= max_cpus * regions_per_thread;
 -
 -        if (region_size >= 2 * 1024u * 1024) {
 -            return max_cpus * regions_per_thread;
 -        }
 -    }
 -    /* If we can't, then just allocate one region per vCPU thread */
 -    return max_cpus;
 -}
 -#endif
 -
 -/*
 - * Initializes region partitioning.
 - *
 - * Called at init time from the parent thread (i.e. the one calling
 - * tcg_context_init), after the target's TCG globals have been set.
 - *
 - * Region partitioning works by splitting code_gen_buffer into separate regions,
 - * and then assigning regions to TCG threads so that the threads can translate
 - * code in parallel without synchronization.
 - *
 - * In softmmu the number of TCG threads is bounded by max_cpus, so we use at
 - * least max_cpus regions in MTTCG. In !MTTCG we use a single region.
 - * Note that the TCG options from the command-line (i.e. -accel accel=tcg,[...])
 - * must have been parsed before calling this function, since it calls
 - * qemu_tcg_mttcg_enabled().
 - *
 - * In user-mode we use a single region.  Having multiple regions in user-mode
 - * is not supported, because the number of vCPU threads (recall that each thread
 - * spawned by the guest corresponds to a vCPU thread) is only bounded by the
 - * OS, and usually this number is huge (tens of thousands is not uncommon).
 - * Thus, given this large bound on the number of vCPU threads and the fact
 - * that code_gen_buffer is allocated at compile-time, we cannot guarantee
 - * that the availability of at least one region per vCPU thread.
 - *
 - * However, this user-mode limitation is unlikely to be a significant problem
 - * in practice. Multi-threaded guests share most if not all of their translated
 - * code, which makes parallel code generation less appealing than in softmmu.
 - */
 -void tcg_region_init(void)
 -{
 -    void *buf = tcg_init_ctx.code_gen_buffer;
 -    void *aligned;
 -    size_t size = tcg_init_ctx.code_gen_buffer_size;
 -    size_t page_size = qemu_real_host_page_size;
 -    size_t region_size;
 -    size_t n_regions;
 -    size_t i;
 -
 -    n_regions = tcg_n_regions();
 -
 -    /* The first region will be 'aligned - buf' bytes larger than the others */
 -    aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
 -    g_assert(aligned < tcg_init_ctx.code_gen_buffer + size);
 -    /*
 -     * Make region_size a multiple of page_size, using aligned as the start.
 -     * As a result of this we might end up with a few extra pages at the end of
 -     * the buffer; we will assign those to the last region.
 -     */
 -    region_size = (size - (aligned - buf)) / n_regions;
 -    region_size = QEMU_ALIGN_DOWN(region_size, page_size);
 -
 -    /* A region must have at least 2 pages; one code, one guard */
 -    g_assert(region_size >= 2 * page_size);
 -
 -    /* init the region struct */
 -    qemu_mutex_init(&region.lock);
 -    region.n = n_regions;
 -    region.size = region_size - page_size;
 -    region.stride = region_size;
 -    region.start = buf;
 -    region.start_aligned = aligned;
 -    /* page-align the end, since its last page will be a guard page */
 -    region.end = QEMU_ALIGN_PTR_DOWN(buf + size, page_size);
 -    /* account for that last guard page */
 -    region.end -= page_size;
 -
 -    /*
 -     * Set guard pages in the rw buffer, as that's the one into which
 -     * buffer overruns could occur.  Do not set guard pages in the rx
 -     * buffer -- let that one use hugepages throughout.
 -     */
 -    for (i = 0; i < region.n; i++) {
 -        void *start, *end;
 -
 -        tcg_region_bounds(i, &start, &end);
 -
 -        /*
 -         * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
 -         * rejects a permission change from RWX -> NONE.  Guard pages are
 -         * nice for bug detection but are not essential; ignore any failure.
 -         */
 -        (void)qemu_mprotect_none(end, page_size);
 -    }
 -
 -    tcg_region_trees_init();
 -
 -    /*
 -     * Leave the initial context initialized to the first region.
 -     * This will be the context into which we generate the prologue.
 -     * It is also the only context for CONFIG_USER_ONLY.
 -     */
 -    tcg_region_initial_alloc__locked(&tcg_init_ctx);
 -}
 -
 -static void tcg_region_prologue_set(TCGContext *s)
 -{
 -    /* Deduct the prologue from the first region.  */
 -    g_assert(region.start == s->code_gen_buffer);
 -    region.start = s->code_ptr;
 -
 -    /* Recompute boundaries of the first region. */
 -    tcg_region_assign(s, 0);
 -
 -    /* Register the balance of the buffer with gdb. */
 -    tcg_register_jit(tcg_splitwx_to_rx(region.start),
 -                     region.end - region.start);
 -}
 -
  #ifdef CONFIG_DEBUG_TCG
  const void *tcg_splitwx_to_rx(void *rw)
  {
@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
  }
  #endif /* !CONFIG_USER_ONLY */
 -/*
 - * Returns the size (in bytes) of all translated code (i.e. from all regions)
 - * currently in the cache.
 - * See also: tcg_code_capacity()
 - * Do not confuse with tcg_current_code_size(); that one applies to a single
 - * TCG context.
 - */
 -size_t tcg_code_size(void)
 -{
 -    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
 -    unsigned int i;
 -    size_t total;
 -
 -    qemu_mutex_lock(&region.lock);
 -    total = region.agg_size_full;
 -    for (i = 0; i < n_ctxs; i++) {
 -        const TCGContext *s = qatomic_read(&tcg_ctxs[i]);
 -        size_t size;
 -
 -        size = qatomic_read(&s->code_gen_ptr) - s->code_gen_buffer;
 -        g_assert(size <= s->code_gen_buffer_size);
 -        total += size;
 -    }
 -    qemu_mutex_unlock(&region.lock);
 -    return total;
 -}
 -
 -/*
 - * Returns the code capacity (in bytes) of the entire cache, i.e. including all
 - * regions.
 - * See also: tcg_code_size()
 - */
 -size_t tcg_code_capacity(void)
 -{
 -    size_t guard_size, capacity;
 -
 -    /* no need for synchronization; these variables are set at init time */
 -    guard_size = region.stride - region.size;
 -    capacity = region.end + guard_size - region.start;
 -    capacity -= region.n * (guard_size + TCG_HIGHWATER);
 -    return capacity;
 -}
 -
 -size_t tcg_tb_phys_invalidate_count(void)
 -{
 -    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
 -    unsigned int i;
 -    size_t total = 0;
 -
 -    for (i = 0; i < n_ctxs; i++) {
 -        const TCGContext *s = qatomic_read(&tcg_ctxs[i]);
 -
 -        total += qatomic_read(&s->tb_phys_invalidate_count);
 -    }
 -    return total;
 -}
 -
  /* pool based memory allocation */
  void *tcg_malloc_internal(TCGContext *s, int size)
  {
 diff --git a/tcg/meson.build b/tcg/meson.build
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/tcg-cpus.c
+--- a/tcg/meson.build
-+++ b/accel/tcg/tcg-cpus.c
++++ b/tcg/meson.build
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ tcg_ss = ss.source_set()
- /*
-- * QEMU System Emulator
+ tcg_ss.add(files(
-+ * QEMU TCG vCPU common functionality
+   'optimize.c',
-+ *
++  'region.c',
-+ * Functionality common to all TCG vCPU variants: mttcg, rr and icount.
+   'tcg.c',
-  *
+   'tcg-common.c',
-  * Copyright (c) 2003-2008 Fabrice Bellard
+   'tcg-op.c',
   * Copyright (c) 2014 Red Hat Inc.
@@ -XXX,XX +XXX,XX @@
  #include "hw/boards.h"
  #include "tcg-cpus.h"
 +#include "tcg-cpus-mttcg.h"
 +#include "tcg-cpus-rr.h"
 -/* Kick all RR vCPUs */
 -static void qemu_cpu_kick_rr_cpus(void)
 -{
 -    CPUState *cpu;
 +/* common functionality among all TCG variants */
 -    CPU_FOREACH(cpu) {
 -        cpu_exit(cpu);
 -    };
 -}
 -
 -static void tcg_kick_vcpu_thread(CPUState *cpu)
 -{
 -    if (qemu_tcg_mttcg_enabled()) {
 -        cpu_exit(cpu);
 -    } else {
 -        qemu_cpu_kick_rr_cpus();
 -    }
 -}
 -
 -/*
 - * TCG vCPU kick timer
 - *
 - * The kick timer is responsible for moving single threaded vCPU
 - * emulation on to the next vCPU. If more than one vCPU is running a
 - * timer event with force a cpu->exit so the next vCPU can get
 - * scheduled.
 - *
 - * The timer is removed if all vCPUs are idle and restarted again once
 - * idleness is complete.
 - */
 -
 -static QEMUTimer *tcg_kick_vcpu_timer;
 -static CPUState *tcg_current_rr_cpu;
 -
 -#define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
 -
 -static inline int64_t qemu_tcg_next_kick(void)
 -{
 -    return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + TCG_KICK_PERIOD;
 -}
 -
 -/* Kick the currently round-robin scheduled vCPU to next */
 -static void qemu_cpu_kick_rr_next_cpu(void)
 -{
 -    CPUState *cpu;
 -    do {
 -        cpu = qatomic_mb_read(&tcg_current_rr_cpu);
 -        if (cpu) {
 -            cpu_exit(cpu);
 -        }
 -    } while (cpu != qatomic_mb_read(&tcg_current_rr_cpu));
 -}
 -
 -static void kick_tcg_thread(void *opaque)
 -{
 -    timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
 -    qemu_cpu_kick_rr_next_cpu();
 -}
 -
 -static void start_tcg_kick_timer(void)
 -{
 -    assert(!mttcg_enabled);
 -    if (!tcg_kick_vcpu_timer && CPU_NEXT(first_cpu)) {
 -        tcg_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
 -                                           kick_tcg_thread, NULL);
 -    }
 -    if (tcg_kick_vcpu_timer && !timer_pending(tcg_kick_vcpu_timer)) {
 -        timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
 -    }
 -}
 -
 -static void stop_tcg_kick_timer(void)
 -{
 -    assert(!mttcg_enabled);
 -    if (tcg_kick_vcpu_timer && timer_pending(tcg_kick_vcpu_timer)) {
 -        timer_del(tcg_kick_vcpu_timer);
 -    }
 -}
 -
 -static void qemu_tcg_destroy_vcpu(CPUState *cpu)
 -{
 -}
 -
 -static void qemu_tcg_rr_wait_io_event(void)
 -{
 -    CPUState *cpu;
 -
 -    while (all_cpu_threads_idle()) {
 -        stop_tcg_kick_timer();
 -        qemu_cond_wait_iothread(first_cpu->halt_cond);
 -    }
 -
 -    start_tcg_kick_timer();
 -
 -    CPU_FOREACH(cpu) {
 -        qemu_wait_io_event_common(cpu);
 -    }
 -}
 -
 -static int64_t tcg_get_icount_limit(void)
 -{
 -    int64_t deadline;
 -
 -    if (replay_mode != REPLAY_MODE_PLAY) {
 -        /*
 -         * Include all the timers, because they may need an attention.
 -         * Too long CPU execution may create unnecessary delay in UI.
 -         */
 -        deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
 -                                              QEMU_TIMER_ATTR_ALL);
 -        /* Check realtime timers, because they help with input processing */
 -        deadline = qemu_soonest_timeout(deadline,
 -                qemu_clock_deadline_ns_all(QEMU_CLOCK_REALTIME,
 -                                           QEMU_TIMER_ATTR_ALL));
 -
 -        /*
 -         * Maintain prior (possibly buggy) behaviour where if no deadline
 -         * was set (as there is no QEMU_CLOCK_VIRTUAL timer) or it is more than
 -         * INT32_MAX nanoseconds ahead, we still use INT32_MAX
 -         * nanoseconds.
 -         */
 -        if ((deadline < 0) || (deadline > INT32_MAX)) {
 -            deadline = INT32_MAX;
 -        }
 -
 -        return icount_round(deadline);
 -    } else {
 -        return replay_get_instructions();
 -    }
 -}
 -
 -static void notify_aio_contexts(void)
 -{
 -    /* Wake up other AioContexts.  */
 -    qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
 -    qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL);
 -}
 -
 -static void handle_icount_deadline(void)
 -{
 -    assert(qemu_in_vcpu_thread());
 -    if (icount_enabled()) {
 -        int64_t deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
 -                                                      QEMU_TIMER_ATTR_ALL);
 -
 -        if (deadline == 0) {
 -            notify_aio_contexts();
 -        }
 -    }
 -}
 -
 -static void prepare_icount_for_run(CPUState *cpu)
 -{
 -    if (icount_enabled()) {
 -        int insns_left;
 -
 -        /*
 -         * These should always be cleared by process_icount_data after
 -         * each vCPU execution. However u16.high can be raised
 -         * asynchronously by cpu_exit/cpu_interrupt/tcg_handle_interrupt
 -         */
 -        g_assert(cpu_neg(cpu)->icount_decr.u16.low == 0);
 -        g_assert(cpu->icount_extra == 0);
 -
 -        cpu->icount_budget = tcg_get_icount_limit();
 -        insns_left = MIN(0xffff, cpu->icount_budget);
 -        cpu_neg(cpu)->icount_decr.u16.low = insns_left;
 -        cpu->icount_extra = cpu->icount_budget - insns_left;
 -
 -        replay_mutex_lock();
 -
 -        if (cpu->icount_budget == 0 && replay_has_checkpoint()) {
 -            notify_aio_contexts();
 -        }
 -    }
 -}
 -
 -static void process_icount_data(CPUState *cpu)
 -{
 -    if (icount_enabled()) {
 -        /* Account for executed instructions */
 -        icount_update(cpu);
 -
 -        /* Reset the counters */
 -        cpu_neg(cpu)->icount_decr.u16.low = 0;
 -        cpu->icount_extra = 0;
 -        cpu->icount_budget = 0;
 -
 -        replay_account_executed_instructions();
 -
 -        replay_mutex_unlock();
 -    }
 -}
 -
 -static int tcg_cpu_exec(CPUState *cpu)
 -{
 -    int ret;
 -#ifdef CONFIG_PROFILER
 -    int64_t ti;
 -#endif
 -
 -    assert(tcg_enabled());
 -#ifdef CONFIG_PROFILER
 -    ti = profile_getclock();
 -#endif
 -    cpu_exec_start(cpu);
 -    ret = cpu_exec(cpu);
 -    cpu_exec_end(cpu);
 -#ifdef CONFIG_PROFILER
 -    qatomic_set(&tcg_ctx->prof.cpu_exec_time,
 -                tcg_ctx->prof.cpu_exec_time + profile_getclock() - ti);
 -#endif
 -    return ret;
 -}
 -
 -/*
 - * Destroy any remaining vCPUs which have been unplugged and have
 - * finished running
 - */
 -static void deal_with_unplugged_cpus(void)
 -{
 -    CPUState *cpu;
 -
 -    CPU_FOREACH(cpu) {
 -        if (cpu->unplug && !cpu_can_run(cpu)) {
 -            qemu_tcg_destroy_vcpu(cpu);
 -            cpu_thread_signal_destroyed(cpu);
 -            break;
 -        }
 -    }
 -}
 -
 -/*
 - * Single-threaded TCG
 - *
 - * In the single-threaded case each vCPU is simulated in turn. If
 - * there is more than a single vCPU we create a simple timer to kick
 - * the vCPU and ensure we don't get stuck in a tight loop in one vCPU.
 - * This is done explicitly rather than relying on side-effects
 - * elsewhere.
 - */
 -
 -static void *tcg_rr_cpu_thread_fn(void *arg)
 -{
 -    CPUState *cpu = arg;
 -
 -    assert(tcg_enabled());
 -    rcu_register_thread();
 -    tcg_register_thread();
 -
 -    qemu_mutex_lock_iothread();
 -    qemu_thread_get_self(cpu->thread);
 -
 -    cpu->thread_id = qemu_get_thread_id();
 -    cpu->can_do_io = 1;
 -    cpu_thread_signal_created(cpu);
 -    qemu_guest_random_seed_thread_part2(cpu->random_seed);
 -
 -    /* wait for initial kick-off after machine start */
 -    while (first_cpu->stopped) {
 -        qemu_cond_wait_iothread(first_cpu->halt_cond);
 -
 -        /* process any pending work */
 -        CPU_FOREACH(cpu) {
 -            current_cpu = cpu;
 -            qemu_wait_io_event_common(cpu);
 -        }
 -    }
 -
 -    start_tcg_kick_timer();
 -
 -    cpu = first_cpu;
 -
 -    /* process any pending work */
 -    cpu->exit_request = 1;
 -
 -    while (1) {
 -        qemu_mutex_unlock_iothread();
 -        replay_mutex_lock();
 -        qemu_mutex_lock_iothread();
 -        /* Account partial waits to QEMU_CLOCK_VIRTUAL.  */
 -        icount_account_warp_timer();
 -
 -        /*
 -         * Run the timers here.  This is much more efficient than
 -         * waking up the I/O thread and waiting for completion.
 -         */
 -        handle_icount_deadline();
 -
 -        replay_mutex_unlock();
 -
 -        if (!cpu) {
 -            cpu = first_cpu;
 -        }
 -
 -        while (cpu && cpu_work_list_empty(cpu) && !cpu->exit_request) {
 -
 -            qatomic_mb_set(&tcg_current_rr_cpu, cpu);
 -            current_cpu = cpu;
 -
 -            qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
 -                              (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
 -
 -            if (cpu_can_run(cpu)) {
 -                int r;
 -
 -                qemu_mutex_unlock_iothread();
 -                prepare_icount_for_run(cpu);
 -
 -                r = tcg_cpu_exec(cpu);
 -
 -                process_icount_data(cpu);
 -                qemu_mutex_lock_iothread();
 -
 -                if (r == EXCP_DEBUG) {
 -                    cpu_handle_guest_debug(cpu);
 -                    break;
 -                } else if (r == EXCP_ATOMIC) {
 -                    qemu_mutex_unlock_iothread();
 -                    cpu_exec_step_atomic(cpu);
 -                    qemu_mutex_lock_iothread();
 -                    break;
 -                }
 -            } else if (cpu->stop) {
 -                if (cpu->unplug) {
 -                    cpu = CPU_NEXT(cpu);
 -                }
 -                break;
 -            }
 -
 -            cpu = CPU_NEXT(cpu);
 -        } /* while (cpu && !cpu->exit_request).. */
 -
 -        /* Does not need qatomic_mb_set because a spurious wakeup is okay.  */
 -        qatomic_set(&tcg_current_rr_cpu, NULL);
 -
 -        if (cpu && cpu->exit_request) {
 -            qatomic_mb_set(&cpu->exit_request, 0);
 -        }
 -
 -        if (icount_enabled() && all_cpu_threads_idle()) {
 -            /*
 -             * When all cpus are sleeping (e.g in WFI), to avoid a deadlock
 -             * in the main_loop, wake it up in order to start the warp timer.
 -             */
 -            qemu_notify_event();
 -        }
 -
 -        qemu_tcg_rr_wait_io_event();
 -        deal_with_unplugged_cpus();
 -    }
 -
 -    rcu_unregister_thread();
 -    return NULL;
 -}
 -
 -/*
 - * Multi-threaded TCG
 - *
 - * In the multi-threaded case each vCPU has its own thread. The TLS
 - * variable current_cpu can be used deep in the code to find the
 - * current CPUState for a given thread.
 - */
 -
 -static void *tcg_cpu_thread_fn(void *arg)
 -{
 -    CPUState *cpu = arg;
 -
 -    assert(tcg_enabled());
 -    g_assert(!icount_enabled());
 -
 -    rcu_register_thread();
 -    tcg_register_thread();
 -
 -    qemu_mutex_lock_iothread();
 -    qemu_thread_get_self(cpu->thread);
 -
 -    cpu->thread_id = qemu_get_thread_id();
 -    cpu->can_do_io = 1;
 -    current_cpu = cpu;
 -    cpu_thread_signal_created(cpu);
 -    qemu_guest_random_seed_thread_part2(cpu->random_seed);
 -
 -    /* process any pending work */
 -    cpu->exit_request = 1;
 -
 -    do {
 -        if (cpu_can_run(cpu)) {
 -            int r;
 -            qemu_mutex_unlock_iothread();
 -            r = tcg_cpu_exec(cpu);
 -            qemu_mutex_lock_iothread();
 -            switch (r) {
 -            case EXCP_DEBUG:
 -                cpu_handle_guest_debug(cpu);
 -                break;
 -            case EXCP_HALTED:
 -                /*
 -                 * during start-up the vCPU is reset and the thread is
 -                 * kicked several times. If we don't ensure we go back
 -                 * to sleep in the halted state we won't cleanly
 -                 * start-up when the vCPU is enabled.
 -                 *
 -                 * cpu->halted should ensure we sleep in wait_io_event
 -                 */
 -                g_assert(cpu->halted);
 -                break;
 -            case EXCP_ATOMIC:
 -                qemu_mutex_unlock_iothread();
 -                cpu_exec_step_atomic(cpu);
 -                qemu_mutex_lock_iothread();
 -            default:
 -                /* Ignore everything else? */
 -                break;
 -            }
 -        }
 -
 -        qatomic_mb_set(&cpu->exit_request, 0);
 -        qemu_wait_io_event(cpu);
 -    } while (!cpu->unplug || cpu_can_run(cpu));
 -
 -    qemu_tcg_destroy_vcpu(cpu);
 -    cpu_thread_signal_destroyed(cpu);
 -    qemu_mutex_unlock_iothread();
 -    rcu_unregister_thread();
 -    return NULL;
 -}
 -
 -static void tcg_start_vcpu_thread(CPUState *cpu)
 +void tcg_start_vcpu_thread(CPUState *cpu)
  {
      char thread_name[VCPU_THREAD_NAME_SIZE];
      static QemuCond *single_tcg_halt_cond;
@@ -XXX,XX +XXX,XX @@ static void tcg_start_vcpu_thread(CPUState *cpu)
      }
  }
 -static int64_t tcg_get_virtual_clock(void)
 +void qemu_tcg_destroy_vcpu(CPUState *cpu)
  {
 -    if (icount_enabled()) {
 -        return icount_get();
 -    }
 -    return cpu_get_clock();
 +    cpu_thread_signal_destroyed(cpu);
  }
 -static int64_t tcg_get_elapsed_ticks(void)
 +int tcg_cpu_exec(CPUState *cpu)
  {
 -    if (icount_enabled()) {
 -        return icount_get();
 -    }
 -    return cpu_get_ticks();
 +    int ret;
 +#ifdef CONFIG_PROFILER
 +    int64_t ti;
 +#endif
 +    assert(tcg_enabled());
 +#ifdef CONFIG_PROFILER
 +    ti = profile_getclock();
 +#endif
 +    cpu_exec_start(cpu);
 +    ret = cpu_exec(cpu);
 +    cpu_exec_end(cpu);
 +#ifdef CONFIG_PROFILER
 +    qatomic_set(&tcg_ctx->prof.cpu_exec_time,
 +                tcg_ctx->prof.cpu_exec_time + profile_getclock() - ti);
 +#endif
 +    return ret;
  }
  /* mask must never be zero, except for A20 change call */
 -static void tcg_handle_interrupt(CPUState *cpu, int mask)
 +void tcg_handle_interrupt(CPUState *cpu, int mask)
  {
 -    int old_mask;
      g_assert(qemu_mutex_iothread_locked());
 -    old_mask = cpu->interrupt_request;
      cpu->interrupt_request |= mask;
      /*
@@ -XXX,XX +XXX,XX @@ static void tcg_handle_interrupt(CPUState *cpu, int mask)
          qemu_cpu_kick(cpu);
      } else {
          qatomic_set(&cpu_neg(cpu)->icount_decr.u16.high, -1);
 -        if (icount_enabled() &&
 -            !cpu->can_do_io
 -            && (mask & ~old_mask) != 0) {
 -            cpu_abort(cpu, "Raised interrupt while not in I/O function");
 -        }
      }
  }
 -
 -const CpusAccel tcg_cpus = {
 -    .create_vcpu_thread = tcg_start_vcpu_thread,
 -    .kick_vcpu_thread = tcg_kick_vcpu_thread,
 -
 -    .handle_interrupt = tcg_handle_interrupt,
 -
 -    .get_virtual_clock = tcg_get_virtual_clock,
 -    .get_elapsed_ticks = tcg_get_elapsed_ticks,
 -};
 diff --git a/softmmu/icount.c b/softmmu/icount.c
 index XXXXXXX..XXXXXXX 100644
 --- a/softmmu/icount.c
 +++ b/softmmu/icount.c
@@ -XXX,XX +XXX,XX @@ void icount_start_warp_timer(void)
  void icount_account_warp_timer(void)
  {
 -    if (!icount_enabled() || !icount_sleep) {
 +    if (!icount_sleep) {
          return;
      }
 diff --git a/accel/tcg/meson.build b/accel/tcg/meson.build
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/meson.build
 +++ b/accel/tcg/meson.build
@@ -XXX,XX +XXX,XX @@ tcg_ss.add(when: 'CONFIG_SOFTMMU', if_false: files('user-exec-stub.c'))
  tcg_ss.add(when: 'CONFIG_PLUGIN', if_true: [files('plugin-gen.c'), libdl])
  specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 -specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files('tcg-all.c', 'cputlb.c', 'tcg-cpus.c'))
 +specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
 +  'tcg-all.c',
 +  'cputlb.c',
 +  'tcg-cpus.c',
 +  'tcg-cpus-mttcg.c',
 +  'tcg-cpus-icount.c',
 +  'tcg-cpus-rr.c'
 +))
 --
 .25.1

-New patch
+[PULL 08/34] accel/tcg: Inline cpu_gen_init
+It consists of one function call and has only one caller.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ accel/tcg/translate-all.c | 7 +------
+file changed, 1 insertion(+), 6 deletions(-)
+diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
+index XXXXXXX..XXXXXXX 100644
+--- a/accel/tcg/translate-all.c
++++ b/accel/tcg/translate-all.c
+@@ -XXX,XX +XXX,XX @@ static void page_table_config_init(void)
+     assert(v_l2_levels >= 0);
+ }
+-static void cpu_gen_init(void)
+-{
+-    tcg_context_init(&tcg_init_ctx);
+-}
+-
+ /* Encode VAL as a signed leb128 sequence at P.
+    Return P incremented past the encoded value.  */
+ static uint8_t *encode_sleb128(uint8_t *p, target_long val)
+@@ -XXX,XX +XXX,XX @@ void tcg_exec_init(unsigned long tb_size, int splitwx)
+     bool ok;
+     tcg_allowed = true;
+-    cpu_gen_init();
++    tcg_context_init(&tcg_init_ctx);
+     page_init();
+     tb_htable_init();
+--
+.25.1

-New patch
+[PULL 09/34] accel/tcg: Move alloc_code_gen_buffer to tcg/region.c
+Buffer management is integral to tcg.  Do not leave the allocation
+to code outside of tcg/.  This is code movement, with further
+cleanups to follow.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ include/tcg/tcg.h         |   2 +-
+ accel/tcg/translate-all.c | 414 +-----------------------------------
+ tcg/region.c              | 431 +++++++++++++++++++++++++++++++++++++-
+files changed, 428 insertions(+), 419 deletions(-)
+diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
+index XXXXXXX..XXXXXXX 100644
+--- a/include/tcg/tcg.h
++++ b/include/tcg/tcg.h
+@@ -XXX,XX +XXX,XX @@ void *tcg_malloc_internal(TCGContext *s, int size);
+ void tcg_pool_reset(TCGContext *s);
+ TranslationBlock *tcg_tb_alloc(TCGContext *s);
+-void tcg_region_init(void);
++void tcg_region_init(size_t tb_size, int splitwx);
+ void tb_destroy(TranslationBlock *tb);
+ void tcg_region_reset_all(void);
+diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
+index XXXXXXX..XXXXXXX 100644
+--- a/accel/tcg/translate-all.c
++++ b/accel/tcg/translate-all.c
+@@ -XXX,XX +XXX,XX @@
+  */
+ #include "qemu/osdep.h"
+-#include "qemu/units.h"
+ #include "qemu-common.h"
+ #define NO_CPU_IO_DEFS
+@@ -XXX,XX +XXX,XX @@
+ #include "exec/cputlb.h"
+ #include "exec/translate-all.h"
+ #include "qemu/bitmap.h"
+-#include "qemu/error-report.h"
+ #include "qemu/qemu-print.h"
+ #include "qemu/timer.h"
+ #include "qemu/main-loop.h"
+@@ -XXX,XX +XXX,XX @@ static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
+     }
+ }
+-/* Minimum size of the code gen buffer.  This number is randomly chosen,
+-   but not so small that we can't have a fair number of TB's live.  */
+-#define MIN_CODE_GEN_BUFFER_SIZE     (1 * MiB)
+-
+-/* Maximum size of the code gen buffer we'd like to use.  Unless otherwise
+-   indicated, this is constrained by the range of direct branches on the
+-   host cpu, as used by the TCG implementation of goto_tb.  */
+-#if defined(__x86_64__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+-#elif defined(__sparc__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+-#elif defined(__powerpc64__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+-#elif defined(__powerpc__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (32 * MiB)
+-#elif defined(__aarch64__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+-#elif defined(__s390x__)
+-  /* We have a +- 4GB range on the branches; leave some slop.  */
+-# define MAX_CODE_GEN_BUFFER_SIZE  (3 * GiB)
+-#elif defined(__mips__)
+-  /* We have a 256MB branch region, but leave room to make sure the
+-     main executable is also within that region.  */
+-# define MAX_CODE_GEN_BUFFER_SIZE  (128 * MiB)
+-#else
+-# define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
+-#endif
+-
+-#if TCG_TARGET_REG_BITS == 32
+-#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (32 * MiB)
+-#ifdef CONFIG_USER_ONLY
+-/*
+- * For user mode on smaller 32 bit systems we may run into trouble
+- * allocating big chunks of data in the right place. On these systems
+- * we utilise a static code generation buffer directly in the binary.
+- */
+-#define USE_STATIC_CODE_GEN_BUFFER
+-#endif
+-#else /* TCG_TARGET_REG_BITS == 64 */
+-#ifdef CONFIG_USER_ONLY
+-/*
+- * As user-mode emulation typically means running multiple instances
+- * of the translator don't go too nuts with our default code gen
+- * buffer lest we make things too hard for the OS.
+- */
+-#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (128 * MiB)
+-#else
+-/*
+- * We expect most system emulation to run one or two guests per host.
+- * Users running large scale system emulation may want to tweak their
+- * runtime setup via the tb-size control on the command line.
+- */
+-#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (1 * GiB)
+-#endif
+-#endif
+-
+-#define DEFAULT_CODE_GEN_BUFFER_SIZE \
+-  (DEFAULT_CODE_GEN_BUFFER_SIZE_1 < MAX_CODE_GEN_BUFFER_SIZE \
+-   ? DEFAULT_CODE_GEN_BUFFER_SIZE_1 : MAX_CODE_GEN_BUFFER_SIZE)
+-
+-static size_t size_code_gen_buffer(size_t tb_size)
+-{
+-    /* Size the buffer.  */
+-    if (tb_size == 0) {
+-        size_t phys_mem = qemu_get_host_physmem();
+-        if (phys_mem == 0) {
+-            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+-        } else {
+-            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
+-        }
+-    }
+-    if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
+-        tb_size = MIN_CODE_GEN_BUFFER_SIZE;
+-    }
+-    if (tb_size > MAX_CODE_GEN_BUFFER_SIZE) {
+-        tb_size = MAX_CODE_GEN_BUFFER_SIZE;
+-    }
+-    return tb_size;
+-}
+-
+-#ifdef __mips__
+-/* In order to use J and JAL within the code_gen_buffer, we require
+-   that the buffer not cross a 256MB boundary.  */
+-static inline bool cross_256mb(void *addr, size_t size)
+-{
+-    return ((uintptr_t)addr ^ ((uintptr_t)addr + size)) & ~0x0ffffffful;
+-}
+-
+-/* We weren't able to allocate a buffer without crossing that boundary,
+-   so make do with the larger portion of the buffer that doesn't cross.
+-   Returns the new base of the buffer, and adjusts code_gen_buffer_size.  */
+-static inline void *split_cross_256mb(void *buf1, size_t size1)
+-{
+-    void *buf2 = (void *)(((uintptr_t)buf1 + size1) & ~0x0ffffffful);
+-    size_t size2 = buf1 + size1 - buf2;
+-
+-    size1 = buf2 - buf1;
+-    if (size1 < size2) {
+-        size1 = size2;
+-        buf1 = buf2;
+-    }
+-
+-    tcg_ctx->code_gen_buffer_size = size1;
+-    return buf1;
+-}
+-#endif
+-
+-#ifdef USE_STATIC_CODE_GEN_BUFFER
+-static uint8_t static_code_gen_buffer[DEFAULT_CODE_GEN_BUFFER_SIZE]
+-    __attribute__((aligned(CODE_GEN_ALIGN)));
+-
+-static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
+-{
+-    void *buf, *end;
+-    size_t size;
+-
+-    if (splitwx > 0) {
+-        error_setg(errp, "jit split-wx not supported");
+-        return false;
+-    }
+-
+-    /* page-align the beginning and end of the buffer */
+-    buf = static_code_gen_buffer;
+-    end = static_code_gen_buffer + sizeof(static_code_gen_buffer);
+-    buf = QEMU_ALIGN_PTR_UP(buf, qemu_real_host_page_size);
+-    end = QEMU_ALIGN_PTR_DOWN(end, qemu_real_host_page_size);
+-
+-    size = end - buf;
+-
+-    /* Honor a command-line option limiting the size of the buffer.  */
+-    if (size > tb_size) {
+-        size = QEMU_ALIGN_DOWN(tb_size, qemu_real_host_page_size);
+-    }
+-    tcg_ctx->code_gen_buffer_size = size;
+-
+-#ifdef __mips__
+-    if (cross_256mb(buf, size)) {
+-        buf = split_cross_256mb(buf, size);
+-        size = tcg_ctx->code_gen_buffer_size;
+-    }
+-#endif
+-
+-    if (qemu_mprotect_rwx(buf, size)) {
+-        error_setg_errno(errp, errno, "mprotect of jit buffer");
+-        return false;
+-    }
+-    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+-
+-    tcg_ctx->code_gen_buffer = buf;
+-    return true;
+-}
+-#elif defined(_WIN32)
+-static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+-{
+-    void *buf;
+-
+-    if (splitwx > 0) {
+-        error_setg(errp, "jit split-wx not supported");
+-        return false;
+-    }
+-
+-    buf = VirtualAlloc(NULL, size, MEM_RESERVE | MEM_COMMIT,
+-                             PAGE_EXECUTE_READWRITE);
+-    if (buf == NULL) {
+-        error_setg_win32(errp, GetLastError(),
+-                         "allocate %zu bytes for jit buffer", size);
+-        return false;
+-    }
+-
+-    tcg_ctx->code_gen_buffer = buf;
+-    tcg_ctx->code_gen_buffer_size = size;
+-    return true;
+-}
+-#else
+-static bool alloc_code_gen_buffer_anon(size_t size, int prot,
+-                                       int flags, Error **errp)
+-{
+-    void *buf;
+-
+-    buf = mmap(NULL, size, prot, flags, -1, 0);
+-    if (buf == MAP_FAILED) {
+-        error_setg_errno(errp, errno,
+-                         "allocate %zu bytes for jit buffer", size);
+-        return false;
+-    }
+-    tcg_ctx->code_gen_buffer_size = size;
+-
+-#ifdef __mips__
+-    if (cross_256mb(buf, size)) {
+-        /*
+-         * Try again, with the original still mapped, to avoid re-acquiring
+-         * the same 256mb crossing.
+-         */
+-        size_t size2;
+-        void *buf2 = mmap(NULL, size, prot, flags, -1, 0);
+-        switch ((int)(buf2 != MAP_FAILED)) {
+-        case 1:
+-            if (!cross_256mb(buf2, size)) {
+-                /* Success!  Use the new buffer.  */
+-                munmap(buf, size);
+-                break;
+-            }
+-            /* Failure.  Work with what we had.  */
+-            munmap(buf2, size);
+-            /* fallthru */
+-        default:
+-            /* Split the original buffer.  Free the smaller half.  */
+-            buf2 = split_cross_256mb(buf, size);
+-            size2 = tcg_ctx->code_gen_buffer_size;
+-            if (buf == buf2) {
+-                munmap(buf + size2, size - size2);
+-            } else {
+-                munmap(buf, size - size2);
+-            }
+-            size = size2;
+-            break;
+-        }
+-        buf = buf2;
+-    }
+-#endif
+-
+-    /* Request large pages for the buffer.  */
+-    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+-
+-    tcg_ctx->code_gen_buffer = buf;
+-    return true;
+-}
+-
+-#ifndef CONFIG_TCG_INTERPRETER
+-#ifdef CONFIG_POSIX
+-#include "qemu/memfd.h"
+-
+-static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
+-{
+-    void *buf_rw = NULL, *buf_rx = MAP_FAILED;
+-    int fd = -1;
+-
+-#ifdef __mips__
+-    /* Find space for the RX mapping, vs the 256MiB regions. */
+-    if (!alloc_code_gen_buffer_anon(size, PROT_NONE,
+-                                    MAP_PRIVATE | MAP_ANONYMOUS |
+-                                    MAP_NORESERVE, errp)) {
+-        return false;
+-    }
+-    /* The size of the mapping may have been adjusted. */
+-    size = tcg_ctx->code_gen_buffer_size;
+-    buf_rx = tcg_ctx->code_gen_buffer;
+-#endif
+-
+-    buf_rw = qemu_memfd_alloc("tcg-jit", size, 0, &fd, errp);
+-    if (buf_rw == NULL) {
+-        goto fail;
+-    }
+-
+-#ifdef __mips__
+-    void *tmp = mmap(buf_rx, size, PROT_READ | PROT_EXEC,
+-                     MAP_SHARED | MAP_FIXED, fd, 0);
+-    if (tmp != buf_rx) {
+-        goto fail_rx;
+-    }
+-#else
+-    buf_rx = mmap(NULL, size, PROT_READ | PROT_EXEC, MAP_SHARED, fd, 0);
+-    if (buf_rx == MAP_FAILED) {
+-        goto fail_rx;
+-    }
+-#endif
+-
+-    close(fd);
+-    tcg_ctx->code_gen_buffer = buf_rw;
+-    tcg_ctx->code_gen_buffer_size = size;
+-    tcg_splitwx_diff = buf_rx - buf_rw;
+-
+-    /* Request large pages for the buffer and the splitwx.  */
+-    qemu_madvise(buf_rw, size, QEMU_MADV_HUGEPAGE);
+-    qemu_madvise(buf_rx, size, QEMU_MADV_HUGEPAGE);
+-    return true;
+-
+- fail_rx:
+-    error_setg_errno(errp, errno, "failed to map shared memory for execute");
+- fail:
+-    if (buf_rx != MAP_FAILED) {
+-        munmap(buf_rx, size);
+-    }
+-    if (buf_rw) {
+-        munmap(buf_rw, size);
+-    }
+-    if (fd >= 0) {
+-        close(fd);
+-    }
+-    return false;
+-}
+-#endif /* CONFIG_POSIX */
+-
+-#ifdef CONFIG_DARWIN
+-#include <mach/mach.h>
+-
+-extern kern_return_t mach_vm_remap(vm_map_t target_task,
+-                                   mach_vm_address_t *target_address,
+-                                   mach_vm_size_t size,
+-                                   mach_vm_offset_t mask,
+-                                   int flags,
+-                                   vm_map_t src_task,
+-                                   mach_vm_address_t src_address,
+-                                   boolean_t copy,
+-                                   vm_prot_t *cur_protection,
+-                                   vm_prot_t *max_protection,
+-                                   vm_inherit_t inheritance);
+-
+-static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
+-{
+-    kern_return_t ret;
+-    mach_vm_address_t buf_rw, buf_rx;
+-    vm_prot_t cur_prot, max_prot;
+-
+-    /* Map the read-write portion via normal anon memory. */
+-    if (!alloc_code_gen_buffer_anon(size, PROT_READ | PROT_WRITE,
+-                                    MAP_PRIVATE | MAP_ANONYMOUS, errp)) {
+-        return false;
+-    }
+-
+-    buf_rw = (mach_vm_address_t)tcg_ctx->code_gen_buffer;
+-    buf_rx = 0;
+-    ret = mach_vm_remap(mach_task_self(),
+-                        &buf_rx,
+-                        size,
+-                        0,
+-                        VM_FLAGS_ANYWHERE,
+-                        mach_task_self(),
+-                        buf_rw,
+-                        false,
+-                        &cur_prot,
+-                        &max_prot,
+-                        VM_INHERIT_NONE);
+-    if (ret != KERN_SUCCESS) {
+-        /* TODO: Convert "ret" to a human readable error message. */
+-        error_setg(errp, "vm_remap for jit splitwx failed");
+-        munmap((void *)buf_rw, size);
+-        return false;
+-    }
+-
+-    if (mprotect((void *)buf_rx, size, PROT_READ | PROT_EXEC) != 0) {
+-        error_setg_errno(errp, errno, "mprotect for jit splitwx");
+-        munmap((void *)buf_rx, size);
+-        munmap((void *)buf_rw, size);
+-        return false;
+-    }
+-
+-    tcg_splitwx_diff = buf_rx - buf_rw;
+-    return true;
+-}
+-#endif /* CONFIG_DARWIN */
+-#endif /* CONFIG_TCG_INTERPRETER */
+-
+-static bool alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
+-{
+-#ifndef CONFIG_TCG_INTERPRETER
+-# ifdef CONFIG_DARWIN
+-    return alloc_code_gen_buffer_splitwx_vmremap(size, errp);
+-# endif
+-# ifdef CONFIG_POSIX
+-    return alloc_code_gen_buffer_splitwx_memfd(size, errp);
+-# endif
+-#endif
+-    error_setg(errp, "jit split-wx not supported");
+-    return false;
+-}
+-
+-static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+-{
+-    ERRP_GUARD();
+-    int prot, flags;
+-
+-    if (splitwx) {
+-        if (alloc_code_gen_buffer_splitwx(size, errp)) {
+-            return true;
+-        }
+-        /*
+-         * If splitwx force-on (1), fail;
+-         * if splitwx default-on (-1), fall through to splitwx off.
+-         */
+-        if (splitwx > 0) {
+-            return false;
+-        }
+-        error_free_or_abort(errp);
+-    }
+-
+-    prot = PROT_READ | PROT_WRITE | PROT_EXEC;
+-    flags = MAP_PRIVATE | MAP_ANONYMOUS;
+-#ifdef CONFIG_TCG_INTERPRETER
+-    /* The tcg interpreter does not need execute permission. */
+-    prot = PROT_READ | PROT_WRITE;
+-#elif defined(CONFIG_DARWIN)
+-    /* Applicable to both iOS and macOS (Apple Silicon). */
+-    if (!splitwx) {
+-        flags |= MAP_JIT;
+-    }
+-#endif
+-
+-    return alloc_code_gen_buffer_anon(size, prot, flags, errp);
+-}
+-#endif /* USE_STATIC_CODE_GEN_BUFFER, WIN32, POSIX */
+-
+ static bool tb_cmp(const void *ap, const void *bp)
+ {
+     const TranslationBlock *a = ap;
+@@ -XXX,XX +XXX,XX @@ static void tb_htable_init(void)
+    size. */
+ void tcg_exec_init(unsigned long tb_size, int splitwx)
+ {
+-    bool ok;
+-
+     tcg_allowed = true;
+     tcg_context_init(&tcg_init_ctx);
+     page_init();
+     tb_htable_init();
+-
+-    ok = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
+-                               splitwx, &error_fatal);
+-    assert(ok);
+-
+-    /* TODO: allocating regions is hand-in-glove with code_gen_buffer. */
+-    tcg_region_init();
++    tcg_region_init(tb_size, splitwx);
+ #if defined(CONFIG_SOFTMMU)
+     /* There's no guest base to take into account, so go ahead and
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@
+  */
+ #include "qemu/osdep.h"
++#include "qemu/units.h"
++#include "qapi/error.h"
+ #include "exec/exec-all.h"
+ #include "tcg/tcg.h"
+ #if !defined(CONFIG_USER_ONLY)
+@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(void)
+ }
+ #endif
++/*
++ * Minimum size of the code gen buffer.  This number is randomly chosen,
++ * but not so small that we can't have a fair number of TB's live.
++ */
++#define MIN_CODE_GEN_BUFFER_SIZE     (1 * MiB)
++
++/*
++ * Maximum size of the code gen buffer we'd like to use.  Unless otherwise
++ * indicated, this is constrained by the range of direct branches on the
++ * host cpu, as used by the TCG implementation of goto_tb.
++ */
++#if defined(__x86_64__)
++# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
++#elif defined(__sparc__)
++# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
++#elif defined(__powerpc64__)
++# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
++#elif defined(__powerpc__)
++# define MAX_CODE_GEN_BUFFER_SIZE  (32 * MiB)
++#elif defined(__aarch64__)
++# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
++#elif defined(__s390x__)
++  /* We have a +- 4GB range on the branches; leave some slop.  */
++# define MAX_CODE_GEN_BUFFER_SIZE  (3 * GiB)
++#elif defined(__mips__)
++  /*
++   * We have a 256MB branch region, but leave room to make sure the
++   * main executable is also within that region.
++   */
++# define MAX_CODE_GEN_BUFFER_SIZE  (128 * MiB)
++#else
++# define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
++#endif
++
++#if TCG_TARGET_REG_BITS == 32
++#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (32 * MiB)
++#ifdef CONFIG_USER_ONLY
++/*
++ * For user mode on smaller 32 bit systems we may run into trouble
++ * allocating big chunks of data in the right place. On these systems
++ * we utilise a static code generation buffer directly in the binary.
++ */
++#define USE_STATIC_CODE_GEN_BUFFER
++#endif
++#else /* TCG_TARGET_REG_BITS == 64 */
++#ifdef CONFIG_USER_ONLY
++/*
++ * As user-mode emulation typically means running multiple instances
++ * of the translator don't go too nuts with our default code gen
++ * buffer lest we make things too hard for the OS.
++ */
++#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (128 * MiB)
++#else
++/*
++ * We expect most system emulation to run one or two guests per host.
++ * Users running large scale system emulation may want to tweak their
++ * runtime setup via the tb-size control on the command line.
++ */
++#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (1 * GiB)
++#endif
++#endif
++
++#define DEFAULT_CODE_GEN_BUFFER_SIZE \
++  (DEFAULT_CODE_GEN_BUFFER_SIZE_1 < MAX_CODE_GEN_BUFFER_SIZE \
++   ? DEFAULT_CODE_GEN_BUFFER_SIZE_1 : MAX_CODE_GEN_BUFFER_SIZE)
++
++static size_t size_code_gen_buffer(size_t tb_size)
++{
++    /* Size the buffer.  */
++    if (tb_size == 0) {
++        size_t phys_mem = qemu_get_host_physmem();
++        if (phys_mem == 0) {
++            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
++        } else {
++            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
++        }
++    }
++    if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
++        tb_size = MIN_CODE_GEN_BUFFER_SIZE;
++    }
++    if (tb_size > MAX_CODE_GEN_BUFFER_SIZE) {
++        tb_size = MAX_CODE_GEN_BUFFER_SIZE;
++    }
++    return tb_size;
++}
++
++#ifdef __mips__
++/*
++ * In order to use J and JAL within the code_gen_buffer, we require
++ * that the buffer not cross a 256MB boundary.
++ */
++static inline bool cross_256mb(void *addr, size_t size)
++{
++    return ((uintptr_t)addr ^ ((uintptr_t)addr + size)) & ~0x0ffffffful;
++}
++
++/*
++ * We weren't able to allocate a buffer without crossing that boundary,
++ * so make do with the larger portion of the buffer that doesn't cross.
++ * Returns the new base of the buffer, and adjusts code_gen_buffer_size.
++ */
++static inline void *split_cross_256mb(void *buf1, size_t size1)
++{
++    void *buf2 = (void *)(((uintptr_t)buf1 + size1) & ~0x0ffffffful);
++    size_t size2 = buf1 + size1 - buf2;
++
++    size1 = buf2 - buf1;
++    if (size1 < size2) {
++        size1 = size2;
++        buf1 = buf2;
++    }
++
++    tcg_ctx->code_gen_buffer_size = size1;
++    return buf1;
++}
++#endif
++
++#ifdef USE_STATIC_CODE_GEN_BUFFER
++static uint8_t static_code_gen_buffer[DEFAULT_CODE_GEN_BUFFER_SIZE]
++    __attribute__((aligned(CODE_GEN_ALIGN)));
++
++static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
++{
++    void *buf, *end;
++    size_t size;
++
++    if (splitwx > 0) {
++        error_setg(errp, "jit split-wx not supported");
++        return false;
++    }
++
++    /* page-align the beginning and end of the buffer */
++    buf = static_code_gen_buffer;
++    end = static_code_gen_buffer + sizeof(static_code_gen_buffer);
++    buf = QEMU_ALIGN_PTR_UP(buf, qemu_real_host_page_size);
++    end = QEMU_ALIGN_PTR_DOWN(end, qemu_real_host_page_size);
++
++    size = end - buf;
++
++    /* Honor a command-line option limiting the size of the buffer.  */
++    if (size > tb_size) {
++        size = QEMU_ALIGN_DOWN(tb_size, qemu_real_host_page_size);
++    }
++    tcg_ctx->code_gen_buffer_size = size;
++
++#ifdef __mips__
++    if (cross_256mb(buf, size)) {
++        buf = split_cross_256mb(buf, size);
++        size = tcg_ctx->code_gen_buffer_size;
++    }
++#endif
++
++    if (qemu_mprotect_rwx(buf, size)) {
++        error_setg_errno(errp, errno, "mprotect of jit buffer");
++        return false;
++    }
++    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
++
++    tcg_ctx->code_gen_buffer = buf;
++    return true;
++}
++#elif defined(_WIN32)
++static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
++{
++    void *buf;
++
++    if (splitwx > 0) {
++        error_setg(errp, "jit split-wx not supported");
++        return false;
++    }
++
++    buf = VirtualAlloc(NULL, size, MEM_RESERVE | MEM_COMMIT,
++                             PAGE_EXECUTE_READWRITE);
++    if (buf == NULL) {
++        error_setg_win32(errp, GetLastError(),
++                         "allocate %zu bytes for jit buffer", size);
++        return false;
++    }
++
++    tcg_ctx->code_gen_buffer = buf;
++    tcg_ctx->code_gen_buffer_size = size;
++    return true;
++}
++#else
++static bool alloc_code_gen_buffer_anon(size_t size, int prot,
++                                       int flags, Error **errp)
++{
++    void *buf;
++
++    buf = mmap(NULL, size, prot, flags, -1, 0);
++    if (buf == MAP_FAILED) {
++        error_setg_errno(errp, errno,
++                         "allocate %zu bytes for jit buffer", size);
++        return false;
++    }
++    tcg_ctx->code_gen_buffer_size = size;
++
++#ifdef __mips__
++    if (cross_256mb(buf, size)) {
++        /*
++         * Try again, with the original still mapped, to avoid re-acquiring
++         * the same 256mb crossing.
++         */
++        size_t size2;
++        void *buf2 = mmap(NULL, size, prot, flags, -1, 0);
++        switch ((int)(buf2 != MAP_FAILED)) {
++        case 1:
++            if (!cross_256mb(buf2, size)) {
++                /* Success!  Use the new buffer.  */
++                munmap(buf, size);
++                break;
++            }
++            /* Failure.  Work with what we had.  */
++            munmap(buf2, size);
++            /* fallthru */
++        default:
++            /* Split the original buffer.  Free the smaller half.  */
++            buf2 = split_cross_256mb(buf, size);
++            size2 = tcg_ctx->code_gen_buffer_size;
++            if (buf == buf2) {
++                munmap(buf + size2, size - size2);
++            } else {
++                munmap(buf, size - size2);
++            }
++            size = size2;
++            break;
++        }
++        buf = buf2;
++    }
++#endif
++
++    /* Request large pages for the buffer.  */
++    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
++
++    tcg_ctx->code_gen_buffer = buf;
++    return true;
++}
++
++#ifndef CONFIG_TCG_INTERPRETER
++#ifdef CONFIG_POSIX
++#include "qemu/memfd.h"
++
++static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
++{
++    void *buf_rw = NULL, *buf_rx = MAP_FAILED;
++    int fd = -1;
++
++#ifdef __mips__
++    /* Find space for the RX mapping, vs the 256MiB regions. */
++    if (!alloc_code_gen_buffer_anon(size, PROT_NONE,
++                                    MAP_PRIVATE | MAP_ANONYMOUS |
++                                    MAP_NORESERVE, errp)) {
++        return false;
++    }
++    /* The size of the mapping may have been adjusted. */
++    size = tcg_ctx->code_gen_buffer_size;
++    buf_rx = tcg_ctx->code_gen_buffer;
++#endif
++
++    buf_rw = qemu_memfd_alloc("tcg-jit", size, 0, &fd, errp);
++    if (buf_rw == NULL) {
++        goto fail;
++    }
++
++#ifdef __mips__
++    void *tmp = mmap(buf_rx, size, PROT_READ | PROT_EXEC,
++                     MAP_SHARED | MAP_FIXED, fd, 0);
++    if (tmp != buf_rx) {
++        goto fail_rx;
++    }
++#else
++    buf_rx = mmap(NULL, size, PROT_READ | PROT_EXEC, MAP_SHARED, fd, 0);
++    if (buf_rx == MAP_FAILED) {
++        goto fail_rx;
++    }
++#endif
++
++    close(fd);
++    tcg_ctx->code_gen_buffer = buf_rw;
++    tcg_ctx->code_gen_buffer_size = size;
++    tcg_splitwx_diff = buf_rx - buf_rw;
++
++    /* Request large pages for the buffer and the splitwx.  */
++    qemu_madvise(buf_rw, size, QEMU_MADV_HUGEPAGE);
++    qemu_madvise(buf_rx, size, QEMU_MADV_HUGEPAGE);
++    return true;
++
++ fail_rx:
++    error_setg_errno(errp, errno, "failed to map shared memory for execute");
++ fail:
++    if (buf_rx != MAP_FAILED) {
++        munmap(buf_rx, size);
++    }
++    if (buf_rw) {
++        munmap(buf_rw, size);
++    }
++    if (fd >= 0) {
++        close(fd);
++    }
++    return false;
++}
++#endif /* CONFIG_POSIX */
++
++#ifdef CONFIG_DARWIN
++#include <mach/mach.h>
++
++extern kern_return_t mach_vm_remap(vm_map_t target_task,
++                                   mach_vm_address_t *target_address,
++                                   mach_vm_size_t size,
++                                   mach_vm_offset_t mask,
++                                   int flags,
++                                   vm_map_t src_task,
++                                   mach_vm_address_t src_address,
++                                   boolean_t copy,
++                                   vm_prot_t *cur_protection,
++                                   vm_prot_t *max_protection,
++                                   vm_inherit_t inheritance);
++
++static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
++{
++    kern_return_t ret;
++    mach_vm_address_t buf_rw, buf_rx;
++    vm_prot_t cur_prot, max_prot;
++
++    /* Map the read-write portion via normal anon memory. */
++    if (!alloc_code_gen_buffer_anon(size, PROT_READ | PROT_WRITE,
++                                    MAP_PRIVATE | MAP_ANONYMOUS, errp)) {
++        return false;
++    }
++
++    buf_rw = (mach_vm_address_t)tcg_ctx->code_gen_buffer;
++    buf_rx = 0;
++    ret = mach_vm_remap(mach_task_self(),
++                        &buf_rx,
++                        size,
++                        0,
++                        VM_FLAGS_ANYWHERE,
++                        mach_task_self(),
++                        buf_rw,
++                        false,
++                        &cur_prot,
++                        &max_prot,
++                        VM_INHERIT_NONE);
++    if (ret != KERN_SUCCESS) {
++        /* TODO: Convert "ret" to a human readable error message. */
++        error_setg(errp, "vm_remap for jit splitwx failed");
++        munmap((void *)buf_rw, size);
++        return false;
++    }
++
++    if (mprotect((void *)buf_rx, size, PROT_READ | PROT_EXEC) != 0) {
++        error_setg_errno(errp, errno, "mprotect for jit splitwx");
++        munmap((void *)buf_rx, size);
++        munmap((void *)buf_rw, size);
++        return false;
++    }
++
++    tcg_splitwx_diff = buf_rx - buf_rw;
++    return true;
++}
++#endif /* CONFIG_DARWIN */
++#endif /* CONFIG_TCG_INTERPRETER */
++
++static bool alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
++{
++#ifndef CONFIG_TCG_INTERPRETER
++# ifdef CONFIG_DARWIN
++    return alloc_code_gen_buffer_splitwx_vmremap(size, errp);
++# endif
++# ifdef CONFIG_POSIX
++    return alloc_code_gen_buffer_splitwx_memfd(size, errp);
++# endif
++#endif
++    error_setg(errp, "jit split-wx not supported");
++    return false;
++}
++
++static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
++{
++    ERRP_GUARD();
++    int prot, flags;
++
++    if (splitwx) {
++        if (alloc_code_gen_buffer_splitwx(size, errp)) {
++            return true;
++        }
++        /*
++         * If splitwx force-on (1), fail;
++         * if splitwx default-on (-1), fall through to splitwx off.
++         */
++        if (splitwx > 0) {
++            return false;
++        }
++        error_free_or_abort(errp);
++    }
++
++    prot = PROT_READ | PROT_WRITE | PROT_EXEC;
++    flags = MAP_PRIVATE | MAP_ANONYMOUS;
++#ifdef CONFIG_TCG_INTERPRETER
++    /* The tcg interpreter does not need execute permission. */
++    prot = PROT_READ | PROT_WRITE;
++#elif defined(CONFIG_DARWIN)
++    /* Applicable to both iOS and macOS (Apple Silicon). */
++    if (!splitwx) {
++        flags |= MAP_JIT;
++    }
++#endif
++
++    return alloc_code_gen_buffer_anon(size, prot, flags, errp);
++}
++#endif /* USE_STATIC_CODE_GEN_BUFFER, WIN32, POSIX */
++
+ /*
+  * Initializes region partitioning.
+  *
+@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(void)
+  * in practice. Multi-threaded guests share most if not all of their translated
+  * code, which makes parallel code generation less appealing than in softmmu.
+  */
+-void tcg_region_init(void)
++void tcg_region_init(size_t tb_size, int splitwx)
+ {
+-    void *buf = tcg_init_ctx.code_gen_buffer;
+-    void *aligned;
+-    size_t size = tcg_init_ctx.code_gen_buffer_size;
+-    size_t page_size = qemu_real_host_page_size;
++    void *buf, *aligned;
++    size_t size;
++    size_t page_size;
+     size_t region_size;
+     size_t n_regions;
+     size_t i;
++    bool ok;
++    ok = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
++                               splitwx, &error_fatal);
++    assert(ok);
++
++    buf = tcg_init_ctx.code_gen_buffer;
++    size = tcg_init_ctx.code_gen_buffer_size;
++    page_size = qemu_real_host_page_size;
+     n_regions = tcg_n_regions();
+     /* The first region will be 'aligned - buf' bytes larger than the others */
+--
+.25.1

-New patch
+[PULL 10/34] accel/tcg: Rename tcg_init to tcg_init_machine
+We shortly want to use tcg_init for something else.
+Since the hook is called init_machine, match that.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ accel/tcg/tcg-all.c | 4 ++--
+file changed, 2 insertions(+), 2 deletions(-)
+diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
+index XXXXXXX..XXXXXXX 100644
+--- a/accel/tcg/tcg-all.c
++++ b/accel/tcg/tcg-all.c
+@@ -XXX,XX +XXX,XX @@ static void tcg_accel_instance_init(Object *obj)
+ bool mttcg_enabled;
+-static int tcg_init(MachineState *ms)
++static int tcg_init_machine(MachineState *ms)
+ {
+     TCGState *s = TCG_STATE(current_accel());
+@@ -XXX,XX +XXX,XX @@ static void tcg_accel_class_init(ObjectClass *oc, void *data)
+ {
+     AccelClass *ac = ACCEL_CLASS(oc);
+     ac->name = "tcg";
+-    ac->init_machine = tcg_init;
++    ac->init_machine = tcg_init_machine;
+     ac->allowed = &tcg_allowed;
+     object_class_property_add_str(oc, "thread",
+--
+.25.1

-New patch
+[PULL 11/34] tcg: Create tcg_init
+Perform both tcg_context_init and tcg_region_init.
+Do not leave this split to the caller.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ include/tcg/tcg.h         | 3 +--
+ tcg/tcg-internal.h        | 1 +
+ accel/tcg/translate-all.c | 3 +--
+ tcg/tcg.c                 | 9 ++++++++-
+files changed, 11 insertions(+), 5 deletions(-)
+diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
+index XXXXXXX..XXXXXXX 100644
+--- a/include/tcg/tcg.h
++++ b/include/tcg/tcg.h
+@@ -XXX,XX +XXX,XX @@ void *tcg_malloc_internal(TCGContext *s, int size);
+ void tcg_pool_reset(TCGContext *s);
+ TranslationBlock *tcg_tb_alloc(TCGContext *s);
+-void tcg_region_init(size_t tb_size, int splitwx);
+ void tb_destroy(TranslationBlock *tb);
+ void tcg_region_reset_all(void);
+@@ -XXX,XX +XXX,XX @@ static inline void *tcg_malloc(int size)
+     }
+ }
+-void tcg_context_init(TCGContext *s);
++void tcg_init(size_t tb_size, int splitwx);
+ void tcg_register_thread(void);
+ void tcg_prologue_init(TCGContext *s);
+ void tcg_func_start(TCGContext *s);
+diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg-internal.h
++++ b/tcg/tcg-internal.h
+@@ -XXX,XX +XXX,XX @@
+ extern TCGContext **tcg_ctxs;
+ extern unsigned int n_tcg_ctxs;
++void tcg_region_init(size_t tb_size, int splitwx);
+ bool tcg_region_alloc(TCGContext *s);
+ void tcg_region_initial_alloc(TCGContext *s);
+ void tcg_region_prologue_set(TCGContext *s);
+diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
+index XXXXXXX..XXXXXXX 100644
+--- a/accel/tcg/translate-all.c
++++ b/accel/tcg/translate-all.c
+@@ -XXX,XX +XXX,XX @@ static void tb_htable_init(void)
+ void tcg_exec_init(unsigned long tb_size, int splitwx)
+ {
+     tcg_allowed = true;
+-    tcg_context_init(&tcg_init_ctx);
+     page_init();
+     tb_htable_init();
+-    tcg_region_init(tb_size, splitwx);
++    tcg_init(tb_size, splitwx);
+ #if defined(CONFIG_SOFTMMU)
+     /* There's no guest base to take into account, so go ahead and
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@ static void process_op_defs(TCGContext *s);
+ static TCGTemp *tcg_global_reg_new_internal(TCGContext *s, TCGType type,
+                                             TCGReg reg, const char *name);
+-void tcg_context_init(TCGContext *s)
++static void tcg_context_init(void)
+ {
++    TCGContext *s = &tcg_init_ctx;
+     int op, total_args, n, i;
+     TCGOpDef *def;
+     TCGArgConstraint *args_ct;
+@@ -XXX,XX +XXX,XX @@ void tcg_context_init(TCGContext *s)
+     cpu_env = temp_tcgv_ptr(ts);
+ }
++void tcg_init(size_t tb_size, int splitwx)
++{
++    tcg_context_init();
++    tcg_region_init(tb_size, splitwx);
++}
++
+ /*
+  * Allocate TBs right before their corresponding translated code, making
+  * sure that TBs and code are on different cache lines.
+--
+.25.1

-[PULL 2/3] accel/tcg: split tcg_start_vcpu_thread
+[PULL 12/34] accel/tcg: Merge tcg_exec_init into tcg_init_machine
-From: Claudio Fontana <cfontana@suse.de>
+There is only one caller, and shortly we will need access
 to the MachineState, which tcg_init_machine already has.
-after the initial split into 3 tcg variants, we proceed to also
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
-split tcg_start_vcpu_thread.
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 We actually split it in 2 this time, since the icount variant
 just uses the round robin function.
 Suggested-by: Richard Henderson <richard.henderson@linaro.org>
 Signed-off-by: Claudio Fontana <cfontana@suse.de>
 Message-Id: <20201015143217.29337-3-cfontana@suse.de>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/tcg-cpus-mttcg.h  | 21 --------------
+ accel/tcg/internal.h      |  2 ++
- accel/tcg/tcg-cpus-rr.h     |  3 +-
+ include/sysemu/tcg.h      |  2 --
- accel/tcg/tcg-cpus.h        |  1 -
+ accel/tcg/tcg-all.c       | 16 +++++++++++++++-
- accel/tcg/tcg-all.c         |  5 ++++
+ accel/tcg/translate-all.c | 21 ++-------------------
- accel/tcg/tcg-cpus-icount.c |  2 +-
+ bsd-user/main.c           |  2 +-
- accel/tcg/tcg-cpus-mttcg.c  | 29 +++++++++++++++++--
+files changed, 20 insertions(+), 23 deletions(-)
  accel/tcg/tcg-cpus-rr.c     | 39 +++++++++++++++++++++++--
  accel/tcg/tcg-cpus.c        | 58 -------------------------------------
 files changed, 71 insertions(+), 87 deletions(-)
  delete mode 100644 accel/tcg/tcg-cpus-mttcg.h
-diff --git a/accel/tcg/tcg-cpus-mttcg.h b/accel/tcg/tcg-cpus-mttcg.h
+diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
-deleted file mode 100644
+index XXXXXXX..XXXXXXX 100644
-index XXXXXXX..XXXXXXX
+--- a/accel/tcg/internal.h
---- a/accel/tcg/tcg-cpus-mttcg.h
++++ b/accel/tcg/internal.h
-+++ /dev/null
+@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu, target_ulong pc,
                                int cflags);
  void QEMU_NORETURN cpu_io_recompile(CPUState *cpu, uintptr_t retaddr);
 +void page_init(void);
 +void tb_htable_init(void);
  #endif /* ACCEL_TCG_INTERNAL_H */
 diff --git a/include/sysemu/tcg.h b/include/sysemu/tcg.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/sysemu/tcg.h
 +++ b/include/sysemu/tcg.h
 @@ -XXX,XX +XXX,XX @@
--/*
+ #ifndef SYSEMU_TCG_H
-- * QEMU TCG Multi Threaded vCPUs implementation
+ #define SYSEMU_TCG_H
-- *
-- * Copyright 2020 SUSE LLC
+-void tcg_exec_init(unsigned long tb_size, int splitwx);
 - *
 - * This work is licensed under the terms of the GNU GPL, version 2 or later.
 - * See the COPYING file in the top-level directory.
 - */
 -
--#ifndef TCG_CPUS_MTTCG_H
+ #ifdef CONFIG_TCG
--#define TCG_CPUS_MTTCG_H
+ extern bool tcg_allowed;
--
+ #define tcg_enabled() (tcg_allowed)
 -/*
 - * In the multi-threaded case each vCPU has its own thread. The TLS
 - * variable current_cpu can be used deep in the code to find the
 - * current CPUState for a given thread.
 - */
 -
 -void *tcg_cpu_thread_fn(void *arg);
 -
 -#endif /* TCG_CPUS_MTTCG_H */
 diff --git a/accel/tcg/tcg-cpus-rr.h b/accel/tcg/tcg-cpus-rr.h
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus-rr.h
 +++ b/accel/tcg/tcg-cpus-rr.h
@@ -XXX,XX +XXX,XX @@
  /* Kick all RR vCPUs. */
  void qemu_cpu_kick_rr_cpus(CPUState *unused);
 -void *tcg_rr_cpu_thread_fn(void *arg);
 +/* start the round robin vcpu thread */
 +void rr_start_vcpu_thread(CPUState *cpu);
  #endif /* TCG_CPUS_RR_H */
 diff --git a/accel/tcg/tcg-cpus.h b/accel/tcg/tcg-cpus.h
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus.h
 +++ b/accel/tcg/tcg-cpus.h
@@ -XXX,XX +XXX,XX @@ extern const CpusAccel tcg_cpus_mttcg;
  extern const CpusAccel tcg_cpus_icount;
  extern const CpusAccel tcg_cpus_rr;
 -void tcg_start_vcpu_thread(CPUState *cpu);
  void qemu_tcg_destroy_vcpu(CPUState *cpu);
  int tcg_cpu_exec(CPUState *cpu);
  void tcg_handle_interrupt(CPUState *cpu, int mask);
 diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-all.c
 +++ b/accel/tcg/tcg-all.c
-@@ -XXX,XX +XXX,XX @@ static int tcg_init(MachineState *ms)
+@@ -XXX,XX +XXX,XX @@
-     tcg_exec_init(s->tb_size * 1024 * 1024);
+ #include "qemu/error-report.h"
  #include "qemu/accel.h"
  #include "qapi/qapi-builtin-visit.h"
 +#include "internal.h"
  struct TCGState {
      AccelState parent_obj;
@@ -XXX,XX +XXX,XX @@ static int tcg_init_machine(MachineState *ms)
  {
      TCGState *s = TCG_STATE(current_accel());
 -    tcg_exec_init(s->tb_size * 1024 * 1024, s->splitwx_enabled);
 +    tcg_allowed = true;
      mttcg_enabled = s->mttcg_enabled;
++
 +    page_init();
 +    tb_htable_init();
 +    tcg_init(s->tb_size * 1024 * 1024, s->splitwx_enabled);
 +
 +#if defined(CONFIG_SOFTMMU)
 +    /*
-+     * Initialize TCG regions
++     * There's no guest base to take into account, so go ahead and
 +     * initialize the prologue now.
 +     */
-+    tcg_region_init();
++    tcg_prologue_init(tcg_ctx);
 +#endif
 +
-     if (mttcg_enabled) {
+     return 0;
-         cpus_register_accel(&tcg_cpus_mttcg);
+ }
-     } else if (icount_enabled()) {
-diff --git a/accel/tcg/tcg-cpus-icount.c b/accel/tcg/tcg-cpus-icount.c
+diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/tcg-cpus-icount.c
+--- a/accel/tcg/translate-all.c
-+++ b/accel/tcg/tcg-cpus-icount.c
++++ b/accel/tcg/translate-all.c
-@@ -XXX,XX +XXX,XX @@ static void icount_handle_interrupt(CPUState *cpu, int mask)
+@@ -XXX,XX +XXX,XX @@ bool cpu_restore_state(CPUState *cpu, uintptr_t host_pc, bool will_exit)
      return false;
  }
- const CpusAccel tcg_cpus_icount = {
+-static void page_init(void)
--    .create_vcpu_thread = tcg_start_vcpu_thread,
++void page_init(void)
 +    .create_vcpu_thread = rr_start_vcpu_thread,
      .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
      .handle_interrupt = icount_handle_interrupt,
 diff --git a/accel/tcg/tcg-cpus-mttcg.c b/accel/tcg/tcg-cpus-mttcg.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus-mttcg.c
 +++ b/accel/tcg/tcg-cpus-mttcg.c
@@ -XXX,XX +XXX,XX @@
  #include "hw/boards.h"
  #include "tcg-cpus.h"
 -#include "tcg-cpus-mttcg.h"
  /*
   * In the multi-threaded case each vCPU has its own thread. The TLS
@@ -XXX,XX +XXX,XX @@
   * current CPUState for a given thread.
   */
 -void *tcg_cpu_thread_fn(void *arg)
 +static void *tcg_cpu_thread_fn(void *arg)
  {
-     CPUState *cpu = arg;
+     page_size_init();
+     page_table_config_init();
-@@ -XXX,XX +XXX,XX @@ static void mttcg_kick_vcpu_thread(CPUState *cpu)
+@@ -XXX,XX +XXX,XX @@ static bool tb_cmp(const void *ap, const void *bp)
-     cpu_exit(cpu);
+         a->page_addr[1] == b->page_addr[1];
  }
-+static void mttcg_start_vcpu_thread(CPUState *cpu)
+-static void tb_htable_init(void)
-+{
++void tb_htable_init(void)
 +    char thread_name[VCPU_THREAD_NAME_SIZE];
 +
 +    g_assert(tcg_enabled());
 +
 +    parallel_cpus = (current_machine->smp.max_cpus > 1);
 +
 +    cpu->thread = g_malloc0(sizeof(QemuThread));
 +    cpu->halt_cond = g_malloc0(sizeof(QemuCond));
 +    qemu_cond_init(cpu->halt_cond);
 +
 +    /* create a thread per vCPU with TCG (MTTCG) */
 +    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG",
 +             cpu->cpu_index);
 +
 +    qemu_thread_create(cpu->thread, thread_name, tcg_cpu_thread_fn,
 +                       cpu, QEMU_THREAD_JOINABLE);
 +
 +#ifdef _WIN32
 +    cpu->hThread = qemu_thread_get_handle(cpu->thread);
 +#endif
 +}
 +
  const CpusAccel tcg_cpus_mttcg = {
 -    .create_vcpu_thread = tcg_start_vcpu_thread,
 +    .create_vcpu_thread = mttcg_start_vcpu_thread,
      .kick_vcpu_thread = mttcg_kick_vcpu_thread,
      .handle_interrupt = tcg_handle_interrupt,
 diff --git a/accel/tcg/tcg-cpus-rr.c b/accel/tcg/tcg-cpus-rr.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus-rr.c
 +++ b/accel/tcg/tcg-cpus-rr.c
@@ -XXX,XX +XXX,XX @@ static void deal_with_unplugged_cpus(void)
   * elsewhere.
   */
 -void *tcg_rr_cpu_thread_fn(void *arg)
 +static void *tcg_rr_cpu_thread_fn(void *arg)
  {
-     CPUState *cpu = arg;
+     unsigned int mode = QHT_MODE_AUTO_RESIZE;
-@@ -XXX,XX +XXX,XX @@ void *tcg_rr_cpu_thread_fn(void *arg)
+     qht_init(&tb_ctx.htable, tb_cmp, CODE_GEN_HTABLE_SIZE, mode);
      return NULL;
  }
-+void rr_start_vcpu_thread(CPUState *cpu)
+-/* Must be called before using the QEMU cpus. 'tb_size' is the size
-+{
+-   (in bytes) allocated to the translation buffer. Zero means default
-+    char thread_name[VCPU_THREAD_NAME_SIZE];
+-   size. */
-+    static QemuCond *single_tcg_halt_cond;
+-void tcg_exec_init(unsigned long tb_size, int splitwx)
 +    static QemuThread *single_tcg_cpu_thread;
 +
 +    g_assert(tcg_enabled());
 +    parallel_cpus = false;
 +
 +    if (!single_tcg_cpu_thread) {
 +        cpu->thread = g_malloc0(sizeof(QemuThread));
 +        cpu->halt_cond = g_malloc0(sizeof(QemuCond));
 +        qemu_cond_init(cpu->halt_cond);
 +
 +        /* share a single thread for all cpus with TCG */
 +        snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "ALL CPUs/TCG");
 +        qemu_thread_create(cpu->thread, thread_name,
 +                           tcg_rr_cpu_thread_fn,
 +                           cpu, QEMU_THREAD_JOINABLE);
 +
 +        single_tcg_halt_cond = cpu->halt_cond;
 +        single_tcg_cpu_thread = cpu->thread;
 +#ifdef _WIN32
 +        cpu->hThread = qemu_thread_get_handle(cpu->thread);
 +#endif
 +    } else {
 +        /* we share the thread */
 +        cpu->thread = single_tcg_cpu_thread;
 +        cpu->halt_cond = single_tcg_halt_cond;
 +        cpu->thread_id = first_cpu->thread_id;
 +        cpu->can_do_io = 1;
 +        cpu->created = true;
 +    }
 +}
 +
  const CpusAccel tcg_cpus_rr = {
 -    .create_vcpu_thread = tcg_start_vcpu_thread,
 +    .create_vcpu_thread = rr_start_vcpu_thread,
      .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
      .handle_interrupt = tcg_handle_interrupt,
 diff --git a/accel/tcg/tcg-cpus.c b/accel/tcg/tcg-cpus.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus.c
 +++ b/accel/tcg/tcg-cpus.c
@@ -XXX,XX +XXX,XX @@
  #include "hw/boards.h"
  #include "tcg-cpus.h"
 -#include "tcg-cpus-mttcg.h"
 -#include "tcg-cpus-rr.h"
  /* common functionality among all TCG variants */
 -void tcg_start_vcpu_thread(CPUState *cpu)
 -{
--    char thread_name[VCPU_THREAD_NAME_SIZE];
+-    tcg_allowed = true;
--    static QemuCond *single_tcg_halt_cond;
+-    page_init();
--    static QemuThread *single_tcg_cpu_thread;
+-    tb_htable_init();
--    static int tcg_region_inited;
+-    tcg_init(tb_size, splitwx);
 -
--    assert(tcg_enabled());
+-#if defined(CONFIG_SOFTMMU)
--    /*
+-    /* There's no guest base to take into account, so go ahead and
--     * Initialize TCG regions--once. Now is a good time, because:
+-       initialize the prologue now.  */
--     * (1) TCG's init context, prologue and target globals have been set up.
+-    tcg_prologue_init(tcg_ctx);
 -     * (2) qemu_tcg_mttcg_enabled() works now (TCG init code runs before the
 -     *     -accel flag is processed, so the check doesn't work then).
 -     */
 -    if (!tcg_region_inited) {
 -        tcg_region_inited = 1;
 -        tcg_region_init();
 -        parallel_cpus = qemu_tcg_mttcg_enabled() && current_machine->smp.max_cpus > 1;
 -    }
 -
 -    if (qemu_tcg_mttcg_enabled() || !single_tcg_cpu_thread) {
 -        cpu->thread = g_malloc0(sizeof(QemuThread));
 -        cpu->halt_cond = g_malloc0(sizeof(QemuCond));
 -        qemu_cond_init(cpu->halt_cond);
 -
 -        if (qemu_tcg_mttcg_enabled()) {
 -            /* create a thread per vCPU with TCG (MTTCG) */
 -            snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG",
 -                 cpu->cpu_index);
 -
 -            qemu_thread_create(cpu->thread, thread_name, tcg_cpu_thread_fn,
 -                               cpu, QEMU_THREAD_JOINABLE);
 -
 -        } else {
 -            /* share a single thread for all cpus with TCG */
 -            snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "ALL CPUs/TCG");
 -            qemu_thread_create(cpu->thread, thread_name,
 -                               tcg_rr_cpu_thread_fn,
 -                               cpu, QEMU_THREAD_JOINABLE);
 -
 -            single_tcg_halt_cond = cpu->halt_cond;
 -            single_tcg_cpu_thread = cpu->thread;
 -        }
 -#ifdef _WIN32
 -        cpu->hThread = qemu_thread_get_handle(cpu->thread);
 -#endif
--    } else {
--        /* For non-MTTCG cases we share the thread */
--        cpu->thread = single_tcg_cpu_thread;
--        cpu->halt_cond = single_tcg_halt_cond;
--        cpu->thread_id = first_cpu->thread_id;
--        cpu->can_do_io = 1;
--        cpu->created = true;
--    }
 -}
 -
- void qemu_tcg_destroy_vcpu(CPUState *cpu)
+ /* call with @p->lock held */
  static inline void invalidate_page_bitmap(PageDesc *p)
  {
-     cpu_thread_signal_destroyed(cpu);
+diff --git a/bsd-user/main.c b/bsd-user/main.c
 index XXXXXXX..XXXXXXX 100644
 --- a/bsd-user/main.c
 +++ b/bsd-user/main.c
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
      envlist_free(envlist);
      /*
 -     * Now that page sizes are configured in tcg_exec_init() we can do
 +     * Now that page sizes are configured we can do
       * proper page alignment for guest_base.
       */
      guest_base = HOST_PAGE_ALIGN(guest_base);
 --
 .25.1

-New patch
+[PULL 13/34] accel/tcg: Use MiB in tcg_init_machine
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ accel/tcg/tcg-all.c | 3 ++-
+file changed, 2 insertions(+), 1 deletion(-)
+diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
+index XXXXXXX..XXXXXXX 100644
+--- a/accel/tcg/tcg-all.c
++++ b/accel/tcg/tcg-all.c
+@@ -XXX,XX +XXX,XX @@
+ #include "qemu/error-report.h"
+ #include "qemu/accel.h"
+ #include "qapi/qapi-builtin-visit.h"
++#include "qemu/units.h"
+ #include "internal.h"
+ struct TCGState {
+@@ -XXX,XX +XXX,XX @@ static int tcg_init_machine(MachineState *ms)
+     page_init();
+     tb_htable_init();
+-    tcg_init(s->tb_size * 1024 * 1024, s->splitwx_enabled);
++    tcg_init(s->tb_size * MiB, s->splitwx_enabled);
+ #if defined(CONFIG_SOFTMMU)
+     /*
+--
+.25.1

-[PULL 3/3] accel/tcg: rename tcg-cpus functions to match module name
+[PULL 14/34] accel/tcg: Pass down max_cpus to tcg_init
-From: Claudio Fontana <cfontana@suse.de>
+Start removing the include of hw/boards.h from tcg/.
 Pass down the max_cpus value from tcg_init_machine,
 where we have the MachineState already.
-Signed-off-by: Claudio Fontana <cfontana@suse.de>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
-Message-Id: <20201015143217.29337-4-cfontana@suse.de>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/tcg-cpus-icount.h |  6 +--
+ include/tcg/tcg.h   |  2 +-
- accel/tcg/tcg-cpus-rr.h     |  2 +-
+ tcg/tcg-internal.h  |  2 +-
- accel/tcg/tcg-cpus.h        |  6 +--
+ accel/tcg/tcg-all.c | 10 +++++++++-
- accel/tcg/tcg-cpus-icount.c | 24 ++++++------
+ tcg/region.c        | 32 +++++++++++---------------------
- accel/tcg/tcg-cpus-mttcg.c  | 10 ++---
+ tcg/tcg.c           | 10 ++++------
- accel/tcg/tcg-cpus-rr.c     | 74 ++++++++++++++++++-------------------
+files changed, 26 insertions(+), 30 deletions(-)
  accel/tcg/tcg-cpus.c        |  6 +--
 files changed, 64 insertions(+), 64 deletions(-)
-diff --git a/accel/tcg/tcg-cpus-icount.h b/accel/tcg/tcg-cpus-icount.h
+diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/tcg-cpus-icount.h
+--- a/include/tcg/tcg.h
-+++ b/accel/tcg/tcg-cpus-icount.h
++++ b/include/tcg/tcg.h
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ static inline void *tcg_malloc(int size)
  #ifndef TCG_CPUS_ICOUNT_H
  #define TCG_CPUS_ICOUNT_H
 -void handle_icount_deadline(void);
 -void prepare_icount_for_run(CPUState *cpu);
 -void process_icount_data(CPUState *cpu);
 +void icount_handle_deadline(void);
 +void icount_prepare_for_run(CPUState *cpu);
 +void icount_process_data(CPUState *cpu);
  #endif /* TCG_CPUS_ICOUNT_H */
 diff --git a/accel/tcg/tcg-cpus-rr.h b/accel/tcg/tcg-cpus-rr.h
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus-rr.h
 +++ b/accel/tcg/tcg-cpus-rr.h
@@ -XXX,XX +XXX,XX @@
  #define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
  /* Kick all RR vCPUs. */
 -void qemu_cpu_kick_rr_cpus(CPUState *unused);
 +void rr_kick_vcpu_thread(CPUState *unused);
  /* start the round robin vcpu thread */
  void rr_start_vcpu_thread(CPUState *cpu);
 diff --git a/accel/tcg/tcg-cpus.h b/accel/tcg/tcg-cpus.h
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus.h
 +++ b/accel/tcg/tcg-cpus.h
@@ -XXX,XX +XXX,XX @@ extern const CpusAccel tcg_cpus_mttcg;
  extern const CpusAccel tcg_cpus_icount;
  extern const CpusAccel tcg_cpus_rr;
 -void qemu_tcg_destroy_vcpu(CPUState *cpu);
 -int tcg_cpu_exec(CPUState *cpu);
 -void tcg_handle_interrupt(CPUState *cpu, int mask);
 +void tcg_cpus_destroy(CPUState *cpu);
 +int tcg_cpus_exec(CPUState *cpu);
 +void tcg_cpus_handle_interrupt(CPUState *cpu, int mask);
  #endif /* TCG_CPUS_H */
 diff --git a/accel/tcg/tcg-cpus-icount.c b/accel/tcg/tcg-cpus-icount.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus-icount.c
 +++ b/accel/tcg/tcg-cpus-icount.c
@@ -XXX,XX +XXX,XX @@
  #include "tcg-cpus-icount.h"
  #include "tcg-cpus-rr.h"
 -static int64_t tcg_get_icount_limit(void)
 +static int64_t icount_get_limit(void)
  {
      int64_t deadline;
@@ -XXX,XX +XXX,XX @@ static int64_t tcg_get_icount_limit(void)
      }
  }
--static void notify_aio_contexts(void)
+-void tcg_init(size_t tb_size, int splitwx);
-+static void icount_notify_aio_contexts(void)
++void tcg_init(size_t tb_size, int splitwx, unsigned max_cpus);
  void tcg_register_thread(void);
  void tcg_prologue_init(TCGContext *s);
  void tcg_func_start(TCGContext *s);
 diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg-internal.h
 +++ b/tcg/tcg-internal.h
@@ -XXX,XX +XXX,XX @@
  extern TCGContext **tcg_ctxs;
  extern unsigned int n_tcg_ctxs;
 -void tcg_region_init(size_t tb_size, int splitwx);
 +void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus);
  bool tcg_region_alloc(TCGContext *s);
  void tcg_region_initial_alloc(TCGContext *s);
  void tcg_region_prologue_set(TCGContext *s);
 diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-all.c
 +++ b/accel/tcg/tcg-all.c
@@ -XXX,XX +XXX,XX @@
  #include "qemu/accel.h"
  #include "qapi/qapi-builtin-visit.h"
  #include "qemu/units.h"
 +#if !defined(CONFIG_USER_ONLY)
 +#include "hw/boards.h"
 +#endif
  #include "internal.h"
  struct TCGState {
@@ -XXX,XX +XXX,XX @@ bool mttcg_enabled;
  static int tcg_init_machine(MachineState *ms)
  {
-     /* Wake up other AioContexts.  */
+     TCGState *s = TCG_STATE(current_accel());
-     qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
++#ifdef CONFIG_USER_ONLY
-     qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL);
++    unsigned max_cpus = 1;
 +#else
 +    unsigned max_cpus = ms->smp.max_cpus;
 +#endif
      tcg_allowed = true;
      mttcg_enabled = s->mttcg_enabled;
      page_init();
      tb_htable_init();
 -    tcg_init(s->tb_size * MiB, s->splitwx_enabled);
 +    tcg_init(s->tb_size * MiB, s->splitwx_enabled, max_cpus);
  #if defined(CONFIG_SOFTMMU)
      /*
 diff --git a/tcg/region.c b/tcg/region.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/region.c
 +++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@
  #include "qapi/error.h"
  #include "exec/exec-all.h"
  #include "tcg/tcg.h"
 -#if !defined(CONFIG_USER_ONLY)
 -#include "hw/boards.h"
 -#endif
  #include "tcg-internal.h"
@@ -XXX,XX +XXX,XX @@ void tcg_region_reset_all(void)
      tcg_region_tree_reset_all();
  }
--void handle_icount_deadline(void)
++static size_t tcg_n_regions(unsigned max_cpus)
-+void icount_handle_deadline(void)
++{
  #ifdef CONFIG_USER_ONLY
 -static size_t tcg_n_regions(void)
 -{
      return 1;
 -}
  #else
 -/*
 - * It is likely that some vCPUs will translate more code than others, so we
 - * first try to set more regions than max_cpus, with those regions being of
 - * reasonable size. If that's not possible we make do by evenly dividing
 - * the code_gen_buffer among the vCPUs.
 - */
 -static size_t tcg_n_regions(void)
 -{
 +    /*
 +     * It is likely that some vCPUs will translate more code than others,
 +     * so we first try to set more regions than max_cpus, with those regions
 +     * being of reasonable size. If that's not possible we make do by evenly
 +     * dividing the code_gen_buffer among the vCPUs.
 +     */
      size_t i;
      /* Use a single region if all we have is one vCPU thread */
 -#if !defined(CONFIG_USER_ONLY)
 -    MachineState *ms = MACHINE(qdev_get_machine());
 -    unsigned int max_cpus = ms->smp.max_cpus;
 -#endif
      if (max_cpus == 1 || !qemu_tcg_mttcg_enabled()) {
          return 1;
      }
@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(void)
      }
      /* If we can't, then just allocate one region per vCPU thread */
      return max_cpus;
 -}
  #endif
 +}
  /*
   * Minimum size of the code gen buffer.  This number is randomly chosen,
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
   * in practice. Multi-threaded guests share most if not all of their translated
   * code, which makes parallel code generation less appealing than in softmmu.
   */
 -void tcg_region_init(size_t tb_size, int splitwx)
 +void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
  {
-     assert(qemu_in_vcpu_thread());
+     void *buf, *aligned;
-     int64_t deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
+     size_t size;
-                                                   QEMU_TIMER_ATTR_ALL);
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx)
+     buf = tcg_init_ctx.code_gen_buffer;
-     if (deadline == 0) {
+     size = tcg_init_ctx.code_gen_buffer_size;
--        notify_aio_contexts();
+     page_size = qemu_real_host_page_size;
-+        icount_notify_aio_contexts();
+-    n_regions = tcg_n_regions();
-     }
++    n_regions = tcg_n_regions(max_cpus);
      /* The first region will be 'aligned - buf' bytes larger than the others */
      aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
 diff --git a/tcg/tcg.c b/tcg/tcg.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg.c
 +++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static void process_op_defs(TCGContext *s);
  static TCGTemp *tcg_global_reg_new_internal(TCGContext *s, TCGType type,
                                              TCGReg reg, const char *name);
 -static void tcg_context_init(void)
 +static void tcg_context_init(unsigned max_cpus)
  {
      TCGContext *s = &tcg_init_ctx;
      int op, total_args, n, i;
@@ -XXX,XX +XXX,XX @@ static void tcg_context_init(void)
      tcg_ctxs = &tcg_ctx;
      n_tcg_ctxs = 1;
  #else
 -    MachineState *ms = MACHINE(qdev_get_machine());
 -    unsigned int max_cpus = ms->smp.max_cpus;
      tcg_ctxs = g_new(TCGContext *, max_cpus);
  #endif
@@ -XXX,XX +XXX,XX @@ static void tcg_context_init(void)
      cpu_env = temp_tcgv_ptr(ts);
  }
--void prepare_icount_for_run(CPUState *cpu)
+-void tcg_init(size_t tb_size, int splitwx)
-+void icount_prepare_for_run(CPUState *cpu)
++void tcg_init(size_t tb_size, int splitwx, unsigned max_cpus)
  {
-     int insns_left;
+-    tcg_context_init();
+-    tcg_region_init(tb_size, splitwx);
-     /*
++    tcg_context_init(max_cpus);
--     * These should always be cleared by process_icount_data after
++    tcg_region_init(tb_size, splitwx, max_cpus);
 +     * These should always be cleared by icount_process_data after
       * each vCPU execution. However u16.high can be raised
 -     * asynchronously by cpu_exit/cpu_interrupt/tcg_handle_interrupt
 +     * asynchronously by cpu_exit/cpu_interrupt/tcg_cpus_handle_interrupt
       */
      g_assert(cpu_neg(cpu)->icount_decr.u16.low == 0);
      g_assert(cpu->icount_extra == 0);
 -    cpu->icount_budget = tcg_get_icount_limit();
 +    cpu->icount_budget = icount_get_limit();
      insns_left = MIN(0xffff, cpu->icount_budget);
      cpu_neg(cpu)->icount_decr.u16.low = insns_left;
      cpu->icount_extra = cpu->icount_budget - insns_left;
@@ -XXX,XX +XXX,XX @@ void prepare_icount_for_run(CPUState *cpu)
      replay_mutex_lock();
      if (cpu->icount_budget == 0 && replay_has_checkpoint()) {
 -        notify_aio_contexts();
 +        icount_notify_aio_contexts();
      }
  }
--void process_icount_data(CPUState *cpu)
+ /*
 +void icount_process_data(CPUState *cpu)
  {
      /* Account for executed instructions */
      icount_update(cpu);
@@ -XXX,XX +XXX,XX @@ static void icount_handle_interrupt(CPUState *cpu, int mask)
  {
      int old_mask = cpu->interrupt_request;
 -    tcg_handle_interrupt(cpu, mask);
 +    tcg_cpus_handle_interrupt(cpu, mask);
      if (qemu_cpu_is_self(cpu) &&
          !cpu->can_do_io
          && (mask & ~old_mask) != 0) {
@@ -XXX,XX +XXX,XX @@ static void icount_handle_interrupt(CPUState *cpu, int mask)
  const CpusAccel tcg_cpus_icount = {
      .create_vcpu_thread = rr_start_vcpu_thread,
 -    .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
 +    .kick_vcpu_thread = rr_kick_vcpu_thread,
      .handle_interrupt = icount_handle_interrupt,
      .get_virtual_clock = icount_get,
 diff --git a/accel/tcg/tcg-cpus-mttcg.c b/accel/tcg/tcg-cpus-mttcg.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus-mttcg.c
 +++ b/accel/tcg/tcg-cpus-mttcg.c
@@ -XXX,XX +XXX,XX @@
   * current CPUState for a given thread.
   */
 -static void *tcg_cpu_thread_fn(void *arg)
 +static void *mttcg_cpu_thread_fn(void *arg)
  {
      CPUState *cpu = arg;
@@ -XXX,XX +XXX,XX @@ static void *tcg_cpu_thread_fn(void *arg)
          if (cpu_can_run(cpu)) {
              int r;
              qemu_mutex_unlock_iothread();
 -            r = tcg_cpu_exec(cpu);
 +            r = tcg_cpus_exec(cpu);
              qemu_mutex_lock_iothread();
              switch (r) {
              case EXCP_DEBUG:
@@ -XXX,XX +XXX,XX @@ static void *tcg_cpu_thread_fn(void *arg)
          qemu_wait_io_event(cpu);
      } while (!cpu->unplug || cpu_can_run(cpu));
 -    qemu_tcg_destroy_vcpu(cpu);
 +    tcg_cpus_destroy(cpu);
      qemu_mutex_unlock_iothread();
      rcu_unregister_thread();
      return NULL;
@@ -XXX,XX +XXX,XX @@ static void mttcg_start_vcpu_thread(CPUState *cpu)
      snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG",
               cpu->cpu_index);
 -    qemu_thread_create(cpu->thread, thread_name, tcg_cpu_thread_fn,
 +    qemu_thread_create(cpu->thread, thread_name, mttcg_cpu_thread_fn,
                         cpu, QEMU_THREAD_JOINABLE);
  #ifdef _WIN32
@@ -XXX,XX +XXX,XX @@ const CpusAccel tcg_cpus_mttcg = {
      .create_vcpu_thread = mttcg_start_vcpu_thread,
      .kick_vcpu_thread = mttcg_kick_vcpu_thread,
 -    .handle_interrupt = tcg_handle_interrupt,
 +    .handle_interrupt = tcg_cpus_handle_interrupt,
  };
 diff --git a/accel/tcg/tcg-cpus-rr.c b/accel/tcg/tcg-cpus-rr.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus-rr.c
 +++ b/accel/tcg/tcg-cpus-rr.c
@@ -XXX,XX +XXX,XX @@
  #include "tcg-cpus-icount.h"
  /* Kick all RR vCPUs */
 -void qemu_cpu_kick_rr_cpus(CPUState *unused)
 +void rr_kick_vcpu_thread(CPUState *unused)
  {
      CPUState *cpu;
@@ -XXX,XX +XXX,XX @@ void qemu_cpu_kick_rr_cpus(CPUState *unused)
   * idleness is complete.
   */
 -static QEMUTimer *tcg_kick_vcpu_timer;
 -static CPUState *tcg_current_rr_cpu;
 +static QEMUTimer *rr_kick_vcpu_timer;
 +static CPUState *rr_current_cpu;
  #define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
 -static inline int64_t qemu_tcg_next_kick(void)
 +static inline int64_t rr_next_kick_time(void)
  {
      return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + TCG_KICK_PERIOD;
  }
  /* Kick the currently round-robin scheduled vCPU to next */
 -static void qemu_cpu_kick_rr_next_cpu(void)
 +static void rr_kick_next_cpu(void)
  {
      CPUState *cpu;
      do {
 -        cpu = qatomic_mb_read(&tcg_current_rr_cpu);
 +        cpu = qatomic_mb_read(&rr_current_cpu);
          if (cpu) {
              cpu_exit(cpu);
          }
 -    } while (cpu != qatomic_mb_read(&tcg_current_rr_cpu));
 +    } while (cpu != qatomic_mb_read(&rr_current_cpu));
  }
 -static void kick_tcg_thread(void *opaque)
 +static void rr_kick_thread(void *opaque)
  {
 -    timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
 -    qemu_cpu_kick_rr_next_cpu();
 +    timer_mod(rr_kick_vcpu_timer, rr_next_kick_time());
 +    rr_kick_next_cpu();
  }
 -static void start_tcg_kick_timer(void)
 +static void rr_start_kick_timer(void)
  {
 -    if (!tcg_kick_vcpu_timer && CPU_NEXT(first_cpu)) {
 -        tcg_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
 -                                           kick_tcg_thread, NULL);
 +    if (!rr_kick_vcpu_timer && CPU_NEXT(first_cpu)) {
 +        rr_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
 +                                           rr_kick_thread, NULL);
      }
 -    if (tcg_kick_vcpu_timer && !timer_pending(tcg_kick_vcpu_timer)) {
 -        timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
 +    if (rr_kick_vcpu_timer && !timer_pending(rr_kick_vcpu_timer)) {
 +        timer_mod(rr_kick_vcpu_timer, rr_next_kick_time());
      }
  }
 -static void stop_tcg_kick_timer(void)
 +static void rr_stop_kick_timer(void)
  {
 -    if (tcg_kick_vcpu_timer && timer_pending(tcg_kick_vcpu_timer)) {
 -        timer_del(tcg_kick_vcpu_timer);
 +    if (rr_kick_vcpu_timer && timer_pending(rr_kick_vcpu_timer)) {
 +        timer_del(rr_kick_vcpu_timer);
      }
  }
 -static void qemu_tcg_rr_wait_io_event(void)
 +static void rr_wait_io_event(void)
  {
      CPUState *cpu;
      while (all_cpu_threads_idle()) {
 -        stop_tcg_kick_timer();
 +        rr_stop_kick_timer();
          qemu_cond_wait_iothread(first_cpu->halt_cond);
      }
 -    start_tcg_kick_timer();
 +    rr_start_kick_timer();
      CPU_FOREACH(cpu) {
          qemu_wait_io_event_common(cpu);
@@ -XXX,XX +XXX,XX @@ static void qemu_tcg_rr_wait_io_event(void)
   * Destroy any remaining vCPUs which have been unplugged and have
   * finished running
   */
 -static void deal_with_unplugged_cpus(void)
 +static void rr_deal_with_unplugged_cpus(void)
  {
      CPUState *cpu;
      CPU_FOREACH(cpu) {
          if (cpu->unplug && !cpu_can_run(cpu)) {
 -            qemu_tcg_destroy_vcpu(cpu);
 +            tcg_cpus_destroy(cpu);
              break;
          }
      }
@@ -XXX,XX +XXX,XX @@ static void deal_with_unplugged_cpus(void)
   * elsewhere.
   */
 -static void *tcg_rr_cpu_thread_fn(void *arg)
 +static void *rr_cpu_thread_fn(void *arg)
  {
      CPUState *cpu = arg;
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
          }
      }
 -    start_tcg_kick_timer();
 +    rr_start_kick_timer();
      cpu = first_cpu;
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
               * Run the timers here.  This is much more efficient than
               * waking up the I/O thread and waiting for completion.
               */
 -            handle_icount_deadline();
 +            icount_handle_deadline();
          }
          replay_mutex_unlock();
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
          while (cpu && cpu_work_list_empty(cpu) && !cpu->exit_request) {
 -            qatomic_mb_set(&tcg_current_rr_cpu, cpu);
 +            qatomic_mb_set(&rr_current_cpu, cpu);
              current_cpu = cpu;
              qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
                  qemu_mutex_unlock_iothread();
                  if (icount_enabled()) {
 -                    prepare_icount_for_run(cpu);
 +                    icount_prepare_for_run(cpu);
                  }
 -                r = tcg_cpu_exec(cpu);
 +                r = tcg_cpus_exec(cpu);
                  if (icount_enabled()) {
 -                    process_icount_data(cpu);
 +                    icount_process_data(cpu);
                  }
                  qemu_mutex_lock_iothread();
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
          } /* while (cpu && !cpu->exit_request).. */
          /* Does not need qatomic_mb_set because a spurious wakeup is okay.  */
 -        qatomic_set(&tcg_current_rr_cpu, NULL);
 +        qatomic_set(&rr_current_cpu, NULL);
          if (cpu && cpu->exit_request) {
              qatomic_mb_set(&cpu->exit_request, 0);
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
              qemu_notify_event();
          }
 -        qemu_tcg_rr_wait_io_event();
 -        deal_with_unplugged_cpus();
 +        rr_wait_io_event();
 +        rr_deal_with_unplugged_cpus();
      }
      rcu_unregister_thread();
@@ -XXX,XX +XXX,XX @@ void rr_start_vcpu_thread(CPUState *cpu)
          /* share a single thread for all cpus with TCG */
          snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "ALL CPUs/TCG");
          qemu_thread_create(cpu->thread, thread_name,
 -                           tcg_rr_cpu_thread_fn,
 +                           rr_cpu_thread_fn,
                             cpu, QEMU_THREAD_JOINABLE);
          single_tcg_halt_cond = cpu->halt_cond;
@@ -XXX,XX +XXX,XX @@ void rr_start_vcpu_thread(CPUState *cpu)
  const CpusAccel tcg_cpus_rr = {
      .create_vcpu_thread = rr_start_vcpu_thread,
 -    .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
 +    .kick_vcpu_thread = rr_kick_vcpu_thread,
 -    .handle_interrupt = tcg_handle_interrupt,
 +    .handle_interrupt = tcg_cpus_handle_interrupt,
  };
 diff --git a/accel/tcg/tcg-cpus.c b/accel/tcg/tcg-cpus.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-cpus.c
 +++ b/accel/tcg/tcg-cpus.c
@@ -XXX,XX +XXX,XX @@
  /* common functionality among all TCG variants */
 -void qemu_tcg_destroy_vcpu(CPUState *cpu)
 +void tcg_cpus_destroy(CPUState *cpu)
  {
      cpu_thread_signal_destroyed(cpu);
  }
 -int tcg_cpu_exec(CPUState *cpu)
 +int tcg_cpus_exec(CPUState *cpu)
  {
      int ret;
  #ifdef CONFIG_PROFILER
@@ -XXX,XX +XXX,XX @@ int tcg_cpu_exec(CPUState *cpu)
  }
  /* mask must never be zero, except for A20 change call */
 -void tcg_handle_interrupt(CPUState *cpu, int mask)
 +void tcg_cpus_handle_interrupt(CPUState *cpu, int mask)
  {
      g_assert(qemu_mutex_iothread_locked());
 --
 .25.1

-New patch
+[PULL 15/34] tcg: Introduce tcg_max_ctxs
+Finish the divorce of tcg/ from hw/, and do not take
+the max cpu value from MachineState; just remember what
+we were passed in tcg_init.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/tcg-internal.h |  3 ++-
+ tcg/region.c       |  6 +++---
+ tcg/tcg.c          | 23 ++++++++++-------------
+files changed, 15 insertions(+), 17 deletions(-)
+diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg-internal.h
++++ b/tcg/tcg-internal.h
+@@ -XXX,XX +XXX,XX @@
+ #define TCG_HIGHWATER 1024
+ extern TCGContext **tcg_ctxs;
+-extern unsigned int n_tcg_ctxs;
++extern unsigned int tcg_cur_ctxs;
++extern unsigned int tcg_max_ctxs;
+ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus);
+ bool tcg_region_alloc(TCGContext *s);
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ void tcg_region_initial_alloc(TCGContext *s)
+ /* Call from a safe-work context */
+ void tcg_region_reset_all(void)
+ {
+-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
++    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
+     unsigned int i;
+     qemu_mutex_lock(&region.lock);
+@@ -XXX,XX +XXX,XX @@ void tcg_region_prologue_set(TCGContext *s)
+  */
+ size_t tcg_code_size(void)
+ {
+-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
++    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
+     unsigned int i;
+     size_t total;
+@@ -XXX,XX +XXX,XX @@ size_t tcg_code_capacity(void)
+ size_t tcg_tb_phys_invalidate_count(void)
+ {
+-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
++    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
+     unsigned int i;
+     size_t total = 0;
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@
+ #define NO_CPU_IO_DEFS
+ #include "exec/exec-all.h"
+-
+-#if !defined(CONFIG_USER_ONLY)
+-#include "hw/boards.h"
+-#endif
+-
+ #include "tcg/tcg-op.h"
+ #if UINTPTR_MAX == UINT32_MAX
+@@ -XXX,XX +XXX,XX @@ static int tcg_out_ldst_finalize(TCGContext *s);
+ #endif
+ TCGContext **tcg_ctxs;
+-unsigned int n_tcg_ctxs;
++unsigned int tcg_cur_ctxs;
++unsigned int tcg_max_ctxs;
+ TCGv_env cpu_env = 0;
+ const void *tcg_code_gen_epilogue;
+ uintptr_t tcg_splitwx_diff;
+@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
+ #else
+ void tcg_register_thread(void)
+ {
+-    MachineState *ms = MACHINE(qdev_get_machine());
+     TCGContext *s = g_malloc(sizeof(*s));
+     unsigned int i, n;
+@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
+     }
+     /* Claim an entry in tcg_ctxs */
+-    n = qatomic_fetch_inc(&n_tcg_ctxs);
+-    g_assert(n < ms->smp.max_cpus);
++    n = qatomic_fetch_inc(&tcg_cur_ctxs);
++    g_assert(n < tcg_max_ctxs);
+     qatomic_set(&tcg_ctxs[n], s);
+     if (n > 0) {
+@@ -XXX,XX +XXX,XX @@ static void tcg_context_init(unsigned max_cpus)
+      */
+ #ifdef CONFIG_USER_ONLY
+     tcg_ctxs = &tcg_ctx;
+-    n_tcg_ctxs = 1;
++    tcg_cur_ctxs = 1;
++    tcg_max_ctxs = 1;
+ #else
+-    tcg_ctxs = g_new(TCGContext *, max_cpus);
++    tcg_max_ctxs = max_cpus;
++    tcg_ctxs = g_new0(TCGContext *, max_cpus);
+ #endif
+     tcg_debug_assert(!tcg_regset_test_reg(s->reserved_regs, TCG_AREG0));
+@@ -XXX,XX +XXX,XX @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op)
+ static inline
+ void tcg_profile_snapshot(TCGProfile *prof, bool counters, bool table)
+ {
+-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
++    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
+     unsigned int i;
+     for (i = 0; i < n_ctxs; i++) {
+@@ -XXX,XX +XXX,XX @@ void tcg_dump_op_count(void)
+ int64_t tcg_cpu_exec_time(void)
+ {
+-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
++    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
+     unsigned int i;
+     int64_t ret = 0;
+--
+.25.1

-New patch
+[PULL 16/34] tcg: Move MAX_CODE_GEN_BUFFER_SIZE to tcg-target.h
+Remove the ifdef ladder and move each define into the
+appropriate header file.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/aarch64/tcg-target.h |  1 +
+ tcg/arm/tcg-target.h     |  1 +
+ tcg/i386/tcg-target.h    |  2 ++
+ tcg/mips/tcg-target.h    |  6 ++++++
+ tcg/ppc/tcg-target.h     |  2 ++
+ tcg/riscv/tcg-target.h   |  1 +
+ tcg/s390/tcg-target.h    |  3 +++
+ tcg/sparc/tcg-target.h   |  1 +
+ tcg/tci/tcg-target.h     |  1 +
+ tcg/region.c             | 33 +++++----------------------------
+files changed, 23 insertions(+), 28 deletions(-)
+diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/aarch64/tcg-target.h
++++ b/tcg/aarch64/tcg-target.h
+@@ -XXX,XX +XXX,XX @@
+ #define TCG_TARGET_INSN_UNIT_SIZE  4
+ #define TCG_TARGET_TLB_DISPLACEMENT_BITS 24
++#define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+ #undef TCG_TARGET_STACK_GROWSUP
+ typedef enum {
+diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/arm/tcg-target.h
++++ b/tcg/arm/tcg-target.h
+@@ -XXX,XX +XXX,XX @@ extern int arm_arch;
+ #undef TCG_TARGET_STACK_GROWSUP
+ #define TCG_TARGET_INSN_UNIT_SIZE 4
+ #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16
++#define MAX_CODE_GEN_BUFFER_SIZE  UINT32_MAX
+ typedef enum {
+     TCG_REG_R0 = 0,
+diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/i386/tcg-target.h
++++ b/tcg/i386/tcg-target.h
+@@ -XXX,XX +XXX,XX @@
+ #ifdef __x86_64__
+ # define TCG_TARGET_REG_BITS  64
+ # define TCG_TARGET_NB_REGS   32
++# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+ #else
+ # define TCG_TARGET_REG_BITS  32
+ # define TCG_TARGET_NB_REGS   24
++# define MAX_CODE_GEN_BUFFER_SIZE  UINT32_MAX
+ #endif
+ typedef enum {
+diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/mips/tcg-target.h
++++ b/tcg/mips/tcg-target.h
+@@ -XXX,XX +XXX,XX @@
+ #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16
+ #define TCG_TARGET_NB_REGS 32
++/*
++ * We have a 256MB branch region, but leave room to make sure the
++ * main executable is also within that region.
++ */
++#define MAX_CODE_GEN_BUFFER_SIZE  (128 * MiB)
++
+ typedef enum {
+     TCG_REG_ZERO = 0,
+     TCG_REG_AT,
+diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/ppc/tcg-target.h
++++ b/tcg/ppc/tcg-target.h
+@@ -XXX,XX +XXX,XX @@
+ #ifdef _ARCH_PPC64
+ # define TCG_TARGET_REG_BITS  64
++# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+ #else
+ # define TCG_TARGET_REG_BITS  32
++# define MAX_CODE_GEN_BUFFER_SIZE  (32 * MiB)
+ #endif
+ #define TCG_TARGET_NB_REGS 64
+diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/riscv/tcg-target.h
++++ b/tcg/riscv/tcg-target.h
+@@ -XXX,XX +XXX,XX @@
+ #define TCG_TARGET_INSN_UNIT_SIZE 4
+ #define TCG_TARGET_TLB_DISPLACEMENT_BITS 20
+ #define TCG_TARGET_NB_REGS 32
++#define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
+ typedef enum {
+     TCG_REG_ZERO,
+diff --git a/tcg/s390/tcg-target.h b/tcg/s390/tcg-target.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/s390/tcg-target.h
++++ b/tcg/s390/tcg-target.h
+@@ -XXX,XX +XXX,XX @@
+ #define TCG_TARGET_INSN_UNIT_SIZE 2
+ #define TCG_TARGET_TLB_DISPLACEMENT_BITS 19
++/* We have a +- 4GB range on the branches; leave some slop.  */
++#define MAX_CODE_GEN_BUFFER_SIZE  (3 * GiB)
++
+ typedef enum TCGReg {
+     TCG_REG_R0 = 0,
+     TCG_REG_R1,
+diff --git a/tcg/sparc/tcg-target.h b/tcg/sparc/tcg-target.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/sparc/tcg-target.h
++++ b/tcg/sparc/tcg-target.h
+@@ -XXX,XX +XXX,XX @@
+ #define TCG_TARGET_INSN_UNIT_SIZE 4
+ #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32
+ #define TCG_TARGET_NB_REGS 32
++#define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+ typedef enum {
+     TCG_REG_G0 = 0,
+diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tci/tcg-target.h
++++ b/tcg/tci/tcg-target.h
+@@ -XXX,XX +XXX,XX @@
+ #define TCG_TARGET_INTERPRETER 1
+ #define TCG_TARGET_INSN_UNIT_SIZE 1
+ #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32
++#define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
+ #if UINTPTR_MAX == UINT32_MAX
+ # define TCG_TARGET_REG_BITS 32
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(unsigned max_cpus)
+ /*
+  * Minimum size of the code gen buffer.  This number is randomly chosen,
+  * but not so small that we can't have a fair number of TB's live.
++ *
++ * Maximum size, MAX_CODE_GEN_BUFFER_SIZE, is defined in tcg-target.h.
++ * Unless otherwise indicated, this is constrained by the range of
++ * direct branches on the host cpu, as used by the TCG implementation
++ * of goto_tb.
+  */
+ #define MIN_CODE_GEN_BUFFER_SIZE     (1 * MiB)
+-/*
+- * Maximum size of the code gen buffer we'd like to use.  Unless otherwise
+- * indicated, this is constrained by the range of direct branches on the
+- * host cpu, as used by the TCG implementation of goto_tb.
+- */
+-#if defined(__x86_64__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+-#elif defined(__sparc__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+-#elif defined(__powerpc64__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+-#elif defined(__powerpc__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (32 * MiB)
+-#elif defined(__aarch64__)
+-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+-#elif defined(__s390x__)
+-  /* We have a +- 4GB range on the branches; leave some slop.  */
+-# define MAX_CODE_GEN_BUFFER_SIZE  (3 * GiB)
+-#elif defined(__mips__)
+-  /*
+-   * We have a 256MB branch region, but leave room to make sure the
+-   * main executable is also within that region.
+-   */
+-# define MAX_CODE_GEN_BUFFER_SIZE  (128 * MiB)
+-#else
+-# define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
+-#endif
+-
+ #if TCG_TARGET_REG_BITS == 32
+ #define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (32 * MiB)
+ #ifdef CONFIG_USER_ONLY
+--
+.25.1

-New patch
+[PULL 17/34] tcg: Replace region.end with region.total_size
+A size is easier to work with than an end point,
+particularly during initial buffer allocation.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/region.c | 30 ++++++++++++++++++------------
+file changed, 18 insertions(+), 12 deletions(-)
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ struct tcg_region_state {
+     /* fields set at init time */
+     void *start;
+     void *start_aligned;
+-    void *end;
+     size_t n;
+     size_t size; /* size of one region */
+     size_t stride; /* .size + guard size */
++    size_t total_size; /* size of entire buffer, >= n * stride */
+     /* fields protected by the lock */
+     size_t current; /* current region index */
+@@ -XXX,XX +XXX,XX @@ static void tcg_region_bounds(size_t curr_region, void **pstart, void **pend)
+     if (curr_region == 0) {
+         start = region.start;
+     }
++    /* The final region may have a few extra pages due to earlier rounding. */
+     if (curr_region == region.n - 1) {
+-        end = region.end;
++        end = region.start_aligned + region.total_size;
+     }
+     *pstart = start;
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+  */
+ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+ {
+-    void *buf, *aligned;
+-    size_t size;
++    void *buf, *aligned, *end;
++    size_t total_size;
+     size_t page_size;
+     size_t region_size;
+     size_t n_regions;
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+     assert(ok);
+     buf = tcg_init_ctx.code_gen_buffer;
+-    size = tcg_init_ctx.code_gen_buffer_size;
++    total_size = tcg_init_ctx.code_gen_buffer_size;
+     page_size = qemu_real_host_page_size;
+     n_regions = tcg_n_regions(max_cpus);
+     /* The first region will be 'aligned - buf' bytes larger than the others */
+     aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
+-    g_assert(aligned < tcg_init_ctx.code_gen_buffer + size);
++    g_assert(aligned < tcg_init_ctx.code_gen_buffer + total_size);
++
+     /*
+      * Make region_size a multiple of page_size, using aligned as the start.
+      * As a result of this we might end up with a few extra pages at the end of
+      * the buffer; we will assign those to the last region.
+      */
+-    region_size = (size - (aligned - buf)) / n_regions;
++    region_size = (total_size - (aligned - buf)) / n_regions;
+     region_size = QEMU_ALIGN_DOWN(region_size, page_size);
+     /* A region must have at least 2 pages; one code, one guard */
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+     region.start = buf;
+     region.start_aligned = aligned;
+     /* page-align the end, since its last page will be a guard page */
+-    region.end = QEMU_ALIGN_PTR_DOWN(buf + size, page_size);
++    end = QEMU_ALIGN_PTR_DOWN(buf + total_size, page_size);
+     /* account for that last guard page */
+-    region.end -= page_size;
++    end -= page_size;
++    total_size = end - aligned;
++    region.total_size = total_size;
+     /*
+      * Set guard pages in the rw buffer, as that's the one into which
+@@ -XXX,XX +XXX,XX @@ void tcg_region_prologue_set(TCGContext *s)
+     /* Register the balance of the buffer with gdb. */
+     tcg_register_jit(tcg_splitwx_to_rx(region.start),
+-                     region.end - region.start);
++                     region.start_aligned + region.total_size - region.start);
+ }
+ /*
+@@ -XXX,XX +XXX,XX @@ size_t tcg_code_capacity(void)
+     /* no need for synchronization; these variables are set at init time */
+     guard_size = region.stride - region.size;
+-    capacity = region.end + guard_size - region.start;
+-    capacity -= region.n * (guard_size + TCG_HIGHWATER);
++    capacity = region.total_size;
++    capacity -= (region.n - 1) * guard_size;
++    capacity -= region.n * TCG_HIGHWATER;
++
+     return capacity;
+ }
+--
+.25.1

-New patch
+[PULL 18/34] tcg: Rename region.start to region.after_prologue
+Give the field a name reflecting its actual meaning.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/region.c | 15 ++++++++-------
+file changed, 8 insertions(+), 7 deletions(-)
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ struct tcg_region_state {
+     QemuMutex lock;
+     /* fields set at init time */
+-    void *start;
+     void *start_aligned;
++    void *after_prologue;
+     size_t n;
+     size_t size; /* size of one region */
+     size_t stride; /* .size + guard size */
+@@ -XXX,XX +XXX,XX @@ static void tcg_region_bounds(size_t curr_region, void **pstart, void **pend)
+     end = start + region.size;
+     if (curr_region == 0) {
+-        start = region.start;
++        start = region.after_prologue;
+     }
+     /* The final region may have a few extra pages due to earlier rounding. */
+     if (curr_region == region.n - 1) {
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+     region.n = n_regions;
+     region.size = region_size - page_size;
+     region.stride = region_size;
+-    region.start = buf;
++    region.after_prologue = buf;
+     region.start_aligned = aligned;
+     /* page-align the end, since its last page will be a guard page */
+     end = QEMU_ALIGN_PTR_DOWN(buf + total_size, page_size);
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+ void tcg_region_prologue_set(TCGContext *s)
+ {
+     /* Deduct the prologue from the first region.  */
+-    g_assert(region.start == s->code_gen_buffer);
+-    region.start = s->code_ptr;
++    g_assert(region.start_aligned == s->code_gen_buffer);
++    region.after_prologue = s->code_ptr;
+     /* Recompute boundaries of the first region. */
+     tcg_region_assign(s, 0);
+     /* Register the balance of the buffer with gdb. */
+-    tcg_register_jit(tcg_splitwx_to_rx(region.start),
+-                     region.start_aligned + region.total_size - region.start);
++    tcg_register_jit(tcg_splitwx_to_rx(region.after_prologue),
++                     region.start_aligned + region.total_size -
++                     region.after_prologue);
+ }
+ /*
+--
+.25.1

-New patch
+[PULL 19/34] tcg: Tidy tcg_n_regions
+Compute the value using straight division and bounds,
+rather than a loop.  Pass in tb_size rather than reading
+from tcg_init_ctx.code_gen_buffer_size,
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/region.c | 29 ++++++++++++-----------------
+file changed, 12 insertions(+), 17 deletions(-)
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ void tcg_region_reset_all(void)
+     tcg_region_tree_reset_all();
+ }
+-static size_t tcg_n_regions(unsigned max_cpus)
++static size_t tcg_n_regions(size_t tb_size, unsigned max_cpus)
+ {
+ #ifdef CONFIG_USER_ONLY
+     return 1;
+ #else
++    size_t n_regions;
++
+     /*
+      * It is likely that some vCPUs will translate more code than others,
+      * so we first try to set more regions than max_cpus, with those regions
+      * being of reasonable size. If that's not possible we make do by evenly
+      * dividing the code_gen_buffer among the vCPUs.
+      */
+-    size_t i;
+-
+     /* Use a single region if all we have is one vCPU thread */
+     if (max_cpus == 1 || !qemu_tcg_mttcg_enabled()) {
+         return 1;
+     }
+-    /* Try to have more regions than max_cpus, with each region being >= 2 MB */
+-    for (i = 8; i > 0; i--) {
+-        size_t regions_per_thread = i;
+-        size_t region_size;
+-
+-        region_size = tcg_init_ctx.code_gen_buffer_size;
+-        region_size /= max_cpus * regions_per_thread;
+-
+-        if (region_size >= 2 * 1024u * 1024) {
+-            return max_cpus * regions_per_thread;
+-        }
++    /*
++     * Try to have more regions than max_cpus, with each region being >= 2 MB.
++     * If we can't, then just allocate one region per vCPU thread.
++     */
++    n_regions = tb_size / (2 * MiB);
++    if (n_regions <= max_cpus) {
++        return max_cpus;
+     }
+-    /* If we can't, then just allocate one region per vCPU thread */
+-    return max_cpus;
++    return MIN(n_regions, max_cpus * 8);
+ #endif
+ }
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+     buf = tcg_init_ctx.code_gen_buffer;
+     total_size = tcg_init_ctx.code_gen_buffer_size;
+     page_size = qemu_real_host_page_size;
+-    n_regions = tcg_n_regions(max_cpus);
++    n_regions = tcg_n_regions(total_size, max_cpus);
+     /* The first region will be 'aligned - buf' bytes larger than the others */
+     aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
+--
+.25.1

-New patch
+[PULL 20/34] tcg: Tidy split_cross_256mb
+Return output buffer and size via output pointer arguments,
+rather than returning size via tcg_ctx->code_gen_buffer_size.
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/region.c | 19 +++++++++----------
+file changed, 9 insertions(+), 10 deletions(-)
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ static inline bool cross_256mb(void *addr, size_t size)
+ /*
+  * We weren't able to allocate a buffer without crossing that boundary,
+  * so make do with the larger portion of the buffer that doesn't cross.
+- * Returns the new base of the buffer, and adjusts code_gen_buffer_size.
++ * Returns the new base and size of the buffer in *obuf and *osize.
+  */
+-static inline void *split_cross_256mb(void *buf1, size_t size1)
++static inline void split_cross_256mb(void **obuf, size_t *osize,
++                                     void *buf1, size_t size1)
+ {
+     void *buf2 = (void *)(((uintptr_t)buf1 + size1) & ~0x0ffffffful);
+     size_t size2 = buf1 + size1 - buf2;
+@@ -XXX,XX +XXX,XX @@ static inline void *split_cross_256mb(void *buf1, size_t size1)
+         buf1 = buf2;
+     }
+-    tcg_ctx->code_gen_buffer_size = size1;
+-    return buf1;
++    *obuf = buf1;
++    *osize = size1;
+ }
+ #endif
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
+     if (size > tb_size) {
+         size = QEMU_ALIGN_DOWN(tb_size, qemu_real_host_page_size);
+     }
+-    tcg_ctx->code_gen_buffer_size = size;
+ #ifdef __mips__
+     if (cross_256mb(buf, size)) {
+-        buf = split_cross_256mb(buf, size);
+-        size = tcg_ctx->code_gen_buffer_size;
++        split_cross_256mb(&buf, &size, buf, size);
+     }
+ #endif
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
+     qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+     tcg_ctx->code_gen_buffer = buf;
++    tcg_ctx->code_gen_buffer_size = size;
+     return true;
+ }
+ #elif defined(_WIN32)
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
+                          "allocate %zu bytes for jit buffer", size);
+         return false;
+     }
+-    tcg_ctx->code_gen_buffer_size = size;
+ #ifdef __mips__
+     if (cross_256mb(buf, size)) {
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
+             /* fallthru */
+         default:
+             /* Split the original buffer.  Free the smaller half.  */
+-            buf2 = split_cross_256mb(buf, size);
+-            size2 = tcg_ctx->code_gen_buffer_size;
++            split_cross_256mb(&buf2, &size2, buf, size);
+             if (buf == buf2) {
+                 munmap(buf + size2, size - size2);
+             } else {
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
+     qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+     tcg_ctx->code_gen_buffer = buf;
++    tcg_ctx->code_gen_buffer_size = size;
+     return true;
+ }
+--
+.25.1

-New patch
+[PULL 21/34] tcg: Move in_code_gen_buffer and tests to region.c
+Shortly, the full code_gen_buffer will only be visible
+to region.c, so move in_code_gen_buffer out-of-line.
+Move the debugging versions of tcg_splitwx_to_{rx,rw}
+to region.c as well, so that the compiler gets to see
+the implementation of in_code_gen_buffer.
+This leaves exactly one use of in_code_gen_buffer outside
+of region.c, in cpu_restore_state.  Which, being on the
+exception path, is not performance critical.
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ include/tcg/tcg.h | 11 +----------
+ tcg/region.c      | 34 ++++++++++++++++++++++++++++++++++
+ tcg/tcg.c         | 23 -----------------------
+files changed, 35 insertions(+), 33 deletions(-)
+diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
+index XXXXXXX..XXXXXXX 100644
+--- a/include/tcg/tcg.h
++++ b/include/tcg/tcg.h
+@@ -XXX,XX +XXX,XX @@ extern const void *tcg_code_gen_epilogue;
+ extern uintptr_t tcg_splitwx_diff;
+ extern TCGv_env cpu_env;
+-static inline bool in_code_gen_buffer(const void *p)
+-{
+-    const TCGContext *s = &tcg_init_ctx;
+-    /*
+-     * Much like it is valid to have a pointer to the byte past the
+-     * end of an array (so long as you don't dereference it), allow
+-     * a pointer to the byte past the end of the code gen buffer.
+-     */
+-    return (size_t)(p - s->code_gen_buffer) <= s->code_gen_buffer_size;
+-}
++bool in_code_gen_buffer(const void *p);
+ #ifdef CONFIG_DEBUG_TCG
+ const void *tcg_splitwx_to_rx(void *rw);
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ static struct tcg_region_state region;
+ static void *region_trees;
+ static size_t tree_size;
++bool in_code_gen_buffer(const void *p)
++{
++    const TCGContext *s = &tcg_init_ctx;
++    /*
++     * Much like it is valid to have a pointer to the byte past the
++     * end of an array (so long as you don't dereference it), allow
++     * a pointer to the byte past the end of the code gen buffer.
++     */
++    return (size_t)(p - s->code_gen_buffer) <= s->code_gen_buffer_size;
++}
++
++#ifdef CONFIG_DEBUG_TCG
++const void *tcg_splitwx_to_rx(void *rw)
++{
++    /* Pass NULL pointers unchanged. */
++    if (rw) {
++        g_assert(in_code_gen_buffer(rw));
++        rw += tcg_splitwx_diff;
++    }
++    return rw;
++}
++
++void *tcg_splitwx_to_rw(const void *rx)
++{
++    /* Pass NULL pointers unchanged. */
++    if (rx) {
++        rx -= tcg_splitwx_diff;
++        /* Assert that we end with a pointer in the rw region. */
++        g_assert(in_code_gen_buffer(rx));
++    }
++    return (void *)rx;
++}
++#endif /* CONFIG_DEBUG_TCG */
++
+ /* compare a pointer @ptr and a tb_tc @s */
+ static int ptr_cmp_tb_tc(const void *ptr, const struct tb_tc *s)
+ {
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef constraint_sets[] = {
+ #include "tcg-target.c.inc"
+-#ifdef CONFIG_DEBUG_TCG
+-const void *tcg_splitwx_to_rx(void *rw)
+-{
+-    /* Pass NULL pointers unchanged. */
+-    if (rw) {
+-        g_assert(in_code_gen_buffer(rw));
+-        rw += tcg_splitwx_diff;
+-    }
+-    return rw;
+-}
+-
+-void *tcg_splitwx_to_rw(const void *rx)
+-{
+-    /* Pass NULL pointers unchanged. */
+-    if (rx) {
+-        rx -= tcg_splitwx_diff;
+-        /* Assert that we end with a pointer in the rw region. */
+-        g_assert(in_code_gen_buffer(rx));
+-    }
+-    return (void *)rx;
+-}
+-#endif /* CONFIG_DEBUG_TCG */
+-
+ static void alloc_tcg_plugin_context(TCGContext *s)
+ {
+ #ifdef CONFIG_PLUGIN
+--
+.25.1

-New patch
+[PULL 22/34] tcg: Allocate code_gen_buffer into struct tcg_region_state
+Do not mess around with setting values within tcg_init_ctx.
+Put the values into 'region' directly, which is where they
+will live for the lifetime of the program.
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/region.c | 64 ++++++++++++++++++++++------------------------------
+file changed, 27 insertions(+), 37 deletions(-)
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ static size_t tree_size;
+ bool in_code_gen_buffer(const void *p)
+ {
+-    const TCGContext *s = &tcg_init_ctx;
+     /*
+      * Much like it is valid to have a pointer to the byte past the
+      * end of an array (so long as you don't dereference it), allow
+      * a pointer to the byte past the end of the code gen buffer.
+      */
+-    return (size_t)(p - s->code_gen_buffer) <= s->code_gen_buffer_size;
++    return (size_t)(p - region.start_aligned) <= region.total_size;
+ }
+ #ifdef CONFIG_DEBUG_TCG
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
+     }
+     qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+-    tcg_ctx->code_gen_buffer = buf;
+-    tcg_ctx->code_gen_buffer_size = size;
++    region.start_aligned = buf;
++    region.total_size = size;
+     return true;
+ }
+ #elif defined(_WIN32)
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+         return false;
+     }
+-    tcg_ctx->code_gen_buffer = buf;
+-    tcg_ctx->code_gen_buffer_size = size;
++    region.start_aligned = buf;
++    region.total_size = size;
+     return true;
+ }
+ #else
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
+     /* Request large pages for the buffer.  */
+     qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+-    tcg_ctx->code_gen_buffer = buf;
+-    tcg_ctx->code_gen_buffer_size = size;
++    region.start_aligned = buf;
++    region.total_size = size;
+     return true;
+ }
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
+         return false;
+     }
+     /* The size of the mapping may have been adjusted. */
+-    size = tcg_ctx->code_gen_buffer_size;
+-    buf_rx = tcg_ctx->code_gen_buffer;
++    buf_rx = region.start_aligned;
++    size = region.total_size;
+ #endif
+     buf_rw = qemu_memfd_alloc("tcg-jit", size, 0, &fd, errp);
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
+ #endif
+     close(fd);
+-    tcg_ctx->code_gen_buffer = buf_rw;
+-    tcg_ctx->code_gen_buffer_size = size;
++    region.start_aligned = buf_rw;
++    region.total_size = size;
+     tcg_splitwx_diff = buf_rx - buf_rw;
+     /* Request large pages for the buffer and the splitwx.  */
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
+         return false;
+     }
+-    buf_rw = (mach_vm_address_t)tcg_ctx->code_gen_buffer;
++    buf_rw = region.start_aligned;
+     buf_rx = 0;
+     ret = mach_vm_remap(mach_task_self(),
+                         &buf_rx,
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+  */
+ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+ {
+-    void *buf, *aligned, *end;
+-    size_t total_size;
+     size_t page_size;
+     size_t region_size;
+-    size_t n_regions;
+     size_t i;
+     bool ok;
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+                                splitwx, &error_fatal);
+     assert(ok);
+-    buf = tcg_init_ctx.code_gen_buffer;
+-    total_size = tcg_init_ctx.code_gen_buffer_size;
+-    page_size = qemu_real_host_page_size;
+-    n_regions = tcg_n_regions(total_size, max_cpus);
+-
+-    /* The first region will be 'aligned - buf' bytes larger than the others */
+-    aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
+-    g_assert(aligned < tcg_init_ctx.code_gen_buffer + total_size);
+-
+     /*
+      * Make region_size a multiple of page_size, using aligned as the start.
+      * As a result of this we might end up with a few extra pages at the end of
+      * the buffer; we will assign those to the last region.
+      */
+-    region_size = (total_size - (aligned - buf)) / n_regions;
++    region.n = tcg_n_regions(region.total_size, max_cpus);
++    page_size = qemu_real_host_page_size;
++    region_size = region.total_size / region.n;
+     region_size = QEMU_ALIGN_DOWN(region_size, page_size);
+     /* A region must have at least 2 pages; one code, one guard */
+     g_assert(region_size >= 2 * page_size);
++    region.stride = region_size;
++
++    /* Reserve space for guard pages. */
++    region.size = region_size - page_size;
++    region.total_size -= page_size;
++
++    /*
++     * The first region will be smaller than the others, via the prologue,
++     * which has yet to be allocated.  For now, the first region begins at
++     * the page boundary.
++     */
++    region.after_prologue = region.start_aligned;
+     /* init the region struct */
+     qemu_mutex_init(&region.lock);
+-    region.n = n_regions;
+-    region.size = region_size - page_size;
+-    region.stride = region_size;
+-    region.after_prologue = buf;
+-    region.start_aligned = aligned;
+-    /* page-align the end, since its last page will be a guard page */
+-    end = QEMU_ALIGN_PTR_DOWN(buf + total_size, page_size);
+-    /* account for that last guard page */
+-    end -= page_size;
+-    total_size = end - aligned;
+-    region.total_size = total_size;
+     /*
+      * Set guard pages in the rw buffer, as that's the one into which
+--
+.25.1

-New patch
+[PULL 23/34] tcg: Return the map protection from alloc_code_gen_buffer
+Change the interface from a boolean error indication to a
 negative error vs a non-negative protection.  For the moment
 this is only interface change, not making use of the new data.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
  tcg/region.c | 63 +++++++++++++++++++++++++++-------------------------
 file changed, 33 insertions(+), 30 deletions(-)
 diff --git a/tcg/region.c b/tcg/region.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/region.c
 +++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static inline void split_cross_256mb(void **obuf, size_t *osize,
  static uint8_t static_code_gen_buffer[DEFAULT_CODE_GEN_BUFFER_SIZE]
      __attribute__((aligned(CODE_GEN_ALIGN)));
 -static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
 +static int alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
  {
      void *buf, *end;
      size_t size;
      if (splitwx > 0) {
          error_setg(errp, "jit split-wx not supported");
 -        return false;
 +        return -1;
      }
      /* page-align the beginning and end of the buffer */
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
      region.start_aligned = buf;
      region.total_size = size;
 -    return true;
 +
 +    return PROT_READ | PROT_WRITE;
  }
  #elif defined(_WIN32)
 -static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
 +static int alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
  {
      void *buf;
      if (splitwx > 0) {
          error_setg(errp, "jit split-wx not supported");
 -        return false;
 +        return -1;
      }
      buf = VirtualAlloc(NULL, size, MEM_RESERVE | MEM_COMMIT,
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
      region.start_aligned = buf;
      region.total_size = size;
 -    return true;
 +
 +    return PAGE_READ | PAGE_WRITE | PAGE_EXEC;
  }
  #else
 -static bool alloc_code_gen_buffer_anon(size_t size, int prot,
 -                                       int flags, Error **errp)
 +static int alloc_code_gen_buffer_anon(size_t size, int prot,
 +                                      int flags, Error **errp)
  {
      void *buf;
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
      if (buf == MAP_FAILED) {
          error_setg_errno(errp, errno,
                           "allocate %zu bytes for jit buffer", size);
 -        return false;
 +        return -1;
      }
  #ifdef __mips__
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
      region.start_aligned = buf;
      region.total_size = size;
 -    return true;
 +    return prot;
  }
  #ifndef CONFIG_TCG_INTERPRETER
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
  #ifdef __mips__
      /* Find space for the RX mapping, vs the 256MiB regions. */
 -    if (!alloc_code_gen_buffer_anon(size, PROT_NONE,
 -                                    MAP_PRIVATE | MAP_ANONYMOUS |
 -                                    MAP_NORESERVE, errp)) {
 +    if (alloc_code_gen_buffer_anon(size, PROT_NONE,
 +                                   MAP_PRIVATE | MAP_ANONYMOUS |
 +                                   MAP_NORESERVE, errp) < 0) {
          return false;
      }
      /* The size of the mapping may have been adjusted. */
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
      /* Request large pages for the buffer and the splitwx.  */
      qemu_madvise(buf_rw, size, QEMU_MADV_HUGEPAGE);
      qemu_madvise(buf_rx, size, QEMU_MADV_HUGEPAGE);
 -    return true;
 +    return PROT_READ | PROT_WRITE;
   fail_rx:
      error_setg_errno(errp, errno, "failed to map shared memory for execute");
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
      if (fd >= 0) {
          close(fd);
      }
 -    return false;
 +    return -1;
  }
  #endif /* CONFIG_POSIX */
@@ -XXX,XX +XXX,XX @@ extern kern_return_t mach_vm_remap(vm_map_t target_task,
                                     vm_prot_t *max_protection,
                                     vm_inherit_t inheritance);
 -static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
 +static int alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
  {
      kern_return_t ret;
      mach_vm_address_t buf_rw, buf_rx;
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
      /* Map the read-write portion via normal anon memory. */
      if (!alloc_code_gen_buffer_anon(size, PROT_READ | PROT_WRITE,
                                      MAP_PRIVATE | MAP_ANONYMOUS, errp)) {
 -        return false;
 +        return -1;
      }
      buf_rw = region.start_aligned;
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
          /* TODO: Convert "ret" to a human readable error message. */
          error_setg(errp, "vm_remap for jit splitwx failed");
          munmap((void *)buf_rw, size);
 -        return false;
 +        return -1;
      }
      if (mprotect((void *)buf_rx, size, PROT_READ | PROT_EXEC) != 0) {
          error_setg_errno(errp, errno, "mprotect for jit splitwx");
          munmap((void *)buf_rx, size);
          munmap((void *)buf_rw, size);
 -        return false;
 +        return -1;
      }
      tcg_splitwx_diff = buf_rx - buf_rw;
 -    return true;
 +    return PROT_READ | PROT_WRITE;
  }
  #endif /* CONFIG_DARWIN */
  #endif /* CONFIG_TCG_INTERPRETER */
 -static bool alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
 +static int alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
  {
  #ifndef CONFIG_TCG_INTERPRETER
  # ifdef CONFIG_DARWIN
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
  # endif
  #endif
      error_setg(errp, "jit split-wx not supported");
 -    return false;
 +    return -1;
  }
 -static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
 +static int alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
  {
      ERRP_GUARD();
      int prot, flags;
      if (splitwx) {
 -        if (alloc_code_gen_buffer_splitwx(size, errp)) {
 -            return true;
 +        prot = alloc_code_gen_buffer_splitwx(size, errp);
 +        if (prot >= 0) {
 +            return prot;
          }
          /*
           * If splitwx force-on (1), fail;
           * if splitwx default-on (-1), fall through to splitwx off.
           */
          if (splitwx > 0) {
 -            return false;
 +            return -1;
          }
          error_free_or_abort(errp);
      }
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
      size_t page_size;
      size_t region_size;
      size_t i;
 -    bool ok;
 +    int have_prot;
 -    ok = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
 -                               splitwx, &error_fatal);
 -    assert(ok);
 +    have_prot = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
 +                                      splitwx, &error_fatal);
 +    assert(have_prot >= 0);
      /*
       * Make region_size a multiple of page_size, using aligned as the start.
 --
 .25.1

-New patch
+[PULL 24/34] tcg: Sink qemu_madvise call to common code
+Move the call out of the N versions of alloc_code_gen_buffer
+and into tcg_region_init.
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/region.c | 14 +++++++-------
+file changed, 7 insertions(+), 7 deletions(-)
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
+         error_setg_errno(errp, errno, "mprotect of jit buffer");
+         return false;
+     }
+-    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+     region.start_aligned = buf;
+     region.total_size = size;
+@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer_anon(size_t size, int prot,
+     }
+ #endif
+-    /* Request large pages for the buffer.  */
+-    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+-
+     region.start_aligned = buf;
+     region.total_size = size;
+     return prot;
+@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
+     region.total_size = size;
+     tcg_splitwx_diff = buf_rx - buf_rw;
+-    /* Request large pages for the buffer and the splitwx.  */
+-    qemu_madvise(buf_rw, size, QEMU_MADV_HUGEPAGE);
+-    qemu_madvise(buf_rx, size, QEMU_MADV_HUGEPAGE);
+     return PROT_READ | PROT_WRITE;
+  fail_rx:
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+                                       splitwx, &error_fatal);
+     assert(have_prot >= 0);
++    /* Request large pages for the buffer and the splitwx.  */
++    qemu_madvise(region.start_aligned, region.total_size, QEMU_MADV_HUGEPAGE);
++    if (tcg_splitwx_diff) {
++        qemu_madvise(region.start_aligned + tcg_splitwx_diff,
++                     region.total_size, QEMU_MADV_HUGEPAGE);
++    }
++
+     /*
+      * Make region_size a multiple of page_size, using aligned as the start.
+      * As a result of this we might end up with a few extra pages at the end of
+--
+.25.1

-New patch
+[PULL 25/34] util/osdep: Add qemu_mprotect_rw
+For --enable-tcg-interpreter on Windows, we will need this.
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ include/qemu/osdep.h | 1 +
+ util/osdep.c         | 9 +++++++++
+files changed, 10 insertions(+)
+diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
+index XXXXXXX..XXXXXXX 100644
+--- a/include/qemu/osdep.h
++++ b/include/qemu/osdep.h
+@@ -XXX,XX +XXX,XX @@ void sigaction_invoke(struct sigaction *action,
+ #endif
+ int qemu_madvise(void *addr, size_t len, int advice);
++int qemu_mprotect_rw(void *addr, size_t size);
+ int qemu_mprotect_rwx(void *addr, size_t size);
+ int qemu_mprotect_none(void *addr, size_t size);
+diff --git a/util/osdep.c b/util/osdep.c
+index XXXXXXX..XXXXXXX 100644
+--- a/util/osdep.c
++++ b/util/osdep.c
+@@ -XXX,XX +XXX,XX @@ static int qemu_mprotect__osdep(void *addr, size_t size, int prot)
+ #endif
+ }
++int qemu_mprotect_rw(void *addr, size_t size)
++{
++#ifdef _WIN32
++    return qemu_mprotect__osdep(addr, size, PAGE_READWRITE);
++#else
++    return qemu_mprotect__osdep(addr, size, PROT_READ | PROT_WRITE);
++#endif
++}
++
+ int qemu_mprotect_rwx(void *addr, size_t size)
+ {
+ #ifdef _WIN32
+--
+.25.1

-New patch
+[PULL 26/34] tcg: Round the tb_size default from qemu_get_host_physmem
+If qemu_get_host_physmem returns an odd number of pages,
+then physmem / 8 will not be a multiple of the page size.
+The following was observed on a gitlab runner:
+ERROR qtest-arm/boot-serial-test - Bail out!
+ERROR:../util/osdep.c:80:qemu_mprotect__osdep: \
+  assertion failed: (!(size & ~qemu_real_host_page_mask))
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/region.c | 47 +++++++++++++++++++++--------------------------
+file changed, 21 insertions(+), 26 deletions(-)
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(size_t tb_size, unsigned max_cpus)
+   (DEFAULT_CODE_GEN_BUFFER_SIZE_1 < MAX_CODE_GEN_BUFFER_SIZE \
+    ? DEFAULT_CODE_GEN_BUFFER_SIZE_1 : MAX_CODE_GEN_BUFFER_SIZE)
+-static size_t size_code_gen_buffer(size_t tb_size)
+-{
+-    /* Size the buffer.  */
+-    if (tb_size == 0) {
+-        size_t phys_mem = qemu_get_host_physmem();
+-        if (phys_mem == 0) {
+-            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+-        } else {
+-            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
+-        }
+-    }
+-    if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
+-        tb_size = MIN_CODE_GEN_BUFFER_SIZE;
+-    }
+-    if (tb_size > MAX_CODE_GEN_BUFFER_SIZE) {
+-        tb_size = MAX_CODE_GEN_BUFFER_SIZE;
+-    }
+-    return tb_size;
+-}
+-
+ #ifdef __mips__
+ /*
+  * In order to use J and JAL within the code_gen_buffer, we require
+@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+  */
+ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+ {
+-    size_t page_size;
++    const size_t page_size = qemu_real_host_page_size;
+     size_t region_size;
+     size_t i;
+     int have_prot;
+-    have_prot = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
+-                                      splitwx, &error_fatal);
++    /* Size the buffer.  */
++    if (tb_size == 0) {
++        size_t phys_mem = qemu_get_host_physmem();
++        if (phys_mem == 0) {
++            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
++        } else {
++            tb_size = QEMU_ALIGN_DOWN(phys_mem / 8, page_size);
++            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, tb_size);
++        }
++    }
++    if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
++        tb_size = MIN_CODE_GEN_BUFFER_SIZE;
++    }
++    if (tb_size > MAX_CODE_GEN_BUFFER_SIZE) {
++        tb_size = MAX_CODE_GEN_BUFFER_SIZE;
++    }
++
++    have_prot = alloc_code_gen_buffer(tb_size, splitwx, &error_fatal);
+     assert(have_prot >= 0);
+     /* Request large pages for the buffer and the splitwx.  */
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+      * As a result of this we might end up with a few extra pages at the end of
+      * the buffer; we will assign those to the last region.
+      */
+-    region.n = tcg_n_regions(region.total_size, max_cpus);
+-    page_size = qemu_real_host_page_size;
+-    region_size = region.total_size / region.n;
++    region.n = tcg_n_regions(tb_size, max_cpus);
++    region_size = tb_size / region.n;
+     region_size = QEMU_ALIGN_DOWN(region_size, page_size);
+     /* A region must have at least 2 pages; one code, one guard */
+--
+.25.1

-New patch
+[PULL 27/34] tcg: Merge buffer protection and guard page protection
+Do not handle protections on a case-by-case basis in the
+various alloc_code_gen_buffer instances; do it within a
+single loop in tcg_region_init.
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/region.c | 45 +++++++++++++++++++++++++++++++--------------
+file changed, 31 insertions(+), 14 deletions(-)
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
+     }
+ #endif
+-    if (qemu_mprotect_rwx(buf, size)) {
+-        error_setg_errno(errp, errno, "mprotect of jit buffer");
+-        return false;
+-    }
+-
+     region.start_aligned = buf;
+     region.total_size = size;
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+ {
+     const size_t page_size = qemu_real_host_page_size;
+     size_t region_size;
+-    size_t i;
+-    int have_prot;
++    int have_prot, need_prot;
+     /* Size the buffer.  */
+     if (tb_size == 0) {
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+      * Set guard pages in the rw buffer, as that's the one into which
+      * buffer overruns could occur.  Do not set guard pages in the rx
+      * buffer -- let that one use hugepages throughout.
++     * Work with the page protections set up with the initial mapping.
+      */
+-    for (i = 0; i < region.n; i++) {
++    need_prot = PAGE_READ | PAGE_WRITE;
++#ifndef CONFIG_TCG_INTERPRETER
++    if (tcg_splitwx_diff == 0) {
++        need_prot |= PAGE_EXEC;
++    }
++#endif
++    for (size_t i = 0, n = region.n; i < n; i++) {
+         void *start, *end;
+         tcg_region_bounds(i, &start, &end);
++        if (have_prot != need_prot) {
++            int rc;
+-        /*
+-         * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
+-         * rejects a permission change from RWX -> NONE.  Guard pages are
+-         * nice for bug detection but are not essential; ignore any failure.
+-         */
+-        (void)qemu_mprotect_none(end, page_size);
++            if (need_prot == (PAGE_READ | PAGE_WRITE | PAGE_EXEC)) {
++                rc = qemu_mprotect_rwx(start, end - start);
++            } else if (need_prot == (PAGE_READ | PAGE_WRITE)) {
++                rc = qemu_mprotect_rw(start, end - start);
++            } else {
++                g_assert_not_reached();
++            }
++            if (rc) {
++                error_setg_errno(&error_fatal, errno,
++                                 "mprotect of jit buffer");
++            }
++        }
++        if (have_prot != 0) {
++            /*
++             * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
++             * rejects a permission change from RWX -> NONE.  Guard pages are
++             * nice for bug detection but are not essential; ignore any failure.
++             */
++            (void)qemu_mprotect_none(end, page_size);
++        }
+     }
+     tcg_region_trees_init();
+--
+.25.1

-New patch
+[PULL 28/34] tcg: When allocating for !splitwx, begin with PROT_NONE
+There's a change in mprotect() behaviour [1] in the latest macOS
+on M1 and it's not yet clear if it's going to be fixed by Apple.
+In this case, instead of changing permissions of N guard pages,
+we change permissions of N rwx regions.  The same number of
+syscalls are required either way.
+[1] https://gist.github.com/hikalium/75ae822466ee4da13cbbe486498a191f
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/region.c | 19 +++++++++----------
+file changed, 9 insertions(+), 10 deletions(-)
+diff --git a/tcg/region.c b/tcg/region.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/region.c
++++ b/tcg/region.c
+@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+         error_free_or_abort(errp);
+     }
+-    prot = PROT_READ | PROT_WRITE | PROT_EXEC;
++    /*
++     * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
++     * rejects a permission change from RWX -> NONE when reserving the
++     * guard pages later.  We can go the other way with the same number
++     * of syscalls, so always begin with PROT_NONE.
++     */
++    prot = PROT_NONE;
+     flags = MAP_PRIVATE | MAP_ANONYMOUS;
+-#ifdef CONFIG_TCG_INTERPRETER
+-    /* The tcg interpreter does not need execute permission. */
+-    prot = PROT_READ | PROT_WRITE;
+-#elif defined(CONFIG_DARWIN)
++#ifdef CONFIG_DARWIN
+     /* Applicable to both iOS and macOS (Apple Silicon). */
+     if (!splitwx) {
+         flags |= MAP_JIT;
+@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
+             }
+         }
+         if (have_prot != 0) {
+-            /*
+-             * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
+-             * rejects a permission change from RWX -> NONE.  Guard pages are
+-             * nice for bug detection but are not essential; ignore any failure.
+-             */
++            /* Guard pages are nice for bug detection but are not essential. */
+             (void)qemu_mprotect_none(end, page_size);
+         }
+     }
+--
+.25.1

-New patch
+[PULL 29/34] tcg: Move tcg_init_ctx and tcg_ctx from accel/tcg/
+These variables belong to the jit side, not the user side.
+Since tcg_init_ctx is no longer used outside of tcg/, move
+the declaration to tcg-internal.h.
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ include/tcg/tcg.h         | 1 -
+ tcg/tcg-internal.h        | 1 +
+ accel/tcg/translate-all.c | 3 ---
+ tcg/tcg.c                 | 3 +++
+files changed, 4 insertions(+), 4 deletions(-)
+diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
+index XXXXXXX..XXXXXXX 100644
+--- a/include/tcg/tcg.h
++++ b/include/tcg/tcg.h
+@@ -XXX,XX +XXX,XX @@ static inline bool temp_readonly(TCGTemp *ts)
+     return ts->kind >= TEMP_FIXED;
+ }
+-extern TCGContext tcg_init_ctx;
+ extern __thread TCGContext *tcg_ctx;
+ extern const void *tcg_code_gen_epilogue;
+ extern uintptr_t tcg_splitwx_diff;
+diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg-internal.h
++++ b/tcg/tcg-internal.h
+@@ -XXX,XX +XXX,XX @@
+ #define TCG_HIGHWATER 1024
++extern TCGContext tcg_init_ctx;
+ extern TCGContext **tcg_ctxs;
+ extern unsigned int tcg_cur_ctxs;
+ extern unsigned int tcg_max_ctxs;
+diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
+index XXXXXXX..XXXXXXX 100644
+--- a/accel/tcg/translate-all.c
++++ b/accel/tcg/translate-all.c
+@@ -XXX,XX +XXX,XX @@ static int v_l2_levels;
+ static void *l1_map[V_L1_MAX_SIZE];
+-/* code generation context */
+-TCGContext tcg_init_ctx;
+-__thread TCGContext *tcg_ctx;
+ TBContext tb_ctx;
+ static void page_table_config_init(void)
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct);
+ static int tcg_out_ldst_finalize(TCGContext *s);
+ #endif
++TCGContext tcg_init_ctx;
++__thread TCGContext *tcg_ctx;
++
+ TCGContext **tcg_ctxs;
+ unsigned int tcg_cur_ctxs;
+ unsigned int tcg_max_ctxs;
+--
+.25.1

-New patch
+[PULL 30/34] tcg: Introduce tcg_remove_ops_after
+Introduce a function to remove everything emitted
+since a given point.
+Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ include/tcg/tcg.h | 10 ++++++++++
+ tcg/tcg.c         | 13 +++++++++++++
+files changed, 23 insertions(+)
+diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
+index XXXXXXX..XXXXXXX 100644
+--- a/include/tcg/tcg.h
++++ b/include/tcg/tcg.h
+@@ -XXX,XX +XXX,XX @@ void tcg_op_remove(TCGContext *s, TCGOp *op);
+ TCGOp *tcg_op_insert_before(TCGContext *s, TCGOp *op, TCGOpcode opc);
+ TCGOp *tcg_op_insert_after(TCGContext *s, TCGOp *op, TCGOpcode opc);
++/**
++ * tcg_remove_ops_after:
++ * @op: target operation
++ *
++ * Discard any opcodes emitted since @op.  Expected usage is to save
++ * a starting point with tcg_last_op(), speculatively emit opcodes,
++ * then decide whether or not to keep those opcodes after the fact.
++ */
++void tcg_remove_ops_after(TCGOp *op);
++
+ void tcg_optimize(TCGContext *s);
+ /* Allocate a new temporary and initialize it with a constant. */
+diff --git a/tcg/tcg.c b/tcg/tcg.c
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.c
++++ b/tcg/tcg.c
+@@ -XXX,XX +XXX,XX @@ void tcg_op_remove(TCGContext *s, TCGOp *op)
+ #endif
+ }
++void tcg_remove_ops_after(TCGOp *op)
++{
++    TCGContext *s = tcg_ctx;
++
++    while (true) {
++        TCGOp *last = tcg_last_op();
++        if (last == op) {
++            return;
++        }
++        tcg_op_remove(s, last);
++    }
++}
++
+ static TCGOp *tcg_op_alloc(TCGOpcode opc)
+ {
+     TCGContext *s = tcg_ctx;
+--
+.25.1

-New patch
+[PULL 31/34] tcg: Fix documentation for tcg_constant_* vs tcg_temp_free_*
+At some point during the development of tcg_constant_*, I changed
+my mind about whether such temps should be able to be passed to
+tcg_temp_free_*.  The final version committed allows this, but the
+commentary was not updated to match.
+Fixes: c0522136adf
+Reported-by: Peter Maydell <peter.maydell@linaro.org>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ include/tcg/tcg.h | 3 ++-
+file changed, 2 insertions(+), 1 deletion(-)
+diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
+index XXXXXXX..XXXXXXX 100644
+--- a/include/tcg/tcg.h
++++ b/include/tcg/tcg.h
+@@ -XXX,XX +XXX,XX @@ TCGv_vec tcg_const_ones_vec_matching(TCGv_vec);
+ /*
+  * Locate or create a read-only temporary that is a constant.
+- * This kind of temporary need not and should not be freed.
++ * This kind of temporary need not be freed, but for convenience
++ * will be silently ignored by tcg_temp_free_*.
+  */
+ TCGTemp *tcg_constant_internal(TCGType type, int64_t val);
+--
+.25.1

-New patch
+[PULL 32/34] tcg/arm: Fix tcg_out_op function signature
+From: "Jose R. Ziviani" <jziviani@suse.de>
+Commit 5e8892db93 fixed several function signatures but tcg_out_op for
+arm is missing. This patch fixes it as well.
+Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
+Message-Id: <20210610224450.23425-1-jziviani@suse.de>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ tcg/arm/tcg-target.c.inc | 3 ++-
+file changed, 2 insertions(+), 1 deletion(-)
+diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/arm/tcg-target.c.inc
++++ b/tcg/arm/tcg-target.c.inc
+@@ -XXX,XX +XXX,XX @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64)
+ static void tcg_out_epilogue(TCGContext *s);
+ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc,
+-                const TCGArg *args, const int *const_args)
++                const TCGArg args[TCG_MAX_OP_ARGS],
++                const int const_args[TCG_MAX_OP_ARGS])
+ {
+     TCGArg a0, a1, a2, a3, a4, a5;
+     int c;
+--
+.25.1

-New patch
+[PULL 33/34] softfloat: Fix tp init in float32_exp2
+Typo in the conversion to FloatParts64.
+Fixes: 572c4d862ff2
+Fixes: Coverity CID 1457457
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Message-Id: <20210607223812.110596-1-richard.henderson@linaro.org>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ fpu/softfloat.c | 2 +-
+file changed, 1 insertion(+), 1 deletion(-)
+diff --git a/fpu/softfloat.c b/fpu/softfloat.c
+index XXXXXXX..XXXXXXX 100644
+--- a/fpu/softfloat.c
++++ b/fpu/softfloat.c
+@@ -XXX,XX +XXX,XX @@ float32 float32_exp2(float32 a, float_status *status)
+     float_raise(float_flag_inexact, status);
+-    float64_unpack_canonical(&xnp, float64_ln2, status);
++    float64_unpack_canonical(&tp, float64_ln2, status);
+     xp = *parts_mul(&xp, &tp, status);
+     xnp = xp;
+--
+.25.1

-New patch
+[PULL 34/34] docs/devel: Explain in more detail the TB chaining mechanisms
+From: Luis Pires <luis.pires@eldorado.org.br>
+Signed-off-by: Luis Pires <luis.pires@eldorado.org.br>
+Message-Id: <20210601125143.191165-1-luis.pires@eldorado.org.br>
+Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+---
+ docs/devel/tcg.rst | 101 ++++++++++++++++++++++++++++++++++++++++-----
+file changed, 90 insertions(+), 11 deletions(-)
+diff --git a/docs/devel/tcg.rst b/docs/devel/tcg.rst
+index XXXXXXX..XXXXXXX 100644
+--- a/docs/devel/tcg.rst
++++ b/docs/devel/tcg.rst
+@@ -XXX,XX +XXX,XX @@ performances.
+ QEMU's dynamic translation backend is called TCG, for "Tiny Code
+ Generator". For more information, please take a look at ``tcg/README``.
+-Some notable features of QEMU's dynamic translator are:
++The following sections outline some notable features and implementation
++details of QEMU's dynamic translator.
+ CPU state optimisations
+ -----------------------
+-The target CPUs have many internal states which change the way it
+-evaluates instructions. In order to achieve a good speed, the
++The target CPUs have many internal states which change the way they
++evaluate instructions. In order to achieve a good speed, the
+ translation phase considers that some state information of the virtual
+ CPU cannot change in it. The state is recorded in the Translation
+ Block (TB). If the state changes (e.g. privilege level), a new TB will
+@@ -XXX,XX +XXX,XX @@ Direct block chaining
+ ---------------------
+ After each translated basic block is executed, QEMU uses the simulated
+-Program Counter (PC) and other cpu state information (such as the CS
++Program Counter (PC) and other CPU state information (such as the CS
+ segment base value) to find the next basic block.
+-In order to accelerate the most common cases where the new simulated PC
+-is known, QEMU can patch a basic block so that it jumps directly to the
+-next one.
++In its simplest, less optimized form, this is done by exiting from the
++current TB, going through the TB epilogue, and then back to the
++main loop. That’s where QEMU looks for the next TB to execute,
++translating it from the guest architecture if it isn’t already available
++in memory. Then QEMU proceeds to execute this next TB, starting at the
++prologue and then moving on to the translated instructions.
+-The most portable code uses an indirect jump. An indirect jump makes
+-it easier to make the jump target modification atomic. On some host
+-architectures (such as x86 or PowerPC), the ``JUMP`` opcode is
+-directly patched so that the block chaining has no overhead.
++Exiting from the TB this way will cause the ``cpu_exec_interrupt()``
++callback to be re-evaluated before executing additional instructions.
++It is mandatory to exit this way after any CPU state changes that may
++unmask interrupts.
++
++In order to accelerate the cases where the TB for the new
++simulated PC is already available, QEMU has mechanisms that allow
++multiple TBs to be chained directly, without having to go back to the
++main loop as described above. These mechanisms are:
++
++``lookup_and_goto_ptr``
++^^^^^^^^^^^^^^^^^^^^^^^
++
++Calling ``tcg_gen_lookup_and_goto_ptr()`` will emit a call to
++``helper_lookup_tb_ptr``. This helper will look for an existing TB that
++matches the current CPU state. If the destination TB is available its
++code address is returned, otherwise the address of the JIT epilogue is
++returned. The call to the helper is always followed by the tcg ``goto_ptr``
++opcode, which branches to the returned address. In this way, we either
++branch to the next TB or return to the main loop.
++
++``goto_tb + exit_tb``
++^^^^^^^^^^^^^^^^^^^^^
++
++The translation code usually implements branching by performing the
++following steps:
++
++1. Call ``tcg_gen_goto_tb()`` passing a jump slot index (either 0 or 1)
++   as a parameter.
++
++2. Emit TCG instructions to update the CPU state with any information
++   that has been assumed constant and is required by the main loop to
++   correctly locate and execute the next TB. For most guests, this is
++   just the PC of the branch destination, but others may store additional
++   data. The information updated in this step must be inferable from both
++   ``cpu_get_tb_cpu_state()`` and ``cpu_restore_state()``.
++
++3. Call ``tcg_gen_exit_tb()`` passing the address of the current TB and
++   the jump slot index again.
++
++Step 1, ``tcg_gen_goto_tb()``, will emit a ``goto_tb`` TCG
++instruction that later on gets translated to a jump to an address
++associated with the specified jump slot. Initially, this is the address
++of step 2's instructions, which update the CPU state information. Step 3,
++``tcg_gen_exit_tb()``, exits from the current TB returning a tagged
++pointer composed of the last executed TB’s address and the jump slot
++index.
++
++The first time this whole sequence is executed, step 1 simply jumps
++to step 2. Then the CPU state information gets updated and we exit from
++the current TB. As a result, the behavior is very similar to the less
++optimized form described earlier in this section.
++
++Next, the main loop looks for the next TB to execute using the
++current CPU state information (creating the TB if it wasn’t already
++available) and, before starting to execute the new TB’s instructions,
++patches the previously executed TB by associating one of its jump
++slots (the one specified in the call to ``tcg_gen_exit_tb()``) with the
++address of the new TB.
++
++The next time this previous TB is executed and we get to that same
++``goto_tb`` step, it will already be patched (assuming the destination TB
++is still in memory) and will jump directly to the first instruction of
++the destination TB, without going back to the main loop.
++
++For the ``goto_tb + exit_tb`` mechanism to be used, the following
++conditions need to be satisfied:
++
++* The change in CPU state must be constant, e.g., a direct branch and
++  not an indirect branch.
++
++* The direct branch cannot cross a page boundary. Memory mappings
++  may change, causing the code at the destination address to change.
++
++Note that, on step 3 (``tcg_gen_exit_tb()``), in addition to the
++jump slot index, the address of the TB just executed is also returned.
++This address corresponds to the TB that will be patched; it may be
++different than the one that was directly executed from the main loop
++if the latter had already been chained to other TBs.
+ Self-modifying code and translated code invalidation
+ ----------------------------------------------------
+--
+.25.1

The following changes since commit 2ecfc0657afa5d29a373271b342f704a1a3c6737:

Merge remote-tracking branch 'remotes/armbru/tags/pull-misc-2020-12-10' into staging (2020-12-10 17:01:05 +0000)

are available in the Git repository at:

https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20201210

for you to fetch changes up to 9e2658d62ebc23efe7df43fc0e306f129510d874:

accel/tcg: rename tcg-cpus functions to match module name (2020-12-10 17:44:10 -0600)

----------------------------------------------------------------
Split CpusAccel for tcg variants

----------------------------------------------------------------
Claudio Fontana (3):
      accel/tcg: split CpusAccel into three TCG variants
      accel/tcg: split tcg_start_vcpu_thread
      accel/tcg: rename tcg-cpus functions to match module name

accel/tcg/tcg-cpus-icount.h |  17 ++
 accel/tcg/tcg-cpus-rr.h     |  21 ++
 accel/tcg/tcg-cpus.h        |  12 +-
 accel/tcg/tcg-all.c         |  13 +-
 accel/tcg/tcg-cpus-icount.c | 147 +++++++++++++
 accel/tcg/tcg-cpus-mttcg.c  | 140 ++++++++++++
 accel/tcg/tcg-cpus-rr.c     | 305 ++++++++++++++++++++++++++
 accel/tcg/tcg-cpus.c        | 506 +-------------------------------------------
 softmmu/icount.c            |   2 +-
 accel/tcg/meson.build       |   9 +-
 10 files changed, 670 insertions(+), 502 deletions(-)
 create mode 100644 accel/tcg/tcg-cpus-icount.h
 create mode 100644 accel/tcg/tcg-cpus-rr.h
 create mode 100644 accel/tcg/tcg-cpus-icount.c
 create mode 100644 accel/tcg/tcg-cpus-mttcg.c
 create mode 100644 accel/tcg/tcg-cpus-rr.c

From: Claudio Fontana <cfontana@suse.de>

split up the CpusAccel tcg_cpus into three TCG variants:

tcg_cpus_rr (single threaded, round robin cpus)
tcg_cpus_icount (same as rr, but with instruction counting enabled)
tcg_cpus_mttcg (multi-threaded cpus)

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Claudio Fontana <cfontana@suse.de>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201015143217.29337-2-cfontana@suse.de>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/tcg-cpus-icount.h |  17 ++
 accel/tcg/tcg-cpus-mttcg.h  |  21 ++
 accel/tcg/tcg-cpus-rr.h     |  20 ++
 accel/tcg/tcg-cpus.h        |  13 +-
 accel/tcg/tcg-all.c         |   8 +-
 accel/tcg/tcg-cpus-icount.c | 147 +++++++++++
 accel/tcg/tcg-cpus-mttcg.c  | 117 +++++++++
 accel/tcg/tcg-cpus-rr.c     | 270 ++++++++++++++++++++
 accel/tcg/tcg-cpus.c        | 484 ++----------------------------------
 softmmu/icount.c            |   2 +-
 accel/tcg/meson.build       |   9 +-
 11 files changed, 646 insertions(+), 462 deletions(-)
 create mode 100644 accel/tcg/tcg-cpus-icount.h
 create mode 100644 accel/tcg/tcg-cpus-mttcg.h
 create mode 100644 accel/tcg/tcg-cpus-rr.h
 create mode 100644 accel/tcg/tcg-cpus-icount.c
 create mode 100644 accel/tcg/tcg-cpus-mttcg.c
 create mode 100644 accel/tcg/tcg-cpus-rr.c

diff --git a/accel/tcg/tcg-cpus-icount.h b/accel/tcg/tcg-cpus-icount.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/accel/tcg/tcg-cpus-icount.h
@@ -XXX,XX +XXX,XX @@
+/*
+ * QEMU TCG Single Threaded vCPUs implementation using instruction counting
+ *
+ * Copyright 2020 SUSE LLC
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef TCG_CPUS_ICOUNT_H
+#define TCG_CPUS_ICOUNT_H
+
+void handle_icount_deadline(void);
+void prepare_icount_for_run(CPUState *cpu);
+void process_icount_data(CPUState *cpu);
+
+#endif /* TCG_CPUS_ICOUNT_H */
diff --git a/accel/tcg/tcg-cpus-mttcg.h b/accel/tcg/tcg-cpus-mttcg.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/accel/tcg/tcg-cpus-mttcg.h
@@ -XXX,XX +XXX,XX @@
+/*
+ * QEMU TCG Multi Threaded vCPUs implementation
+ *
+ * Copyright 2020 SUSE LLC
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef TCG_CPUS_MTTCG_H
+#define TCG_CPUS_MTTCG_H
+
+/*
+ * In the multi-threaded case each vCPU has its own thread. The TLS
+ * variable current_cpu can be used deep in the code to find the
+ * current CPUState for a given thread.
+ */
+
+void *tcg_cpu_thread_fn(void *arg);
+
+#endif /* TCG_CPUS_MTTCG_H */
diff --git a/accel/tcg/tcg-cpus-rr.h b/accel/tcg/tcg-cpus-rr.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/accel/tcg/tcg-cpus-rr.h
@@ -XXX,XX +XXX,XX @@
+/*
+ * QEMU TCG Single Threaded vCPUs implementation
+ *
+ * Copyright 2020 SUSE LLC
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef TCG_CPUS_RR_H
+#define TCG_CPUS_RR_H
+
+#define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
+
+/* Kick all RR vCPUs. */
+void qemu_cpu_kick_rr_cpus(CPUState *unused);
+
+void *tcg_rr_cpu_thread_fn(void *arg);
+
+#endif /* TCG_CPUS_RR_H */
diff --git a/accel/tcg/tcg-cpus.h b/accel/tcg/tcg-cpus.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus.h
+++ b/accel/tcg/tcg-cpus.h
@@ -XXX,XX +XXX,XX @@
 /*
- * Accelerator CPUS Interface
+ * QEMU TCG vCPU common functionality
+ *
+ * Functionality common to all TCG vcpu variants: mttcg, rr and icount.
  *
  * Copyright 2020 SUSE LLC
  *
@@ -XXX,XX +XXX,XX @@
 
 #include "sysemu/cpus.h"
 
-extern const CpusAccel tcg_cpus;
+extern const CpusAccel tcg_cpus_mttcg;
+extern const CpusAccel tcg_cpus_icount;
+extern const CpusAccel tcg_cpus_rr;
+
+void tcg_start_vcpu_thread(CPUState *cpu);
+void qemu_tcg_destroy_vcpu(CPUState *cpu);
+int tcg_cpu_exec(CPUState *cpu);
+void tcg_handle_interrupt(CPUState *cpu, int mask);
 
 #endif /* TCG_CPUS_H */
diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -XXX,XX +XXX,XX @@ static int tcg_init(MachineState *ms)
 
     tcg_exec_init(s->tb_size * 1024 * 1024);
     mttcg_enabled = s->mttcg_enabled;
-    cpus_register_accel(&tcg_cpus);
 
+    if (mttcg_enabled) {
+        cpus_register_accel(&tcg_cpus_mttcg);
+    } else if (icount_enabled()) {
+        cpus_register_accel(&tcg_cpus_icount);
+    } else {
+        cpus_register_accel(&tcg_cpus_rr);
+    }
     return 0;
 }
 
diff --git a/accel/tcg/tcg-cpus-icount.c b/accel/tcg/tcg-cpus-icount.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/accel/tcg/tcg-cpus-icount.c
@@ -XXX,XX +XXX,XX @@
+/*
+ * QEMU TCG Single Threaded vCPUs implementation using instruction counting
+ *
+ * Copyright (c) 2003-2008 Fabrice Bellard
+ * Copyright (c) 2014 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "sysemu/tcg.h"
+#include "sysemu/replay.h"
+#include "qemu/main-loop.h"
+#include "qemu/guest-random.h"
+#include "exec/exec-all.h"
+#include "hw/boards.h"
+
+#include "tcg-cpus.h"
+#include "tcg-cpus-icount.h"
+#include "tcg-cpus-rr.h"
+
+static int64_t tcg_get_icount_limit(void)
+{
+    int64_t deadline;
+
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        /*
+         * Include all the timers, because they may need an attention.
+         * Too long CPU execution may create unnecessary delay in UI.
+         */
+        deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
+                                              QEMU_TIMER_ATTR_ALL);
+        /* Check realtime timers, because they help with input processing */
+        deadline = qemu_soonest_timeout(deadline,
+                qemu_clock_deadline_ns_all(QEMU_CLOCK_REALTIME,
+                                           QEMU_TIMER_ATTR_ALL));
+
+        /*
+         * Maintain prior (possibly buggy) behaviour where if no deadline
+         * was set (as there is no QEMU_CLOCK_VIRTUAL timer) or it is more than
+         * INT32_MAX nanoseconds ahead, we still use INT32_MAX
+         * nanoseconds.
+         */
+        if ((deadline < 0) || (deadline > INT32_MAX)) {
+            deadline = INT32_MAX;
+        }
+
+        return icount_round(deadline);
+    } else {
+        return replay_get_instructions();
+    }
+}
+
+static void notify_aio_contexts(void)
+{
+    /* Wake up other AioContexts.  */
+    qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
+    qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL);
+}
+
+void handle_icount_deadline(void)
+{
+    assert(qemu_in_vcpu_thread());
+    int64_t deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
+                                                  QEMU_TIMER_ATTR_ALL);
+
+    if (deadline == 0) {
+        notify_aio_contexts();
+    }
+}
+
+void prepare_icount_for_run(CPUState *cpu)
+{
+    int insns_left;
+
+    /*
+     * These should always be cleared by process_icount_data after
+     * each vCPU execution. However u16.high can be raised
+     * asynchronously by cpu_exit/cpu_interrupt/tcg_handle_interrupt
+     */
+    g_assert(cpu_neg(cpu)->icount_decr.u16.low == 0);
+    g_assert(cpu->icount_extra == 0);
+
+    cpu->icount_budget = tcg_get_icount_limit();
+    insns_left = MIN(0xffff, cpu->icount_budget);
+    cpu_neg(cpu)->icount_decr.u16.low = insns_left;
+    cpu->icount_extra = cpu->icount_budget - insns_left;
+
+    replay_mutex_lock();
+
+    if (cpu->icount_budget == 0 && replay_has_checkpoint()) {
+        notify_aio_contexts();
+    }
+}
+
+void process_icount_data(CPUState *cpu)
+{
+    /* Account for executed instructions */
+    icount_update(cpu);
+
+    /* Reset the counters */
+    cpu_neg(cpu)->icount_decr.u16.low = 0;
+    cpu->icount_extra = 0;
+    cpu->icount_budget = 0;
+
+    replay_account_executed_instructions();
+
+    replay_mutex_unlock();
+}
+
+static void icount_handle_interrupt(CPUState *cpu, int mask)
+{
+    int old_mask = cpu->interrupt_request;
+
+    tcg_handle_interrupt(cpu, mask);
+    if (qemu_cpu_is_self(cpu) &&
+        !cpu->can_do_io
+        && (mask & ~old_mask) != 0) {
+        cpu_abort(cpu, "Raised interrupt while not in I/O function");
+    }
+}
+
+const CpusAccel tcg_cpus_icount = {
+    .create_vcpu_thread = tcg_start_vcpu_thread,
+    .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
+
+    .handle_interrupt = icount_handle_interrupt,
+    .get_virtual_clock = icount_get,
+    .get_elapsed_ticks = icount_get,
+};
diff --git a/accel/tcg/tcg-cpus-mttcg.c b/accel/tcg/tcg-cpus-mttcg.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/accel/tcg/tcg-cpus-mttcg.c
@@ -XXX,XX +XXX,XX @@
+/*
+ * QEMU TCG Multi Threaded vCPUs implementation
+ *
+ * Copyright (c) 2003-2008 Fabrice Bellard
+ * Copyright (c) 2014 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "sysemu/tcg.h"
+#include "sysemu/replay.h"
+#include "qemu/main-loop.h"
+#include "qemu/guest-random.h"
+#include "exec/exec-all.h"
+#include "hw/boards.h"
+
+#include "tcg-cpus.h"
+#include "tcg-cpus-mttcg.h"
+
+/*
+ * In the multi-threaded case each vCPU has its own thread. The TLS
+ * variable current_cpu can be used deep in the code to find the
+ * current CPUState for a given thread.
+ */
+
+void *tcg_cpu_thread_fn(void *arg)
+{
+    CPUState *cpu = arg;
+
+    assert(tcg_enabled());
+    g_assert(!icount_enabled());
+
+    rcu_register_thread();
+    tcg_register_thread();
+
+    qemu_mutex_lock_iothread();
+    qemu_thread_get_self(cpu->thread);
+
+    cpu->thread_id = qemu_get_thread_id();
+    cpu->can_do_io = 1;
+    current_cpu = cpu;
+    cpu_thread_signal_created(cpu);
+    qemu_guest_random_seed_thread_part2(cpu->random_seed);
+
+    /* process any pending work */
+    cpu->exit_request = 1;
+
+    do {
+        if (cpu_can_run(cpu)) {
+            int r;
+            qemu_mutex_unlock_iothread();
+            r = tcg_cpu_exec(cpu);
+            qemu_mutex_lock_iothread();
+            switch (r) {
+            case EXCP_DEBUG:
+                cpu_handle_guest_debug(cpu);
+                break;
+            case EXCP_HALTED:
+                /*
+                 * during start-up the vCPU is reset and the thread is
+                 * kicked several times. If we don't ensure we go back
+                 * to sleep in the halted state we won't cleanly
+                 * start-up when the vCPU is enabled.
+                 *
+                 * cpu->halted should ensure we sleep in wait_io_event
+                 */
+                g_assert(cpu->halted);
+                break;
+            case EXCP_ATOMIC:
+                qemu_mutex_unlock_iothread();
+                cpu_exec_step_atomic(cpu);
+                qemu_mutex_lock_iothread();
+            default:
+                /* Ignore everything else? */
+                break;
+            }
+        }
+
+        qatomic_mb_set(&cpu->exit_request, 0);
+        qemu_wait_io_event(cpu);
+    } while (!cpu->unplug || cpu_can_run(cpu));
+
+    qemu_tcg_destroy_vcpu(cpu);
+    qemu_mutex_unlock_iothread();
+    rcu_unregister_thread();
+    return NULL;
+}
+
+static void mttcg_kick_vcpu_thread(CPUState *cpu)
+{
+    cpu_exit(cpu);
+}
+
+const CpusAccel tcg_cpus_mttcg = {
+    .create_vcpu_thread = tcg_start_vcpu_thread,
+    .kick_vcpu_thread = mttcg_kick_vcpu_thread,
+
+    .handle_interrupt = tcg_handle_interrupt,
+};
diff --git a/accel/tcg/tcg-cpus-rr.c b/accel/tcg/tcg-cpus-rr.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/accel/tcg/tcg-cpus-rr.c
@@ -XXX,XX +XXX,XX @@
+/*
+ * QEMU TCG Single Threaded vCPUs implementation
+ *
+ * Copyright (c) 2003-2008 Fabrice Bellard
+ * Copyright (c) 2014 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "sysemu/tcg.h"
+#include "sysemu/replay.h"
+#include "qemu/main-loop.h"
+#include "qemu/guest-random.h"
+#include "exec/exec-all.h"
+#include "hw/boards.h"
+
+#include "tcg-cpus.h"
+#include "tcg-cpus-rr.h"
+#include "tcg-cpus-icount.h"
+
+/* Kick all RR vCPUs */
+void qemu_cpu_kick_rr_cpus(CPUState *unused)
+{
+    CPUState *cpu;
+
+    CPU_FOREACH(cpu) {
+        cpu_exit(cpu);
+    };
+}
+
+/*
+ * TCG vCPU kick timer
+ *
+ * The kick timer is responsible for moving single threaded vCPU
+ * emulation on to the next vCPU. If more than one vCPU is running a
+ * timer event with force a cpu->exit so the next vCPU can get
+ * scheduled.
+ *
+ * The timer is removed if all vCPUs are idle and restarted again once
+ * idleness is complete.
+ */
+
+static QEMUTimer *tcg_kick_vcpu_timer;
+static CPUState *tcg_current_rr_cpu;
+
+#define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
+
+static inline int64_t qemu_tcg_next_kick(void)
+{
+    return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + TCG_KICK_PERIOD;
+}
+
+/* Kick the currently round-robin scheduled vCPU to next */
+static void qemu_cpu_kick_rr_next_cpu(void)
+{
+    CPUState *cpu;
+    do {
+        cpu = qatomic_mb_read(&tcg_current_rr_cpu);
+        if (cpu) {
+            cpu_exit(cpu);
+        }
+    } while (cpu != qatomic_mb_read(&tcg_current_rr_cpu));
+}
+
+static void kick_tcg_thread(void *opaque)
+{
+    timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
+    qemu_cpu_kick_rr_next_cpu();
+}
+
+static void start_tcg_kick_timer(void)
+{
+    if (!tcg_kick_vcpu_timer && CPU_NEXT(first_cpu)) {
+        tcg_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
+                                           kick_tcg_thread, NULL);
+    }
+    if (tcg_kick_vcpu_timer && !timer_pending(tcg_kick_vcpu_timer)) {
+        timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
+    }
+}
+
+static void stop_tcg_kick_timer(void)
+{
+    if (tcg_kick_vcpu_timer && timer_pending(tcg_kick_vcpu_timer)) {
+        timer_del(tcg_kick_vcpu_timer);
+    }
+}
+
+static void qemu_tcg_rr_wait_io_event(void)
+{
+    CPUState *cpu;
+
+    while (all_cpu_threads_idle()) {
+        stop_tcg_kick_timer();
+        qemu_cond_wait_iothread(first_cpu->halt_cond);
+    }
+
+    start_tcg_kick_timer();
+
+    CPU_FOREACH(cpu) {
+        qemu_wait_io_event_common(cpu);
+    }
+}
+
+/*
+ * Destroy any remaining vCPUs which have been unplugged and have
+ * finished running
+ */
+static void deal_with_unplugged_cpus(void)
+{
+    CPUState *cpu;
+
+    CPU_FOREACH(cpu) {
+        if (cpu->unplug && !cpu_can_run(cpu)) {
+            qemu_tcg_destroy_vcpu(cpu);
+            break;
+        }
+    }
+}
+
+/*
+ * In the single-threaded case each vCPU is simulated in turn. If
+ * there is more than a single vCPU we create a simple timer to kick
+ * the vCPU and ensure we don't get stuck in a tight loop in one vCPU.
+ * This is done explicitly rather than relying on side-effects
+ * elsewhere.
+ */
+
+void *tcg_rr_cpu_thread_fn(void *arg)
+{
+    CPUState *cpu = arg;
+
+    assert(tcg_enabled());
+    rcu_register_thread();
+    tcg_register_thread();
+
+    qemu_mutex_lock_iothread();
+    qemu_thread_get_self(cpu->thread);
+
+    cpu->thread_id = qemu_get_thread_id();
+    cpu->can_do_io = 1;
+    cpu_thread_signal_created(cpu);
+    qemu_guest_random_seed_thread_part2(cpu->random_seed);
+
+    /* wait for initial kick-off after machine start */
+    while (first_cpu->stopped) {
+        qemu_cond_wait_iothread(first_cpu->halt_cond);
+
+        /* process any pending work */
+        CPU_FOREACH(cpu) {
+            current_cpu = cpu;
+            qemu_wait_io_event_common(cpu);
+        }
+    }
+
+    start_tcg_kick_timer();
+
+    cpu = first_cpu;
+
+    /* process any pending work */
+    cpu->exit_request = 1;
+
+    while (1) {
+        qemu_mutex_unlock_iothread();
+        replay_mutex_lock();
+        qemu_mutex_lock_iothread();
+
+        if (icount_enabled()) {
+            /* Account partial waits to QEMU_CLOCK_VIRTUAL.  */
+            icount_account_warp_timer();
+            /*
+             * Run the timers here.  This is much more efficient than
+             * waking up the I/O thread and waiting for completion.
+             */
+            handle_icount_deadline();
+        }
+
+        replay_mutex_unlock();
+
+        if (!cpu) {
+            cpu = first_cpu;
+        }
+
+        while (cpu && cpu_work_list_empty(cpu) && !cpu->exit_request) {
+
+            qatomic_mb_set(&tcg_current_rr_cpu, cpu);
+            current_cpu = cpu;
+
+            qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
+                              (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
+
+            if (cpu_can_run(cpu)) {
+                int r;
+
+                qemu_mutex_unlock_iothread();
+                if (icount_enabled()) {
+                    prepare_icount_for_run(cpu);
+                }
+                r = tcg_cpu_exec(cpu);
+                if (icount_enabled()) {
+                    process_icount_data(cpu);
+                }
+                qemu_mutex_lock_iothread();
+
+                if (r == EXCP_DEBUG) {
+                    cpu_handle_guest_debug(cpu);
+                    break;
+                } else if (r == EXCP_ATOMIC) {
+                    qemu_mutex_unlock_iothread();
+                    cpu_exec_step_atomic(cpu);
+                    qemu_mutex_lock_iothread();
+                    break;
+                }
+            } else if (cpu->stop) {
+                if (cpu->unplug) {
+                    cpu = CPU_NEXT(cpu);
+                }
+                break;
+            }
+
+            cpu = CPU_NEXT(cpu);
+        } /* while (cpu && !cpu->exit_request).. */
+
+        /* Does not need qatomic_mb_set because a spurious wakeup is okay.  */
+        qatomic_set(&tcg_current_rr_cpu, NULL);
+
+        if (cpu && cpu->exit_request) {
+            qatomic_mb_set(&cpu->exit_request, 0);
+        }
+
+        if (icount_enabled() && all_cpu_threads_idle()) {
+            /*
+             * When all cpus are sleeping (e.g in WFI), to avoid a deadlock
+             * in the main_loop, wake it up in order to start the warp timer.
+             */
+            qemu_notify_event();
+        }
+
+        qemu_tcg_rr_wait_io_event();
+        deal_with_unplugged_cpus();
+    }
+
+    rcu_unregister_thread();
+    return NULL;
+}
+
+const CpusAccel tcg_cpus_rr = {
+    .create_vcpu_thread = tcg_start_vcpu_thread,
+    .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
+
+    .handle_interrupt = tcg_handle_interrupt,
+};
diff --git a/accel/tcg/tcg-cpus.c b/accel/tcg/tcg-cpus.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus.c
+++ b/accel/tcg/tcg-cpus.c
@@ -XXX,XX +XXX,XX @@
 /*
- * QEMU System Emulator
+ * QEMU TCG vCPU common functionality
+ *
+ * Functionality common to all TCG vCPU variants: mttcg, rr and icount.
  *
  * Copyright (c) 2003-2008 Fabrice Bellard
  * Copyright (c) 2014 Red Hat Inc.
@@ -XXX,XX +XXX,XX @@
 #include "hw/boards.h"
 
 #include "tcg-cpus.h"
+#include "tcg-cpus-mttcg.h"
+#include "tcg-cpus-rr.h"
 
-/* Kick all RR vCPUs */
-static void qemu_cpu_kick_rr_cpus(void)
-{
-    CPUState *cpu;
+/* common functionality among all TCG variants */
 
-    CPU_FOREACH(cpu) {
-        cpu_exit(cpu);
-    };
-}
-
-static void tcg_kick_vcpu_thread(CPUState *cpu)
-{
-    if (qemu_tcg_mttcg_enabled()) {
-        cpu_exit(cpu);
-    } else {
-        qemu_cpu_kick_rr_cpus();
-    }
-}
-
-/*
- * TCG vCPU kick timer
- *
- * The kick timer is responsible for moving single threaded vCPU
- * emulation on to the next vCPU. If more than one vCPU is running a
- * timer event with force a cpu->exit so the next vCPU can get
- * scheduled.
- *
- * The timer is removed if all vCPUs are idle and restarted again once
- * idleness is complete.
- */
-
-static QEMUTimer *tcg_kick_vcpu_timer;
-static CPUState *tcg_current_rr_cpu;
-
-#define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
-
-static inline int64_t qemu_tcg_next_kick(void)
-{
-    return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + TCG_KICK_PERIOD;
-}
-
-/* Kick the currently round-robin scheduled vCPU to next */
-static void qemu_cpu_kick_rr_next_cpu(void)
-{
-    CPUState *cpu;
-    do {
-        cpu = qatomic_mb_read(&tcg_current_rr_cpu);
-        if (cpu) {
-            cpu_exit(cpu);
-        }
-    } while (cpu != qatomic_mb_read(&tcg_current_rr_cpu));
-}
-
-static void kick_tcg_thread(void *opaque)
-{
-    timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
-    qemu_cpu_kick_rr_next_cpu();
-}
-
-static void start_tcg_kick_timer(void)
-{
-    assert(!mttcg_enabled);
-    if (!tcg_kick_vcpu_timer && CPU_NEXT(first_cpu)) {
-        tcg_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
-                                           kick_tcg_thread, NULL);
-    }
-    if (tcg_kick_vcpu_timer && !timer_pending(tcg_kick_vcpu_timer)) {
-        timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
-    }
-}
-
-static void stop_tcg_kick_timer(void)
-{
-    assert(!mttcg_enabled);
-    if (tcg_kick_vcpu_timer && timer_pending(tcg_kick_vcpu_timer)) {
-        timer_del(tcg_kick_vcpu_timer);
-    }
-}
-
-static void qemu_tcg_destroy_vcpu(CPUState *cpu)
-{
-}
-
-static void qemu_tcg_rr_wait_io_event(void)
-{
-    CPUState *cpu;
-
-    while (all_cpu_threads_idle()) {
-        stop_tcg_kick_timer();
-        qemu_cond_wait_iothread(first_cpu->halt_cond);
-    }
-
-    start_tcg_kick_timer();
-
-    CPU_FOREACH(cpu) {
-        qemu_wait_io_event_common(cpu);
-    }
-}
-
-static int64_t tcg_get_icount_limit(void)
-{
-    int64_t deadline;
-
-    if (replay_mode != REPLAY_MODE_PLAY) {
-        /*
-         * Include all the timers, because they may need an attention.
-         * Too long CPU execution may create unnecessary delay in UI.
-         */
-        deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
-                                              QEMU_TIMER_ATTR_ALL);
-        /* Check realtime timers, because they help with input processing */
-        deadline = qemu_soonest_timeout(deadline,
-                qemu_clock_deadline_ns_all(QEMU_CLOCK_REALTIME,
-                                           QEMU_TIMER_ATTR_ALL));
-
-        /*
-         * Maintain prior (possibly buggy) behaviour where if no deadline
-         * was set (as there is no QEMU_CLOCK_VIRTUAL timer) or it is more than
-         * INT32_MAX nanoseconds ahead, we still use INT32_MAX
-         * nanoseconds.
-         */
-        if ((deadline < 0) || (deadline > INT32_MAX)) {
-            deadline = INT32_MAX;
-        }
-
-        return icount_round(deadline);
-    } else {
-        return replay_get_instructions();
-    }
-}
-
-static void notify_aio_contexts(void)
-{
-    /* Wake up other AioContexts.  */
-    qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
-    qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL);
-}
-
-static void handle_icount_deadline(void)
-{
-    assert(qemu_in_vcpu_thread());
-    if (icount_enabled()) {
-        int64_t deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
-                                                      QEMU_TIMER_ATTR_ALL);
-
-        if (deadline == 0) {
-            notify_aio_contexts();
-        }
-    }
-}
-
-static void prepare_icount_for_run(CPUState *cpu)
-{
-    if (icount_enabled()) {
-        int insns_left;
-
-        /*
-         * These should always be cleared by process_icount_data after
-         * each vCPU execution. However u16.high can be raised
-         * asynchronously by cpu_exit/cpu_interrupt/tcg_handle_interrupt
-         */
-        g_assert(cpu_neg(cpu)->icount_decr.u16.low == 0);
-        g_assert(cpu->icount_extra == 0);
-
-        cpu->icount_budget = tcg_get_icount_limit();
-        insns_left = MIN(0xffff, cpu->icount_budget);
-        cpu_neg(cpu)->icount_decr.u16.low = insns_left;
-        cpu->icount_extra = cpu->icount_budget - insns_left;
-
-        replay_mutex_lock();
-
-        if (cpu->icount_budget == 0 && replay_has_checkpoint()) {
-            notify_aio_contexts();
-        }
-    }
-}
-
-static void process_icount_data(CPUState *cpu)
-{
-    if (icount_enabled()) {
-        /* Account for executed instructions */
-        icount_update(cpu);
-
-        /* Reset the counters */
-        cpu_neg(cpu)->icount_decr.u16.low = 0;
-        cpu->icount_extra = 0;
-        cpu->icount_budget = 0;
-
-        replay_account_executed_instructions();
-
-        replay_mutex_unlock();
-    }
-}
-
-static int tcg_cpu_exec(CPUState *cpu)
-{
-    int ret;
-#ifdef CONFIG_PROFILER
-    int64_t ti;
-#endif
-
-    assert(tcg_enabled());
-#ifdef CONFIG_PROFILER
-    ti = profile_getclock();
-#endif
-    cpu_exec_start(cpu);
-    ret = cpu_exec(cpu);
-    cpu_exec_end(cpu);
-#ifdef CONFIG_PROFILER
-    qatomic_set(&tcg_ctx->prof.cpu_exec_time,
-                tcg_ctx->prof.cpu_exec_time + profile_getclock() - ti);
-#endif
-    return ret;
-}
-
-/*
- * Destroy any remaining vCPUs which have been unplugged and have
- * finished running
- */
-static void deal_with_unplugged_cpus(void)
-{
-    CPUState *cpu;
-
-    CPU_FOREACH(cpu) {
-        if (cpu->unplug && !cpu_can_run(cpu)) {
-            qemu_tcg_destroy_vcpu(cpu);
-            cpu_thread_signal_destroyed(cpu);
-            break;
-        }
-    }
-}
-
-/*
- * Single-threaded TCG
- *
- * In the single-threaded case each vCPU is simulated in turn. If
- * there is more than a single vCPU we create a simple timer to kick
- * the vCPU and ensure we don't get stuck in a tight loop in one vCPU.
- * This is done explicitly rather than relying on side-effects
- * elsewhere.
- */
-
-static void *tcg_rr_cpu_thread_fn(void *arg)
-{
-    CPUState *cpu = arg;
-
-    assert(tcg_enabled());
-    rcu_register_thread();
-    tcg_register_thread();
-
-    qemu_mutex_lock_iothread();
-    qemu_thread_get_self(cpu->thread);
-
-    cpu->thread_id = qemu_get_thread_id();
-    cpu->can_do_io = 1;
-    cpu_thread_signal_created(cpu);
-    qemu_guest_random_seed_thread_part2(cpu->random_seed);
-
-    /* wait for initial kick-off after machine start */
-    while (first_cpu->stopped) {
-        qemu_cond_wait_iothread(first_cpu->halt_cond);
-
-        /* process any pending work */
-        CPU_FOREACH(cpu) {
-            current_cpu = cpu;
-            qemu_wait_io_event_common(cpu);
-        }
-    }
-
-    start_tcg_kick_timer();
-
-    cpu = first_cpu;
-
-    /* process any pending work */
-    cpu->exit_request = 1;
-
-    while (1) {
-        qemu_mutex_unlock_iothread();
-        replay_mutex_lock();
-        qemu_mutex_lock_iothread();
-        /* Account partial waits to QEMU_CLOCK_VIRTUAL.  */
-        icount_account_warp_timer();
-
-        /*
-         * Run the timers here.  This is much more efficient than
-         * waking up the I/O thread and waiting for completion.
-         */
-        handle_icount_deadline();
-
-        replay_mutex_unlock();
-
-        if (!cpu) {
-            cpu = first_cpu;
-        }
-
-        while (cpu && cpu_work_list_empty(cpu) && !cpu->exit_request) {
-
-            qatomic_mb_set(&tcg_current_rr_cpu, cpu);
-            current_cpu = cpu;
-
-            qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
-                              (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
-
-            if (cpu_can_run(cpu)) {
-                int r;
-
-                qemu_mutex_unlock_iothread();
-                prepare_icount_for_run(cpu);
-
-                r = tcg_cpu_exec(cpu);
-
-                process_icount_data(cpu);
-                qemu_mutex_lock_iothread();
-
-                if (r == EXCP_DEBUG) {
-                    cpu_handle_guest_debug(cpu);
-                    break;
-                } else if (r == EXCP_ATOMIC) {
-                    qemu_mutex_unlock_iothread();
-                    cpu_exec_step_atomic(cpu);
-                    qemu_mutex_lock_iothread();
-                    break;
-                }
-            } else if (cpu->stop) {
-                if (cpu->unplug) {
-                    cpu = CPU_NEXT(cpu);
-                }
-                break;
-            }
-
-            cpu = CPU_NEXT(cpu);
-        } /* while (cpu && !cpu->exit_request).. */
-
-        /* Does not need qatomic_mb_set because a spurious wakeup is okay.  */
-        qatomic_set(&tcg_current_rr_cpu, NULL);
-
-        if (cpu && cpu->exit_request) {
-            qatomic_mb_set(&cpu->exit_request, 0);
-        }
-
-        if (icount_enabled() && all_cpu_threads_idle()) {
-            /*
-             * When all cpus are sleeping (e.g in WFI), to avoid a deadlock
-             * in the main_loop, wake it up in order to start the warp timer.
-             */
-            qemu_notify_event();
-        }
-
-        qemu_tcg_rr_wait_io_event();
-        deal_with_unplugged_cpus();
-    }
-
-    rcu_unregister_thread();
-    return NULL;
-}
-
-/*
- * Multi-threaded TCG
- *
- * In the multi-threaded case each vCPU has its own thread. The TLS
- * variable current_cpu can be used deep in the code to find the
- * current CPUState for a given thread.
- */
-
-static void *tcg_cpu_thread_fn(void *arg)
-{
-    CPUState *cpu = arg;
-
-    assert(tcg_enabled());
-    g_assert(!icount_enabled());
-
-    rcu_register_thread();
-    tcg_register_thread();
-
-    qemu_mutex_lock_iothread();
-    qemu_thread_get_self(cpu->thread);
-
-    cpu->thread_id = qemu_get_thread_id();
-    cpu->can_do_io = 1;
-    current_cpu = cpu;
-    cpu_thread_signal_created(cpu);
-    qemu_guest_random_seed_thread_part2(cpu->random_seed);
-
-    /* process any pending work */
-    cpu->exit_request = 1;
-
-    do {
-        if (cpu_can_run(cpu)) {
-            int r;
-            qemu_mutex_unlock_iothread();
-            r = tcg_cpu_exec(cpu);
-            qemu_mutex_lock_iothread();
-            switch (r) {
-            case EXCP_DEBUG:
-                cpu_handle_guest_debug(cpu);
-                break;
-            case EXCP_HALTED:
-                /*
-                 * during start-up the vCPU is reset and the thread is
-                 * kicked several times. If we don't ensure we go back
-                 * to sleep in the halted state we won't cleanly
-                 * start-up when the vCPU is enabled.
-                 *
-                 * cpu->halted should ensure we sleep in wait_io_event
-                 */
-                g_assert(cpu->halted);
-                break;
-            case EXCP_ATOMIC:
-                qemu_mutex_unlock_iothread();
-                cpu_exec_step_atomic(cpu);
-                qemu_mutex_lock_iothread();
-            default:
-                /* Ignore everything else? */
-                break;
-            }
-        }
-
-        qatomic_mb_set(&cpu->exit_request, 0);
-        qemu_wait_io_event(cpu);
-    } while (!cpu->unplug || cpu_can_run(cpu));
-
-    qemu_tcg_destroy_vcpu(cpu);
-    cpu_thread_signal_destroyed(cpu);
-    qemu_mutex_unlock_iothread();
-    rcu_unregister_thread();
-    return NULL;
-}
-
-static void tcg_start_vcpu_thread(CPUState *cpu)
+void tcg_start_vcpu_thread(CPUState *cpu)
 {
     char thread_name[VCPU_THREAD_NAME_SIZE];
     static QemuCond *single_tcg_halt_cond;
@@ -XXX,XX +XXX,XX @@ static void tcg_start_vcpu_thread(CPUState *cpu)
     }
 }
 
-static int64_t tcg_get_virtual_clock(void)
+void qemu_tcg_destroy_vcpu(CPUState *cpu)
 {
-    if (icount_enabled()) {
-        return icount_get();
-    }
-    return cpu_get_clock();
+    cpu_thread_signal_destroyed(cpu);
 }
 
-static int64_t tcg_get_elapsed_ticks(void)
+int tcg_cpu_exec(CPUState *cpu)
 {
-    if (icount_enabled()) {
-        return icount_get();
-    }
-    return cpu_get_ticks();
+    int ret;
+#ifdef CONFIG_PROFILER
+    int64_t ti;
+#endif
+    assert(tcg_enabled());
+#ifdef CONFIG_PROFILER
+    ti = profile_getclock();
+#endif
+    cpu_exec_start(cpu);
+    ret = cpu_exec(cpu);
+    cpu_exec_end(cpu);
+#ifdef CONFIG_PROFILER
+    qatomic_set(&tcg_ctx->prof.cpu_exec_time,
+                tcg_ctx->prof.cpu_exec_time + profile_getclock() - ti);
+#endif
+    return ret;
 }
 
 /* mask must never be zero, except for A20 change call */
-static void tcg_handle_interrupt(CPUState *cpu, int mask)
+void tcg_handle_interrupt(CPUState *cpu, int mask)
 {
-    int old_mask;
     g_assert(qemu_mutex_iothread_locked());
 
-    old_mask = cpu->interrupt_request;
     cpu->interrupt_request |= mask;
 
     /*
@@ -XXX,XX +XXX,XX @@ static void tcg_handle_interrupt(CPUState *cpu, int mask)
         qemu_cpu_kick(cpu);
     } else {
         qatomic_set(&cpu_neg(cpu)->icount_decr.u16.high, -1);
-        if (icount_enabled() &&
-            !cpu->can_do_io
-            && (mask & ~old_mask) != 0) {
-            cpu_abort(cpu, "Raised interrupt while not in I/O function");
-        }
     }
 }
-
-const CpusAccel tcg_cpus = {
-    .create_vcpu_thread = tcg_start_vcpu_thread,
-    .kick_vcpu_thread = tcg_kick_vcpu_thread,
-
-    .handle_interrupt = tcg_handle_interrupt,
-
-    .get_virtual_clock = tcg_get_virtual_clock,
-    .get_elapsed_ticks = tcg_get_elapsed_ticks,
-};
diff --git a/softmmu/icount.c b/softmmu/icount.c
index XXXXXXX..XXXXXXX 100644
--- a/softmmu/icount.c
+++ b/softmmu/icount.c
@@ -XXX,XX +XXX,XX @@ void icount_start_warp_timer(void)
 
 void icount_account_warp_timer(void)
 {
-    if (!icount_enabled() || !icount_sleep) {
+    if (!icount_sleep) {
         return;
     }
 
diff --git a/accel/tcg/meson.build b/accel/tcg/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/meson.build
+++ b/accel/tcg/meson.build
@@ -XXX,XX +XXX,XX @@ tcg_ss.add(when: 'CONFIG_SOFTMMU', if_false: files('user-exec-stub.c'))
 tcg_ss.add(when: 'CONFIG_PLUGIN', if_true: [files('plugin-gen.c'), libdl])
 specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 
-specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files('tcg-all.c', 'cputlb.c', 'tcg-cpus.c'))
+specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
+  'tcg-all.c',
+  'cputlb.c',
+  'tcg-cpus.c',
+  'tcg-cpus-mttcg.c',
+  'tcg-cpus-icount.c',
+  'tcg-cpus-rr.c'
+))
-- 
2.25.1

From: Claudio Fontana <cfontana@suse.de>

after the initial split into 3 tcg variants, we proceed to also
split tcg_start_vcpu_thread.

We actually split it in 2 this time, since the icount variant
just uses the round robin function.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Claudio Fontana <cfontana@suse.de>
Message-Id: <20201015143217.29337-3-cfontana@suse.de>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/tcg-cpus-mttcg.h  | 21 --------------
 accel/tcg/tcg-cpus-rr.h     |  3 +-
 accel/tcg/tcg-cpus.h        |  1 -
 accel/tcg/tcg-all.c         |  5 ++++
 accel/tcg/tcg-cpus-icount.c |  2 +-
 accel/tcg/tcg-cpus-mttcg.c  | 29 +++++++++++++++++--
 accel/tcg/tcg-cpus-rr.c     | 39 +++++++++++++++++++++++--
 accel/tcg/tcg-cpus.c        | 58 -------------------------------------
 8 files changed, 71 insertions(+), 87 deletions(-)
 delete mode 100644 accel/tcg/tcg-cpus-mttcg.h

diff --git a/accel/tcg/tcg-cpus-mttcg.h b/accel/tcg/tcg-cpus-mttcg.h
deleted file mode 100644
index XXXXXXX..XXXXXXX
--- a/accel/tcg/tcg-cpus-mttcg.h
+++ /dev/null
@@ -XXX,XX +XXX,XX @@
-/*
- * QEMU TCG Multi Threaded vCPUs implementation
- *
- * Copyright 2020 SUSE LLC
- *
- * This work is licensed under the terms of the GNU GPL, version 2 or later.
- * See the COPYING file in the top-level directory.
- */
-
-#ifndef TCG_CPUS_MTTCG_H
-#define TCG_CPUS_MTTCG_H
-
-/*
- * In the multi-threaded case each vCPU has its own thread. The TLS
- * variable current_cpu can be used deep in the code to find the
- * current CPUState for a given thread.
- */
-
-void *tcg_cpu_thread_fn(void *arg);
-
-#endif /* TCG_CPUS_MTTCG_H */
diff --git a/accel/tcg/tcg-cpus-rr.h b/accel/tcg/tcg-cpus-rr.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus-rr.h
+++ b/accel/tcg/tcg-cpus-rr.h
@@ -XXX,XX +XXX,XX @@
 /* Kick all RR vCPUs. */
 void qemu_cpu_kick_rr_cpus(CPUState *unused);
 
-void *tcg_rr_cpu_thread_fn(void *arg);
+/* start the round robin vcpu thread */
+void rr_start_vcpu_thread(CPUState *cpu);
 
 #endif /* TCG_CPUS_RR_H */
diff --git a/accel/tcg/tcg-cpus.h b/accel/tcg/tcg-cpus.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus.h
+++ b/accel/tcg/tcg-cpus.h
@@ -XXX,XX +XXX,XX @@ extern const CpusAccel tcg_cpus_mttcg;
 extern const CpusAccel tcg_cpus_icount;
 extern const CpusAccel tcg_cpus_rr;
 
-void tcg_start_vcpu_thread(CPUState *cpu);
 void qemu_tcg_destroy_vcpu(CPUState *cpu);
 int tcg_cpu_exec(CPUState *cpu);
 void tcg_handle_interrupt(CPUState *cpu, int mask);
diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -XXX,XX +XXX,XX @@ static int tcg_init(MachineState *ms)
     tcg_exec_init(s->tb_size * 1024 * 1024);
     mttcg_enabled = s->mttcg_enabled;
 
+    /*
+     * Initialize TCG regions
+     */
+    tcg_region_init();
+
     if (mttcg_enabled) {
         cpus_register_accel(&tcg_cpus_mttcg);
     } else if (icount_enabled()) {
diff --git a/accel/tcg/tcg-cpus-icount.c b/accel/tcg/tcg-cpus-icount.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus-icount.c
+++ b/accel/tcg/tcg-cpus-icount.c
@@ -XXX,XX +XXX,XX @@ static void icount_handle_interrupt(CPUState *cpu, int mask)
 }
 
 const CpusAccel tcg_cpus_icount = {
-    .create_vcpu_thread = tcg_start_vcpu_thread,
+    .create_vcpu_thread = rr_start_vcpu_thread,
     .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
 
     .handle_interrupt = icount_handle_interrupt,
diff --git a/accel/tcg/tcg-cpus-mttcg.c b/accel/tcg/tcg-cpus-mttcg.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus-mttcg.c
+++ b/accel/tcg/tcg-cpus-mttcg.c
@@ -XXX,XX +XXX,XX @@
 #include "hw/boards.h"
 
 #include "tcg-cpus.h"
-#include "tcg-cpus-mttcg.h"
 
 /*
  * In the multi-threaded case each vCPU has its own thread. The TLS
@@ -XXX,XX +XXX,XX @@
  * current CPUState for a given thread.
  */
 
-void *tcg_cpu_thread_fn(void *arg)
+static void *tcg_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
 
@@ -XXX,XX +XXX,XX @@ static void mttcg_kick_vcpu_thread(CPUState *cpu)
     cpu_exit(cpu);
 }
 
+static void mttcg_start_vcpu_thread(CPUState *cpu)
+{
+    char thread_name[VCPU_THREAD_NAME_SIZE];
+
+    g_assert(tcg_enabled());
+
+    parallel_cpus = (current_machine->smp.max_cpus > 1);
+
+    cpu->thread = g_malloc0(sizeof(QemuThread));
+    cpu->halt_cond = g_malloc0(sizeof(QemuCond));
+    qemu_cond_init(cpu->halt_cond);
+
+    /* create a thread per vCPU with TCG (MTTCG) */
+    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG",
+             cpu->cpu_index);
+
+    qemu_thread_create(cpu->thread, thread_name, tcg_cpu_thread_fn,
+                       cpu, QEMU_THREAD_JOINABLE);
+
+#ifdef _WIN32
+    cpu->hThread = qemu_thread_get_handle(cpu->thread);
+#endif
+}
+
 const CpusAccel tcg_cpus_mttcg = {
-    .create_vcpu_thread = tcg_start_vcpu_thread,
+    .create_vcpu_thread = mttcg_start_vcpu_thread,
     .kick_vcpu_thread = mttcg_kick_vcpu_thread,
 
     .handle_interrupt = tcg_handle_interrupt,
diff --git a/accel/tcg/tcg-cpus-rr.c b/accel/tcg/tcg-cpus-rr.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus-rr.c
+++ b/accel/tcg/tcg-cpus-rr.c
@@ -XXX,XX +XXX,XX @@ static void deal_with_unplugged_cpus(void)
  * elsewhere.
  */
 
-void *tcg_rr_cpu_thread_fn(void *arg)
+static void *tcg_rr_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
 
@@ -XXX,XX +XXX,XX @@ void *tcg_rr_cpu_thread_fn(void *arg)
     return NULL;
 }
 
+void rr_start_vcpu_thread(CPUState *cpu)
+{
+    char thread_name[VCPU_THREAD_NAME_SIZE];
+    static QemuCond *single_tcg_halt_cond;
+    static QemuThread *single_tcg_cpu_thread;
+
+    g_assert(tcg_enabled());
+    parallel_cpus = false;
+
+    if (!single_tcg_cpu_thread) {
+        cpu->thread = g_malloc0(sizeof(QemuThread));
+        cpu->halt_cond = g_malloc0(sizeof(QemuCond));
+        qemu_cond_init(cpu->halt_cond);
+
+        /* share a single thread for all cpus with TCG */
+        snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "ALL CPUs/TCG");
+        qemu_thread_create(cpu->thread, thread_name,
+                           tcg_rr_cpu_thread_fn,
+                           cpu, QEMU_THREAD_JOINABLE);
+
+        single_tcg_halt_cond = cpu->halt_cond;
+        single_tcg_cpu_thread = cpu->thread;
+#ifdef _WIN32
+        cpu->hThread = qemu_thread_get_handle(cpu->thread);
+#endif
+    } else {
+        /* we share the thread */
+        cpu->thread = single_tcg_cpu_thread;
+        cpu->halt_cond = single_tcg_halt_cond;
+        cpu->thread_id = first_cpu->thread_id;
+        cpu->can_do_io = 1;
+        cpu->created = true;
+    }
+}
+
 const CpusAccel tcg_cpus_rr = {
-    .create_vcpu_thread = tcg_start_vcpu_thread,
+    .create_vcpu_thread = rr_start_vcpu_thread,
     .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
 
     .handle_interrupt = tcg_handle_interrupt,
diff --git a/accel/tcg/tcg-cpus.c b/accel/tcg/tcg-cpus.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus.c
+++ b/accel/tcg/tcg-cpus.c
@@ -XXX,XX +XXX,XX @@
 #include "hw/boards.h"
 
 #include "tcg-cpus.h"
-#include "tcg-cpus-mttcg.h"
-#include "tcg-cpus-rr.h"
 
 /* common functionality among all TCG variants */
 
-void tcg_start_vcpu_thread(CPUState *cpu)
-{
-    char thread_name[VCPU_THREAD_NAME_SIZE];
-    static QemuCond *single_tcg_halt_cond;
-    static QemuThread *single_tcg_cpu_thread;
-    static int tcg_region_inited;
-
-    assert(tcg_enabled());
-    /*
-     * Initialize TCG regions--once. Now is a good time, because:
-     * (1) TCG's init context, prologue and target globals have been set up.
-     * (2) qemu_tcg_mttcg_enabled() works now (TCG init code runs before the
-     *     -accel flag is processed, so the check doesn't work then).
-     */
-    if (!tcg_region_inited) {
-        tcg_region_inited = 1;
-        tcg_region_init();
-        parallel_cpus = qemu_tcg_mttcg_enabled() && current_machine->smp.max_cpus > 1;
-    }
-
-    if (qemu_tcg_mttcg_enabled() || !single_tcg_cpu_thread) {
-        cpu->thread = g_malloc0(sizeof(QemuThread));
-        cpu->halt_cond = g_malloc0(sizeof(QemuCond));
-        qemu_cond_init(cpu->halt_cond);
-
-        if (qemu_tcg_mttcg_enabled()) {
-            /* create a thread per vCPU with TCG (MTTCG) */
-            snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG",
-                 cpu->cpu_index);
-
-            qemu_thread_create(cpu->thread, thread_name, tcg_cpu_thread_fn,
-                               cpu, QEMU_THREAD_JOINABLE);
-
-        } else {
-            /* share a single thread for all cpus with TCG */
-            snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "ALL CPUs/TCG");
-            qemu_thread_create(cpu->thread, thread_name,
-                               tcg_rr_cpu_thread_fn,
-                               cpu, QEMU_THREAD_JOINABLE);
-
-            single_tcg_halt_cond = cpu->halt_cond;
-            single_tcg_cpu_thread = cpu->thread;
-        }
-#ifdef _WIN32
-        cpu->hThread = qemu_thread_get_handle(cpu->thread);
-#endif
-    } else {
-        /* For non-MTTCG cases we share the thread */
-        cpu->thread = single_tcg_cpu_thread;
-        cpu->halt_cond = single_tcg_halt_cond;
-        cpu->thread_id = first_cpu->thread_id;
-        cpu->can_do_io = 1;
-        cpu->created = true;
-    }
-}
-
 void qemu_tcg_destroy_vcpu(CPUState *cpu)
 {
     cpu_thread_signal_destroyed(cpu);
-- 
2.25.1

From: Claudio Fontana <cfontana@suse.de>

Signed-off-by: Claudio Fontana <cfontana@suse.de>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201015143217.29337-4-cfontana@suse.de>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/tcg-cpus-icount.h |  6 +--
 accel/tcg/tcg-cpus-rr.h     |  2 +-
 accel/tcg/tcg-cpus.h        |  6 +--
 accel/tcg/tcg-cpus-icount.c | 24 ++++++------
 accel/tcg/tcg-cpus-mttcg.c  | 10 ++---
 accel/tcg/tcg-cpus-rr.c     | 74 ++++++++++++++++++-------------------
 accel/tcg/tcg-cpus.c        |  6 +--
 7 files changed, 64 insertions(+), 64 deletions(-)

diff --git a/accel/tcg/tcg-cpus-icount.h b/accel/tcg/tcg-cpus-icount.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus-icount.h
+++ b/accel/tcg/tcg-cpus-icount.h
@@ -XXX,XX +XXX,XX @@
 #ifndef TCG_CPUS_ICOUNT_H
 #define TCG_CPUS_ICOUNT_H
 
-void handle_icount_deadline(void);
-void prepare_icount_for_run(CPUState *cpu);
-void process_icount_data(CPUState *cpu);
+void icount_handle_deadline(void);
+void icount_prepare_for_run(CPUState *cpu);
+void icount_process_data(CPUState *cpu);
 
 #endif /* TCG_CPUS_ICOUNT_H */
diff --git a/accel/tcg/tcg-cpus-rr.h b/accel/tcg/tcg-cpus-rr.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus-rr.h
+++ b/accel/tcg/tcg-cpus-rr.h
@@ -XXX,XX +XXX,XX @@
 #define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
 
 /* Kick all RR vCPUs. */
-void qemu_cpu_kick_rr_cpus(CPUState *unused);
+void rr_kick_vcpu_thread(CPUState *unused);
 
 /* start the round robin vcpu thread */
 void rr_start_vcpu_thread(CPUState *cpu);
diff --git a/accel/tcg/tcg-cpus.h b/accel/tcg/tcg-cpus.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus.h
+++ b/accel/tcg/tcg-cpus.h
@@ -XXX,XX +XXX,XX @@ extern const CpusAccel tcg_cpus_mttcg;
 extern const CpusAccel tcg_cpus_icount;
 extern const CpusAccel tcg_cpus_rr;
 
-void qemu_tcg_destroy_vcpu(CPUState *cpu);
-int tcg_cpu_exec(CPUState *cpu);
-void tcg_handle_interrupt(CPUState *cpu, int mask);
+void tcg_cpus_destroy(CPUState *cpu);
+int tcg_cpus_exec(CPUState *cpu);
+void tcg_cpus_handle_interrupt(CPUState *cpu, int mask);
 
 #endif /* TCG_CPUS_H */
diff --git a/accel/tcg/tcg-cpus-icount.c b/accel/tcg/tcg-cpus-icount.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus-icount.c
+++ b/accel/tcg/tcg-cpus-icount.c
@@ -XXX,XX +XXX,XX @@
 #include "tcg-cpus-icount.h"
 #include "tcg-cpus-rr.h"
 
-static int64_t tcg_get_icount_limit(void)
+static int64_t icount_get_limit(void)
 {
     int64_t deadline;
 
@@ -XXX,XX +XXX,XX @@ static int64_t tcg_get_icount_limit(void)
     }
 }
 
-static void notify_aio_contexts(void)
+static void icount_notify_aio_contexts(void)
 {
     /* Wake up other AioContexts.  */
     qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
     qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL);
 }
 
-void handle_icount_deadline(void)
+void icount_handle_deadline(void)
 {
     assert(qemu_in_vcpu_thread());
     int64_t deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL,
                                                   QEMU_TIMER_ATTR_ALL);
 
     if (deadline == 0) {
-        notify_aio_contexts();
+        icount_notify_aio_contexts();
     }
 }
 
-void prepare_icount_for_run(CPUState *cpu)
+void icount_prepare_for_run(CPUState *cpu)
 {
     int insns_left;
 
     /*
-     * These should always be cleared by process_icount_data after
+     * These should always be cleared by icount_process_data after
      * each vCPU execution. However u16.high can be raised
-     * asynchronously by cpu_exit/cpu_interrupt/tcg_handle_interrupt
+     * asynchronously by cpu_exit/cpu_interrupt/tcg_cpus_handle_interrupt
      */
     g_assert(cpu_neg(cpu)->icount_decr.u16.low == 0);
     g_assert(cpu->icount_extra == 0);
 
-    cpu->icount_budget = tcg_get_icount_limit();
+    cpu->icount_budget = icount_get_limit();
     insns_left = MIN(0xffff, cpu->icount_budget);
     cpu_neg(cpu)->icount_decr.u16.low = insns_left;
     cpu->icount_extra = cpu->icount_budget - insns_left;
@@ -XXX,XX +XXX,XX @@ void prepare_icount_for_run(CPUState *cpu)
     replay_mutex_lock();
 
     if (cpu->icount_budget == 0 && replay_has_checkpoint()) {
-        notify_aio_contexts();
+        icount_notify_aio_contexts();
     }
 }
 
-void process_icount_data(CPUState *cpu)
+void icount_process_data(CPUState *cpu)
 {
     /* Account for executed instructions */
     icount_update(cpu);
@@ -XXX,XX +XXX,XX @@ static void icount_handle_interrupt(CPUState *cpu, int mask)
 {
     int old_mask = cpu->interrupt_request;
 
-    tcg_handle_interrupt(cpu, mask);
+    tcg_cpus_handle_interrupt(cpu, mask);
     if (qemu_cpu_is_self(cpu) &&
         !cpu->can_do_io
         && (mask & ~old_mask) != 0) {
@@ -XXX,XX +XXX,XX @@ static void icount_handle_interrupt(CPUState *cpu, int mask)
 
 const CpusAccel tcg_cpus_icount = {
     .create_vcpu_thread = rr_start_vcpu_thread,
-    .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
+    .kick_vcpu_thread = rr_kick_vcpu_thread,
 
     .handle_interrupt = icount_handle_interrupt,
     .get_virtual_clock = icount_get,
diff --git a/accel/tcg/tcg-cpus-mttcg.c b/accel/tcg/tcg-cpus-mttcg.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus-mttcg.c
+++ b/accel/tcg/tcg-cpus-mttcg.c
@@ -XXX,XX +XXX,XX @@
  * current CPUState for a given thread.
  */
 
-static void *tcg_cpu_thread_fn(void *arg)
+static void *mttcg_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
 
@@ -XXX,XX +XXX,XX @@ static void *tcg_cpu_thread_fn(void *arg)
         if (cpu_can_run(cpu)) {
             int r;
             qemu_mutex_unlock_iothread();
-            r = tcg_cpu_exec(cpu);
+            r = tcg_cpus_exec(cpu);
             qemu_mutex_lock_iothread();
             switch (r) {
             case EXCP_DEBUG:
@@ -XXX,XX +XXX,XX @@ static void *tcg_cpu_thread_fn(void *arg)
         qemu_wait_io_event(cpu);
     } while (!cpu->unplug || cpu_can_run(cpu));
 
-    qemu_tcg_destroy_vcpu(cpu);
+    tcg_cpus_destroy(cpu);
     qemu_mutex_unlock_iothread();
     rcu_unregister_thread();
     return NULL;
@@ -XXX,XX +XXX,XX @@ static void mttcg_start_vcpu_thread(CPUState *cpu)
     snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG",
              cpu->cpu_index);
 
-    qemu_thread_create(cpu->thread, thread_name, tcg_cpu_thread_fn,
+    qemu_thread_create(cpu->thread, thread_name, mttcg_cpu_thread_fn,
                        cpu, QEMU_THREAD_JOINABLE);
 
 #ifdef _WIN32
@@ -XXX,XX +XXX,XX @@ const CpusAccel tcg_cpus_mttcg = {
     .create_vcpu_thread = mttcg_start_vcpu_thread,
     .kick_vcpu_thread = mttcg_kick_vcpu_thread,
 
-    .handle_interrupt = tcg_handle_interrupt,
+    .handle_interrupt = tcg_cpus_handle_interrupt,
 };
diff --git a/accel/tcg/tcg-cpus-rr.c b/accel/tcg/tcg-cpus-rr.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus-rr.c
+++ b/accel/tcg/tcg-cpus-rr.c
@@ -XXX,XX +XXX,XX @@
 #include "tcg-cpus-icount.h"
 
 /* Kick all RR vCPUs */
-void qemu_cpu_kick_rr_cpus(CPUState *unused)
+void rr_kick_vcpu_thread(CPUState *unused)
 {
     CPUState *cpu;
 
@@ -XXX,XX +XXX,XX @@ void qemu_cpu_kick_rr_cpus(CPUState *unused)
  * idleness is complete.
  */
 
-static QEMUTimer *tcg_kick_vcpu_timer;
-static CPUState *tcg_current_rr_cpu;
+static QEMUTimer *rr_kick_vcpu_timer;
+static CPUState *rr_current_cpu;
 
 #define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10)
 
-static inline int64_t qemu_tcg_next_kick(void)
+static inline int64_t rr_next_kick_time(void)
 {
     return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + TCG_KICK_PERIOD;
 }
 
 /* Kick the currently round-robin scheduled vCPU to next */
-static void qemu_cpu_kick_rr_next_cpu(void)
+static void rr_kick_next_cpu(void)
 {
     CPUState *cpu;
     do {
-        cpu = qatomic_mb_read(&tcg_current_rr_cpu);
+        cpu = qatomic_mb_read(&rr_current_cpu);
         if (cpu) {
             cpu_exit(cpu);
         }
-    } while (cpu != qatomic_mb_read(&tcg_current_rr_cpu));
+    } while (cpu != qatomic_mb_read(&rr_current_cpu));
 }
 
-static void kick_tcg_thread(void *opaque)
+static void rr_kick_thread(void *opaque)
 {
-    timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
-    qemu_cpu_kick_rr_next_cpu();
+    timer_mod(rr_kick_vcpu_timer, rr_next_kick_time());
+    rr_kick_next_cpu();
 }
 
-static void start_tcg_kick_timer(void)
+static void rr_start_kick_timer(void)
 {
-    if (!tcg_kick_vcpu_timer && CPU_NEXT(first_cpu)) {
-        tcg_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
-                                           kick_tcg_thread, NULL);
+    if (!rr_kick_vcpu_timer && CPU_NEXT(first_cpu)) {
+        rr_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL,
+                                           rr_kick_thread, NULL);
     }
-    if (tcg_kick_vcpu_timer && !timer_pending(tcg_kick_vcpu_timer)) {
-        timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick());
+    if (rr_kick_vcpu_timer && !timer_pending(rr_kick_vcpu_timer)) {
+        timer_mod(rr_kick_vcpu_timer, rr_next_kick_time());
     }
 }
 
-static void stop_tcg_kick_timer(void)
+static void rr_stop_kick_timer(void)
 {
-    if (tcg_kick_vcpu_timer && timer_pending(tcg_kick_vcpu_timer)) {
-        timer_del(tcg_kick_vcpu_timer);
+    if (rr_kick_vcpu_timer && timer_pending(rr_kick_vcpu_timer)) {
+        timer_del(rr_kick_vcpu_timer);
     }
 }
 
-static void qemu_tcg_rr_wait_io_event(void)
+static void rr_wait_io_event(void)
 {
     CPUState *cpu;
 
     while (all_cpu_threads_idle()) {
-        stop_tcg_kick_timer();
+        rr_stop_kick_timer();
         qemu_cond_wait_iothread(first_cpu->halt_cond);
     }
 
-    start_tcg_kick_timer();
+    rr_start_kick_timer();
 
     CPU_FOREACH(cpu) {
         qemu_wait_io_event_common(cpu);
@@ -XXX,XX +XXX,XX @@ static void qemu_tcg_rr_wait_io_event(void)
  * Destroy any remaining vCPUs which have been unplugged and have
  * finished running
  */
-static void deal_with_unplugged_cpus(void)
+static void rr_deal_with_unplugged_cpus(void)
 {
     CPUState *cpu;
 
     CPU_FOREACH(cpu) {
         if (cpu->unplug && !cpu_can_run(cpu)) {
-            qemu_tcg_destroy_vcpu(cpu);
+            tcg_cpus_destroy(cpu);
             break;
         }
     }
@@ -XXX,XX +XXX,XX @@ static void deal_with_unplugged_cpus(void)
  * elsewhere.
  */
 
-static void *tcg_rr_cpu_thread_fn(void *arg)
+static void *rr_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
 
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
         }
     }
 
-    start_tcg_kick_timer();
+    rr_start_kick_timer();
 
     cpu = first_cpu;
 
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
              * Run the timers here.  This is much more efficient than
              * waking up the I/O thread and waiting for completion.
              */
-            handle_icount_deadline();
+            icount_handle_deadline();
         }
 
         replay_mutex_unlock();
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
 
         while (cpu && cpu_work_list_empty(cpu) && !cpu->exit_request) {
 
-            qatomic_mb_set(&tcg_current_rr_cpu, cpu);
+            qatomic_mb_set(&rr_current_cpu, cpu);
             current_cpu = cpu;
 
             qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
 
                 qemu_mutex_unlock_iothread();
                 if (icount_enabled()) {
-                    prepare_icount_for_run(cpu);
+                    icount_prepare_for_run(cpu);
                 }
-                r = tcg_cpu_exec(cpu);
+                r = tcg_cpus_exec(cpu);
                 if (icount_enabled()) {
-                    process_icount_data(cpu);
+                    icount_process_data(cpu);
                 }
                 qemu_mutex_lock_iothread();
 
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
         } /* while (cpu && !cpu->exit_request).. */
 
         /* Does not need qatomic_mb_set because a spurious wakeup is okay.  */
-        qatomic_set(&tcg_current_rr_cpu, NULL);
+        qatomic_set(&rr_current_cpu, NULL);
 
         if (cpu && cpu->exit_request) {
             qatomic_mb_set(&cpu->exit_request, 0);
@@ -XXX,XX +XXX,XX @@ static void *tcg_rr_cpu_thread_fn(void *arg)
             qemu_notify_event();
         }
 
-        qemu_tcg_rr_wait_io_event();
-        deal_with_unplugged_cpus();
+        rr_wait_io_event();
+        rr_deal_with_unplugged_cpus();
     }
 
     rcu_unregister_thread();
@@ -XXX,XX +XXX,XX @@ void rr_start_vcpu_thread(CPUState *cpu)
         /* share a single thread for all cpus with TCG */
         snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "ALL CPUs/TCG");
         qemu_thread_create(cpu->thread, thread_name,
-                           tcg_rr_cpu_thread_fn,
+                           rr_cpu_thread_fn,
                            cpu, QEMU_THREAD_JOINABLE);
 
         single_tcg_halt_cond = cpu->halt_cond;
@@ -XXX,XX +XXX,XX @@ void rr_start_vcpu_thread(CPUState *cpu)
 
 const CpusAccel tcg_cpus_rr = {
     .create_vcpu_thread = rr_start_vcpu_thread,
-    .kick_vcpu_thread = qemu_cpu_kick_rr_cpus,
+    .kick_vcpu_thread = rr_kick_vcpu_thread,
 
-    .handle_interrupt = tcg_handle_interrupt,
+    .handle_interrupt = tcg_cpus_handle_interrupt,
 };
diff --git a/accel/tcg/tcg-cpus.c b/accel/tcg/tcg-cpus.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-cpus.c
+++ b/accel/tcg/tcg-cpus.c
@@ -XXX,XX +XXX,XX @@
 
 /* common functionality among all TCG variants */
 
-void qemu_tcg_destroy_vcpu(CPUState *cpu)
+void tcg_cpus_destroy(CPUState *cpu)
 {
     cpu_thread_signal_destroyed(cpu);
 }
 
-int tcg_cpu_exec(CPUState *cpu)
+int tcg_cpus_exec(CPUState *cpu)
 {
     int ret;
 #ifdef CONFIG_PROFILER
@@ -XXX,XX +XXX,XX @@ int tcg_cpu_exec(CPUState *cpu)
 }
 
 /* mask must never be zero, except for A20 change call */
-void tcg_handle_interrupt(CPUState *cpu, int mask)
+void tcg_cpus_handle_interrupt(CPUState *cpu, int mask)
 {
     g_assert(qemu_mutex_iothread_locked());
 
-- 
2.25.1

This is mostly my code_gen_buffer cleanup, plus a few other random
changes thrown in.  Including a fix for a recent float32_exp2 bug.

The following changes since commit 894fc4fd670aaf04a67dc7507739f914ff4bacf2:

Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2021-06-11 09:21:48 +0100)

are available in the Git repository at:

https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20210611

for you to fetch changes up to 60afaddc208d34f6dc86dd974f6e02724fba6eb6:

docs/devel: Explain in more detail the TB chaining mechanisms (2021-06-11 09:41:25 -0700)

----------------------------------------------------------------
Clean up code_gen_buffer allocation.
Add tcg_remove_ops_after.
Fix tcg_constant_* documentation.
Improve TB chaining documentation.
Fix float32_exp2.

----------------------------------------------------------------
Jose R. Ziviani (1):
      tcg/arm: Fix tcg_out_op function signature

Luis Pires (1):
      docs/devel: Explain in more detail the TB chaining mechanisms

Richard Henderson (32):
      meson: Split out tcg/meson.build
      meson: Split out fpu/meson.build
      tcg: Re-order tcg_region_init vs tcg_prologue_init
      tcg: Remove error return from tcg_region_initial_alloc__locked
      tcg: Split out tcg_region_initial_alloc
      tcg: Split out tcg_region_prologue_set
      tcg: Split out region.c
      accel/tcg: Inline cpu_gen_init
      accel/tcg: Move alloc_code_gen_buffer to tcg/region.c
      accel/tcg: Rename tcg_init to tcg_init_machine
      tcg: Create tcg_init
      accel/tcg: Merge tcg_exec_init into tcg_init_machine
      accel/tcg: Use MiB in tcg_init_machine
      accel/tcg: Pass down max_cpus to tcg_init
      tcg: Introduce tcg_max_ctxs
      tcg: Move MAX_CODE_GEN_BUFFER_SIZE to tcg-target.h
      tcg: Replace region.end with region.total_size
      tcg: Rename region.start to region.after_prologue
      tcg: Tidy tcg_n_regions
      tcg: Tidy split_cross_256mb
      tcg: Move in_code_gen_buffer and tests to region.c
      tcg: Allocate code_gen_buffer into struct tcg_region_state
      tcg: Return the map protection from alloc_code_gen_buffer
      tcg: Sink qemu_madvise call to common code
      util/osdep: Add qemu_mprotect_rw
      tcg: Round the tb_size default from qemu_get_host_physmem
      tcg: Merge buffer protection and guard page protection
      tcg: When allocating for !splitwx, begin with PROT_NONE
      tcg: Move tcg_init_ctx and tcg_ctx from accel/tcg/
      tcg: Introduce tcg_remove_ops_after
      tcg: Fix documentation for tcg_constant_* vs tcg_temp_free_*
      softfloat: Fix tp init in float32_exp2

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 meson.build     |  8 +-------
 tcg/meson.build | 13 +++++++++++++
 2 files changed, 14 insertions(+), 7 deletions(-)
 create mode 100644 tcg/meson.build

diff --git a/meson.build b/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/meson.build
+++ b/meson.build
@@ -XXX,XX +XXX,XX @@ common_ss.add(capstone)
 specific_ss.add(files('cpu.c', 'disas.c', 'gdbstub.c'), capstone)
 specific_ss.add(when: 'CONFIG_TCG', if_true: files(
   'fpu/softfloat.c',
-  'tcg/optimize.c',
-  'tcg/tcg-common.c',
-  'tcg/tcg-op-gvec.c',
-  'tcg/tcg-op-vec.c',
-  'tcg/tcg-op.c',
-  'tcg/tcg.c',
 ))
-specific_ss.add(when: 'CONFIG_TCG_INTERPRETER', if_true: files('tcg/tci.c'))
 
 # Work around a gcc bug/misfeature wherein constant propagation looks
 # through an alias:
@@ -XXX,XX +XXX,XX @@ subdir('net')
 subdir('replay')
 subdir('semihosting')
 subdir('hw')
+subdir('tcg')
 subdir('accel')
 subdir('plugins')
 subdir('bsd-user')
diff --git a/tcg/meson.build b/tcg/meson.build
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tcg/meson.build
@@ -XXX,XX +XXX,XX @@
+tcg_ss = ss.source_set()
+
+tcg_ss.add(files(
+  'optimize.c',
+  'tcg.c',
+  'tcg-common.c',
+  'tcg-op.c',
+  'tcg-op-gvec.c',
+  'tcg-op-vec.c',
+))
+tcg_ss.add(when: 'CONFIG_TCG_INTERPRETER', if_true: files('tci.c'))
+
+specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
-- 
2.25.1

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 meson.build     | 4 +---
 fpu/meson.build | 1 +
 2 files changed, 2 insertions(+), 3 deletions(-)
 create mode 100644 fpu/meson.build

diff --git a/meson.build b/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/meson.build
+++ b/meson.build
@@ -XXX,XX +XXX,XX @@ subdir('softmmu')
 
 common_ss.add(capstone)
 specific_ss.add(files('cpu.c', 'disas.c', 'gdbstub.c'), capstone)
-specific_ss.add(when: 'CONFIG_TCG', if_true: files(
-  'fpu/softfloat.c',
-))
 
 # Work around a gcc bug/misfeature wherein constant propagation looks
 # through an alias:
@@ -XXX,XX +XXX,XX @@ subdir('replay')
 subdir('semihosting')
 subdir('hw')
 subdir('tcg')
+subdir('fpu')
 subdir('accel')
 subdir('plugins')
 subdir('bsd-user')
diff --git a/fpu/meson.build b/fpu/meson.build
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/fpu/meson.build
@@ -0,0 +1 @@
+specific_ss.add(when: 'CONFIG_TCG', if_true: files('softfloat.c'))
-- 
2.25.1

Instead of delaying tcg_region_init until after tcg_prologue_init
is complete, do tcg_region_init first and let tcg_prologue_init
shrink the first region by the size of the generated prologue.

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/tcg-all.c       | 11 ---------
 accel/tcg/translate-all.c |  3 +++
 bsd-user/main.c           |  1 -
 linux-user/main.c         |  1 -
 tcg/tcg.c                 | 52 ++++++++++++++-------------------------
 5 files changed, 22 insertions(+), 46 deletions(-)

diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -XXX,XX +XXX,XX @@ static int tcg_init(MachineState *ms)
 
     tcg_exec_init(s->tb_size * 1024 * 1024, s->splitwx_enabled);
     mttcg_enabled = s->mttcg_enabled;
-
-    /*
-     * Initialize TCG regions only for softmmu.
-     *
-     * This needs to be done later for user mode, because the prologue
-     * generation needs to be delayed so that GUEST_BASE is already set.
-     */
-#ifndef CONFIG_USER_ONLY
-    tcg_region_init();
-#endif /* !CONFIG_USER_ONLY */
-
     return 0;
 }
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ void tcg_exec_init(unsigned long tb_size, int splitwx)
                                splitwx, &error_fatal);
     assert(ok);
 
+    /* TODO: allocating regions is hand-in-glove with code_gen_buffer. */
+    tcg_region_init();
+
 #if defined(CONFIG_SOFTMMU)
     /* There's no guest base to take into account, so go ahead and
        initialize the prologue now.  */
diff --git a/bsd-user/main.c b/bsd-user/main.c
index XXXXXXX..XXXXXXX 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
      * the real value of GUEST_BASE into account.
      */
     tcg_prologue_init(tcg_ctx);
-    tcg_region_init();
 
     /* build Task State */
     memset(ts, 0, sizeof(TaskState));
diff --git a/linux-user/main.c b/linux-user/main.c
index XXXXXXX..XXXXXXX 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv, char **envp)
        generating the prologue until now so that the prologue can take
        the real value of GUEST_BASE into account.  */
     tcg_prologue_init(tcg_ctx);
-    tcg_region_init();
 
     target_cpu_copy_regs(env, regs);
 
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tcg_tb_alloc(TCGContext *s)
 
 void tcg_prologue_init(TCGContext *s)
 {
-    size_t prologue_size, total_size;
-    void *buf0, *buf1;
+    size_t prologue_size;
 
     /* Put the prologue at the beginning of code_gen_buffer.  */
-    buf0 = s->code_gen_buffer;
-    total_size = s->code_gen_buffer_size;
-    s->code_ptr = buf0;
-    s->code_buf = buf0;
+    tcg_region_assign(s, 0);
+    s->code_ptr = s->code_gen_ptr;
+    s->code_buf = s->code_gen_ptr;
     s->data_gen_ptr = NULL;
 
-    /*
-     * The region trees are not yet configured, but tcg_splitwx_to_rx
-     * needs the bounds for an assert.
-     */
-    region.start = buf0;
-    region.end = buf0 + total_size;
-
 #ifndef CONFIG_TCG_INTERPRETER
-    tcg_qemu_tb_exec = (tcg_prologue_fn *)tcg_splitwx_to_rx(buf0);
+    tcg_qemu_tb_exec = (tcg_prologue_fn *)tcg_splitwx_to_rx(s->code_ptr);
 #endif
 
-    /* Compute a high-water mark, at which we voluntarily flush the buffer
-       and start over.  The size here is arbitrary, significantly larger
-       than we expect the code generation for any one opcode to require.  */
-    s->code_gen_highwater = s->code_gen_buffer + (total_size - TCG_HIGHWATER);
-
 #ifdef TCG_TARGET_NEED_POOL_LABELS
     s->pool_labels = NULL;
 #endif
@@ -XXX,XX +XXX,XX @@ void tcg_prologue_init(TCGContext *s)
     }
 #endif
 
-    buf1 = s->code_ptr;
+    prologue_size = tcg_current_code_size(s);
+
 #ifndef CONFIG_TCG_INTERPRETER
-    flush_idcache_range((uintptr_t)tcg_splitwx_to_rx(buf0), (uintptr_t)buf0,
-                        tcg_ptr_byte_diff(buf1, buf0));
+    flush_idcache_range((uintptr_t)tcg_splitwx_to_rx(s->code_buf),
+                        (uintptr_t)s->code_buf, prologue_size);
 #endif
 
-    /* Deduct the prologue from the buffer.  */
-    prologue_size = tcg_current_code_size(s);
-    s->code_gen_ptr = buf1;
-    s->code_gen_buffer = buf1;
-    s->code_buf = buf1;
-    total_size -= prologue_size;
-    s->code_gen_buffer_size = total_size;
+    /* Deduct the prologue from the first region.  */
+    region.start = s->code_ptr;
 
-    tcg_register_jit(tcg_splitwx_to_rx(s->code_gen_buffer), total_size);
+    /* Recompute boundaries of the first region. */
+    tcg_region_assign(s, 0);
+
+    tcg_register_jit(tcg_splitwx_to_rx(region.start),
+                     region.end - region.start);
 
 #ifdef DEBUG_DISAS
     if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM)) {
         FILE *logfile = qemu_log_lock();
         qemu_log("PROLOGUE: [size=%zu]\n", prologue_size);
         if (s->data_gen_ptr) {
-            size_t code_size = s->data_gen_ptr - buf0;
+            size_t code_size = s->data_gen_ptr - s->code_gen_ptr;
             size_t data_size = prologue_size - code_size;
             size_t i;
 
-            log_disas(buf0, code_size);
+            log_disas(s->code_gen_ptr, code_size);
 
             for (i = 0; i < data_size; i += sizeof(tcg_target_ulong)) {
                 if (sizeof(tcg_target_ulong) == 8) {
@@ -XXX,XX +XXX,XX @@ void tcg_prologue_init(TCGContext *s)
                 }
             }
         } else {
-            log_disas(buf0, prologue_size);
+            log_disas(s->code_gen_ptr, prologue_size);
         }
         qemu_log("\n");
         qemu_log_flush();
-- 
2.25.1

All callers immediately assert on error, so move the assert
into the function itself.

diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static bool tcg_region_alloc(TCGContext *s)
  * Perform a context's first region allocation.
  * This function does _not_ increment region.agg_size_full.
  */
-static inline bool tcg_region_initial_alloc__locked(TCGContext *s)
+static void tcg_region_initial_alloc__locked(TCGContext *s)
 {
-    return tcg_region_alloc__locked(s);
+    bool err = tcg_region_alloc__locked(s);
+    g_assert(!err);
 }
 
 /* Call from a safe-work context */
@@ -XXX,XX +XXX,XX @@ void tcg_region_reset_all(void)
 
     for (i = 0; i < n_ctxs; i++) {
         TCGContext *s = qatomic_read(&tcg_ctxs[i]);
-        bool err = tcg_region_initial_alloc__locked(s);
-
-        g_assert(!err);
+        tcg_region_initial_alloc__locked(s);
     }
     qemu_mutex_unlock(&region.lock);
 
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(void)
 
     /* In user-mode we support only one ctx, so do the initial allocation now */
 #ifdef CONFIG_USER_ONLY
-    {
-        bool err = tcg_region_initial_alloc__locked(tcg_ctx);
-
-        g_assert(!err);
-    }
+    tcg_region_initial_alloc__locked(tcg_ctx);
 #endif
 }
 
@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
     MachineState *ms = MACHINE(qdev_get_machine());
     TCGContext *s = g_malloc(sizeof(*s));
     unsigned int i, n;
-    bool err;
 
     *s = tcg_init_ctx;
 
@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
 
     tcg_ctx = s;
     qemu_mutex_lock(&region.lock);
-    err = tcg_region_initial_alloc__locked(tcg_ctx);
-    g_assert(!err);
+    tcg_region_initial_alloc__locked(s);
     qemu_mutex_unlock(&region.lock);
 }
 #endif /* !CONFIG_USER_ONLY */
-- 
2.25.1

This has only one user, and currently needs an ifdef,
but will make more sense after some code motion.

diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static void tcg_region_initial_alloc__locked(TCGContext *s)
     g_assert(!err);
 }
 
+#ifndef CONFIG_USER_ONLY
+static void tcg_region_initial_alloc(TCGContext *s)
+{
+    qemu_mutex_lock(&region.lock);
+    tcg_region_initial_alloc__locked(s);
+    qemu_mutex_unlock(&region.lock);
+}
+#endif
+
 /* Call from a safe-work context */
 void tcg_region_reset_all(void)
 {
@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
     }
 
     tcg_ctx = s;
-    qemu_mutex_lock(&region.lock);
-    tcg_region_initial_alloc__locked(s);
-    qemu_mutex_unlock(&region.lock);
+    tcg_region_initial_alloc(s);
 }
 #endif /* !CONFIG_USER_ONLY */
 
-- 
2.25.1

This has only one user, but will make more sense after some
code motion.

Always leave the tcg_init_ctx initialized to the first region,
in preparation for tcg_prologue_init().  This also requires
that we don't re-allocate the region for the first cpu, lest
we hit the assertion for total number of regions allocated .

diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(void)
 
     tcg_region_trees_init();
 
-    /* In user-mode we support only one ctx, so do the initial allocation now */
-#ifdef CONFIG_USER_ONLY
-    tcg_region_initial_alloc__locked(tcg_ctx);
-#endif
+    /*
+     * Leave the initial context initialized to the first region.
+     * This will be the context into which we generate the prologue.
+     * It is also the only context for CONFIG_USER_ONLY.
+     */
+    tcg_region_initial_alloc__locked(&tcg_init_ctx);
+}
+
+static void tcg_region_prologue_set(TCGContext *s)
+{
+    /* Deduct the prologue from the first region.  */
+    g_assert(region.start == s->code_gen_buffer);
+    region.start = s->code_ptr;
+
+    /* Recompute boundaries of the first region. */
+    tcg_region_assign(s, 0);
+
+    /* Register the balance of the buffer with gdb. */
+    tcg_register_jit(tcg_splitwx_to_rx(region.start),
+                     region.end - region.start);
 }
 
 #ifdef CONFIG_DEBUG_TCG
@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
 
     if (n > 0) {
         alloc_tcg_plugin_context(s);
+        tcg_region_initial_alloc(s);
     }
 
     tcg_ctx = s;
-    tcg_region_initial_alloc(s);
 }
 #endif /* !CONFIG_USER_ONLY */
 
@@ -XXX,XX +XXX,XX @@ void tcg_prologue_init(TCGContext *s)
 {
     size_t prologue_size;
 
-    /* Put the prologue at the beginning of code_gen_buffer.  */
-    tcg_region_assign(s, 0);
     s->code_ptr = s->code_gen_ptr;
     s->code_buf = s->code_gen_ptr;
     s->data_gen_ptr = NULL;
@@ -XXX,XX +XXX,XX @@ void tcg_prologue_init(TCGContext *s)
                         (uintptr_t)s->code_buf, prologue_size);
 #endif
 
-    /* Deduct the prologue from the first region.  */
-    region.start = s->code_ptr;
-
-    /* Recompute boundaries of the first region. */
-    tcg_region_assign(s, 0);
-
-    tcg_register_jit(tcg_splitwx_to_rx(region.start),
-                     region.end - region.start);
+    tcg_region_prologue_set(s);
 
 #ifdef DEBUG_DISAS
     if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM)) {
-- 
2.25.1

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/tcg-internal.h |  37 +++
 tcg/region.c       | 572 +++++++++++++++++++++++++++++++++++++++++++++
 tcg/tcg.c          | 547 +------------------------------------------
 tcg/meson.build    |   1 +
 4 files changed, 613 insertions(+), 544 deletions(-)
 create mode 100644 tcg/tcg-internal.h
 create mode 100644 tcg/region.c

diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tcg/tcg-internal.h
@@ -XXX,XX +XXX,XX @@
+/*
+ * Internal declarations for Tiny Code Generator for QEMU
+ *
+ * Copyright (c) 2008 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef TCG_INTERNAL_H
+#define TCG_INTERNAL_H 1
+
+#define TCG_HIGHWATER 1024
+
+extern TCGContext **tcg_ctxs;
+extern unsigned int n_tcg_ctxs;
+
+bool tcg_region_alloc(TCGContext *s);
+void tcg_region_initial_alloc(TCGContext *s);
+void tcg_region_prologue_set(TCGContext *s);
+
+#endif /* TCG_INTERNAL_H */
diff --git a/tcg/region.c b/tcg/region.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@
+/*
+ * Memory region management for Tiny Code Generator for QEMU
+ *
+ * Copyright (c) 2008 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "exec/exec-all.h"
+#include "tcg/tcg.h"
+#if !defined(CONFIG_USER_ONLY)
+#include "hw/boards.h"
+#endif
+#include "tcg-internal.h"
+
+
+struct tcg_region_tree {
+    QemuMutex lock;
+    GTree *tree;
+    /* padding to avoid false sharing is computed at run-time */
+};
+
+/*
+ * We divide code_gen_buffer into equally-sized "regions" that TCG threads
+ * dynamically allocate from as demand dictates. Given appropriate region
+ * sizing, this minimizes flushes even when some TCG threads generate a lot
+ * more code than others.
+ */
+struct tcg_region_state {
+    QemuMutex lock;
+
+    /* fields set at init time */
+    void *start;
+    void *start_aligned;
+    void *end;
+    size_t n;
+    size_t size; /* size of one region */
+    size_t stride; /* .size + guard size */
+
+    /* fields protected by the lock */
+    size_t current; /* current region index */
+    size_t agg_size_full; /* aggregate size of full regions */
+};
+
+static struct tcg_region_state region;
+
+/*
+ * This is an array of struct tcg_region_tree's, with padding.
+ * We use void * to simplify the computation of region_trees[i]; each
+ * struct is found every tree_size bytes.
+ */
+static void *region_trees;
+static size_t tree_size;
+
+/* compare a pointer @ptr and a tb_tc @s */
+static int ptr_cmp_tb_tc(const void *ptr, const struct tb_tc *s)
+{
+    if (ptr >= s->ptr + s->size) {
+        return 1;
+    } else if (ptr < s->ptr) {
+        return -1;
+    }
+    return 0;
+}
+
+static gint tb_tc_cmp(gconstpointer ap, gconstpointer bp)
+{
+    const struct tb_tc *a = ap;
+    const struct tb_tc *b = bp;
+
+    /*
+     * When both sizes are set, we know this isn't a lookup.
+     * This is the most likely case: every TB must be inserted; lookups
+     * are a lot less frequent.
+     */
+    if (likely(a->size && b->size)) {
+        if (a->ptr > b->ptr) {
+            return 1;
+        } else if (a->ptr < b->ptr) {
+            return -1;
+        }
+        /* a->ptr == b->ptr should happen only on deletions */
+        g_assert(a->size == b->size);
+        return 0;
+    }
+    /*
+     * All lookups have either .size field set to 0.
+     * From the glib sources we see that @ap is always the lookup key. However
+     * the docs provide no guarantee, so we just mark this case as likely.
+     */
+    if (likely(a->size == 0)) {
+        return ptr_cmp_tb_tc(a->ptr, b);
+    }
+    return ptr_cmp_tb_tc(b->ptr, a);
+}
+
+static void tcg_region_trees_init(void)
+{
+    size_t i;
+
+    tree_size = ROUND_UP(sizeof(struct tcg_region_tree), qemu_dcache_linesize);
+    region_trees = qemu_memalign(qemu_dcache_linesize, region.n * tree_size);
+    for (i = 0; i < region.n; i++) {
+        struct tcg_region_tree *rt = region_trees + i * tree_size;
+
+        qemu_mutex_init(&rt->lock);
+        rt->tree = g_tree_new(tb_tc_cmp);
+    }
+}
+
+static struct tcg_region_tree *tc_ptr_to_region_tree(const void *p)
+{
+    size_t region_idx;
+
+    /*
+     * Like tcg_splitwx_to_rw, with no assert.  The pc may come from
+     * a signal handler over which the caller has no control.
+     */
+    if (!in_code_gen_buffer(p)) {
+        p -= tcg_splitwx_diff;
+        if (!in_code_gen_buffer(p)) {
+            return NULL;
+        }
+    }
+
+    if (p < region.start_aligned) {
+        region_idx = 0;
+    } else {
+        ptrdiff_t offset = p - region.start_aligned;
+
+        if (offset > region.stride * (region.n - 1)) {
+            region_idx = region.n - 1;
+        } else {
+            region_idx = offset / region.stride;
+        }
+    }
+    return region_trees + region_idx * tree_size;
+}
+
+void tcg_tb_insert(TranslationBlock *tb)
+{
+    struct tcg_region_tree *rt = tc_ptr_to_region_tree(tb->tc.ptr);
+
+    g_assert(rt != NULL);
+    qemu_mutex_lock(&rt->lock);
+    g_tree_insert(rt->tree, &tb->tc, tb);
+    qemu_mutex_unlock(&rt->lock);
+}
+
+void tcg_tb_remove(TranslationBlock *tb)
+{
+    struct tcg_region_tree *rt = tc_ptr_to_region_tree(tb->tc.ptr);
+
+    g_assert(rt != NULL);
+    qemu_mutex_lock(&rt->lock);
+    g_tree_remove(rt->tree, &tb->tc);
+    qemu_mutex_unlock(&rt->lock);
+}
+
+/*
+ * Find the TB 'tb' such that
+ * tb->tc.ptr <= tc_ptr < tb->tc.ptr + tb->tc.size
+ * Return NULL if not found.
+ */
+TranslationBlock *tcg_tb_lookup(uintptr_t tc_ptr)
+{
+    struct tcg_region_tree *rt = tc_ptr_to_region_tree((void *)tc_ptr);
+    TranslationBlock *tb;
+    struct tb_tc s = { .ptr = (void *)tc_ptr };
+
+    if (rt == NULL) {
+        return NULL;
+    }
+
+    qemu_mutex_lock(&rt->lock);
+    tb = g_tree_lookup(rt->tree, &s);
+    qemu_mutex_unlock(&rt->lock);
+    return tb;
+}
+
+static void tcg_region_tree_lock_all(void)
+{
+    size_t i;
+
+    for (i = 0; i < region.n; i++) {
+        struct tcg_region_tree *rt = region_trees + i * tree_size;
+
+        qemu_mutex_lock(&rt->lock);
+    }
+}
+
+static void tcg_region_tree_unlock_all(void)
+{
+    size_t i;
+
+    for (i = 0; i < region.n; i++) {
+        struct tcg_region_tree *rt = region_trees + i * tree_size;
+
+        qemu_mutex_unlock(&rt->lock);
+    }
+}
+
+void tcg_tb_foreach(GTraverseFunc func, gpointer user_data)
+{
+    size_t i;
+
+    tcg_region_tree_lock_all();
+    for (i = 0; i < region.n; i++) {
+        struct tcg_region_tree *rt = region_trees + i * tree_size;
+
+        g_tree_foreach(rt->tree, func, user_data);
+    }
+    tcg_region_tree_unlock_all();
+}
+
+size_t tcg_nb_tbs(void)
+{
+    size_t nb_tbs = 0;
+    size_t i;
+
+    tcg_region_tree_lock_all();
+    for (i = 0; i < region.n; i++) {
+        struct tcg_region_tree *rt = region_trees + i * tree_size;
+
+        nb_tbs += g_tree_nnodes(rt->tree);
+    }
+    tcg_region_tree_unlock_all();
+    return nb_tbs;
+}
+
+static gboolean tcg_region_tree_traverse(gpointer k, gpointer v, gpointer data)
+{
+    TranslationBlock *tb = v;
+
+    tb_destroy(tb);
+    return FALSE;
+}
+
+static void tcg_region_tree_reset_all(void)
+{
+    size_t i;
+
+    tcg_region_tree_lock_all();
+    for (i = 0; i < region.n; i++) {
+        struct tcg_region_tree *rt = region_trees + i * tree_size;
+
+        g_tree_foreach(rt->tree, tcg_region_tree_traverse, NULL);
+        /* Increment the refcount first so that destroy acts as a reset */
+        g_tree_ref(rt->tree);
+        g_tree_destroy(rt->tree);
+    }
+    tcg_region_tree_unlock_all();
+}
+
+static void tcg_region_bounds(size_t curr_region, void **pstart, void **pend)
+{
+    void *start, *end;
+
+    start = region.start_aligned + curr_region * region.stride;
+    end = start + region.size;
+
+    if (curr_region == 0) {
+        start = region.start;
+    }
+    if (curr_region == region.n - 1) {
+        end = region.end;
+    }
+
+    *pstart = start;
+    *pend = end;
+}
+
+static void tcg_region_assign(TCGContext *s, size_t curr_region)
+{
+    void *start, *end;
+
+    tcg_region_bounds(curr_region, &start, &end);
+
+    s->code_gen_buffer = start;
+    s->code_gen_ptr = start;
+    s->code_gen_buffer_size = end - start;
+    s->code_gen_highwater = end - TCG_HIGHWATER;
+}
+
+static bool tcg_region_alloc__locked(TCGContext *s)
+{
+    if (region.current == region.n) {
+        return true;
+    }
+    tcg_region_assign(s, region.current);
+    region.current++;
+    return false;
+}
+
+/*
+ * Request a new region once the one in use has filled up.
+ * Returns true on error.
+ */
+bool tcg_region_alloc(TCGContext *s)
+{
+    bool err;
+    /* read the region size now; alloc__locked will overwrite it on success */
+    size_t size_full = s->code_gen_buffer_size;
+
+    qemu_mutex_lock(&region.lock);
+    err = tcg_region_alloc__locked(s);
+    if (!err) {
+        region.agg_size_full += size_full - TCG_HIGHWATER;
+    }
+    qemu_mutex_unlock(&region.lock);
+    return err;
+}
+
+/*
+ * Perform a context's first region allocation.
+ * This function does _not_ increment region.agg_size_full.
+ */
+static void tcg_region_initial_alloc__locked(TCGContext *s)
+{
+    bool err = tcg_region_alloc__locked(s);
+    g_assert(!err);
+}
+
+void tcg_region_initial_alloc(TCGContext *s)
+{
+    qemu_mutex_lock(&region.lock);
+    tcg_region_initial_alloc__locked(s);
+    qemu_mutex_unlock(&region.lock);
+}
+
+/* Call from a safe-work context */
+void tcg_region_reset_all(void)
+{
+    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
+    unsigned int i;
+
+    qemu_mutex_lock(&region.lock);
+    region.current = 0;
+    region.agg_size_full = 0;
+
+    for (i = 0; i < n_ctxs; i++) {
+        TCGContext *s = qatomic_read(&tcg_ctxs[i]);
+        tcg_region_initial_alloc__locked(s);
+    }
+    qemu_mutex_unlock(&region.lock);
+
+    tcg_region_tree_reset_all();
+}
+
+#ifdef CONFIG_USER_ONLY
+static size_t tcg_n_regions(void)
+{
+    return 1;
+}
+#else
+/*
+ * It is likely that some vCPUs will translate more code than others, so we
+ * first try to set more regions than max_cpus, with those regions being of
+ * reasonable size. If that's not possible we make do by evenly dividing
+ * the code_gen_buffer among the vCPUs.
+ */
+static size_t tcg_n_regions(void)
+{
+    size_t i;
+
+    /* Use a single region if all we have is one vCPU thread */
+#if !defined(CONFIG_USER_ONLY)
+    MachineState *ms = MACHINE(qdev_get_machine());
+    unsigned int max_cpus = ms->smp.max_cpus;
+#endif
+    if (max_cpus == 1 || !qemu_tcg_mttcg_enabled()) {
+        return 1;
+    }
+
+    /* Try to have more regions than max_cpus, with each region being >= 2 MB */
+    for (i = 8; i > 0; i--) {
+        size_t regions_per_thread = i;
+        size_t region_size;
+
+        region_size = tcg_init_ctx.code_gen_buffer_size;
+        region_size /= max_cpus * regions_per_thread;
+
+        if (region_size >= 2 * 1024u * 1024) {
+            return max_cpus * regions_per_thread;
+        }
+    }
+    /* If we can't, then just allocate one region per vCPU thread */
+    return max_cpus;
+}
+#endif
+
+/*
+ * Initializes region partitioning.
+ *
+ * Called at init time from the parent thread (i.e. the one calling
+ * tcg_context_init), after the target's TCG globals have been set.
+ *
+ * Region partitioning works by splitting code_gen_buffer into separate regions,
+ * and then assigning regions to TCG threads so that the threads can translate
+ * code in parallel without synchronization.
+ *
+ * In softmmu the number of TCG threads is bounded by max_cpus, so we use at
+ * least max_cpus regions in MTTCG. In !MTTCG we use a single region.
+ * Note that the TCG options from the command-line (i.e. -accel accel=tcg,[...])
+ * must have been parsed before calling this function, since it calls
+ * qemu_tcg_mttcg_enabled().
+ *
+ * In user-mode we use a single region.  Having multiple regions in user-mode
+ * is not supported, because the number of vCPU threads (recall that each thread
+ * spawned by the guest corresponds to a vCPU thread) is only bounded by the
+ * OS, and usually this number is huge (tens of thousands is not uncommon).
+ * Thus, given this large bound on the number of vCPU threads and the fact
+ * that code_gen_buffer is allocated at compile-time, we cannot guarantee
+ * that the availability of at least one region per vCPU thread.
+ *
+ * However, this user-mode limitation is unlikely to be a significant problem
+ * in practice. Multi-threaded guests share most if not all of their translated
+ * code, which makes parallel code generation less appealing than in softmmu.
+ */
+void tcg_region_init(void)
+{
+    void *buf = tcg_init_ctx.code_gen_buffer;
+    void *aligned;
+    size_t size = tcg_init_ctx.code_gen_buffer_size;
+    size_t page_size = qemu_real_host_page_size;
+    size_t region_size;
+    size_t n_regions;
+    size_t i;
+
+    n_regions = tcg_n_regions();
+
+    /* The first region will be 'aligned - buf' bytes larger than the others */
+    aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
+    g_assert(aligned < tcg_init_ctx.code_gen_buffer + size);
+    /*
+     * Make region_size a multiple of page_size, using aligned as the start.
+     * As a result of this we might end up with a few extra pages at the end of
+     * the buffer; we will assign those to the last region.
+     */
+    region_size = (size - (aligned - buf)) / n_regions;
+    region_size = QEMU_ALIGN_DOWN(region_size, page_size);
+
+    /* A region must have at least 2 pages; one code, one guard */
+    g_assert(region_size >= 2 * page_size);
+
+    /* init the region struct */
+    qemu_mutex_init(&region.lock);
+    region.n = n_regions;
+    region.size = region_size - page_size;
+    region.stride = region_size;
+    region.start = buf;
+    region.start_aligned = aligned;
+    /* page-align the end, since its last page will be a guard page */
+    region.end = QEMU_ALIGN_PTR_DOWN(buf + size, page_size);
+    /* account for that last guard page */
+    region.end -= page_size;
+
+    /*
+     * Set guard pages in the rw buffer, as that's the one into which
+     * buffer overruns could occur.  Do not set guard pages in the rx
+     * buffer -- let that one use hugepages throughout.
+     */
+    for (i = 0; i < region.n; i++) {
+        void *start, *end;
+
+        tcg_region_bounds(i, &start, &end);
+
+        /*
+         * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
+         * rejects a permission change from RWX -> NONE.  Guard pages are
+         * nice for bug detection but are not essential; ignore any failure.
+         */
+        (void)qemu_mprotect_none(end, page_size);
+    }
+
+    tcg_region_trees_init();
+
+    /*
+     * Leave the initial context initialized to the first region.
+     * This will be the context into which we generate the prologue.
+     * It is also the only context for CONFIG_USER_ONLY.
+     */
+    tcg_region_initial_alloc__locked(&tcg_init_ctx);
+}
+
+void tcg_region_prologue_set(TCGContext *s)
+{
+    /* Deduct the prologue from the first region.  */
+    g_assert(region.start == s->code_gen_buffer);
+    region.start = s->code_ptr;
+
+    /* Recompute boundaries of the first region. */
+    tcg_region_assign(s, 0);
+
+    /* Register the balance of the buffer with gdb. */
+    tcg_register_jit(tcg_splitwx_to_rx(region.start),
+                     region.end - region.start);
+}
+
+/*
+ * Returns the size (in bytes) of all translated code (i.e. from all regions)
+ * currently in the cache.
+ * See also: tcg_code_capacity()
+ * Do not confuse with tcg_current_code_size(); that one applies to a single
+ * TCG context.
+ */
+size_t tcg_code_size(void)
+{
+    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
+    unsigned int i;
+    size_t total;
+
+    qemu_mutex_lock(&region.lock);
+    total = region.agg_size_full;
+    for (i = 0; i < n_ctxs; i++) {
+        const TCGContext *s = qatomic_read(&tcg_ctxs[i]);
+        size_t size;
+
+        size = qatomic_read(&s->code_gen_ptr) - s->code_gen_buffer;
+        g_assert(size <= s->code_gen_buffer_size);
+        total += size;
+    }
+    qemu_mutex_unlock(&region.lock);
+    return total;
+}
+
+/*
+ * Returns the code capacity (in bytes) of the entire cache, i.e. including all
+ * regions.
+ * See also: tcg_code_size()
+ */
+size_t tcg_code_capacity(void)
+{
+    size_t guard_size, capacity;
+
+    /* no need for synchronization; these variables are set at init time */
+    guard_size = region.stride - region.size;
+    capacity = region.end + guard_size - region.start;
+    capacity -= region.n * (guard_size + TCG_HIGHWATER);
+    return capacity;
+}
+
+size_t tcg_tb_phys_invalidate_count(void)
+{
+    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
+    unsigned int i;
+    size_t total = 0;
+
+    for (i = 0; i < n_ctxs; i++) {
+        const TCGContext *s = qatomic_read(&tcg_ctxs[i]);
+
+        total += qatomic_read(&s->tb_phys_invalidate_count);
+    }
+    return total;
+}
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@
 
 #include "elf.h"
 #include "exec/log.h"
+#include "tcg-internal.h"
 
 /* Forward declarations for functions declared in tcg-target.c.inc and
    used here. */
@@ -XXX,XX +XXX,XX @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct);
 static int tcg_out_ldst_finalize(TCGContext *s);
 #endif
 
-#define TCG_HIGHWATER 1024
-
-static TCGContext **tcg_ctxs;
-static unsigned int n_tcg_ctxs;
+TCGContext **tcg_ctxs;
+unsigned int n_tcg_ctxs;
 TCGv_env cpu_env = 0;
 const void *tcg_code_gen_epilogue;
 uintptr_t tcg_splitwx_diff;
@@ -XXX,XX +XXX,XX @@ uintptr_t tcg_splitwx_diff;
 tcg_prologue_fn *tcg_qemu_tb_exec;
 #endif
 
-struct tcg_region_tree {
-    QemuMutex lock;
-    GTree *tree;
-    /* padding to avoid false sharing is computed at run-time */
-};
-
-/*
- * We divide code_gen_buffer into equally-sized "regions" that TCG threads
- * dynamically allocate from as demand dictates. Given appropriate region
- * sizing, this minimizes flushes even when some TCG threads generate a lot
- * more code than others.
- */
-struct tcg_region_state {
-    QemuMutex lock;
-
-    /* fields set at init time */
-    void *start;
-    void *start_aligned;
-    void *end;
-    size_t n;
-    size_t size; /* size of one region */
-    size_t stride; /* .size + guard size */
-
-    /* fields protected by the lock */
-    size_t current; /* current region index */
-    size_t agg_size_full; /* aggregate size of full regions */
-};
-
-static struct tcg_region_state region;
-/*
- * This is an array of struct tcg_region_tree's, with padding.
- * We use void * to simplify the computation of region_trees[i]; each
- * struct is found every tree_size bytes.
- */
-static void *region_trees;
-static size_t tree_size;
 static TCGRegSet tcg_target_available_regs[TCG_TYPE_COUNT];
 static TCGRegSet tcg_target_call_clobber_regs;
 
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef constraint_sets[] = {
 
 #include "tcg-target.c.inc"
 
-/* compare a pointer @ptr and a tb_tc @s */
-static int ptr_cmp_tb_tc(const void *ptr, const struct tb_tc *s)
-{
-    if (ptr >= s->ptr + s->size) {
-        return 1;
-    } else if (ptr < s->ptr) {
-        return -1;
-    }
-    return 0;
-}
-
-static gint tb_tc_cmp(gconstpointer ap, gconstpointer bp)
-{
-    const struct tb_tc *a = ap;
-    const struct tb_tc *b = bp;
-
-    /*
-     * When both sizes are set, we know this isn't a lookup.
-     * This is the most likely case: every TB must be inserted; lookups
-     * are a lot less frequent.
-     */
-    if (likely(a->size && b->size)) {
-        if (a->ptr > b->ptr) {
-            return 1;
-        } else if (a->ptr < b->ptr) {
-            return -1;
-        }
-        /* a->ptr == b->ptr should happen only on deletions */
-        g_assert(a->size == b->size);
-        return 0;
-    }
-    /*
-     * All lookups have either .size field set to 0.
-     * From the glib sources we see that @ap is always the lookup key. However
-     * the docs provide no guarantee, so we just mark this case as likely.
-     */
-    if (likely(a->size == 0)) {
-        return ptr_cmp_tb_tc(a->ptr, b);
-    }
-    return ptr_cmp_tb_tc(b->ptr, a);
-}
-
-static void tcg_region_trees_init(void)
-{
-    size_t i;
-
-    tree_size = ROUND_UP(sizeof(struct tcg_region_tree), qemu_dcache_linesize);
-    region_trees = qemu_memalign(qemu_dcache_linesize, region.n * tree_size);
-    for (i = 0; i < region.n; i++) {
-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-
-        qemu_mutex_init(&rt->lock);
-        rt->tree = g_tree_new(tb_tc_cmp);
-    }
-}
-
-static struct tcg_region_tree *tc_ptr_to_region_tree(const void *p)
-{
-    size_t region_idx;
-
-    /*
-     * Like tcg_splitwx_to_rw, with no assert.  The pc may come from
-     * a signal handler over which the caller has no control.
-     */
-    if (!in_code_gen_buffer(p)) {
-        p -= tcg_splitwx_diff;
-        if (!in_code_gen_buffer(p)) {
-            return NULL;
-        }
-    }
-
-    if (p < region.start_aligned) {
-        region_idx = 0;
-    } else {
-        ptrdiff_t offset = p - region.start_aligned;
-
-        if (offset > region.stride * (region.n - 1)) {
-            region_idx = region.n - 1;
-        } else {
-            region_idx = offset / region.stride;
-        }
-    }
-    return region_trees + region_idx * tree_size;
-}
-
-void tcg_tb_insert(TranslationBlock *tb)
-{
-    struct tcg_region_tree *rt = tc_ptr_to_region_tree(tb->tc.ptr);
-
-    g_assert(rt != NULL);
-    qemu_mutex_lock(&rt->lock);
-    g_tree_insert(rt->tree, &tb->tc, tb);
-    qemu_mutex_unlock(&rt->lock);
-}
-
-void tcg_tb_remove(TranslationBlock *tb)
-{
-    struct tcg_region_tree *rt = tc_ptr_to_region_tree(tb->tc.ptr);
-
-    g_assert(rt != NULL);
-    qemu_mutex_lock(&rt->lock);
-    g_tree_remove(rt->tree, &tb->tc);
-    qemu_mutex_unlock(&rt->lock);
-}
-
-/*
- * Find the TB 'tb' such that
- * tb->tc.ptr <= tc_ptr < tb->tc.ptr + tb->tc.size
- * Return NULL if not found.
- */
-TranslationBlock *tcg_tb_lookup(uintptr_t tc_ptr)
-{
-    struct tcg_region_tree *rt = tc_ptr_to_region_tree((void *)tc_ptr);
-    TranslationBlock *tb;
-    struct tb_tc s = { .ptr = (void *)tc_ptr };
-
-    if (rt == NULL) {
-        return NULL;
-    }
-
-    qemu_mutex_lock(&rt->lock);
-    tb = g_tree_lookup(rt->tree, &s);
-    qemu_mutex_unlock(&rt->lock);
-    return tb;
-}
-
-static void tcg_region_tree_lock_all(void)
-{
-    size_t i;
-
-    for (i = 0; i < region.n; i++) {
-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-
-        qemu_mutex_lock(&rt->lock);
-    }
-}
-
-static void tcg_region_tree_unlock_all(void)
-{
-    size_t i;
-
-    for (i = 0; i < region.n; i++) {
-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-
-        qemu_mutex_unlock(&rt->lock);
-    }
-}
-
-void tcg_tb_foreach(GTraverseFunc func, gpointer user_data)
-{
-    size_t i;
-
-    tcg_region_tree_lock_all();
-    for (i = 0; i < region.n; i++) {
-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-
-        g_tree_foreach(rt->tree, func, user_data);
-    }
-    tcg_region_tree_unlock_all();
-}
-
-size_t tcg_nb_tbs(void)
-{
-    size_t nb_tbs = 0;
-    size_t i;
-
-    tcg_region_tree_lock_all();
-    for (i = 0; i < region.n; i++) {
-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-
-        nb_tbs += g_tree_nnodes(rt->tree);
-    }
-    tcg_region_tree_unlock_all();
-    return nb_tbs;
-}
-
-static gboolean tcg_region_tree_traverse(gpointer k, gpointer v, gpointer data)
-{
-    TranslationBlock *tb = v;
-
-    tb_destroy(tb);
-    return FALSE;
-}
-
-static void tcg_region_tree_reset_all(void)
-{
-    size_t i;
-
-    tcg_region_tree_lock_all();
-    for (i = 0; i < region.n; i++) {
-        struct tcg_region_tree *rt = region_trees + i * tree_size;
-
-        g_tree_foreach(rt->tree, tcg_region_tree_traverse, NULL);
-        /* Increment the refcount first so that destroy acts as a reset */
-        g_tree_ref(rt->tree);
-        g_tree_destroy(rt->tree);
-    }
-    tcg_region_tree_unlock_all();
-}
-
-static void tcg_region_bounds(size_t curr_region, void **pstart, void **pend)
-{
-    void *start, *end;
-
-    start = region.start_aligned + curr_region * region.stride;
-    end = start + region.size;
-
-    if (curr_region == 0) {
-        start = region.start;
-    }
-    if (curr_region == region.n - 1) {
-        end = region.end;
-    }
-
-    *pstart = start;
-    *pend = end;
-}
-
-static void tcg_region_assign(TCGContext *s, size_t curr_region)
-{
-    void *start, *end;
-
-    tcg_region_bounds(curr_region, &start, &end);
-
-    s->code_gen_buffer = start;
-    s->code_gen_ptr = start;
-    s->code_gen_buffer_size = end - start;
-    s->code_gen_highwater = end - TCG_HIGHWATER;
-}
-
-static bool tcg_region_alloc__locked(TCGContext *s)
-{
-    if (region.current == region.n) {
-        return true;
-    }
-    tcg_region_assign(s, region.current);
-    region.current++;
-    return false;
-}
-
-/*
- * Request a new region once the one in use has filled up.
- * Returns true on error.
- */
-static bool tcg_region_alloc(TCGContext *s)
-{
-    bool err;
-    /* read the region size now; alloc__locked will overwrite it on success */
-    size_t size_full = s->code_gen_buffer_size;
-
-    qemu_mutex_lock(&region.lock);
-    err = tcg_region_alloc__locked(s);
-    if (!err) {
-        region.agg_size_full += size_full - TCG_HIGHWATER;
-    }
-    qemu_mutex_unlock(&region.lock);
-    return err;
-}
-
-/*
- * Perform a context's first region allocation.
- * This function does _not_ increment region.agg_size_full.
- */
-static void tcg_region_initial_alloc__locked(TCGContext *s)
-{
-    bool err = tcg_region_alloc__locked(s);
-    g_assert(!err);
-}
-
-#ifndef CONFIG_USER_ONLY
-static void tcg_region_initial_alloc(TCGContext *s)
-{
-    qemu_mutex_lock(&region.lock);
-    tcg_region_initial_alloc__locked(s);
-    qemu_mutex_unlock(&region.lock);
-}
-#endif
-
-/* Call from a safe-work context */
-void tcg_region_reset_all(void)
-{
-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
-    unsigned int i;
-
-    qemu_mutex_lock(&region.lock);
-    region.current = 0;
-    region.agg_size_full = 0;
-
-    for (i = 0; i < n_ctxs; i++) {
-        TCGContext *s = qatomic_read(&tcg_ctxs[i]);
-        tcg_region_initial_alloc__locked(s);
-    }
-    qemu_mutex_unlock(&region.lock);
-
-    tcg_region_tree_reset_all();
-}
-
-#ifdef CONFIG_USER_ONLY
-static size_t tcg_n_regions(void)
-{
-    return 1;
-}
-#else
-/*
- * It is likely that some vCPUs will translate more code than others, so we
- * first try to set more regions than max_cpus, with those regions being of
- * reasonable size. If that's not possible we make do by evenly dividing
- * the code_gen_buffer among the vCPUs.
- */
-static size_t tcg_n_regions(void)
-{
-    size_t i;
-
-    /* Use a single region if all we have is one vCPU thread */
-#if !defined(CONFIG_USER_ONLY)
-    MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int max_cpus = ms->smp.max_cpus;
-#endif
-    if (max_cpus == 1 || !qemu_tcg_mttcg_enabled()) {
-        return 1;
-    }
-
-    /* Try to have more regions than max_cpus, with each region being >= 2 MB */
-    for (i = 8; i > 0; i--) {
-        size_t regions_per_thread = i;
-        size_t region_size;
-
-        region_size = tcg_init_ctx.code_gen_buffer_size;
-        region_size /= max_cpus * regions_per_thread;
-
-        if (region_size >= 2 * 1024u * 1024) {
-            return max_cpus * regions_per_thread;
-        }
-    }
-    /* If we can't, then just allocate one region per vCPU thread */
-    return max_cpus;
-}
-#endif
-
-/*
- * Initializes region partitioning.
- *
- * Called at init time from the parent thread (i.e. the one calling
- * tcg_context_init), after the target's TCG globals have been set.
- *
- * Region partitioning works by splitting code_gen_buffer into separate regions,
- * and then assigning regions to TCG threads so that the threads can translate
- * code in parallel without synchronization.
- *
- * In softmmu the number of TCG threads is bounded by max_cpus, so we use at
- * least max_cpus regions in MTTCG. In !MTTCG we use a single region.
- * Note that the TCG options from the command-line (i.e. -accel accel=tcg,[...])
- * must have been parsed before calling this function, since it calls
- * qemu_tcg_mttcg_enabled().
- *
- * In user-mode we use a single region.  Having multiple regions in user-mode
- * is not supported, because the number of vCPU threads (recall that each thread
- * spawned by the guest corresponds to a vCPU thread) is only bounded by the
- * OS, and usually this number is huge (tens of thousands is not uncommon).
- * Thus, given this large bound on the number of vCPU threads and the fact
- * that code_gen_buffer is allocated at compile-time, we cannot guarantee
- * that the availability of at least one region per vCPU thread.
- *
- * However, this user-mode limitation is unlikely to be a significant problem
- * in practice. Multi-threaded guests share most if not all of their translated
- * code, which makes parallel code generation less appealing than in softmmu.
- */
-void tcg_region_init(void)
-{
-    void *buf = tcg_init_ctx.code_gen_buffer;
-    void *aligned;
-    size_t size = tcg_init_ctx.code_gen_buffer_size;
-    size_t page_size = qemu_real_host_page_size;
-    size_t region_size;
-    size_t n_regions;
-    size_t i;
-
-    n_regions = tcg_n_regions();
-
-    /* The first region will be 'aligned - buf' bytes larger than the others */
-    aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
-    g_assert(aligned < tcg_init_ctx.code_gen_buffer + size);
-    /*
-     * Make region_size a multiple of page_size, using aligned as the start.
-     * As a result of this we might end up with a few extra pages at the end of
-     * the buffer; we will assign those to the last region.
-     */
-    region_size = (size - (aligned - buf)) / n_regions;
-    region_size = QEMU_ALIGN_DOWN(region_size, page_size);
-
-    /* A region must have at least 2 pages; one code, one guard */
-    g_assert(region_size >= 2 * page_size);
-
-    /* init the region struct */
-    qemu_mutex_init(&region.lock);
-    region.n = n_regions;
-    region.size = region_size - page_size;
-    region.stride = region_size;
-    region.start = buf;
-    region.start_aligned = aligned;
-    /* page-align the end, since its last page will be a guard page */
-    region.end = QEMU_ALIGN_PTR_DOWN(buf + size, page_size);
-    /* account for that last guard page */
-    region.end -= page_size;
-
-    /*
-     * Set guard pages in the rw buffer, as that's the one into which
-     * buffer overruns could occur.  Do not set guard pages in the rx
-     * buffer -- let that one use hugepages throughout.
-     */
-    for (i = 0; i < region.n; i++) {
-        void *start, *end;
-
-        tcg_region_bounds(i, &start, &end);
-
-        /*
-         * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
-         * rejects a permission change from RWX -> NONE.  Guard pages are
-         * nice for bug detection but are not essential; ignore any failure.
-         */
-        (void)qemu_mprotect_none(end, page_size);
-    }
-
-    tcg_region_trees_init();
-
-    /*
-     * Leave the initial context initialized to the first region.
-     * This will be the context into which we generate the prologue.
-     * It is also the only context for CONFIG_USER_ONLY.
-     */
-    tcg_region_initial_alloc__locked(&tcg_init_ctx);
-}
-
-static void tcg_region_prologue_set(TCGContext *s)
-{
-    /* Deduct the prologue from the first region.  */
-    g_assert(region.start == s->code_gen_buffer);
-    region.start = s->code_ptr;
-
-    /* Recompute boundaries of the first region. */
-    tcg_region_assign(s, 0);
-
-    /* Register the balance of the buffer with gdb. */
-    tcg_register_jit(tcg_splitwx_to_rx(region.start),
-                     region.end - region.start);
-}
-
 #ifdef CONFIG_DEBUG_TCG
 const void *tcg_splitwx_to_rx(void *rw)
 {
@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
 }
 #endif /* !CONFIG_USER_ONLY */
 
-/*
- * Returns the size (in bytes) of all translated code (i.e. from all regions)
- * currently in the cache.
- * See also: tcg_code_capacity()
- * Do not confuse with tcg_current_code_size(); that one applies to a single
- * TCG context.
- */
-size_t tcg_code_size(void)
-{
-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
-    unsigned int i;
-    size_t total;
-
-    qemu_mutex_lock(&region.lock);
-    total = region.agg_size_full;
-    for (i = 0; i < n_ctxs; i++) {
-        const TCGContext *s = qatomic_read(&tcg_ctxs[i]);
-        size_t size;
-
-        size = qatomic_read(&s->code_gen_ptr) - s->code_gen_buffer;
-        g_assert(size <= s->code_gen_buffer_size);
-        total += size;
-    }
-    qemu_mutex_unlock(&region.lock);
-    return total;
-}
-
-/*
- * Returns the code capacity (in bytes) of the entire cache, i.e. including all
- * regions.
- * See also: tcg_code_size()
- */
-size_t tcg_code_capacity(void)
-{
-    size_t guard_size, capacity;
-
-    /* no need for synchronization; these variables are set at init time */
-    guard_size = region.stride - region.size;
-    capacity = region.end + guard_size - region.start;
-    capacity -= region.n * (guard_size + TCG_HIGHWATER);
-    return capacity;
-}
-
-size_t tcg_tb_phys_invalidate_count(void)
-{
-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
-    unsigned int i;
-    size_t total = 0;
-
-    for (i = 0; i < n_ctxs; i++) {
-        const TCGContext *s = qatomic_read(&tcg_ctxs[i]);
-
-        total += qatomic_read(&s->tb_phys_invalidate_count);
-    }
-    return total;
-}
-
 /* pool based memory allocation */
 void *tcg_malloc_internal(TCGContext *s, int size)
 {
diff --git a/tcg/meson.build b/tcg/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/tcg/meson.build
+++ b/tcg/meson.build
@@ -XXX,XX +XXX,XX @@ tcg_ss = ss.source_set()
 
 tcg_ss.add(files(
   'optimize.c',
+  'region.c',
   'tcg.c',
   'tcg-common.c',
   'tcg-op.c',
-- 
2.25.1

It consists of one function call and has only one caller.

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static void page_table_config_init(void)
     assert(v_l2_levels >= 0);
 }
 
-static void cpu_gen_init(void)
-{
-    tcg_context_init(&tcg_init_ctx);
-}
-
 /* Encode VAL as a signed leb128 sequence at P.
    Return P incremented past the encoded value.  */
 static uint8_t *encode_sleb128(uint8_t *p, target_long val)
@@ -XXX,XX +XXX,XX @@ void tcg_exec_init(unsigned long tb_size, int splitwx)
     bool ok;
 
     tcg_allowed = true;
-    cpu_gen_init();
+    tcg_context_init(&tcg_init_ctx);
     page_init();
     tb_htable_init();
 
-- 
2.25.1

Buffer management is integral to tcg.  Do not leave the allocation
to code outside of tcg/.  This is code movement, with further
cleanups to follow.

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg.h         |   2 +-
 accel/tcg/translate-all.c | 414 +-----------------------------------
 tcg/region.c              | 431 +++++++++++++++++++++++++++++++++++++-
 3 files changed, 428 insertions(+), 419 deletions(-)

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ void *tcg_malloc_internal(TCGContext *s, int size);
 void tcg_pool_reset(TCGContext *s);
 TranslationBlock *tcg_tb_alloc(TCGContext *s);
 
-void tcg_region_init(void);
+void tcg_region_init(size_t tb_size, int splitwx);
 void tb_destroy(TranslationBlock *tb);
 void tcg_region_reset_all(void);
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@
  */
 
 #include "qemu/osdep.h"
-#include "qemu/units.h"
 #include "qemu-common.h"
 
 #define NO_CPU_IO_DEFS
@@ -XXX,XX +XXX,XX @@
 #include "exec/cputlb.h"
 #include "exec/translate-all.h"
 #include "qemu/bitmap.h"
-#include "qemu/error-report.h"
 #include "qemu/qemu-print.h"
 #include "qemu/timer.h"
 #include "qemu/main-loop.h"
@@ -XXX,XX +XXX,XX @@ static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
     }
 }
 
-/* Minimum size of the code gen buffer.  This number is randomly chosen,
-   but not so small that we can't have a fair number of TB's live.  */
-#define MIN_CODE_GEN_BUFFER_SIZE     (1 * MiB)
-
-/* Maximum size of the code gen buffer we'd like to use.  Unless otherwise
-   indicated, this is constrained by the range of direct branches on the
-   host cpu, as used by the TCG implementation of goto_tb.  */
-#if defined(__x86_64__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
-#elif defined(__sparc__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
-#elif defined(__powerpc64__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
-#elif defined(__powerpc__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (32 * MiB)
-#elif defined(__aarch64__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
-#elif defined(__s390x__)
-  /* We have a +- 4GB range on the branches; leave some slop.  */
-# define MAX_CODE_GEN_BUFFER_SIZE  (3 * GiB)
-#elif defined(__mips__)
-  /* We have a 256MB branch region, but leave room to make sure the
-     main executable is also within that region.  */
-# define MAX_CODE_GEN_BUFFER_SIZE  (128 * MiB)
-#else
-# define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
-#endif
-
-#if TCG_TARGET_REG_BITS == 32
-#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (32 * MiB)
-#ifdef CONFIG_USER_ONLY
-/*
- * For user mode on smaller 32 bit systems we may run into trouble
- * allocating big chunks of data in the right place. On these systems
- * we utilise a static code generation buffer directly in the binary.
- */
-#define USE_STATIC_CODE_GEN_BUFFER
-#endif
-#else /* TCG_TARGET_REG_BITS == 64 */
-#ifdef CONFIG_USER_ONLY
-/*
- * As user-mode emulation typically means running multiple instances
- * of the translator don't go too nuts with our default code gen
- * buffer lest we make things too hard for the OS.
- */
-#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (128 * MiB)
-#else
-/*
- * We expect most system emulation to run one or two guests per host.
- * Users running large scale system emulation may want to tweak their
- * runtime setup via the tb-size control on the command line.
- */
-#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (1 * GiB)
-#endif
-#endif
-
-#define DEFAULT_CODE_GEN_BUFFER_SIZE \
-  (DEFAULT_CODE_GEN_BUFFER_SIZE_1 < MAX_CODE_GEN_BUFFER_SIZE \
-   ? DEFAULT_CODE_GEN_BUFFER_SIZE_1 : MAX_CODE_GEN_BUFFER_SIZE)
-
-static size_t size_code_gen_buffer(size_t tb_size)
-{
-    /* Size the buffer.  */
-    if (tb_size == 0) {
-        size_t phys_mem = qemu_get_host_physmem();
-        if (phys_mem == 0) {
-            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
-        } else {
-            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
-        }
-    }
-    if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
-        tb_size = MIN_CODE_GEN_BUFFER_SIZE;
-    }
-    if (tb_size > MAX_CODE_GEN_BUFFER_SIZE) {
-        tb_size = MAX_CODE_GEN_BUFFER_SIZE;
-    }
-    return tb_size;
-}
-
-#ifdef __mips__
-/* In order to use J and JAL within the code_gen_buffer, we require
-   that the buffer not cross a 256MB boundary.  */
-static inline bool cross_256mb(void *addr, size_t size)
-{
-    return ((uintptr_t)addr ^ ((uintptr_t)addr + size)) & ~0x0ffffffful;
-}
-
-/* We weren't able to allocate a buffer without crossing that boundary,
-   so make do with the larger portion of the buffer that doesn't cross.
-   Returns the new base of the buffer, and adjusts code_gen_buffer_size.  */
-static inline void *split_cross_256mb(void *buf1, size_t size1)
-{
-    void *buf2 = (void *)(((uintptr_t)buf1 + size1) & ~0x0ffffffful);
-    size_t size2 = buf1 + size1 - buf2;
-
-    size1 = buf2 - buf1;
-    if (size1 < size2) {
-        size1 = size2;
-        buf1 = buf2;
-    }
-
-    tcg_ctx->code_gen_buffer_size = size1;
-    return buf1;
-}
-#endif
-
-#ifdef USE_STATIC_CODE_GEN_BUFFER
-static uint8_t static_code_gen_buffer[DEFAULT_CODE_GEN_BUFFER_SIZE]
-    __attribute__((aligned(CODE_GEN_ALIGN)));
-
-static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
-{
-    void *buf, *end;
-    size_t size;
-
-    if (splitwx > 0) {
-        error_setg(errp, "jit split-wx not supported");
-        return false;
-    }
-
-    /* page-align the beginning and end of the buffer */
-    buf = static_code_gen_buffer;
-    end = static_code_gen_buffer + sizeof(static_code_gen_buffer);
-    buf = QEMU_ALIGN_PTR_UP(buf, qemu_real_host_page_size);
-    end = QEMU_ALIGN_PTR_DOWN(end, qemu_real_host_page_size);
-
-    size = end - buf;
-
-    /* Honor a command-line option limiting the size of the buffer.  */
-    if (size > tb_size) {
-        size = QEMU_ALIGN_DOWN(tb_size, qemu_real_host_page_size);
-    }
-    tcg_ctx->code_gen_buffer_size = size;
-
-#ifdef __mips__
-    if (cross_256mb(buf, size)) {
-        buf = split_cross_256mb(buf, size);
-        size = tcg_ctx->code_gen_buffer_size;
-    }
-#endif
-
-    if (qemu_mprotect_rwx(buf, size)) {
-        error_setg_errno(errp, errno, "mprotect of jit buffer");
-        return false;
-    }
-    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
-
-    tcg_ctx->code_gen_buffer = buf;
-    return true;
-}
-#elif defined(_WIN32)
-static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
-{
-    void *buf;
-
-    if (splitwx > 0) {
-        error_setg(errp, "jit split-wx not supported");
-        return false;
-    }
-
-    buf = VirtualAlloc(NULL, size, MEM_RESERVE | MEM_COMMIT,
-                             PAGE_EXECUTE_READWRITE);
-    if (buf == NULL) {
-        error_setg_win32(errp, GetLastError(),
-                         "allocate %zu bytes for jit buffer", size);
-        return false;
-    }
-
-    tcg_ctx->code_gen_buffer = buf;
-    tcg_ctx->code_gen_buffer_size = size;
-    return true;
-}
-#else
-static bool alloc_code_gen_buffer_anon(size_t size, int prot,
-                                       int flags, Error **errp)
-{
-    void *buf;
-
-    buf = mmap(NULL, size, prot, flags, -1, 0);
-    if (buf == MAP_FAILED) {
-        error_setg_errno(errp, errno,
-                         "allocate %zu bytes for jit buffer", size);
-        return false;
-    }
-    tcg_ctx->code_gen_buffer_size = size;
-
-#ifdef __mips__
-    if (cross_256mb(buf, size)) {
-        /*
-         * Try again, with the original still mapped, to avoid re-acquiring
-         * the same 256mb crossing.
-         */
-        size_t size2;
-        void *buf2 = mmap(NULL, size, prot, flags, -1, 0);
-        switch ((int)(buf2 != MAP_FAILED)) {
-        case 1:
-            if (!cross_256mb(buf2, size)) {
-                /* Success!  Use the new buffer.  */
-                munmap(buf, size);
-                break;
-            }
-            /* Failure.  Work with what we had.  */
-            munmap(buf2, size);
-            /* fallthru */
-        default:
-            /* Split the original buffer.  Free the smaller half.  */
-            buf2 = split_cross_256mb(buf, size);
-            size2 = tcg_ctx->code_gen_buffer_size;
-            if (buf == buf2) {
-                munmap(buf + size2, size - size2);
-            } else {
-                munmap(buf, size - size2);
-            }
-            size = size2;
-            break;
-        }
-        buf = buf2;
-    }
-#endif
-
-    /* Request large pages for the buffer.  */
-    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
-
-    tcg_ctx->code_gen_buffer = buf;
-    return true;
-}
-
-#ifndef CONFIG_TCG_INTERPRETER
-#ifdef CONFIG_POSIX
-#include "qemu/memfd.h"
-
-static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
-{
-    void *buf_rw = NULL, *buf_rx = MAP_FAILED;
-    int fd = -1;
-
-#ifdef __mips__
-    /* Find space for the RX mapping, vs the 256MiB regions. */
-    if (!alloc_code_gen_buffer_anon(size, PROT_NONE,
-                                    MAP_PRIVATE | MAP_ANONYMOUS |
-                                    MAP_NORESERVE, errp)) {
-        return false;
-    }
-    /* The size of the mapping may have been adjusted. */
-    size = tcg_ctx->code_gen_buffer_size;
-    buf_rx = tcg_ctx->code_gen_buffer;
-#endif
-
-    buf_rw = qemu_memfd_alloc("tcg-jit", size, 0, &fd, errp);
-    if (buf_rw == NULL) {
-        goto fail;
-    }
-
-#ifdef __mips__
-    void *tmp = mmap(buf_rx, size, PROT_READ | PROT_EXEC,
-                     MAP_SHARED | MAP_FIXED, fd, 0);
-    if (tmp != buf_rx) {
-        goto fail_rx;
-    }
-#else
-    buf_rx = mmap(NULL, size, PROT_READ | PROT_EXEC, MAP_SHARED, fd, 0);
-    if (buf_rx == MAP_FAILED) {
-        goto fail_rx;
-    }
-#endif
-
-    close(fd);
-    tcg_ctx->code_gen_buffer = buf_rw;
-    tcg_ctx->code_gen_buffer_size = size;
-    tcg_splitwx_diff = buf_rx - buf_rw;
-
-    /* Request large pages for the buffer and the splitwx.  */
-    qemu_madvise(buf_rw, size, QEMU_MADV_HUGEPAGE);
-    qemu_madvise(buf_rx, size, QEMU_MADV_HUGEPAGE);
-    return true;
-
- fail_rx:
-    error_setg_errno(errp, errno, "failed to map shared memory for execute");
- fail:
-    if (buf_rx != MAP_FAILED) {
-        munmap(buf_rx, size);
-    }
-    if (buf_rw) {
-        munmap(buf_rw, size);
-    }
-    if (fd >= 0) {
-        close(fd);
-    }
-    return false;
-}
-#endif /* CONFIG_POSIX */
-
-#ifdef CONFIG_DARWIN
-#include <mach/mach.h>
-
-extern kern_return_t mach_vm_remap(vm_map_t target_task,
-                                   mach_vm_address_t *target_address,
-                                   mach_vm_size_t size,
-                                   mach_vm_offset_t mask,
-                                   int flags,
-                                   vm_map_t src_task,
-                                   mach_vm_address_t src_address,
-                                   boolean_t copy,
-                                   vm_prot_t *cur_protection,
-                                   vm_prot_t *max_protection,
-                                   vm_inherit_t inheritance);
-
-static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
-{
-    kern_return_t ret;
-    mach_vm_address_t buf_rw, buf_rx;
-    vm_prot_t cur_prot, max_prot;
-
-    /* Map the read-write portion via normal anon memory. */
-    if (!alloc_code_gen_buffer_anon(size, PROT_READ | PROT_WRITE,
-                                    MAP_PRIVATE | MAP_ANONYMOUS, errp)) {
-        return false;
-    }
-
-    buf_rw = (mach_vm_address_t)tcg_ctx->code_gen_buffer;
-    buf_rx = 0;
-    ret = mach_vm_remap(mach_task_self(),
-                        &buf_rx,
-                        size,
-                        0,
-                        VM_FLAGS_ANYWHERE,
-                        mach_task_self(),
-                        buf_rw,
-                        false,
-                        &cur_prot,
-                        &max_prot,
-                        VM_INHERIT_NONE);
-    if (ret != KERN_SUCCESS) {
-        /* TODO: Convert "ret" to a human readable error message. */
-        error_setg(errp, "vm_remap for jit splitwx failed");
-        munmap((void *)buf_rw, size);
-        return false;
-    }
-
-    if (mprotect((void *)buf_rx, size, PROT_READ | PROT_EXEC) != 0) {
-        error_setg_errno(errp, errno, "mprotect for jit splitwx");
-        munmap((void *)buf_rx, size);
-        munmap((void *)buf_rw, size);
-        return false;
-    }
-
-    tcg_splitwx_diff = buf_rx - buf_rw;
-    return true;
-}
-#endif /* CONFIG_DARWIN */
-#endif /* CONFIG_TCG_INTERPRETER */
-
-static bool alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
-{
-#ifndef CONFIG_TCG_INTERPRETER
-# ifdef CONFIG_DARWIN
-    return alloc_code_gen_buffer_splitwx_vmremap(size, errp);
-# endif
-# ifdef CONFIG_POSIX
-    return alloc_code_gen_buffer_splitwx_memfd(size, errp);
-# endif
-#endif
-    error_setg(errp, "jit split-wx not supported");
-    return false;
-}
-
-static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
-{
-    ERRP_GUARD();
-    int prot, flags;
-
-    if (splitwx) {
-        if (alloc_code_gen_buffer_splitwx(size, errp)) {
-            return true;
-        }
-        /*
-         * If splitwx force-on (1), fail;
-         * if splitwx default-on (-1), fall through to splitwx off.
-         */
-        if (splitwx > 0) {
-            return false;
-        }
-        error_free_or_abort(errp);
-    }
-
-    prot = PROT_READ | PROT_WRITE | PROT_EXEC;
-    flags = MAP_PRIVATE | MAP_ANONYMOUS;
-#ifdef CONFIG_TCG_INTERPRETER
-    /* The tcg interpreter does not need execute permission. */
-    prot = PROT_READ | PROT_WRITE;
-#elif defined(CONFIG_DARWIN)
-    /* Applicable to both iOS and macOS (Apple Silicon). */
-    if (!splitwx) {
-        flags |= MAP_JIT;
-    }
-#endif
-
-    return alloc_code_gen_buffer_anon(size, prot, flags, errp);
-}
-#endif /* USE_STATIC_CODE_GEN_BUFFER, WIN32, POSIX */
-
 static bool tb_cmp(const void *ap, const void *bp)
 {
     const TranslationBlock *a = ap;
@@ -XXX,XX +XXX,XX @@ static void tb_htable_init(void)
    size. */
 void tcg_exec_init(unsigned long tb_size, int splitwx)
 {
-    bool ok;
-
     tcg_allowed = true;
     tcg_context_init(&tcg_init_ctx);
     page_init();
     tb_htable_init();
-
-    ok = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
-                               splitwx, &error_fatal);
-    assert(ok);
-
-    /* TODO: allocating regions is hand-in-glove with code_gen_buffer. */
-    tcg_region_init();
+    tcg_region_init(tb_size, splitwx);
 
 #if defined(CONFIG_SOFTMMU)
     /* There's no guest base to take into account, so go ahead and
diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/units.h"
+#include "qapi/error.h"
 #include "exec/exec-all.h"
 #include "tcg/tcg.h"
 #if !defined(CONFIG_USER_ONLY)
@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(void)
 }
 #endif
 
+/*
+ * Minimum size of the code gen buffer.  This number is randomly chosen,
+ * but not so small that we can't have a fair number of TB's live.
+ */
+#define MIN_CODE_GEN_BUFFER_SIZE     (1 * MiB)
+
+/*
+ * Maximum size of the code gen buffer we'd like to use.  Unless otherwise
+ * indicated, this is constrained by the range of direct branches on the
+ * host cpu, as used by the TCG implementation of goto_tb.
+ */
+#if defined(__x86_64__)
+# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+#elif defined(__sparc__)
+# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+#elif defined(__powerpc64__)
+# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+#elif defined(__powerpc__)
+# define MAX_CODE_GEN_BUFFER_SIZE  (32 * MiB)
+#elif defined(__aarch64__)
+# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
+#elif defined(__s390x__)
+  /* We have a +- 4GB range on the branches; leave some slop.  */
+# define MAX_CODE_GEN_BUFFER_SIZE  (3 * GiB)
+#elif defined(__mips__)
+  /*
+   * We have a 256MB branch region, but leave room to make sure the
+   * main executable is also within that region.
+   */
+# define MAX_CODE_GEN_BUFFER_SIZE  (128 * MiB)
+#else
+# define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
+#endif
+
+#if TCG_TARGET_REG_BITS == 32
+#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (32 * MiB)
+#ifdef CONFIG_USER_ONLY
+/*
+ * For user mode on smaller 32 bit systems we may run into trouble
+ * allocating big chunks of data in the right place. On these systems
+ * we utilise a static code generation buffer directly in the binary.
+ */
+#define USE_STATIC_CODE_GEN_BUFFER
+#endif
+#else /* TCG_TARGET_REG_BITS == 64 */
+#ifdef CONFIG_USER_ONLY
+/*
+ * As user-mode emulation typically means running multiple instances
+ * of the translator don't go too nuts with our default code gen
+ * buffer lest we make things too hard for the OS.
+ */
+#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (128 * MiB)
+#else
+/*
+ * We expect most system emulation to run one or two guests per host.
+ * Users running large scale system emulation may want to tweak their
+ * runtime setup via the tb-size control on the command line.
+ */
+#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (1 * GiB)
+#endif
+#endif
+
+#define DEFAULT_CODE_GEN_BUFFER_SIZE \
+  (DEFAULT_CODE_GEN_BUFFER_SIZE_1 < MAX_CODE_GEN_BUFFER_SIZE \
+   ? DEFAULT_CODE_GEN_BUFFER_SIZE_1 : MAX_CODE_GEN_BUFFER_SIZE)
+
+static size_t size_code_gen_buffer(size_t tb_size)
+{
+    /* Size the buffer.  */
+    if (tb_size == 0) {
+        size_t phys_mem = qemu_get_host_physmem();
+        if (phys_mem == 0) {
+            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+        } else {
+            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
+        }
+    }
+    if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
+        tb_size = MIN_CODE_GEN_BUFFER_SIZE;
+    }
+    if (tb_size > MAX_CODE_GEN_BUFFER_SIZE) {
+        tb_size = MAX_CODE_GEN_BUFFER_SIZE;
+    }
+    return tb_size;
+}
+
+#ifdef __mips__
+/*
+ * In order to use J and JAL within the code_gen_buffer, we require
+ * that the buffer not cross a 256MB boundary.
+ */
+static inline bool cross_256mb(void *addr, size_t size)
+{
+    return ((uintptr_t)addr ^ ((uintptr_t)addr + size)) & ~0x0ffffffful;
+}
+
+/*
+ * We weren't able to allocate a buffer without crossing that boundary,
+ * so make do with the larger portion of the buffer that doesn't cross.
+ * Returns the new base of the buffer, and adjusts code_gen_buffer_size.
+ */
+static inline void *split_cross_256mb(void *buf1, size_t size1)
+{
+    void *buf2 = (void *)(((uintptr_t)buf1 + size1) & ~0x0ffffffful);
+    size_t size2 = buf1 + size1 - buf2;
+
+    size1 = buf2 - buf1;
+    if (size1 < size2) {
+        size1 = size2;
+        buf1 = buf2;
+    }
+
+    tcg_ctx->code_gen_buffer_size = size1;
+    return buf1;
+}
+#endif
+
+#ifdef USE_STATIC_CODE_GEN_BUFFER
+static uint8_t static_code_gen_buffer[DEFAULT_CODE_GEN_BUFFER_SIZE]
+    __attribute__((aligned(CODE_GEN_ALIGN)));
+
+static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
+{
+    void *buf, *end;
+    size_t size;
+
+    if (splitwx > 0) {
+        error_setg(errp, "jit split-wx not supported");
+        return false;
+    }
+
+    /* page-align the beginning and end of the buffer */
+    buf = static_code_gen_buffer;
+    end = static_code_gen_buffer + sizeof(static_code_gen_buffer);
+    buf = QEMU_ALIGN_PTR_UP(buf, qemu_real_host_page_size);
+    end = QEMU_ALIGN_PTR_DOWN(end, qemu_real_host_page_size);
+
+    size = end - buf;
+
+    /* Honor a command-line option limiting the size of the buffer.  */
+    if (size > tb_size) {
+        size = QEMU_ALIGN_DOWN(tb_size, qemu_real_host_page_size);
+    }
+    tcg_ctx->code_gen_buffer_size = size;
+
+#ifdef __mips__
+    if (cross_256mb(buf, size)) {
+        buf = split_cross_256mb(buf, size);
+        size = tcg_ctx->code_gen_buffer_size;
+    }
+#endif
+
+    if (qemu_mprotect_rwx(buf, size)) {
+        error_setg_errno(errp, errno, "mprotect of jit buffer");
+        return false;
+    }
+    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+
+    tcg_ctx->code_gen_buffer = buf;
+    return true;
+}
+#elif defined(_WIN32)
+static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+{
+    void *buf;
+
+    if (splitwx > 0) {
+        error_setg(errp, "jit split-wx not supported");
+        return false;
+    }
+
+    buf = VirtualAlloc(NULL, size, MEM_RESERVE | MEM_COMMIT,
+                             PAGE_EXECUTE_READWRITE);
+    if (buf == NULL) {
+        error_setg_win32(errp, GetLastError(),
+                         "allocate %zu bytes for jit buffer", size);
+        return false;
+    }
+
+    tcg_ctx->code_gen_buffer = buf;
+    tcg_ctx->code_gen_buffer_size = size;
+    return true;
+}
+#else
+static bool alloc_code_gen_buffer_anon(size_t size, int prot,
+                                       int flags, Error **errp)
+{
+    void *buf;
+
+    buf = mmap(NULL, size, prot, flags, -1, 0);
+    if (buf == MAP_FAILED) {
+        error_setg_errno(errp, errno,
+                         "allocate %zu bytes for jit buffer", size);
+        return false;
+    }
+    tcg_ctx->code_gen_buffer_size = size;
+
+#ifdef __mips__
+    if (cross_256mb(buf, size)) {
+        /*
+         * Try again, with the original still mapped, to avoid re-acquiring
+         * the same 256mb crossing.
+         */
+        size_t size2;
+        void *buf2 = mmap(NULL, size, prot, flags, -1, 0);
+        switch ((int)(buf2 != MAP_FAILED)) {
+        case 1:
+            if (!cross_256mb(buf2, size)) {
+                /* Success!  Use the new buffer.  */
+                munmap(buf, size);
+                break;
+            }
+            /* Failure.  Work with what we had.  */
+            munmap(buf2, size);
+            /* fallthru */
+        default:
+            /* Split the original buffer.  Free the smaller half.  */
+            buf2 = split_cross_256mb(buf, size);
+            size2 = tcg_ctx->code_gen_buffer_size;
+            if (buf == buf2) {
+                munmap(buf + size2, size - size2);
+            } else {
+                munmap(buf, size - size2);
+            }
+            size = size2;
+            break;
+        }
+        buf = buf2;
+    }
+#endif
+
+    /* Request large pages for the buffer.  */
+    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
+
+    tcg_ctx->code_gen_buffer = buf;
+    return true;
+}
+
+#ifndef CONFIG_TCG_INTERPRETER
+#ifdef CONFIG_POSIX
+#include "qemu/memfd.h"
+
+static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
+{
+    void *buf_rw = NULL, *buf_rx = MAP_FAILED;
+    int fd = -1;
+
+#ifdef __mips__
+    /* Find space for the RX mapping, vs the 256MiB regions. */
+    if (!alloc_code_gen_buffer_anon(size, PROT_NONE,
+                                    MAP_PRIVATE | MAP_ANONYMOUS |
+                                    MAP_NORESERVE, errp)) {
+        return false;
+    }
+    /* The size of the mapping may have been adjusted. */
+    size = tcg_ctx->code_gen_buffer_size;
+    buf_rx = tcg_ctx->code_gen_buffer;
+#endif
+
+    buf_rw = qemu_memfd_alloc("tcg-jit", size, 0, &fd, errp);
+    if (buf_rw == NULL) {
+        goto fail;
+    }
+
+#ifdef __mips__
+    void *tmp = mmap(buf_rx, size, PROT_READ | PROT_EXEC,
+                     MAP_SHARED | MAP_FIXED, fd, 0);
+    if (tmp != buf_rx) {
+        goto fail_rx;
+    }
+#else
+    buf_rx = mmap(NULL, size, PROT_READ | PROT_EXEC, MAP_SHARED, fd, 0);
+    if (buf_rx == MAP_FAILED) {
+        goto fail_rx;
+    }
+#endif
+
+    close(fd);
+    tcg_ctx->code_gen_buffer = buf_rw;
+    tcg_ctx->code_gen_buffer_size = size;
+    tcg_splitwx_diff = buf_rx - buf_rw;
+
+    /* Request large pages for the buffer and the splitwx.  */
+    qemu_madvise(buf_rw, size, QEMU_MADV_HUGEPAGE);
+    qemu_madvise(buf_rx, size, QEMU_MADV_HUGEPAGE);
+    return true;
+
+ fail_rx:
+    error_setg_errno(errp, errno, "failed to map shared memory for execute");
+ fail:
+    if (buf_rx != MAP_FAILED) {
+        munmap(buf_rx, size);
+    }
+    if (buf_rw) {
+        munmap(buf_rw, size);
+    }
+    if (fd >= 0) {
+        close(fd);
+    }
+    return false;
+}
+#endif /* CONFIG_POSIX */
+
+#ifdef CONFIG_DARWIN
+#include <mach/mach.h>
+
+extern kern_return_t mach_vm_remap(vm_map_t target_task,
+                                   mach_vm_address_t *target_address,
+                                   mach_vm_size_t size,
+                                   mach_vm_offset_t mask,
+                                   int flags,
+                                   vm_map_t src_task,
+                                   mach_vm_address_t src_address,
+                                   boolean_t copy,
+                                   vm_prot_t *cur_protection,
+                                   vm_prot_t *max_protection,
+                                   vm_inherit_t inheritance);
+
+static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
+{
+    kern_return_t ret;
+    mach_vm_address_t buf_rw, buf_rx;
+    vm_prot_t cur_prot, max_prot;
+
+    /* Map the read-write portion via normal anon memory. */
+    if (!alloc_code_gen_buffer_anon(size, PROT_READ | PROT_WRITE,
+                                    MAP_PRIVATE | MAP_ANONYMOUS, errp)) {
+        return false;
+    }
+
+    buf_rw = (mach_vm_address_t)tcg_ctx->code_gen_buffer;
+    buf_rx = 0;
+    ret = mach_vm_remap(mach_task_self(),
+                        &buf_rx,
+                        size,
+                        0,
+                        VM_FLAGS_ANYWHERE,
+                        mach_task_self(),
+                        buf_rw,
+                        false,
+                        &cur_prot,
+                        &max_prot,
+                        VM_INHERIT_NONE);
+    if (ret != KERN_SUCCESS) {
+        /* TODO: Convert "ret" to a human readable error message. */
+        error_setg(errp, "vm_remap for jit splitwx failed");
+        munmap((void *)buf_rw, size);
+        return false;
+    }
+
+    if (mprotect((void *)buf_rx, size, PROT_READ | PROT_EXEC) != 0) {
+        error_setg_errno(errp, errno, "mprotect for jit splitwx");
+        munmap((void *)buf_rx, size);
+        munmap((void *)buf_rw, size);
+        return false;
+    }
+
+    tcg_splitwx_diff = buf_rx - buf_rw;
+    return true;
+}
+#endif /* CONFIG_DARWIN */
+#endif /* CONFIG_TCG_INTERPRETER */
+
+static bool alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
+{
+#ifndef CONFIG_TCG_INTERPRETER
+# ifdef CONFIG_DARWIN
+    return alloc_code_gen_buffer_splitwx_vmremap(size, errp);
+# endif
+# ifdef CONFIG_POSIX
+    return alloc_code_gen_buffer_splitwx_memfd(size, errp);
+# endif
+#endif
+    error_setg(errp, "jit split-wx not supported");
+    return false;
+}
+
+static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+{
+    ERRP_GUARD();
+    int prot, flags;
+
+    if (splitwx) {
+        if (alloc_code_gen_buffer_splitwx(size, errp)) {
+            return true;
+        }
+        /*
+         * If splitwx force-on (1), fail;
+         * if splitwx default-on (-1), fall through to splitwx off.
+         */
+        if (splitwx > 0) {
+            return false;
+        }
+        error_free_or_abort(errp);
+    }
+
+    prot = PROT_READ | PROT_WRITE | PROT_EXEC;
+    flags = MAP_PRIVATE | MAP_ANONYMOUS;
+#ifdef CONFIG_TCG_INTERPRETER
+    /* The tcg interpreter does not need execute permission. */
+    prot = PROT_READ | PROT_WRITE;
+#elif defined(CONFIG_DARWIN)
+    /* Applicable to both iOS and macOS (Apple Silicon). */
+    if (!splitwx) {
+        flags |= MAP_JIT;
+    }
+#endif
+
+    return alloc_code_gen_buffer_anon(size, prot, flags, errp);
+}
+#endif /* USE_STATIC_CODE_GEN_BUFFER, WIN32, POSIX */
+
 /*
  * Initializes region partitioning.
  *
@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(void)
  * in practice. Multi-threaded guests share most if not all of their translated
  * code, which makes parallel code generation less appealing than in softmmu.
  */
-void tcg_region_init(void)
+void tcg_region_init(size_t tb_size, int splitwx)
 {
-    void *buf = tcg_init_ctx.code_gen_buffer;
-    void *aligned;
-    size_t size = tcg_init_ctx.code_gen_buffer_size;
-    size_t page_size = qemu_real_host_page_size;
+    void *buf, *aligned;
+    size_t size;
+    size_t page_size;
     size_t region_size;
     size_t n_regions;
     size_t i;
+    bool ok;
 
+    ok = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
+                               splitwx, &error_fatal);
+    assert(ok);
+
+    buf = tcg_init_ctx.code_gen_buffer;
+    size = tcg_init_ctx.code_gen_buffer_size;
+    page_size = qemu_real_host_page_size;
     n_regions = tcg_n_regions();
 
     /* The first region will be 'aligned - buf' bytes larger than the others */
-- 
2.25.1

We shortly want to use tcg_init for something else.
Since the hook is called init_machine, match that.

diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -XXX,XX +XXX,XX @@ static void tcg_accel_instance_init(Object *obj)
 
 bool mttcg_enabled;
 
-static int tcg_init(MachineState *ms)
+static int tcg_init_machine(MachineState *ms)
 {
     TCGState *s = TCG_STATE(current_accel());
 
@@ -XXX,XX +XXX,XX @@ static void tcg_accel_class_init(ObjectClass *oc, void *data)
 {
     AccelClass *ac = ACCEL_CLASS(oc);
     ac->name = "tcg";
-    ac->init_machine = tcg_init;
+    ac->init_machine = tcg_init_machine;
     ac->allowed = &tcg_allowed;
 
     object_class_property_add_str(oc, "thread",
-- 
2.25.1

Perform both tcg_context_init and tcg_region_init.
Do not leave this split to the caller.

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg.h         | 3 +--
 tcg/tcg-internal.h        | 1 +
 accel/tcg/translate-all.c | 3 +--
 tcg/tcg.c                 | 9 ++++++++-
 4 files changed, 11 insertions(+), 5 deletions(-)

There is only one caller, and shortly we will need access
to the MachineState, which tcg_init_machine already has.

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/internal.h      |  2 ++
 include/sysemu/tcg.h      |  2 --
 accel/tcg/tcg-all.c       | 16 +++++++++++++++-
 accel/tcg/translate-all.c | 21 ++-------------------
 bsd-user/main.c           |  2 +-
 5 files changed, 20 insertions(+), 23 deletions(-)

diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/internal.h
+++ b/accel/tcg/internal.h
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu, target_ulong pc,
                               int cflags);
 
 void QEMU_NORETURN cpu_io_recompile(CPUState *cpu, uintptr_t retaddr);
+void page_init(void);
+void tb_htable_init(void);
 
 #endif /* ACCEL_TCG_INTERNAL_H */
diff --git a/include/sysemu/tcg.h b/include/sysemu/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/sysemu/tcg.h
+++ b/include/sysemu/tcg.h
@@ -XXX,XX +XXX,XX @@
 #ifndef SYSEMU_TCG_H
 #define SYSEMU_TCG_H
 
-void tcg_exec_init(unsigned long tb_size, int splitwx);
-
 #ifdef CONFIG_TCG
 extern bool tcg_allowed;
 #define tcg_enabled() (tcg_allowed)
diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -XXX,XX +XXX,XX @@
 #include "qemu/error-report.h"
 #include "qemu/accel.h"
 #include "qapi/qapi-builtin-visit.h"
+#include "internal.h"
 
 struct TCGState {
     AccelState parent_obj;
@@ -XXX,XX +XXX,XX @@ static int tcg_init_machine(MachineState *ms)
 {
     TCGState *s = TCG_STATE(current_accel());
 
-    tcg_exec_init(s->tb_size * 1024 * 1024, s->splitwx_enabled);
+    tcg_allowed = true;
     mttcg_enabled = s->mttcg_enabled;
+
+    page_init();
+    tb_htable_init();
+    tcg_init(s->tb_size * 1024 * 1024, s->splitwx_enabled);
+
+#if defined(CONFIG_SOFTMMU)
+    /*
+     * There's no guest base to take into account, so go ahead and
+     * initialize the prologue now.
+     */
+    tcg_prologue_init(tcg_ctx);
+#endif
+
     return 0;
 }
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ bool cpu_restore_state(CPUState *cpu, uintptr_t host_pc, bool will_exit)
     return false;
 }
 
-static void page_init(void)
+void page_init(void)
 {
     page_size_init();
     page_table_config_init();
@@ -XXX,XX +XXX,XX @@ static bool tb_cmp(const void *ap, const void *bp)
         a->page_addr[1] == b->page_addr[1];
 }
 
-static void tb_htable_init(void)
+void tb_htable_init(void)
 {
     unsigned int mode = QHT_MODE_AUTO_RESIZE;
 
     qht_init(&tb_ctx.htable, tb_cmp, CODE_GEN_HTABLE_SIZE, mode);
 }
 
-/* Must be called before using the QEMU cpus. 'tb_size' is the size
-   (in bytes) allocated to the translation buffer. Zero means default
-   size. */
-void tcg_exec_init(unsigned long tb_size, int splitwx)
-{
-    tcg_allowed = true;
-    page_init();
-    tb_htable_init();
-    tcg_init(tb_size, splitwx);
-
-#if defined(CONFIG_SOFTMMU)
-    /* There's no guest base to take into account, so go ahead and
-       initialize the prologue now.  */
-    tcg_prologue_init(tcg_ctx);
-#endif
-}
-
 /* call with @p->lock held */
 static inline void invalidate_page_bitmap(PageDesc *p)
 {
diff --git a/bsd-user/main.c b/bsd-user/main.c
index XXXXXXX..XXXXXXX 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
     envlist_free(envlist);
 
     /*
-     * Now that page sizes are configured in tcg_exec_init() we can do
+     * Now that page sizes are configured we can do
      * proper page alignment for guest_base.
      */
     guest_base = HOST_PAGE_ALIGN(guest_base);
-- 
2.25.1

Start removing the include of hw/boards.h from tcg/.
Pass down the max_cpus value from tcg_init_machine,
where we have the MachineState already.

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg.h   |  2 +-
 tcg/tcg-internal.h  |  2 +-
 accel/tcg/tcg-all.c | 10 +++++++++-
 tcg/region.c        | 32 +++++++++++---------------------
 tcg/tcg.c           | 10 ++++------
 5 files changed, 26 insertions(+), 30 deletions(-)

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ static inline void *tcg_malloc(int size)
     }
 }
 
-void tcg_init(size_t tb_size, int splitwx);
+void tcg_init(size_t tb_size, int splitwx, unsigned max_cpus);
 void tcg_register_thread(void);
 void tcg_prologue_init(TCGContext *s);
 void tcg_func_start(TCGContext *s);
diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-internal.h
+++ b/tcg/tcg-internal.h
@@ -XXX,XX +XXX,XX @@
 extern TCGContext **tcg_ctxs;
 extern unsigned int n_tcg_ctxs;
 
-void tcg_region_init(size_t tb_size, int splitwx);
+void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus);
 bool tcg_region_alloc(TCGContext *s);
 void tcg_region_initial_alloc(TCGContext *s);
 void tcg_region_prologue_set(TCGContext *s);
diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -XXX,XX +XXX,XX @@
 #include "qemu/accel.h"
 #include "qapi/qapi-builtin-visit.h"
 #include "qemu/units.h"
+#if !defined(CONFIG_USER_ONLY)
+#include "hw/boards.h"
+#endif
 #include "internal.h"
 
 struct TCGState {
@@ -XXX,XX +XXX,XX @@ bool mttcg_enabled;
 static int tcg_init_machine(MachineState *ms)
 {
     TCGState *s = TCG_STATE(current_accel());
+#ifdef CONFIG_USER_ONLY
+    unsigned max_cpus = 1;
+#else
+    unsigned max_cpus = ms->smp.max_cpus;
+#endif
 
     tcg_allowed = true;
     mttcg_enabled = s->mttcg_enabled;
 
     page_init();
     tb_htable_init();
-    tcg_init(s->tb_size * MiB, s->splitwx_enabled);
+    tcg_init(s->tb_size * MiB, s->splitwx_enabled, max_cpus);
 
 #if defined(CONFIG_SOFTMMU)
     /*
diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@
 #include "qapi/error.h"
 #include "exec/exec-all.h"
 #include "tcg/tcg.h"
-#if !defined(CONFIG_USER_ONLY)
-#include "hw/boards.h"
-#endif
 #include "tcg-internal.h"
 
 
@@ -XXX,XX +XXX,XX @@ void tcg_region_reset_all(void)
     tcg_region_tree_reset_all();
 }
 
+static size_t tcg_n_regions(unsigned max_cpus)
+{
 #ifdef CONFIG_USER_ONLY
-static size_t tcg_n_regions(void)
-{
     return 1;
-}
 #else
-/*
- * It is likely that some vCPUs will translate more code than others, so we
- * first try to set more regions than max_cpus, with those regions being of
- * reasonable size. If that's not possible we make do by evenly dividing
- * the code_gen_buffer among the vCPUs.
- */
-static size_t tcg_n_regions(void)
-{
+    /*
+     * It is likely that some vCPUs will translate more code than others,
+     * so we first try to set more regions than max_cpus, with those regions
+     * being of reasonable size. If that's not possible we make do by evenly
+     * dividing the code_gen_buffer among the vCPUs.
+     */
     size_t i;
 
     /* Use a single region if all we have is one vCPU thread */
-#if !defined(CONFIG_USER_ONLY)
-    MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int max_cpus = ms->smp.max_cpus;
-#endif
     if (max_cpus == 1 || !qemu_tcg_mttcg_enabled()) {
         return 1;
     }
@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(void)
     }
     /* If we can't, then just allocate one region per vCPU thread */
     return max_cpus;
-}
 #endif
+}
 
 /*
  * Minimum size of the code gen buffer.  This number is randomly chosen,
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
  * in practice. Multi-threaded guests share most if not all of their translated
  * code, which makes parallel code generation less appealing than in softmmu.
  */
-void tcg_region_init(size_t tb_size, int splitwx)
+void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
 {
     void *buf, *aligned;
     size_t size;
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx)
     buf = tcg_init_ctx.code_gen_buffer;
     size = tcg_init_ctx.code_gen_buffer_size;
     page_size = qemu_real_host_page_size;
-    n_regions = tcg_n_regions();
+    n_regions = tcg_n_regions(max_cpus);
 
     /* The first region will be 'aligned - buf' bytes larger than the others */
     aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static void process_op_defs(TCGContext *s);
 static TCGTemp *tcg_global_reg_new_internal(TCGContext *s, TCGType type,
                                             TCGReg reg, const char *name);
 
-static void tcg_context_init(void)
+static void tcg_context_init(unsigned max_cpus)
 {
     TCGContext *s = &tcg_init_ctx;
     int op, total_args, n, i;
@@ -XXX,XX +XXX,XX @@ static void tcg_context_init(void)
     tcg_ctxs = &tcg_ctx;
     n_tcg_ctxs = 1;
 #else
-    MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int max_cpus = ms->smp.max_cpus;
     tcg_ctxs = g_new(TCGContext *, max_cpus);
 #endif
 
@@ -XXX,XX +XXX,XX @@ static void tcg_context_init(void)
     cpu_env = temp_tcgv_ptr(ts);
 }
 
-void tcg_init(size_t tb_size, int splitwx)
+void tcg_init(size_t tb_size, int splitwx, unsigned max_cpus)
 {
-    tcg_context_init();
-    tcg_region_init(tb_size, splitwx);
+    tcg_context_init(max_cpus);
+    tcg_region_init(tb_size, splitwx, max_cpus);
 }
 
 /*
-- 
2.25.1

Finish the divorce of tcg/ from hw/, and do not take
the max cpu value from MachineState; just remember what
we were passed in tcg_init.

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/tcg-internal.h |  3 ++-
 tcg/region.c       |  6 +++---
 tcg/tcg.c          | 23 ++++++++++-------------
 3 files changed, 15 insertions(+), 17 deletions(-)

diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-internal.h
+++ b/tcg/tcg-internal.h
@@ -XXX,XX +XXX,XX @@
 #define TCG_HIGHWATER 1024
 
 extern TCGContext **tcg_ctxs;
-extern unsigned int n_tcg_ctxs;
+extern unsigned int tcg_cur_ctxs;
+extern unsigned int tcg_max_ctxs;
 
 void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus);
 bool tcg_region_alloc(TCGContext *s);
diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ void tcg_region_initial_alloc(TCGContext *s)
 /* Call from a safe-work context */
 void tcg_region_reset_all(void)
 {
-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
+    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
     unsigned int i;
 
     qemu_mutex_lock(&region.lock);
@@ -XXX,XX +XXX,XX @@ void tcg_region_prologue_set(TCGContext *s)
  */
 size_t tcg_code_size(void)
 {
-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
+    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
     unsigned int i;
     size_t total;
 
@@ -XXX,XX +XXX,XX @@ size_t tcg_code_capacity(void)
 
 size_t tcg_tb_phys_invalidate_count(void)
 {
-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
+    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
     unsigned int i;
     size_t total = 0;
 
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@
 #define NO_CPU_IO_DEFS
 
 #include "exec/exec-all.h"
-
-#if !defined(CONFIG_USER_ONLY)
-#include "hw/boards.h"
-#endif
-
 #include "tcg/tcg-op.h"
 
 #if UINTPTR_MAX == UINT32_MAX
@@ -XXX,XX +XXX,XX @@ static int tcg_out_ldst_finalize(TCGContext *s);
 #endif
 
 TCGContext **tcg_ctxs;
-unsigned int n_tcg_ctxs;
+unsigned int tcg_cur_ctxs;
+unsigned int tcg_max_ctxs;
 TCGv_env cpu_env = 0;
 const void *tcg_code_gen_epilogue;
 uintptr_t tcg_splitwx_diff;
@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
 #else
 void tcg_register_thread(void)
 {
-    MachineState *ms = MACHINE(qdev_get_machine());
     TCGContext *s = g_malloc(sizeof(*s));
     unsigned int i, n;
 
@@ -XXX,XX +XXX,XX @@ void tcg_register_thread(void)
     }
 
     /* Claim an entry in tcg_ctxs */
-    n = qatomic_fetch_inc(&n_tcg_ctxs);
-    g_assert(n < ms->smp.max_cpus);
+    n = qatomic_fetch_inc(&tcg_cur_ctxs);
+    g_assert(n < tcg_max_ctxs);
     qatomic_set(&tcg_ctxs[n], s);
 
     if (n > 0) {
@@ -XXX,XX +XXX,XX @@ static void tcg_context_init(unsigned max_cpus)
      */
 #ifdef CONFIG_USER_ONLY
     tcg_ctxs = &tcg_ctx;
-    n_tcg_ctxs = 1;
+    tcg_cur_ctxs = 1;
+    tcg_max_ctxs = 1;
 #else
-    tcg_ctxs = g_new(TCGContext *, max_cpus);
+    tcg_max_ctxs = max_cpus;
+    tcg_ctxs = g_new0(TCGContext *, max_cpus);
 #endif
 
     tcg_debug_assert(!tcg_regset_test_reg(s->reserved_regs, TCG_AREG0));
@@ -XXX,XX +XXX,XX @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op)
 static inline
 void tcg_profile_snapshot(TCGProfile *prof, bool counters, bool table)
 {
-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
+    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
     unsigned int i;
 
     for (i = 0; i < n_ctxs; i++) {
@@ -XXX,XX +XXX,XX @@ void tcg_dump_op_count(void)
 
 int64_t tcg_cpu_exec_time(void)
 {
-    unsigned int n_ctxs = qatomic_read(&n_tcg_ctxs);
+    unsigned int n_ctxs = qatomic_read(&tcg_cur_ctxs);
     unsigned int i;
     int64_t ret = 0;
 
-- 
2.25.1

Remove the ifdef ladder and move each define into the
appropriate header file.

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/aarch64/tcg-target.h |  1 +
 tcg/arm/tcg-target.h     |  1 +
 tcg/i386/tcg-target.h    |  2 ++
 tcg/mips/tcg-target.h    |  6 ++++++
 tcg/ppc/tcg-target.h     |  2 ++
 tcg/riscv/tcg-target.h   |  1 +
 tcg/s390/tcg-target.h    |  3 +++
 tcg/sparc/tcg-target.h   |  1 +
 tcg/tci/tcg-target.h     |  1 +
 tcg/region.c             | 33 +++++----------------------------
 10 files changed, 23 insertions(+), 28 deletions(-)

diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/aarch64/tcg-target.h
+++ b/tcg/aarch64/tcg-target.h
@@ -XXX,XX +XXX,XX @@
 
 #define TCG_TARGET_INSN_UNIT_SIZE  4
 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 24
+#define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
 #undef TCG_TARGET_STACK_GROWSUP
 
 typedef enum {
diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/arm/tcg-target.h
+++ b/tcg/arm/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern int arm_arch;
 #undef TCG_TARGET_STACK_GROWSUP
 #define TCG_TARGET_INSN_UNIT_SIZE 4
 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16
+#define MAX_CODE_GEN_BUFFER_SIZE  UINT32_MAX
 
 typedef enum {
     TCG_REG_R0 = 0,
diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/i386/tcg-target.h
+++ b/tcg/i386/tcg-target.h
@@ -XXX,XX +XXX,XX @@
 #ifdef __x86_64__
 # define TCG_TARGET_REG_BITS  64
 # define TCG_TARGET_NB_REGS   32
+# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
 #else
 # define TCG_TARGET_REG_BITS  32
 # define TCG_TARGET_NB_REGS   24
+# define MAX_CODE_GEN_BUFFER_SIZE  UINT32_MAX
 #endif
 
 typedef enum {
diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/mips/tcg-target.h
+++ b/tcg/mips/tcg-target.h
@@ -XXX,XX +XXX,XX @@
 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16
 #define TCG_TARGET_NB_REGS 32
 
+/*
+ * We have a 256MB branch region, but leave room to make sure the
+ * main executable is also within that region.
+ */
+#define MAX_CODE_GEN_BUFFER_SIZE  (128 * MiB)
+
 typedef enum {
     TCG_REG_ZERO = 0,
     TCG_REG_AT,
diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/ppc/tcg-target.h
+++ b/tcg/ppc/tcg-target.h
@@ -XXX,XX +XXX,XX @@
 
 #ifdef _ARCH_PPC64
 # define TCG_TARGET_REG_BITS  64
+# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
 #else
 # define TCG_TARGET_REG_BITS  32
+# define MAX_CODE_GEN_BUFFER_SIZE  (32 * MiB)
 #endif
 
 #define TCG_TARGET_NB_REGS 64
diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/riscv/tcg-target.h
+++ b/tcg/riscv/tcg-target.h
@@ -XXX,XX +XXX,XX @@
 #define TCG_TARGET_INSN_UNIT_SIZE 4
 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 20
 #define TCG_TARGET_NB_REGS 32
+#define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
 
 typedef enum {
     TCG_REG_ZERO,
diff --git a/tcg/s390/tcg-target.h b/tcg/s390/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/s390/tcg-target.h
+++ b/tcg/s390/tcg-target.h
@@ -XXX,XX +XXX,XX @@
 #define TCG_TARGET_INSN_UNIT_SIZE 2
 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 19
 
+/* We have a +- 4GB range on the branches; leave some slop.  */
+#define MAX_CODE_GEN_BUFFER_SIZE  (3 * GiB)
+
 typedef enum TCGReg {
     TCG_REG_R0 = 0,
     TCG_REG_R1,
diff --git a/tcg/sparc/tcg-target.h b/tcg/sparc/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/sparc/tcg-target.h
+++ b/tcg/sparc/tcg-target.h
@@ -XXX,XX +XXX,XX @@
 #define TCG_TARGET_INSN_UNIT_SIZE 4
 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32
 #define TCG_TARGET_NB_REGS 32
+#define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
 
 typedef enum {
     TCG_REG_G0 = 0,
diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tci/tcg-target.h
+++ b/tcg/tci/tcg-target.h
@@ -XXX,XX +XXX,XX @@
 #define TCG_TARGET_INTERPRETER 1
 #define TCG_TARGET_INSN_UNIT_SIZE 1
 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32
+#define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
 
 #if UINTPTR_MAX == UINT32_MAX
 # define TCG_TARGET_REG_BITS 32
diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(unsigned max_cpus)
 /*
  * Minimum size of the code gen buffer.  This number is randomly chosen,
  * but not so small that we can't have a fair number of TB's live.
+ *
+ * Maximum size, MAX_CODE_GEN_BUFFER_SIZE, is defined in tcg-target.h.
+ * Unless otherwise indicated, this is constrained by the range of
+ * direct branches on the host cpu, as used by the TCG implementation
+ * of goto_tb.
  */
 #define MIN_CODE_GEN_BUFFER_SIZE     (1 * MiB)
 
-/*
- * Maximum size of the code gen buffer we'd like to use.  Unless otherwise
- * indicated, this is constrained by the range of direct branches on the
- * host cpu, as used by the TCG implementation of goto_tb.
- */
-#if defined(__x86_64__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
-#elif defined(__sparc__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
-#elif defined(__powerpc64__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
-#elif defined(__powerpc__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (32 * MiB)
-#elif defined(__aarch64__)
-# define MAX_CODE_GEN_BUFFER_SIZE  (2 * GiB)
-#elif defined(__s390x__)
-  /* We have a +- 4GB range on the branches; leave some slop.  */
-# define MAX_CODE_GEN_BUFFER_SIZE  (3 * GiB)
-#elif defined(__mips__)
-  /*
-   * We have a 256MB branch region, but leave room to make sure the
-   * main executable is also within that region.
-   */
-# define MAX_CODE_GEN_BUFFER_SIZE  (128 * MiB)
-#else
-# define MAX_CODE_GEN_BUFFER_SIZE  ((size_t)-1)
-#endif
-
 #if TCG_TARGET_REG_BITS == 32
 #define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (32 * MiB)
 #ifdef CONFIG_USER_ONLY
-- 
2.25.1

A size is easier to work with than an end point,
particularly during initial buffer allocation.

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ struct tcg_region_state {
     /* fields set at init time */
     void *start;
     void *start_aligned;
-    void *end;
     size_t n;
     size_t size; /* size of one region */
     size_t stride; /* .size + guard size */
+    size_t total_size; /* size of entire buffer, >= n * stride */
 
     /* fields protected by the lock */
     size_t current; /* current region index */
@@ -XXX,XX +XXX,XX @@ static void tcg_region_bounds(size_t curr_region, void **pstart, void **pend)
     if (curr_region == 0) {
         start = region.start;
     }
+    /* The final region may have a few extra pages due to earlier rounding. */
     if (curr_region == region.n - 1) {
-        end = region.end;
+        end = region.start_aligned + region.total_size;
     }
 
     *pstart = start;
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
  */
 void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
 {
-    void *buf, *aligned;
-    size_t size;
+    void *buf, *aligned, *end;
+    size_t total_size;
     size_t page_size;
     size_t region_size;
     size_t n_regions;
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
     assert(ok);
 
     buf = tcg_init_ctx.code_gen_buffer;
-    size = tcg_init_ctx.code_gen_buffer_size;
+    total_size = tcg_init_ctx.code_gen_buffer_size;
     page_size = qemu_real_host_page_size;
     n_regions = tcg_n_regions(max_cpus);
 
     /* The first region will be 'aligned - buf' bytes larger than the others */
     aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
-    g_assert(aligned < tcg_init_ctx.code_gen_buffer + size);
+    g_assert(aligned < tcg_init_ctx.code_gen_buffer + total_size);
+
     /*
      * Make region_size a multiple of page_size, using aligned as the start.
      * As a result of this we might end up with a few extra pages at the end of
      * the buffer; we will assign those to the last region.
      */
-    region_size = (size - (aligned - buf)) / n_regions;
+    region_size = (total_size - (aligned - buf)) / n_regions;
     region_size = QEMU_ALIGN_DOWN(region_size, page_size);
 
     /* A region must have at least 2 pages; one code, one guard */
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
     region.start = buf;
     region.start_aligned = aligned;
     /* page-align the end, since its last page will be a guard page */
-    region.end = QEMU_ALIGN_PTR_DOWN(buf + size, page_size);
+    end = QEMU_ALIGN_PTR_DOWN(buf + total_size, page_size);
     /* account for that last guard page */
-    region.end -= page_size;
+    end -= page_size;
+    total_size = end - aligned;
+    region.total_size = total_size;
 
     /*
      * Set guard pages in the rw buffer, as that's the one into which
@@ -XXX,XX +XXX,XX @@ void tcg_region_prologue_set(TCGContext *s)
 
     /* Register the balance of the buffer with gdb. */
     tcg_register_jit(tcg_splitwx_to_rx(region.start),
-                     region.end - region.start);
+                     region.start_aligned + region.total_size - region.start);
 }
 
 /*
@@ -XXX,XX +XXX,XX @@ size_t tcg_code_capacity(void)
 
     /* no need for synchronization; these variables are set at init time */
     guard_size = region.stride - region.size;
-    capacity = region.end + guard_size - region.start;
-    capacity -= region.n * (guard_size + TCG_HIGHWATER);
+    capacity = region.total_size;
+    capacity -= (region.n - 1) * guard_size;
+    capacity -= region.n * TCG_HIGHWATER;
+
     return capacity;
 }
 
-- 
2.25.1

Give the field a name reflecting its actual meaning.

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ struct tcg_region_state {
     QemuMutex lock;
 
     /* fields set at init time */
-    void *start;
     void *start_aligned;
+    void *after_prologue;
     size_t n;
     size_t size; /* size of one region */
     size_t stride; /* .size + guard size */
@@ -XXX,XX +XXX,XX @@ static void tcg_region_bounds(size_t curr_region, void **pstart, void **pend)
     end = start + region.size;
 
     if (curr_region == 0) {
-        start = region.start;
+        start = region.after_prologue;
     }
     /* The final region may have a few extra pages due to earlier rounding. */
     if (curr_region == region.n - 1) {
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
     region.n = n_regions;
     region.size = region_size - page_size;
     region.stride = region_size;
-    region.start = buf;
+    region.after_prologue = buf;
     region.start_aligned = aligned;
     /* page-align the end, since its last page will be a guard page */
     end = QEMU_ALIGN_PTR_DOWN(buf + total_size, page_size);
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
 void tcg_region_prologue_set(TCGContext *s)
 {
     /* Deduct the prologue from the first region.  */
-    g_assert(region.start == s->code_gen_buffer);
-    region.start = s->code_ptr;
+    g_assert(region.start_aligned == s->code_gen_buffer);
+    region.after_prologue = s->code_ptr;
 
     /* Recompute boundaries of the first region. */
     tcg_region_assign(s, 0);
 
     /* Register the balance of the buffer with gdb. */
-    tcg_register_jit(tcg_splitwx_to_rx(region.start),
-                     region.start_aligned + region.total_size - region.start);
+    tcg_register_jit(tcg_splitwx_to_rx(region.after_prologue),
+                     region.start_aligned + region.total_size -
+                     region.after_prologue);
 }
 
 /*
-- 
2.25.1

Compute the value using straight division and bounds,
rather than a loop.  Pass in tb_size rather than reading
from tcg_init_ctx.code_gen_buffer_size,

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ void tcg_region_reset_all(void)
     tcg_region_tree_reset_all();
 }
 
-static size_t tcg_n_regions(unsigned max_cpus)
+static size_t tcg_n_regions(size_t tb_size, unsigned max_cpus)
 {
 #ifdef CONFIG_USER_ONLY
     return 1;
 #else
+    size_t n_regions;
+
     /*
      * It is likely that some vCPUs will translate more code than others,
      * so we first try to set more regions than max_cpus, with those regions
      * being of reasonable size. If that's not possible we make do by evenly
      * dividing the code_gen_buffer among the vCPUs.
      */
-    size_t i;
-
     /* Use a single region if all we have is one vCPU thread */
     if (max_cpus == 1 || !qemu_tcg_mttcg_enabled()) {
         return 1;
     }
 
-    /* Try to have more regions than max_cpus, with each region being >= 2 MB */
-    for (i = 8; i > 0; i--) {
-        size_t regions_per_thread = i;
-        size_t region_size;
-
-        region_size = tcg_init_ctx.code_gen_buffer_size;
-        region_size /= max_cpus * regions_per_thread;
-
-        if (region_size >= 2 * 1024u * 1024) {
-            return max_cpus * regions_per_thread;
-        }
+    /*
+     * Try to have more regions than max_cpus, with each region being >= 2 MB.
+     * If we can't, then just allocate one region per vCPU thread.
+     */
+    n_regions = tb_size / (2 * MiB);
+    if (n_regions <= max_cpus) {
+        return max_cpus;
     }
-    /* If we can't, then just allocate one region per vCPU thread */
-    return max_cpus;
+    return MIN(n_regions, max_cpus * 8);
 #endif
 }
 
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
     buf = tcg_init_ctx.code_gen_buffer;
     total_size = tcg_init_ctx.code_gen_buffer_size;
     page_size = qemu_real_host_page_size;
-    n_regions = tcg_n_regions(max_cpus);
+    n_regions = tcg_n_regions(total_size, max_cpus);
 
     /* The first region will be 'aligned - buf' bytes larger than the others */
     aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
-- 
2.25.1

Return output buffer and size via output pointer arguments,
rather than returning size via tcg_ctx->code_gen_buffer_size.

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static inline bool cross_256mb(void *addr, size_t size)
 /*
  * We weren't able to allocate a buffer without crossing that boundary,
  * so make do with the larger portion of the buffer that doesn't cross.
- * Returns the new base of the buffer, and adjusts code_gen_buffer_size.
+ * Returns the new base and size of the buffer in *obuf and *osize.
  */
-static inline void *split_cross_256mb(void *buf1, size_t size1)
+static inline void split_cross_256mb(void **obuf, size_t *osize,
+                                     void *buf1, size_t size1)
 {
     void *buf2 = (void *)(((uintptr_t)buf1 + size1) & ~0x0ffffffful);
     size_t size2 = buf1 + size1 - buf2;
@@ -XXX,XX +XXX,XX @@ static inline void *split_cross_256mb(void *buf1, size_t size1)
         buf1 = buf2;
     }
 
-    tcg_ctx->code_gen_buffer_size = size1;
-    return buf1;
+    *obuf = buf1;
+    *osize = size1;
 }
 #endif
 
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
     if (size > tb_size) {
         size = QEMU_ALIGN_DOWN(tb_size, qemu_real_host_page_size);
     }
-    tcg_ctx->code_gen_buffer_size = size;
 
 #ifdef __mips__
     if (cross_256mb(buf, size)) {
-        buf = split_cross_256mb(buf, size);
-        size = tcg_ctx->code_gen_buffer_size;
+        split_cross_256mb(&buf, &size, buf, size);
     }
 #endif
 
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
     qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
 
     tcg_ctx->code_gen_buffer = buf;
+    tcg_ctx->code_gen_buffer_size = size;
     return true;
 }
 #elif defined(_WIN32)
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
                          "allocate %zu bytes for jit buffer", size);
         return false;
     }
-    tcg_ctx->code_gen_buffer_size = size;
 
 #ifdef __mips__
     if (cross_256mb(buf, size)) {
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
             /* fallthru */
         default:
             /* Split the original buffer.  Free the smaller half.  */
-            buf2 = split_cross_256mb(buf, size);
-            size2 = tcg_ctx->code_gen_buffer_size;
+            split_cross_256mb(&buf2, &size2, buf, size);
             if (buf == buf2) {
                 munmap(buf + size2, size - size2);
             } else {
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
     qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
 
     tcg_ctx->code_gen_buffer = buf;
+    tcg_ctx->code_gen_buffer_size = size;
     return true;
 }
 
-- 
2.25.1

Shortly, the full code_gen_buffer will only be visible
to region.c, so move in_code_gen_buffer out-of-line.

Move the debugging versions of tcg_splitwx_to_{rx,rw}
to region.c as well, so that the compiler gets to see
the implementation of in_code_gen_buffer.

This leaves exactly one use of in_code_gen_buffer outside
of region.c, in cpu_restore_state.  Which, being on the
exception path, is not performance critical.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg.h | 11 +----------
 tcg/region.c      | 34 ++++++++++++++++++++++++++++++++++
 tcg/tcg.c         | 23 -----------------------
 3 files changed, 35 insertions(+), 33 deletions(-)

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ extern const void *tcg_code_gen_epilogue;
 extern uintptr_t tcg_splitwx_diff;
 extern TCGv_env cpu_env;
 
-static inline bool in_code_gen_buffer(const void *p)
-{
-    const TCGContext *s = &tcg_init_ctx;
-    /*
-     * Much like it is valid to have a pointer to the byte past the
-     * end of an array (so long as you don't dereference it), allow
-     * a pointer to the byte past the end of the code gen buffer.
-     */
-    return (size_t)(p - s->code_gen_buffer) <= s->code_gen_buffer_size;
-}
+bool in_code_gen_buffer(const void *p);
 
 #ifdef CONFIG_DEBUG_TCG
 const void *tcg_splitwx_to_rx(void *rw);
diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static struct tcg_region_state region;
 static void *region_trees;
 static size_t tree_size;
 
+bool in_code_gen_buffer(const void *p)
+{
+    const TCGContext *s = &tcg_init_ctx;
+    /*
+     * Much like it is valid to have a pointer to the byte past the
+     * end of an array (so long as you don't dereference it), allow
+     * a pointer to the byte past the end of the code gen buffer.
+     */
+    return (size_t)(p - s->code_gen_buffer) <= s->code_gen_buffer_size;
+}
+
+#ifdef CONFIG_DEBUG_TCG
+const void *tcg_splitwx_to_rx(void *rw)
+{
+    /* Pass NULL pointers unchanged. */
+    if (rw) {
+        g_assert(in_code_gen_buffer(rw));
+        rw += tcg_splitwx_diff;
+    }
+    return rw;
+}
+
+void *tcg_splitwx_to_rw(const void *rx)
+{
+    /* Pass NULL pointers unchanged. */
+    if (rx) {
+        rx -= tcg_splitwx_diff;
+        /* Assert that we end with a pointer in the rw region. */
+        g_assert(in_code_gen_buffer(rx));
+    }
+    return (void *)rx;
+}
+#endif /* CONFIG_DEBUG_TCG */
+
 /* compare a pointer @ptr and a tb_tc @s */
 static int ptr_cmp_tb_tc(const void *ptr, const struct tb_tc *s)
 {
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef constraint_sets[] = {
 
 #include "tcg-target.c.inc"
 
-#ifdef CONFIG_DEBUG_TCG
-const void *tcg_splitwx_to_rx(void *rw)
-{
-    /* Pass NULL pointers unchanged. */
-    if (rw) {
-        g_assert(in_code_gen_buffer(rw));
-        rw += tcg_splitwx_diff;
-    }
-    return rw;
-}
-
-void *tcg_splitwx_to_rw(const void *rx)
-{
-    /* Pass NULL pointers unchanged. */
-    if (rx) {
-        rx -= tcg_splitwx_diff;
-        /* Assert that we end with a pointer in the rw region. */
-        g_assert(in_code_gen_buffer(rx));
-    }
-    return (void *)rx;
-}
-#endif /* CONFIG_DEBUG_TCG */
-
 static void alloc_tcg_plugin_context(TCGContext *s)
 {
 #ifdef CONFIG_PLUGIN
-- 
2.25.1

Do not mess around with setting values within tcg_init_ctx.
Put the values into 'region' directly, which is where they
will live for the lifetime of the program.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/region.c | 64 ++++++++++++++++++++++------------------------------
 1 file changed, 27 insertions(+), 37 deletions(-)

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static size_t tree_size;
 
 bool in_code_gen_buffer(const void *p)
 {
-    const TCGContext *s = &tcg_init_ctx;
     /*
      * Much like it is valid to have a pointer to the byte past the
      * end of an array (so long as you don't dereference it), allow
      * a pointer to the byte past the end of the code gen buffer.
      */
-    return (size_t)(p - s->code_gen_buffer) <= s->code_gen_buffer_size;
+    return (size_t)(p - region.start_aligned) <= region.total_size;
 }
 
 #ifdef CONFIG_DEBUG_TCG
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
     }
     qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
 
-    tcg_ctx->code_gen_buffer = buf;
-    tcg_ctx->code_gen_buffer_size = size;
+    region.start_aligned = buf;
+    region.total_size = size;
     return true;
 }
 #elif defined(_WIN32)
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
         return false;
     }
 
-    tcg_ctx->code_gen_buffer = buf;
-    tcg_ctx->code_gen_buffer_size = size;
+    region.start_aligned = buf;
+    region.total_size = size;
     return true;
 }
 #else
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
     /* Request large pages for the buffer.  */
     qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
 
-    tcg_ctx->code_gen_buffer = buf;
-    tcg_ctx->code_gen_buffer_size = size;
+    region.start_aligned = buf;
+    region.total_size = size;
     return true;
 }
 
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
         return false;
     }
     /* The size of the mapping may have been adjusted. */
-    size = tcg_ctx->code_gen_buffer_size;
-    buf_rx = tcg_ctx->code_gen_buffer;
+    buf_rx = region.start_aligned;
+    size = region.total_size;
 #endif
 
     buf_rw = qemu_memfd_alloc("tcg-jit", size, 0, &fd, errp);
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
 #endif
 
     close(fd);
-    tcg_ctx->code_gen_buffer = buf_rw;
-    tcg_ctx->code_gen_buffer_size = size;
+    region.start_aligned = buf_rw;
+    region.total_size = size;
     tcg_splitwx_diff = buf_rx - buf_rw;
 
     /* Request large pages for the buffer and the splitwx.  */
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
         return false;
     }
 
-    buf_rw = (mach_vm_address_t)tcg_ctx->code_gen_buffer;
+    buf_rw = region.start_aligned;
     buf_rx = 0;
     ret = mach_vm_remap(mach_task_self(),
                         &buf_rx,
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
  */
 void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
 {
-    void *buf, *aligned, *end;
-    size_t total_size;
     size_t page_size;
     size_t region_size;
-    size_t n_regions;
     size_t i;
     bool ok;
 
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
                                splitwx, &error_fatal);
     assert(ok);
 
-    buf = tcg_init_ctx.code_gen_buffer;
-    total_size = tcg_init_ctx.code_gen_buffer_size;
-    page_size = qemu_real_host_page_size;
-    n_regions = tcg_n_regions(total_size, max_cpus);
-
-    /* The first region will be 'aligned - buf' bytes larger than the others */
-    aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
-    g_assert(aligned < tcg_init_ctx.code_gen_buffer + total_size);
-
     /*
      * Make region_size a multiple of page_size, using aligned as the start.
      * As a result of this we might end up with a few extra pages at the end of
      * the buffer; we will assign those to the last region.
      */
-    region_size = (total_size - (aligned - buf)) / n_regions;
+    region.n = tcg_n_regions(region.total_size, max_cpus);
+    page_size = qemu_real_host_page_size;
+    region_size = region.total_size / region.n;
     region_size = QEMU_ALIGN_DOWN(region_size, page_size);
 
     /* A region must have at least 2 pages; one code, one guard */
     g_assert(region_size >= 2 * page_size);
+    region.stride = region_size;
+
+    /* Reserve space for guard pages. */
+    region.size = region_size - page_size;
+    region.total_size -= page_size;
+
+    /*
+     * The first region will be smaller than the others, via the prologue,
+     * which has yet to be allocated.  For now, the first region begins at
+     * the page boundary.
+     */
+    region.after_prologue = region.start_aligned;
 
     /* init the region struct */
     qemu_mutex_init(&region.lock);
-    region.n = n_regions;
-    region.size = region_size - page_size;
-    region.stride = region_size;
-    region.after_prologue = buf;
-    region.start_aligned = aligned;
-    /* page-align the end, since its last page will be a guard page */
-    end = QEMU_ALIGN_PTR_DOWN(buf + total_size, page_size);
-    /* account for that last guard page */
-    end -= page_size;
-    total_size = end - aligned;
-    region.total_size = total_size;
 
     /*
      * Set guard pages in the rw buffer, as that's the one into which
-- 
2.25.1

Change the interface from a boolean error indication to a
negative error vs a non-negative protection.  For the moment
this is only interface change, not making use of the new data.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/region.c | 63 +++++++++++++++++++++++++++-------------------------
 1 file changed, 33 insertions(+), 30 deletions(-)

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static inline void split_cross_256mb(void **obuf, size_t *osize,
 static uint8_t static_code_gen_buffer[DEFAULT_CODE_GEN_BUFFER_SIZE]
     __attribute__((aligned(CODE_GEN_ALIGN)));
 
-static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
+static int alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
 {
     void *buf, *end;
     size_t size;
 
     if (splitwx > 0) {
         error_setg(errp, "jit split-wx not supported");
-        return false;
+        return -1;
     }
 
     /* page-align the beginning and end of the buffer */
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
 
     region.start_aligned = buf;
     region.total_size = size;
-    return true;
+
+    return PROT_READ | PROT_WRITE;
 }
 #elif defined(_WIN32)
-static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+static int alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
 {
     void *buf;
 
     if (splitwx > 0) {
         error_setg(errp, "jit split-wx not supported");
-        return false;
+        return -1;
     }
 
     buf = VirtualAlloc(NULL, size, MEM_RESERVE | MEM_COMMIT,
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
 
     region.start_aligned = buf;
     region.total_size = size;
-    return true;
+
+    return PAGE_READ | PAGE_WRITE | PAGE_EXEC;
 }
 #else
-static bool alloc_code_gen_buffer_anon(size_t size, int prot,
-                                       int flags, Error **errp)
+static int alloc_code_gen_buffer_anon(size_t size, int prot,
+                                      int flags, Error **errp)
 {
     void *buf;
 
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
     if (buf == MAP_FAILED) {
         error_setg_errno(errp, errno,
                          "allocate %zu bytes for jit buffer", size);
-        return false;
+        return -1;
     }
 
 #ifdef __mips__
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_anon(size_t size, int prot,
 
     region.start_aligned = buf;
     region.total_size = size;
-    return true;
+    return prot;
 }
 
 #ifndef CONFIG_TCG_INTERPRETER
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
 
 #ifdef __mips__
     /* Find space for the RX mapping, vs the 256MiB regions. */
-    if (!alloc_code_gen_buffer_anon(size, PROT_NONE,
-                                    MAP_PRIVATE | MAP_ANONYMOUS |
-                                    MAP_NORESERVE, errp)) {
+    if (alloc_code_gen_buffer_anon(size, PROT_NONE,
+                                   MAP_PRIVATE | MAP_ANONYMOUS |
+                                   MAP_NORESERVE, errp) < 0) {
         return false;
     }
     /* The size of the mapping may have been adjusted. */
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
     /* Request large pages for the buffer and the splitwx.  */
     qemu_madvise(buf_rw, size, QEMU_MADV_HUGEPAGE);
     qemu_madvise(buf_rx, size, QEMU_MADV_HUGEPAGE);
-    return true;
+    return PROT_READ | PROT_WRITE;
 
  fail_rx:
     error_setg_errno(errp, errno, "failed to map shared memory for execute");
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
     if (fd >= 0) {
         close(fd);
     }
-    return false;
+    return -1;
 }
 #endif /* CONFIG_POSIX */
 
@@ -XXX,XX +XXX,XX @@ extern kern_return_t mach_vm_remap(vm_map_t target_task,
                                    vm_prot_t *max_protection,
                                    vm_inherit_t inheritance);
 
-static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
+static int alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
 {
     kern_return_t ret;
     mach_vm_address_t buf_rw, buf_rx;
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
     /* Map the read-write portion via normal anon memory. */
     if (!alloc_code_gen_buffer_anon(size, PROT_READ | PROT_WRITE,
                                     MAP_PRIVATE | MAP_ANONYMOUS, errp)) {
-        return false;
+        return -1;
     }
 
     buf_rw = region.start_aligned;
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_vmremap(size_t size, Error **errp)
         /* TODO: Convert "ret" to a human readable error message. */
         error_setg(errp, "vm_remap for jit splitwx failed");
         munmap((void *)buf_rw, size);
-        return false;
+        return -1;
     }
 
     if (mprotect((void *)buf_rx, size, PROT_READ | PROT_EXEC) != 0) {
         error_setg_errno(errp, errno, "mprotect for jit splitwx");
         munmap((void *)buf_rx, size);
         munmap((void *)buf_rw, size);
-        return false;
+        return -1;
     }
 
     tcg_splitwx_diff = buf_rx - buf_rw;
-    return true;
+    return PROT_READ | PROT_WRITE;
 }
 #endif /* CONFIG_DARWIN */
 #endif /* CONFIG_TCG_INTERPRETER */
 
-static bool alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
+static int alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
 {
 #ifndef CONFIG_TCG_INTERPRETER
 # ifdef CONFIG_DARWIN
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx(size_t size, Error **errp)
 # endif
 #endif
     error_setg(errp, "jit split-wx not supported");
-    return false;
+    return -1;
 }
 
-static bool alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
+static int alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
 {
     ERRP_GUARD();
     int prot, flags;
 
     if (splitwx) {
-        if (alloc_code_gen_buffer_splitwx(size, errp)) {
-            return true;
+        prot = alloc_code_gen_buffer_splitwx(size, errp);
+        if (prot >= 0) {
+            return prot;
         }
         /*
          * If splitwx force-on (1), fail;
          * if splitwx default-on (-1), fall through to splitwx off.
          */
         if (splitwx > 0) {
-            return false;
+            return -1;
         }
         error_free_or_abort(errp);
     }
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
     size_t page_size;
     size_t region_size;
     size_t i;
-    bool ok;
+    int have_prot;
 
-    ok = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
-                               splitwx, &error_fatal);
-    assert(ok);
+    have_prot = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
+                                      splitwx, &error_fatal);
+    assert(have_prot >= 0);
 
     /*
      * Make region_size a multiple of page_size, using aligned as the start.
-- 
2.25.1

Move the call out of the N versions of alloc_code_gen_buffer
and into tcg_region_init.

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
         error_setg_errno(errp, errno, "mprotect of jit buffer");
         return false;
     }
-    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
 
     region.start_aligned = buf;
     region.total_size = size;
@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer_anon(size_t size, int prot,
     }
 #endif
 
-    /* Request large pages for the buffer.  */
-    qemu_madvise(buf, size, QEMU_MADV_HUGEPAGE);
-
     region.start_aligned = buf;
     region.total_size = size;
     return prot;
@@ -XXX,XX +XXX,XX @@ static bool alloc_code_gen_buffer_splitwx_memfd(size_t size, Error **errp)
     region.total_size = size;
     tcg_splitwx_diff = buf_rx - buf_rw;
 
-    /* Request large pages for the buffer and the splitwx.  */
-    qemu_madvise(buf_rw, size, QEMU_MADV_HUGEPAGE);
-    qemu_madvise(buf_rx, size, QEMU_MADV_HUGEPAGE);
     return PROT_READ | PROT_WRITE;
 
  fail_rx:
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
                                       splitwx, &error_fatal);
     assert(have_prot >= 0);
 
+    /* Request large pages for the buffer and the splitwx.  */
+    qemu_madvise(region.start_aligned, region.total_size, QEMU_MADV_HUGEPAGE);
+    if (tcg_splitwx_diff) {
+        qemu_madvise(region.start_aligned + tcg_splitwx_diff,
+                     region.total_size, QEMU_MADV_HUGEPAGE);
+    }
+
     /*
      * Make region_size a multiple of page_size, using aligned as the start.
      * As a result of this we might end up with a few extra pages at the end of
-- 
2.25.1

For --enable-tcg-interpreter on Windows, we will need this.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/osdep.h | 1 +
 util/osdep.c         | 9 +++++++++
 2 files changed, 10 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -XXX,XX +XXX,XX @@ void sigaction_invoke(struct sigaction *action,
 #endif
 
 int qemu_madvise(void *addr, size_t len, int advice);
+int qemu_mprotect_rw(void *addr, size_t size);
 int qemu_mprotect_rwx(void *addr, size_t size);
 int qemu_mprotect_none(void *addr, size_t size);
 
diff --git a/util/osdep.c b/util/osdep.c
index XXXXXXX..XXXXXXX 100644
--- a/util/osdep.c
+++ b/util/osdep.c
@@ -XXX,XX +XXX,XX @@ static int qemu_mprotect__osdep(void *addr, size_t size, int prot)
 #endif
 }
 
+int qemu_mprotect_rw(void *addr, size_t size)
+{
+#ifdef _WIN32
+    return qemu_mprotect__osdep(addr, size, PAGE_READWRITE);
+#else
+    return qemu_mprotect__osdep(addr, size, PROT_READ | PROT_WRITE);
+#endif
+}
+
 int qemu_mprotect_rwx(void *addr, size_t size)
 {
 #ifdef _WIN32
-- 
2.25.1

If qemu_get_host_physmem returns an odd number of pages,
then physmem / 8 will not be a multiple of the page size.

The following was observed on a gitlab runner:

ERROR qtest-arm/boot-serial-test - Bail out!
ERROR:../util/osdep.c:80:qemu_mprotect__osdep: \
  assertion failed: (!(size & ~qemu_real_host_page_mask))

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/region.c | 47 +++++++++++++++++++++--------------------------
 1 file changed, 21 insertions(+), 26 deletions(-)

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static size_t tcg_n_regions(size_t tb_size, unsigned max_cpus)
   (DEFAULT_CODE_GEN_BUFFER_SIZE_1 < MAX_CODE_GEN_BUFFER_SIZE \
    ? DEFAULT_CODE_GEN_BUFFER_SIZE_1 : MAX_CODE_GEN_BUFFER_SIZE)
 
-static size_t size_code_gen_buffer(size_t tb_size)
-{
-    /* Size the buffer.  */
-    if (tb_size == 0) {
-        size_t phys_mem = qemu_get_host_physmem();
-        if (phys_mem == 0) {
-            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
-        } else {
-            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8);
-        }
-    }
-    if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
-        tb_size = MIN_CODE_GEN_BUFFER_SIZE;
-    }
-    if (tb_size > MAX_CODE_GEN_BUFFER_SIZE) {
-        tb_size = MAX_CODE_GEN_BUFFER_SIZE;
-    }
-    return tb_size;
-}
-
 #ifdef __mips__
 /*
  * In order to use J and JAL within the code_gen_buffer, we require
@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
  */
 void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
 {
-    size_t page_size;
+    const size_t page_size = qemu_real_host_page_size;
     size_t region_size;
     size_t i;
     int have_prot;
 
-    have_prot = alloc_code_gen_buffer(size_code_gen_buffer(tb_size),
-                                      splitwx, &error_fatal);
+    /* Size the buffer.  */
+    if (tb_size == 0) {
+        size_t phys_mem = qemu_get_host_physmem();
+        if (phys_mem == 0) {
+            tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+        } else {
+            tb_size = QEMU_ALIGN_DOWN(phys_mem / 8, page_size);
+            tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, tb_size);
+        }
+    }
+    if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
+        tb_size = MIN_CODE_GEN_BUFFER_SIZE;
+    }
+    if (tb_size > MAX_CODE_GEN_BUFFER_SIZE) {
+        tb_size = MAX_CODE_GEN_BUFFER_SIZE;
+    }
+
+    have_prot = alloc_code_gen_buffer(tb_size, splitwx, &error_fatal);
     assert(have_prot >= 0);
 
     /* Request large pages for the buffer and the splitwx.  */
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
      * As a result of this we might end up with a few extra pages at the end of
      * the buffer; we will assign those to the last region.
      */
-    region.n = tcg_n_regions(region.total_size, max_cpus);
-    page_size = qemu_real_host_page_size;
-    region_size = region.total_size / region.n;
+    region.n = tcg_n_regions(tb_size, max_cpus);
+    region_size = tb_size / region.n;
     region_size = QEMU_ALIGN_DOWN(region_size, page_size);
 
     /* A region must have at least 2 pages; one code, one guard */
-- 
2.25.1

Do not handle protections on a case-by-case basis in the
various alloc_code_gen_buffer instances; do it within a
single loop in tcg_region_init.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/region.c | 45 +++++++++++++++++++++++++++++++--------------
 1 file changed, 31 insertions(+), 14 deletions(-)

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer(size_t tb_size, int splitwx, Error **errp)
     }
 #endif
 
-    if (qemu_mprotect_rwx(buf, size)) {
-        error_setg_errno(errp, errno, "mprotect of jit buffer");
-        return false;
-    }
-
     region.start_aligned = buf;
     region.total_size = size;
 
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
 {
     const size_t page_size = qemu_real_host_page_size;
     size_t region_size;
-    size_t i;
-    int have_prot;
+    int have_prot, need_prot;
 
     /* Size the buffer.  */
     if (tb_size == 0) {
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
      * Set guard pages in the rw buffer, as that's the one into which
      * buffer overruns could occur.  Do not set guard pages in the rx
      * buffer -- let that one use hugepages throughout.
+     * Work with the page protections set up with the initial mapping.
      */
-    for (i = 0; i < region.n; i++) {
+    need_prot = PAGE_READ | PAGE_WRITE;
+#ifndef CONFIG_TCG_INTERPRETER
+    if (tcg_splitwx_diff == 0) {
+        need_prot |= PAGE_EXEC;
+    }
+#endif
+    for (size_t i = 0, n = region.n; i < n; i++) {
         void *start, *end;
 
         tcg_region_bounds(i, &start, &end);
+        if (have_prot != need_prot) {
+            int rc;
 
-        /*
-         * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
-         * rejects a permission change from RWX -> NONE.  Guard pages are
-         * nice for bug detection but are not essential; ignore any failure.
-         */
-        (void)qemu_mprotect_none(end, page_size);
+            if (need_prot == (PAGE_READ | PAGE_WRITE | PAGE_EXEC)) {
+                rc = qemu_mprotect_rwx(start, end - start);
+            } else if (need_prot == (PAGE_READ | PAGE_WRITE)) {
+                rc = qemu_mprotect_rw(start, end - start);
+            } else {
+                g_assert_not_reached();
+            }
+            if (rc) {
+                error_setg_errno(&error_fatal, errno,
+                                 "mprotect of jit buffer");
+            }
+        }
+        if (have_prot != 0) {
+            /*
+             * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
+             * rejects a permission change from RWX -> NONE.  Guard pages are
+             * nice for bug detection but are not essential; ignore any failure.
+             */
+            (void)qemu_mprotect_none(end, page_size);
+        }
     }
 
     tcg_region_trees_init();
-- 
2.25.1

There's a change in mprotect() behaviour [1] in the latest macOS
on M1 and it's not yet clear if it's going to be fixed by Apple.

In this case, instead of changing permissions of N guard pages,
we change permissions of N rwx regions.  The same number of
syscalls are required either way.

[1] https://gist.github.com/hikalium/75ae822466ee4da13cbbe486498a191f

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/region.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/tcg/region.c b/tcg/region.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/region.c
+++ b/tcg/region.c
@@ -XXX,XX +XXX,XX @@ static int alloc_code_gen_buffer(size_t size, int splitwx, Error **errp)
         error_free_or_abort(errp);
     }
 
-    prot = PROT_READ | PROT_WRITE | PROT_EXEC;
+    /*
+     * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
+     * rejects a permission change from RWX -> NONE when reserving the
+     * guard pages later.  We can go the other way with the same number
+     * of syscalls, so always begin with PROT_NONE.
+     */
+    prot = PROT_NONE;
     flags = MAP_PRIVATE | MAP_ANONYMOUS;
-#ifdef CONFIG_TCG_INTERPRETER
-    /* The tcg interpreter does not need execute permission. */
-    prot = PROT_READ | PROT_WRITE;
-#elif defined(CONFIG_DARWIN)
+#ifdef CONFIG_DARWIN
     /* Applicable to both iOS and macOS (Apple Silicon). */
     if (!splitwx) {
         flags |= MAP_JIT;
@@ -XXX,XX +XXX,XX @@ void tcg_region_init(size_t tb_size, int splitwx, unsigned max_cpus)
             }
         }
         if (have_prot != 0) {
-            /*
-             * macOS 11.2 has a bug (Apple Feedback FB8994773) in which mprotect
-             * rejects a permission change from RWX -> NONE.  Guard pages are
-             * nice for bug detection but are not essential; ignore any failure.
-             */
+            /* Guard pages are nice for bug detection but are not essential. */
             (void)qemu_mprotect_none(end, page_size);
         }
     }
-- 
2.25.1

These variables belong to the jit side, not the user side.

Since tcg_init_ctx is no longer used outside of tcg/, move
the declaration to tcg-internal.h.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg.h         | 1 -
 tcg/tcg-internal.h        | 1 +
 accel/tcg/translate-all.c | 3 ---
 tcg/tcg.c                 | 3 +++
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ static inline bool temp_readonly(TCGTemp *ts)
     return ts->kind >= TEMP_FIXED;
 }
 
-extern TCGContext tcg_init_ctx;
 extern __thread TCGContext *tcg_ctx;
 extern const void *tcg_code_gen_epilogue;
 extern uintptr_t tcg_splitwx_diff;
diff --git a/tcg/tcg-internal.h b/tcg/tcg-internal.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-internal.h
+++ b/tcg/tcg-internal.h
@@ -XXX,XX +XXX,XX @@
 
 #define TCG_HIGHWATER 1024
 
+extern TCGContext tcg_init_ctx;
 extern TCGContext **tcg_ctxs;
 extern unsigned int tcg_cur_ctxs;
 extern unsigned int tcg_max_ctxs;
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static int v_l2_levels;
 
 static void *l1_map[V_L1_MAX_SIZE];
 
-/* code generation context */
-TCGContext tcg_init_ctx;
-__thread TCGContext *tcg_ctx;
 TBContext tb_ctx;
 
 static void page_table_config_init(void)
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct);
 static int tcg_out_ldst_finalize(TCGContext *s);
 #endif
 
+TCGContext tcg_init_ctx;
+__thread TCGContext *tcg_ctx;
+
 TCGContext **tcg_ctxs;
 unsigned int tcg_cur_ctxs;
 unsigned int tcg_max_ctxs;
-- 
2.25.1

Introduce a function to remove everything emitted
since a given point.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg.h | 10 ++++++++++
 tcg/tcg.c         | 13 +++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ void tcg_op_remove(TCGContext *s, TCGOp *op);
 TCGOp *tcg_op_insert_before(TCGContext *s, TCGOp *op, TCGOpcode opc);
 TCGOp *tcg_op_insert_after(TCGContext *s, TCGOp *op, TCGOpcode opc);
 
+/**
+ * tcg_remove_ops_after:
+ * @op: target operation
+ *
+ * Discard any opcodes emitted since @op.  Expected usage is to save
+ * a starting point with tcg_last_op(), speculatively emit opcodes,
+ * then decide whether or not to keep those opcodes after the fact.
+ */
+void tcg_remove_ops_after(TCGOp *op);
+
 void tcg_optimize(TCGContext *s);
 
 /* Allocate a new temporary and initialize it with a constant. */
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ void tcg_op_remove(TCGContext *s, TCGOp *op)
 #endif
 }
 
+void tcg_remove_ops_after(TCGOp *op)
+{
+    TCGContext *s = tcg_ctx;
+
+    while (true) {
+        TCGOp *last = tcg_last_op();
+        if (last == op) {
+            return;
+        }
+        tcg_op_remove(s, last);
+    }
+}
+
 static TCGOp *tcg_op_alloc(TCGOpcode opc)
 {
     TCGContext *s = tcg_ctx;
-- 
2.25.1

At some point during the development of tcg_constant_*, I changed
my mind about whether such temps should be able to be passed to
tcg_temp_free_*.  The final version committed allows this, but the
commentary was not updated to match.

Fixes: c0522136adf
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/tcg/tcg.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ TCGv_vec tcg_const_ones_vec_matching(TCGv_vec);
 
 /*
  * Locate or create a read-only temporary that is a constant.
- * This kind of temporary need not and should not be freed.
+ * This kind of temporary need not be freed, but for convenience
+ * will be silently ignored by tcg_temp_free_*.
  */
 TCGTemp *tcg_constant_internal(TCGType type, int64_t val);
 
-- 
2.25.1

From: "Jose R. Ziviani" <jziviani@suse.de>

Commit 5e8892db93 fixed several function signatures but tcg_out_op for
arm is missing. This patch fixes it as well.

Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
Message-Id: <20210610224450.23425-1-jziviani@suse.de>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/arm/tcg-target.c.inc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc
index XXXXXXX..XXXXXXX 100644
--- a/tcg/arm/tcg-target.c.inc
+++ b/tcg/arm/tcg-target.c.inc
@@ -XXX,XX +XXX,XX @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64)
 static void tcg_out_epilogue(TCGContext *s);
 
 static inline void tcg_out_op(TCGContext *s, TCGOpcode opc,
-                const TCGArg *args, const int *const_args)
+                const TCGArg args[TCG_MAX_OP_ARGS],
+                const int const_args[TCG_MAX_OP_ARGS])
 {
     TCGArg a0, a1, a2, a3, a4, a5;
     int c;
-- 
2.25.1

From: Luis Pires <luis.pires@eldorado.org.br>

Signed-off-by: Luis Pires <luis.pires@eldorado.org.br>
Message-Id: <20210601125143.191165-1-luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 docs/devel/tcg.rst | 101 ++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 90 insertions(+), 11 deletions(-)

diff --git a/docs/devel/tcg.rst b/docs/devel/tcg.rst
index XXXXXXX..XXXXXXX 100644
--- a/docs/devel/tcg.rst
+++ b/docs/devel/tcg.rst
@@ -XXX,XX +XXX,XX @@ performances.
 QEMU's dynamic translation backend is called TCG, for "Tiny Code
 Generator". For more information, please take a look at ``tcg/README``.
 
-Some notable features of QEMU's dynamic translator are:
+The following sections outline some notable features and implementation
+details of QEMU's dynamic translator.
 
 CPU state optimisations
 -----------------------
 
-The target CPUs have many internal states which change the way it
-evaluates instructions. In order to achieve a good speed, the
+The target CPUs have many internal states which change the way they
+evaluate instructions. In order to achieve a good speed, the
 translation phase considers that some state information of the virtual
 CPU cannot change in it. The state is recorded in the Translation
 Block (TB). If the state changes (e.g. privilege level), a new TB will
@@ -XXX,XX +XXX,XX @@ Direct block chaining
 ---------------------
 
 After each translated basic block is executed, QEMU uses the simulated
-Program Counter (PC) and other cpu state information (such as the CS
+Program Counter (PC) and other CPU state information (such as the CS
 segment base value) to find the next basic block.
 
-In order to accelerate the most common cases where the new simulated PC
-is known, QEMU can patch a basic block so that it jumps directly to the
-next one.
+In its simplest, less optimized form, this is done by exiting from the
+current TB, going through the TB epilogue, and then back to the
+main loop. That’s where QEMU looks for the next TB to execute,
+translating it from the guest architecture if it isn’t already available
+in memory. Then QEMU proceeds to execute this next TB, starting at the
+prologue and then moving on to the translated instructions.
 
-The most portable code uses an indirect jump. An indirect jump makes
-it easier to make the jump target modification atomic. On some host
-architectures (such as x86 or PowerPC), the ``JUMP`` opcode is
-directly patched so that the block chaining has no overhead.
+Exiting from the TB this way will cause the ``cpu_exec_interrupt()``
+callback to be re-evaluated before executing additional instructions.
+It is mandatory to exit this way after any CPU state changes that may
+unmask interrupts.
+
+In order to accelerate the cases where the TB for the new
+simulated PC is already available, QEMU has mechanisms that allow
+multiple TBs to be chained directly, without having to go back to the
+main loop as described above. These mechanisms are:
+
+``lookup_and_goto_ptr``
+^^^^^^^^^^^^^^^^^^^^^^^
+
+Calling ``tcg_gen_lookup_and_goto_ptr()`` will emit a call to
+``helper_lookup_tb_ptr``. This helper will look for an existing TB that
+matches the current CPU state. If the destination TB is available its
+code address is returned, otherwise the address of the JIT epilogue is
+returned. The call to the helper is always followed by the tcg ``goto_ptr``
+opcode, which branches to the returned address. In this way, we either
+branch to the next TB or return to the main loop.
+
+``goto_tb + exit_tb``
+^^^^^^^^^^^^^^^^^^^^^
+
+The translation code usually implements branching by performing the
+following steps:
+
+1. Call ``tcg_gen_goto_tb()`` passing a jump slot index (either 0 or 1)
+   as a parameter.
+
+2. Emit TCG instructions to update the CPU state with any information
+   that has been assumed constant and is required by the main loop to
+   correctly locate and execute the next TB. For most guests, this is
+   just the PC of the branch destination, but others may store additional
+   data. The information updated in this step must be inferable from both
+   ``cpu_get_tb_cpu_state()`` and ``cpu_restore_state()``.
+
+3. Call ``tcg_gen_exit_tb()`` passing the address of the current TB and
+   the jump slot index again.
+
+Step 1, ``tcg_gen_goto_tb()``, will emit a ``goto_tb`` TCG
+instruction that later on gets translated to a jump to an address
+associated with the specified jump slot. Initially, this is the address
+of step 2's instructions, which update the CPU state information. Step 3,
+``tcg_gen_exit_tb()``, exits from the current TB returning a tagged
+pointer composed of the last executed TB’s address and the jump slot
+index.
+
+The first time this whole sequence is executed, step 1 simply jumps
+to step 2. Then the CPU state information gets updated and we exit from
+the current TB. As a result, the behavior is very similar to the less
+optimized form described earlier in this section.
+
+Next, the main loop looks for the next TB to execute using the
+current CPU state information (creating the TB if it wasn’t already
+available) and, before starting to execute the new TB’s instructions,
+patches the previously executed TB by associating one of its jump
+slots (the one specified in the call to ``tcg_gen_exit_tb()``) with the
+address of the new TB.
+
+The next time this previous TB is executed and we get to that same
+``goto_tb`` step, it will already be patched (assuming the destination TB
+is still in memory) and will jump directly to the first instruction of
+the destination TB, without going back to the main loop.
+
+For the ``goto_tb + exit_tb`` mechanism to be used, the following
+conditions need to be satisfied:
+
+* The change in CPU state must be constant, e.g., a direct branch and
+  not an indirect branch.
+
+* The direct branch cannot cross a page boundary. Memory mappings
+  may change, causing the code at the destination address to change.
+
+Note that, on step 3 (``tcg_gen_exit_tb()``), in addition to the
+jump slot index, the address of the TB just executed is also returned.
+This address corresponds to the TB that will be patched; it may be
+different than the one that was directly executed from the main loop
+if the latter had already been chained to other TBs.
 
 Self-modifying code and translated code invalidation
 ----------------------------------------------------
-- 
2.25.1