Series comparison

-[PULL 00/16] tcg patch queue
+[PULL v2 00/12] tcg patch queue
-This is v4 of my notdirty + rom patch set with two suggested name
+V2 fixes a build problem that affected win32.
 changes (qemu_build_not_reached, TLB_DISCARD_WRITE) from David and Alex.
 r~
-The following changes since commit 240ab11fb72049d6373cbbec8d788f8e411a00bc:
+The following changes since commit 187f35512106501fe9a11057f4d8705431e0026d:
-  Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20190924' into staging (2019-09-24 15:36:31 +0100)
+  Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-next-251019-3' into staging (2019-10-26 10:13:48 +0100)
 are available in the Git repository at:
-  https://github.com/rth7680/qemu.git tags/pull-tcg-20190925
+  https://github.com/rth7680/qemu.git tags/pull-tcg-20191028
-for you to fetch changes up to ae57db63acf5a0399232f852acc5c1d83ef63400:
+for you to fetch changes up to fe9b676fb3160496b4b2bf0c57d33be724bf04c3:
-  cputlb: Pass retaddr to tb_check_watchpoint (2019-09-25 10:56:28 -0700)
+  translate-all: Remove tb_alloc (2019-10-28 10:35:23 +0100)
 ----------------------------------------------------------------
-Fixes for TLB_BSWAP
+Improvements for TARGET_PAGE_BITS_VARY
-Coversion of NOTDIRTY and ROM handling to cputlb
+Fix for TCI ld16u_i64.
-Followup cleanups to cputlb
+Fix for segv on icount execute from i/o memory.
 Two misc cleanups.
 ----------------------------------------------------------------
-Richard Henderson (16):
+Alex Bennée (1):
-      exec: Use TARGET_PAGE_BITS_MIN for TLB flags
+      cputlb: ensure _cmmu helper functions follow the naming standard
       cputlb: Disable __always_inline__ without optimization
       qemu/compiler.h: Add qemu_build_not_reached
       cputlb: Use qemu_build_not_reached in load/store_helpers
       cputlb: Split out load/store_memop
       cputlb: Introduce TLB_BSWAP
       exec: Adjust notdirty tracing
       cputlb: Move ROM handling from I/O path to TLB path
       cputlb: Move NOTDIRTY handling from I/O path to TLB path
       cputlb: Partially inline memory_region_section_get_iotlb
       cputlb: Merge and move memory_notdirty_write_{prepare,complete}
       cputlb: Handle TLB_NOTDIRTY in probe_access
       cputlb: Remove cpu->mem_io_vaddr
       cputlb: Remove tb_invalidate_phys_page_range is_cpu_write_access
       cputlb: Pass retaddr to tb_invalidate_phys_page_fast
       cputlb: Pass retaddr to tb_check_watchpoint
- accel/tcg/translate-all.h      |   8 +-
+Clement Deschamps (1):
- include/exec/cpu-all.h         |  23 ++-
+      translate-all: fix uninitialized tb->orig_tb
  include/exec/cpu-common.h      |   3 -
  include/exec/exec-all.h        |   6 +-
  include/exec/memory-internal.h |  65 --------
  include/hw/core/cpu.h          |   2 -
  include/qemu/compiler.h        |  26 +++
  accel/tcg/cputlb.c             | 348 +++++++++++++++++++++++++----------------
  accel/tcg/translate-all.c      |  51 +++---
  exec.c                         | 158 +------------------
  hw/core/cpu.c                  |   1 -
  memory.c                       |  20 ---
  trace-events                   |   4 +-
 files changed, 288 insertions(+), 427 deletions(-)
+Richard Henderson (8):
+      exec: Split out variable page size support to exec-vary.c
+      configure: Detect compiler support for __attribute__((alias))
+      exec: Use const alias for TARGET_PAGE_BITS_VARY
+      exec: Restrict TARGET_PAGE_BITS_VARY assert to CONFIG_DEBUG_TCG
+      exec: Promote TARGET_PAGE_MASK to target_long
+      exec: Cache TARGET_PAGE_MASK for TARGET_PAGE_BITS_VARY
+      cputlb: Fix tlb_vaddr_to_host
+      translate-all: Remove tb_alloc
+Stefan Weil (1):
+      tci: Add implementation for INDEX_op_ld16u_i64
+Wei Yang (1):
+      cpu: use ROUND_UP() to define xxx_PAGE_ALIGN
+ Makefile.target                  |   2 +-
+ include/exec/cpu-all.h           |  33 ++++++++----
+ include/exec/cpu_ldst_template.h |   4 +-
+ include/qemu-common.h            |   6 +++
+ tcg/tcg.h                        |  20 +++++---
+ accel/tcg/cputlb.c               |  26 ++++++++--
+ accel/tcg/translate-all.c        |  21 ++------
+ exec-vary.c                      | 108 +++++++++++++++++++++++++++++++++++++++
+ exec.c                           |  34 ------------
+ target/cris/translate_v10.inc.c  |   3 +-
+ tcg/tci.c                        |  15 ++++++
+ configure                        |  19 +++++++
+files changed, 214 insertions(+), 77 deletions(-)
+ create mode 100644 exec-vary.c

-[PULL 11/16] cputlb: Merge and move memory_notdirty_write_{prepare, complete}
+[PULL v2 01/12] tci: Add implementation for INDEX_op_ld16u_i64
-Since 9458a9a1df1a, all readers of the dirty bitmaps wait
+From: Stefan Weil <sw@weilnetz.de>
 for the rcu lock, which means that they wait until the end
 of any executing TranslationBlock.
-As a consequence, there is no need for the actual access
+This fixes "make check-tcg" on a Debian x86_64 host.
 to happen in between the _prepare and _complete.  Therefore,
 we can improve things by merging the two functions into
 notdirty_write and dropping the NotDirtyInfo structure.
-In addition, the only users of notdirty_write are in cputlb.c,
+Signed-off-by: Stefan Weil <sw@weilnetz.de>
-so move the merged function there.  Pass in the CPUIOTLBEntry
+Tested-by: Thomas Huth <thuth@redhat.com>
-from which the ram_addr_t may be computed.
+Message-Id: <20190410194838.10123-1-sw@weilnetz.de>
 Reviewed-by: David Hildenbrand <david@redhat.com>
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/memory-internal.h | 65 -----------------------------
+ tcg/tci.c | 15 +++++++++++++++
- accel/tcg/cputlb.c             | 76 +++++++++++++++++++---------------
+file changed, 15 insertions(+)
  exec.c                         | 44 --------------------
 files changed, 42 insertions(+), 143 deletions(-)
-diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
+diff --git a/tcg/tci.c b/tcg/tci.c
 index XXXXXXX..XXXXXXX 100644
---- a/include/exec/memory-internal.h
+--- a/tcg/tci.c
-+++ b/include/exec/memory-internal.h
++++ b/tcg/tci.c
-@@ -XXX,XX +XXX,XX @@ void address_space_dispatch_free(AddressSpaceDispatch *d);
+@@ -XXX,XX +XXX,XX @@ static void tci_write_reg8(tcg_target_ulong *regs, TCGReg index, uint8_t value)
+     tci_write_reg(regs, index, value);
  void mtree_print_dispatch(struct AddressSpaceDispatch *d,
                            MemoryRegion *root);
 -
 -struct page_collection;
 -
 -/* Opaque struct for passing info from memory_notdirty_write_prepare()
 - * to memory_notdirty_write_complete(). Callers should treat all fields
 - * as private, with the exception of @active.
 - *
 - * @active is a field which is not touched by either the prepare or
 - * complete functions, but which the caller can use if it wishes to
 - * track whether it has called prepare for this struct and so needs
 - * to later call the complete function.
 - */
 -typedef struct {
 -    CPUState *cpu;
 -    struct page_collection *pages;
 -    ram_addr_t ram_addr;
 -    vaddr mem_vaddr;
 -    unsigned size;
 -    bool active;
 -} NotDirtyInfo;
 -
 -/**
 - * memory_notdirty_write_prepare: call before writing to non-dirty memory
 - * @ndi: pointer to opaque NotDirtyInfo struct
 - * @cpu: CPU doing the write
 - * @mem_vaddr: virtual address of write
 - * @ram_addr: the ram address of the write
 - * @size: size of write in bytes
 - *
 - * Any code which writes to the host memory corresponding to
 - * guest RAM which has been marked as NOTDIRTY must wrap those
 - * writes in calls to memory_notdirty_write_prepare() and
 - * memory_notdirty_write_complete():
 - *
 - *  NotDirtyInfo ndi;
 - *  memory_notdirty_write_prepare(&ndi, ....);
 - *  ... perform write here ...
 - *  memory_notdirty_write_complete(&ndi);
 - *
 - * These calls will ensure that we flush any TCG translated code for
 - * the memory being written, update the dirty bits and (if possible)
 - * remove the slowpath callback for writing to the memory.
 - *
 - * This must only be called if we are using TCG; it will assert otherwise.
 - *
 - * We may take locks in the prepare call, so callers must ensure that
 - * they don't exit (via longjump or otherwise) without calling complete.
 - *
 - * This call must only be made inside an RCU critical section.
 - * (Note that while we're executing a TCG TB we're always in an
 - * RCU critical section, which is likely to be the case for callers
 - * of these functions.)
 - */
 -void memory_notdirty_write_prepare(NotDirtyInfo *ndi,
 -                                   CPUState *cpu,
 -                                   vaddr mem_vaddr,
 -                                   ram_addr_t ram_addr,
 -                                   unsigned size);
 -/**
 - * memory_notdirty_write_complete: finish write to non-dirty memory
 - * @ndi: pointer to the opaque NotDirtyInfo struct which was initialized
 - * by memory_not_dirty_write_prepare().
 - */
 -void memory_notdirty_write_complete(NotDirtyInfo *ndi);
 -
  #endif
  #endif
 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cputlb.c
 +++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@
  #include "exec/helper-proto.h"
  #include "qemu/atomic.h"
  #include "qemu/atomic128.h"
 +#include "translate-all.h"
  /* DEBUG defines, enable DEBUG_TLB_LOG to log to the CPU_LOG_MMU target */
  /* #define DEBUG_TLB */
@@ -XXX,XX +XXX,XX @@ tb_page_addr_t get_page_addr_code(CPUArchState *env, target_ulong addr)
      return qemu_ram_addr_from_host_nofail(p);
  }
-+static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
++static void
-+                           CPUIOTLBEntry *iotlbentry, uintptr_t retaddr)
++tci_write_reg16(tcg_target_ulong *regs, TCGReg index, uint16_t value)
 +{
-+    ram_addr_t ram_addr = mem_vaddr + iotlbentry->addr;
++    tci_write_reg(regs, index, value);
 +
 +    trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size);
 +
 +    if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
 +        struct page_collection *pages
 +            = page_collection_lock(ram_addr, ram_addr + size);
 +
 +        /* We require mem_io_pc in tb_invalidate_phys_page_range.  */
 +        cpu->mem_io_pc = retaddr;
 +
 +        tb_invalidate_phys_page_fast(pages, ram_addr, size);
 +        page_collection_unlock(pages);
 +    }
 +
 +    /*
 +     * Set both VGA and migration bits for simplicity and to remove
 +     * the notdirty callback faster.
 +     */
 +    cpu_physical_memory_set_dirty_range(ram_addr, size, DIRTY_CLIENTS_NOCODE);
 +
 +    /* We remove the notdirty callback only if the code has been flushed. */
 +    if (!cpu_physical_memory_is_clean(ram_addr)) {
 +        trace_memory_notdirty_set_dirty(mem_vaddr);
 +        tlb_set_dirty(cpu, mem_vaddr);
 +    }
 +}
 +
- /*
+ static void
-  * Probe for whether the specified guest access is permitted. If it is not
+ tci_write_reg32(tcg_target_ulong *regs, TCGReg index, uint32_t value)
   * permitted then an exception will be taken in the same way as if this
@@ -XXX,XX +XXX,XX @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr,
  /* Probe for a read-modify-write atomic operation.  Do not allow unaligned
   * operations, or io operations to proceed.  Return the host address.  */
  static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
 -                               TCGMemOpIdx oi, uintptr_t retaddr,
 -                               NotDirtyInfo *ndi)
 +                               TCGMemOpIdx oi, uintptr_t retaddr)
  {
-     size_t mmu_idx = get_mmuidx(oi);
+@@ -XXX,XX +XXX,XX @@ uintptr_t tcg_qemu_tb_exec(CPUArchState *env, uint8_t *tb_ptr)
-     uintptr_t index = tlb_index(env, mmu_idx, addr);
+             tci_write_reg8(regs, t0, *(uint8_t *)(t1 + t2));
-@@ -XXX,XX +XXX,XX @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
+             break;
+         case INDEX_op_ld8s_i32:
-     hostaddr = (void *)((uintptr_t)addr + tlbe->addend);
++            TODO();
++            break;
--    ndi->active = false;
+         case INDEX_op_ld16u_i32:
-     if (unlikely(tlb_addr & TLB_NOTDIRTY)) {
+             TODO();
--        ndi->active = true;
+             break;
--        memory_notdirty_write_prepare(ndi, env_cpu(env), addr,
+@@ -XXX,XX +XXX,XX @@ uintptr_t tcg_qemu_tb_exec(CPUArchState *env, uint8_t *tb_ptr)
--                                      qemu_ram_addr_from_host_nofail(hostaddr),
+             tci_write_reg8(regs, t0, *(uint8_t *)(t1 + t2));
--                                      1 << s_bits);
+             break;
-+        notdirty_write(env_cpu(env), addr, 1 << s_bits,
+         case INDEX_op_ld8s_i64:
-+                       &env_tlb(env)->d[mmu_idx].iotlb[index], retaddr);
++            TODO();
-     }
++            break;
+         case INDEX_op_ld16u_i64:
-     return hostaddr;
++            t0 = *tb_ptr++;
-@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
++            t1 = tci_read_r(regs, &tb_ptr);
-             return;
++            t2 = tci_read_s32(&tb_ptr);
-         }
++            tci_write_reg16(regs, t0, *(uint16_t *)(t1 + t2));
++            break;
--        haddr = (void *)((uintptr_t)addr + entry->addend);
+         case INDEX_op_ld16s_i64:
--
+             TODO();
-         /* Handle clean RAM pages.  */
+             break;
          if (tlb_addr & TLB_NOTDIRTY) {
 -            NotDirtyInfo ndi;
 -
 -            /* We require mem_io_pc in tb_invalidate_phys_page_range.  */
 -            env_cpu(env)->mem_io_pc = retaddr;
 -
 -            memory_notdirty_write_prepare(&ndi, env_cpu(env), addr,
 -                                          addr + iotlbentry->addr, size);
 -
 -            if (unlikely(need_swap)) {
 -                store_memop(haddr, val, op ^ MO_BSWAP);
 -            } else {
 -                store_memop(haddr, val, op);
 -            }
 -
 -            memory_notdirty_write_complete(&ndi);
 -            return;
 +            notdirty_write(env_cpu(env), addr, size, iotlbentry, retaddr);
          }
 +        haddr = (void *)((uintptr_t)addr + entry->addend);
 +
          /*
           * Keep these two store_memop separate to ensure that the compiler
           * is able to fold the entire function to a single instruction.
@@ -XXX,XX +XXX,XX @@ void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val,
  #define EXTRA_ARGS     , TCGMemOpIdx oi, uintptr_t retaddr
  #define ATOMIC_NAME(X) \
      HELPER(glue(glue(glue(atomic_ ## X, SUFFIX), END), _mmu))
 -#define ATOMIC_MMU_DECLS NotDirtyInfo ndi
 -#define ATOMIC_MMU_LOOKUP atomic_mmu_lookup(env, addr, oi, retaddr, &ndi)
 -#define ATOMIC_MMU_CLEANUP                              \
 -    do {                                                \
 -        if (unlikely(ndi.active)) {                     \
 -            memory_notdirty_write_complete(&ndi);       \
 -        }                                               \
 -    } while (0)
 +#define ATOMIC_MMU_DECLS
 +#define ATOMIC_MMU_LOOKUP atomic_mmu_lookup(env, addr, oi, retaddr)
 +#define ATOMIC_MMU_CLEANUP
  #define DATA_SIZE 1
  #include "atomic_template.h"
@@ -XXX,XX +XXX,XX @@ void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val,
  #undef ATOMIC_MMU_LOOKUP
  #define EXTRA_ARGS         , TCGMemOpIdx oi
  #define ATOMIC_NAME(X)     HELPER(glue(glue(atomic_ ## X, SUFFIX), END))
 -#define ATOMIC_MMU_LOOKUP  atomic_mmu_lookup(env, addr, oi, GETPC(), &ndi)
 +#define ATOMIC_MMU_LOOKUP  atomic_mmu_lookup(env, addr, oi, GETPC())
  #define DATA_SIZE 1
  #include "atomic_template.h"
 diff --git a/exec.c b/exec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/exec.c
 +++ b/exec.c
@@ -XXX,XX +XXX,XX @@ ram_addr_t qemu_ram_addr_from_host(void *ptr)
      return block->offset + offset;
  }
 -/* Called within RCU critical section. */
 -void memory_notdirty_write_prepare(NotDirtyInfo *ndi,
 -                          CPUState *cpu,
 -                          vaddr mem_vaddr,
 -                          ram_addr_t ram_addr,
 -                          unsigned size)
 -{
 -    ndi->cpu = cpu;
 -    ndi->ram_addr = ram_addr;
 -    ndi->mem_vaddr = mem_vaddr;
 -    ndi->size = size;
 -    ndi->pages = NULL;
 -
 -    trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size);
 -
 -    assert(tcg_enabled());
 -    if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
 -        ndi->pages = page_collection_lock(ram_addr, ram_addr + size);
 -        tb_invalidate_phys_page_fast(ndi->pages, ram_addr, size);
 -    }
 -}
 -
 -/* Called within RCU critical section. */
 -void memory_notdirty_write_complete(NotDirtyInfo *ndi)
 -{
 -    if (ndi->pages) {
 -        assert(tcg_enabled());
 -        page_collection_unlock(ndi->pages);
 -        ndi->pages = NULL;
 -    }
 -
 -    /* Set both VGA and migration bits for simplicity and to remove
 -     * the notdirty callback faster.
 -     */
 -    cpu_physical_memory_set_dirty_range(ndi->ram_addr, ndi->size,
 -                                        DIRTY_CLIENTS_NOCODE);
 -    /* we remove the notdirty callback only if the code has been
 -       flushed */
 -    if (!cpu_physical_memory_is_clean(ndi->ram_addr)) {
 -        trace_memory_notdirty_set_dirty(ndi->mem_vaddr);
 -        tlb_set_dirty(ndi->cpu, ndi->mem_vaddr);
 -    }
 -}
 -
  /* Generate a debug exception if a watchpoint has been hit.  */
  void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
                            MemTxAttrs attrs, int flags, uintptr_t ra)
 --
 .17.1

-[PULL 05/16] cputlb: Split out load/store_memop
+[PULL v2 02/12] cputlb: ensure _cmmu helper functions follow the naming standard
-We will shortly be using these more than once.
+From: Alex Bennée <alex.bennee@linaro.org>
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+We document this in docs/devel/load-stores.rst so lets follow it. The
-Reviewed-by: David Hildenbrand <david@redhat.com>
+bit and 64 bit access functions have historically not included the
 sign so we leave those as is. We also introduce some signed helpers
 which are used for loading immediate values in the translator.
 Fixes: 282dffc8
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
 Message-Id: <20191021150910.23216-1-alex.bennee@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/cputlb.c | 107 +++++++++++++++++++++++----------------------
+ include/exec/cpu_ldst_template.h |  4 ++--
-file changed, 55 insertions(+), 52 deletions(-)
+ tcg/tcg.h                        | 20 ++++++++++++++------
  accel/tcg/cputlb.c               | 24 +++++++++++++++++++++---
  target/cris/translate_v10.inc.c  |  3 +--
 files changed, 38 insertions(+), 13 deletions(-)
+diff --git a/include/exec/cpu_ldst_template.h b/include/exec/cpu_ldst_template.h
+index XXXXXXX..XXXXXXX 100644
+--- a/include/exec/cpu_ldst_template.h
++++ b/include/exec/cpu_ldst_template.h
+@@ -XXX,XX +XXX,XX @@
+ #ifdef SOFTMMU_CODE_ACCESS
+ #define ADDR_READ addr_code
+ #define MMUSUFFIX _cmmu
+-#define URETSUFFIX SUFFIX
+-#define SRETSUFFIX SUFFIX
++#define URETSUFFIX USUFFIX
++#define SRETSUFFIX glue(s, SUFFIX)
+ #else
+ #define ADDR_READ addr_read
+ #define MMUSUFFIX _mmu
+diff --git a/tcg/tcg.h b/tcg/tcg.h
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/tcg.h
++++ b/tcg/tcg.h
+@@ -XXX,XX +XXX,XX @@ void helper_be_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val,
+ void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val,
+                        TCGMemOpIdx oi, uintptr_t retaddr);
+-uint8_t helper_ret_ldb_cmmu(CPUArchState *env, target_ulong addr,
++uint8_t helper_ret_ldub_cmmu(CPUArchState *env, target_ulong addr,
+                             TCGMemOpIdx oi, uintptr_t retaddr);
+-uint16_t helper_le_ldw_cmmu(CPUArchState *env, target_ulong addr,
++int8_t helper_ret_ldsb_cmmu(CPUArchState *env, target_ulong addr,
+                             TCGMemOpIdx oi, uintptr_t retaddr);
++uint16_t helper_le_lduw_cmmu(CPUArchState *env, target_ulong addr,
++                             TCGMemOpIdx oi, uintptr_t retaddr);
++int16_t helper_le_ldsw_cmmu(CPUArchState *env, target_ulong addr,
++                             TCGMemOpIdx oi, uintptr_t retaddr);
+ uint32_t helper_le_ldl_cmmu(CPUArchState *env, target_ulong addr,
+                             TCGMemOpIdx oi, uintptr_t retaddr);
+ uint64_t helper_le_ldq_cmmu(CPUArchState *env, target_ulong addr,
+                             TCGMemOpIdx oi, uintptr_t retaddr);
+-uint16_t helper_be_ldw_cmmu(CPUArchState *env, target_ulong addr,
+-                            TCGMemOpIdx oi, uintptr_t retaddr);
++uint16_t helper_be_lduw_cmmu(CPUArchState *env, target_ulong addr,
++                             TCGMemOpIdx oi, uintptr_t retaddr);
++int16_t helper_be_ldsw_cmmu(CPUArchState *env, target_ulong addr,
++                             TCGMemOpIdx oi, uintptr_t retaddr);
+ uint32_t helper_be_ldl_cmmu(CPUArchState *env, target_ulong addr,
+                             TCGMemOpIdx oi, uintptr_t retaddr);
+ uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr,
+@@ -XXX,XX +XXX,XX @@ uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr,
+ # define helper_ret_stw_mmu   helper_be_stw_mmu
+ # define helper_ret_stl_mmu   helper_be_stl_mmu
+ # define helper_ret_stq_mmu   helper_be_stq_mmu
+-# define helper_ret_ldw_cmmu  helper_be_ldw_cmmu
++# define helper_ret_lduw_cmmu  helper_be_lduw_cmmu
++# define helper_ret_ldsw_cmmu  helper_be_ldsw_cmmu
+ # define helper_ret_ldl_cmmu  helper_be_ldl_cmmu
+ # define helper_ret_ldq_cmmu  helper_be_ldq_cmmu
+ #else
+@@ -XXX,XX +XXX,XX @@ uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr,
+ # define helper_ret_stw_mmu   helper_le_stw_mmu
+ # define helper_ret_stl_mmu   helper_le_stl_mmu
+ # define helper_ret_stq_mmu   helper_le_stq_mmu
+-# define helper_ret_ldw_cmmu  helper_le_ldw_cmmu
++# define helper_ret_lduw_cmmu  helper_le_lduw_cmmu
++# define helper_ret_ldsw_cmmu  helper_le_ldsw_cmmu
+ # define helper_ret_ldl_cmmu  helper_le_ldl_cmmu
+ # define helper_ret_ldq_cmmu  helper_le_ldq_cmmu
+ #endif
 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cputlb.c
 +++ b/accel/tcg/cputlb.c
-@@ -XXX,XX +XXX,XX @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
+@@ -XXX,XX +XXX,XX @@ static uint64_t full_ldub_cmmu(CPUArchState *env, target_ulong addr,
- typedef uint64_t FullLoadHelper(CPUArchState *env, target_ulong addr,
+     return load_helper(env, addr, oi, retaddr, MO_8, true, full_ldub_cmmu);
-                                 TCGMemOpIdx oi, uintptr_t retaddr);
+ }
-+static inline uint64_t QEMU_ALWAYS_INLINE
+-uint8_t helper_ret_ldb_cmmu(CPUArchState *env, target_ulong addr,
-+load_memop(const void *haddr, MemOp op)
++uint8_t helper_ret_ldub_cmmu(CPUArchState *env, target_ulong addr,
                              TCGMemOpIdx oi, uintptr_t retaddr)
  {
      return full_ldub_cmmu(env, addr, oi, retaddr);
  }
 +int8_t helper_ret_ldsb_cmmu(CPUArchState *env, target_ulong addr,
 +                            TCGMemOpIdx oi, uintptr_t retaddr)
 +{
-+    switch (op) {
++    return (int8_t) full_ldub_cmmu(env, addr, oi, retaddr);
 +    case MO_UB:
 +        return ldub_p(haddr);
 +    case MO_BEUW:
 +        return lduw_be_p(haddr);
 +    case MO_LEUW:
 +        return lduw_le_p(haddr);
 +    case MO_BEUL:
 +        return (uint32_t)ldl_be_p(haddr);
 +    case MO_LEUL:
 +        return (uint32_t)ldl_le_p(haddr);
 +    case MO_BEQ:
 +        return ldq_be_p(haddr);
 +    case MO_LEQ:
 +        return ldq_le_p(haddr);
 +    default:
 +        qemu_build_not_reached();
 +    }
 +}
 +
- static inline uint64_t QEMU_ALWAYS_INLINE
+ static uint64_t full_le_lduw_cmmu(CPUArchState *env, target_ulong addr,
- load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
+                                   TCGMemOpIdx oi, uintptr_t retaddr)
-             uintptr_t retaddr, MemOp op, bool code_read,
+ {
-@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
+@@ -XXX,XX +XXX,XX @@ static uint64_t full_le_lduw_cmmu(CPUArchState *env, target_ulong addr,
+                        full_le_lduw_cmmu);
   do_aligned_access:
      haddr = (void *)((uintptr_t)addr + entry->addend);
 -    switch (op) {
 -    case MO_UB:
 -        res = ldub_p(haddr);
 -        break;
 -    case MO_BEUW:
 -        res = lduw_be_p(haddr);
 -        break;
 -    case MO_LEUW:
 -        res = lduw_le_p(haddr);
 -        break;
 -    case MO_BEUL:
 -        res = (uint32_t)ldl_be_p(haddr);
 -        break;
 -    case MO_LEUL:
 -        res = (uint32_t)ldl_le_p(haddr);
 -        break;
 -    case MO_BEQ:
 -        res = ldq_be_p(haddr);
 -        break;
 -    case MO_LEQ:
 -        res = ldq_le_p(haddr);
 -        break;
 -    default:
 -        qemu_build_not_reached();
 -    }
 -
 -    return res;
 +    return load_memop(haddr, op);
  }
- /*
+-uint16_t helper_le_ldw_cmmu(CPUArchState *env, target_ulong addr,
-@@ -XXX,XX +XXX,XX @@ tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *env, target_ulong addr,
++uint16_t helper_le_lduw_cmmu(CPUArchState *env, target_ulong addr,
-  * Store Helpers
+                             TCGMemOpIdx oi, uintptr_t retaddr)
-  */
+ {
+     return full_le_lduw_cmmu(env, addr, oi, retaddr);
-+static inline void QEMU_ALWAYS_INLINE
+ }
-+store_memop(void *haddr, uint64_t val, MemOp op)
 +int16_t helper_le_ldsw_cmmu(CPUArchState *env, target_ulong addr,
 +                            TCGMemOpIdx oi, uintptr_t retaddr)
 +{
-+    switch (op) {
++    return (int16_t) full_le_lduw_cmmu(env, addr, oi, retaddr);
 +    case MO_UB:
 +        stb_p(haddr, val);
 +        break;
 +    case MO_BEUW:
 +        stw_be_p(haddr, val);
 +        break;
 +    case MO_LEUW:
 +        stw_le_p(haddr, val);
 +        break;
 +    case MO_BEUL:
 +        stl_be_p(haddr, val);
 +        break;
 +    case MO_LEUL:
 +        stl_le_p(haddr, val);
 +        break;
 +    case MO_BEQ:
 +        stq_be_p(haddr, val);
 +        break;
 +    case MO_LEQ:
 +        stq_le_p(haddr, val);
 +        break;
 +    default:
 +        qemu_build_not_reached();
 +    }
 +}
 +
- static inline void QEMU_ALWAYS_INLINE
+ static uint64_t full_be_lduw_cmmu(CPUArchState *env, target_ulong addr,
- store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
+                                   TCGMemOpIdx oi, uintptr_t retaddr)
-              TCGMemOpIdx oi, uintptr_t retaddr, MemOp op)
+ {
-@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
+@@ -XXX,XX +XXX,XX @@ static uint64_t full_be_lduw_cmmu(CPUArchState *env, target_ulong addr,
+                        full_be_lduw_cmmu);
   do_aligned_access:
      haddr = (void *)((uintptr_t)addr + entry->addend);
 -    switch (op) {
 -    case MO_UB:
 -        stb_p(haddr, val);
 -        break;
 -    case MO_BEUW:
 -        stw_be_p(haddr, val);
 -        break;
 -    case MO_LEUW:
 -        stw_le_p(haddr, val);
 -        break;
 -    case MO_BEUL:
 -        stl_be_p(haddr, val);
 -        break;
 -    case MO_LEUL:
 -        stl_le_p(haddr, val);
 -        break;
 -    case MO_BEQ:
 -        stq_be_p(haddr, val);
 -        break;
 -    case MO_LEQ:
 -        stq_le_p(haddr, val);
 -        break;
 -    default:
 -        qemu_build_not_reached();
 -    }
 +    store_memop(haddr, val, op);
  }
- void helper_ret_stb_mmu(CPUArchState *env, target_ulong addr, uint8_t val,
+-uint16_t helper_be_ldw_cmmu(CPUArchState *env, target_ulong addr,
 +uint16_t helper_be_lduw_cmmu(CPUArchState *env, target_ulong addr,
                              TCGMemOpIdx oi, uintptr_t retaddr)
  {
      return full_be_lduw_cmmu(env, addr, oi, retaddr);
  }
 +int16_t helper_be_ldsw_cmmu(CPUArchState *env, target_ulong addr,
 +                            TCGMemOpIdx oi, uintptr_t retaddr)
 +{
 +    return (int16_t) full_be_lduw_cmmu(env, addr, oi, retaddr);
 +}
 +
  static uint64_t full_le_ldul_cmmu(CPUArchState *env, target_ulong addr,
                                    TCGMemOpIdx oi, uintptr_t retaddr)
  {
 diff --git a/target/cris/translate_v10.inc.c b/target/cris/translate_v10.inc.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/cris/translate_v10.inc.c
 +++ b/target/cris/translate_v10.inc.c
@@ -XXX,XX +XXX,XX @@ static unsigned int dec10_ind(CPUCRISState *env, DisasContext *dc)
          case CRISV10_IND_BCC_M:
              cris_cc_mask(dc, 0);
 -            imm = cpu_ldsw_code(env, dc->pc + 2);
 -            simm = (int16_t)imm;
 +            simm = cpu_ldsw_code(env, dc->pc + 2);
              simm += 4;
              LOG_DIS("bcc_m: b%s %x\n", cc_name(dc->cond), dc->pc + simm);
 --
 .17.1

-[PULL 15/16] cputlb: Pass retaddr to tb_invalidate_phys_page_fast
+[PULL v2 03/12] cpu: use ROUND_UP() to define xxx_PAGE_ALIGN
-Rather than rely on cpu->mem_io_pc, pass retaddr down directly.
+From: Wei Yang <richardw.yang@linux.intel.com>
-Within tb_invalidate_phys_page_range__locked, the is_cpu_write_access
+Use ROUND_UP() to define, which is a little bit easy to read.
 parameter is non-zero exactly when retaddr would be non-zero, so that
 is a simple replacement.
 Recognize that current_tb_not_found is true only when mem_io_pc
 (and now retaddr) are also non-zero, so remove a redundant test.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Reviewed-by: David Hildenbrand <david@redhat.com>
+Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
 Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
 Reviewed-by: Juan Quintela <quintela@redhat.com>
 Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
 Message-Id: <20191013021145.16011-2-richardw.yang@linux.intel.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/translate-all.h |  3 ++-
+ include/exec/cpu-all.h | 7 +++----
- accel/tcg/cputlb.c        |  6 +-----
+file changed, 3 insertions(+), 4 deletions(-)
  accel/tcg/translate-all.c | 39 +++++++++++++++++++--------------------
 files changed, 22 insertions(+), 26 deletions(-)
-diff --git a/accel/tcg/translate-all.h b/accel/tcg/translate-all.h
+diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/translate-all.h
+--- a/include/exec/cpu-all.h
-+++ b/accel/tcg/translate-all.h
++++ b/include/exec/cpu-all.h
-@@ -XXX,XX +XXX,XX @@ struct page_collection *page_collection_lock(tb_page_addr_t start,
+@@ -XXX,XX +XXX,XX @@ extern int target_page_bits;
-                                              tb_page_addr_t end);
- void page_collection_unlock(struct page_collection *set);
+ #define TARGET_PAGE_SIZE (1 << TARGET_PAGE_BITS)
- void tb_invalidate_phys_page_fast(struct page_collection *pages,
+ #define TARGET_PAGE_MASK ~(TARGET_PAGE_SIZE - 1)
--                                  tb_page_addr_t start, int len);
+-#define TARGET_PAGE_ALIGN(addr) (((addr) + TARGET_PAGE_SIZE - 1) & TARGET_PAGE_MASK)
-+                                  tb_page_addr_t start, int len,
++#define TARGET_PAGE_ALIGN(addr) ROUND_UP((addr), TARGET_PAGE_SIZE)
-+                                  uintptr_t retaddr);
- void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end);
+ /* Using intptr_t ensures that qemu_*_page_mask is sign-extended even
- void tb_check_watchpoint(CPUState *cpu);
+  * when intptr_t is 32-bit and we are aligning a long long.
+@@ -XXX,XX +XXX,XX @@ extern int target_page_bits;
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
+ extern uintptr_t qemu_host_page_size;
-index XXXXXXX..XXXXXXX 100644
+ extern intptr_t qemu_host_page_mask;
---- a/accel/tcg/cputlb.c
-+++ b/accel/tcg/cputlb.c
+-#define HOST_PAGE_ALIGN(addr) (((addr) + qemu_host_page_size - 1) & qemu_host_page_mask)
-@@ -XXX,XX +XXX,XX @@ static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
+-#define REAL_HOST_PAGE_ALIGN(addr) (((addr) + qemu_real_host_page_size - 1) & \
-     if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
+-                                    qemu_real_host_page_mask)
-         struct page_collection *pages
++#define HOST_PAGE_ALIGN(addr) ROUND_UP((addr), qemu_host_page_size)
-             = page_collection_lock(ram_addr, ram_addr + size);
++#define REAL_HOST_PAGE_ALIGN(addr) ROUND_UP((addr), qemu_real_host_page_size)
--
--        /* We require mem_io_pc in tb_invalidate_phys_page_range.  */
+ /* same as PROT_xxx */
--        cpu->mem_io_pc = retaddr;
+ #define PAGE_READ      0x0001
 -
 -        tb_invalidate_phys_page_fast(pages, ram_addr, size);
 +        tb_invalidate_phys_page_fast(pages, ram_addr, size, retaddr);
          page_collection_unlock(pages);
      }
 diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/translate-all.c
 +++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static void
  tb_invalidate_phys_page_range__locked(struct page_collection *pages,
                                        PageDesc *p, tb_page_addr_t start,
                                        tb_page_addr_t end,
 -                                      int is_cpu_write_access)
 +                                      uintptr_t retaddr)
  {
      TranslationBlock *tb;
      tb_page_addr_t tb_start, tb_end;
@@ -XXX,XX +XXX,XX @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
  #ifdef TARGET_HAS_PRECISE_SMC
      CPUState *cpu = current_cpu;
      CPUArchState *env = NULL;
 -    int current_tb_not_found = is_cpu_write_access;
 +    bool current_tb_not_found = retaddr != 0;
 +    bool current_tb_modified = false;
      TranslationBlock *current_tb = NULL;
 -    int current_tb_modified = 0;
      target_ulong current_pc = 0;
      target_ulong current_cs_base = 0;
      uint32_t current_flags = 0;
@@ -XXX,XX +XXX,XX @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
          if (!(tb_end <= start || tb_start >= end)) {
  #ifdef TARGET_HAS_PRECISE_SMC
              if (current_tb_not_found) {
 -                current_tb_not_found = 0;
 -                current_tb = NULL;
 -                if (cpu->mem_io_pc) {
 -                    /* now we have a real cpu fault */
 -                    current_tb = tcg_tb_lookup(cpu->mem_io_pc);
 -                }
 +                current_tb_not_found = false;
 +                /* now we have a real cpu fault */
 +                current_tb = tcg_tb_lookup(retaddr);
              }
              if (current_tb == tb &&
                  (tb_cflags(current_tb) & CF_COUNT_MASK) != 1) {
 -                /* If we are modifying the current TB, we must stop
 -                its execution. We could be more precise by checking
 -                that the modification is after the current PC, but it
 -                would require a specialized function to partially
 -                restore the CPU state */
 -
 -                current_tb_modified = 1;
 -                cpu_restore_state_from_tb(cpu, current_tb,
 -                                          cpu->mem_io_pc, true);
 +                /*
 +                 * If we are modifying the current TB, we must stop
 +                 * its execution. We could be more precise by checking
 +                 * that the modification is after the current PC, but it
 +                 * would require a specialized function to partially
 +                 * restore the CPU state.
 +                 */
 +                current_tb_modified = true;
 +                cpu_restore_state_from_tb(cpu, current_tb, retaddr, true);
                  cpu_get_tb_cpu_state(env, &current_pc, &current_cs_base,
                                       &current_flags);
              }
@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_range(target_ulong start, target_ulong end)
   * Call with all @pages in the range [@start, @start + len[ locked.
   */
  void tb_invalidate_phys_page_fast(struct page_collection *pages,
 -                                  tb_page_addr_t start, int len)
 +                                  tb_page_addr_t start, int len,
 +                                  uintptr_t retaddr)
  {
      PageDesc *p;
@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_page_fast(struct page_collection *pages,
          }
      } else {
      do_invalidate:
 -        tb_invalidate_phys_page_range__locked(pages, p, start, start + len, 1);
 +        tb_invalidate_phys_page_range__locked(pages, p, start, start + len,
 +                                              retaddr);
      }
  }
  #else
 --
 .17.1

-[PULL 09/16] cputlb: Move NOTDIRTY handling from I/O path to TLB path
+[PULL v2 04/12] exec: Split out variable page size support to exec-vary.c
-Pages that we want to track for NOTDIRTY are RAM.  We do not
+The next patch will play a trick with "const" that will
-really need to go through the I/O path to handle them.
+confuse the compiler about the uses of target_page_bits
 within exec.c.  Moving everything to a new file prevents
 this confusion.
-Acked-by: David Hildenbrand <david@redhat.com>
+No functional change so far.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/cpu-common.h |  2 --
+ Makefile.target       |  2 +-
- accel/tcg/cputlb.c        | 26 +++++++++++++++++---
+ include/qemu-common.h |  6 +++++
- exec.c                    | 50 ---------------------------------------
+ exec-vary.c           | 57 +++++++++++++++++++++++++++++++++++++++++++
- memory.c                  | 16 -------------
+ exec.c                | 34 --------------------------
-files changed, 23 insertions(+), 71 deletions(-)
+files changed, 64 insertions(+), 35 deletions(-)
  create mode 100644 exec-vary.c
-diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
+diff --git a/Makefile.target b/Makefile.target
 index XXXXXXX..XXXXXXX 100644
---- a/include/exec/cpu-common.h
+--- a/Makefile.target
-+++ b/include/exec/cpu-common.h
++++ b/Makefile.target
-@@ -XXX,XX +XXX,XX @@ void qemu_flush_coalesced_mmio_buffer(void);
+@@ -XXX,XX +XXX,XX @@ obj-y += trace/
- void cpu_flush_icache_range(hwaddr start, hwaddr len);
+ #########################################################
+ # cpu emulator library
--extern struct MemoryRegion io_mem_notdirty;
+-obj-y += exec.o
--
++obj-y += exec.o exec-vary.o
- typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);
+ obj-y += accel/
+ obj-$(CONFIG_TCG) += tcg/tcg.o tcg/tcg-op.o tcg/tcg-op-vec.o tcg/tcg-op-gvec.o
- int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque);
+ obj-$(CONFIG_TCG) += tcg/tcg-common.o tcg/optimize.o
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
+diff --git a/include/qemu-common.h b/include/qemu-common.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
+--- a/include/qemu-common.h
-+++ b/accel/tcg/cputlb.c
++++ b/include/qemu-common.h
-@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
+@@ -XXX,XX +XXX,XX @@ void cpu_exec_step_atomic(CPUState *cpu);
-     mr = section->mr;
+  */
-     mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
+ bool set_preferred_target_page_bits(int bits);
-     cpu->mem_io_pc = retaddr;
--    if (mr != &io_mem_notdirty && !cpu->can_do_io) {
++/**
-+    if (!cpu->can_do_io) {
++ * finalize_target_page_bits:
-         cpu_io_recompile(cpu, retaddr);
++ * Commit the final value set by set_preferred_target_page_bits.
-     }
++ */
++void finalize_target_page_bits(void);
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
      section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
      mr = section->mr;
      mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
 -    if (mr != &io_mem_notdirty && !cpu->can_do_io) {
 +    if (!cpu->can_do_io) {
          cpu_io_recompile(cpu, retaddr);
      }
      cpu->mem_io_vaddr = addr;
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
          need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
          /* Handle I/O access.  */
 -        if (likely(tlb_addr & (TLB_MMIO | TLB_NOTDIRTY))) {
 +        if (tlb_addr & TLB_MMIO) {
              io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr,
                        op ^ (need_swap * MO_BSWAP));
              return;
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
          haddr = (void *)((uintptr_t)addr + entry->addend);
 +        /* Handle clean RAM pages.  */
 +        if (tlb_addr & TLB_NOTDIRTY) {
 +            NotDirtyInfo ndi;
 +
-+            /* We require mem_io_pc in tb_invalidate_phys_page_range.  */
+ /**
-+            env_cpu(env)->mem_io_pc = retaddr;
+  * Sends a (part of) iovec down a socket, yielding when the socket is full, or
   * Receives data into a (part of) iovec from a socket,
 diff --git a/exec-vary.c b/exec-vary.c
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/exec-vary.c
@@ -XXX,XX +XXX,XX @@
 +/*
 + * Variable page size handling
 + *
 + *  Copyright (c) 2003 Fabrice Bellard
 + *
 + * This library is free software; you can redistribute it and/or
 + * modify it under the terms of the GNU Lesser General Public
 + * License as published by the Free Software Foundation; either
 + * version 2 of the License, or (at your option) any later version.
 + *
 + * This library is distributed in the hope that it will be useful,
 + * but WITHOUT ANY WARRANTY; without even the implied warranty of
 + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 + * Lesser General Public License for more details.
 + *
 + * You should have received a copy of the GNU Lesser General Public
 + * License along with this library; if not, see <http://www.gnu.org/licenses/>.
 + */
 +
-+            memory_notdirty_write_prepare(&ndi, env_cpu(env), addr,
++#include "qemu/osdep.h"
-+                                          addr + iotlbentry->addr, size);
++#include "qemu-common.h"
 +#include "exec/exec-all.h"
 +
-+            if (unlikely(need_swap)) {
++#ifdef TARGET_PAGE_BITS_VARY
-+                store_memop(haddr, val, op ^ MO_BSWAP);
++int target_page_bits;
-+            } else {
++bool target_page_bits_decided;
-+                store_memop(haddr, val, op);
++#endif
 +            }
 +
-+            memory_notdirty_write_complete(&ndi);
++bool set_preferred_target_page_bits(int bits)
-+            return;
++{
 +    /*
 +     * The target page size is the lowest common denominator for all
 +     * the CPUs in the system, so we can only make it smaller, never
 +     * larger. And we can't make it smaller once we've committed to
 +     * a particular size.
 +     */
 +#ifdef TARGET_PAGE_BITS_VARY
 +    assert(bits >= TARGET_PAGE_BITS_MIN);
 +    if (target_page_bits == 0 || target_page_bits > bits) {
 +        if (target_page_bits_decided) {
 +            return false;
 +        }
++        target_page_bits = bits;
++    }
++#endif
++    return true;
++}
 +
-         /*
++void finalize_target_page_bits(void)
-          * Keep these two store_memop separate to ensure that the compiler
++{
-          * is able to fold the entire function to a single instruction.
++#ifdef TARGET_PAGE_BITS_VARY
 +    if (target_page_bits == 0) {
 +        target_page_bits = TARGET_PAGE_BITS_MIN;
 +    }
 +    target_page_bits_decided = true;
 +#endif
 +}
 diff --git a/exec.c b/exec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/exec.c
 +++ b/exec.c
-@@ -XXX,XX +XXX,XX @@ static MemoryRegion *system_io;
+@@ -XXX,XX +XXX,XX @@ AddressSpace address_space_memory;
  AddressSpace address_space_io;
  AddressSpace address_space_memory;
 -MemoryRegion io_mem_notdirty;
  static MemoryRegion io_mem_unassigned;
  #endif
-@@ -XXX,XX +XXX,XX @@ typedef struct subpage_t {
+-#ifdef TARGET_PAGE_BITS_VARY
- } subpage_t;
+-int target_page_bits;
+-bool target_page_bits_decided;
- #define PHYS_SECTION_UNASSIGNED 0
+-#endif
--#define PHYS_SECTION_NOTDIRTY 1
+-
+ CPUTailQ cpus = QTAILQ_HEAD_INITIALIZER(cpus);
- static void io_mem_init(void);
- static void memory_map_init(void);
+ /* current CPU in the current thread. It is only valid inside
-@@ -XXX,XX +XXX,XX @@ hwaddr memory_region_section_get_iotlb(CPUState *cpu,
+@@ -XXX,XX +XXX,XX @@ int use_icount;
-     if (memory_region_is_ram(section->mr)) {
+ uintptr_t qemu_host_page_size;
-         /* Normal RAM.  */
+ intptr_t qemu_host_page_mask;
-         iotlb = memory_region_get_ram_addr(section->mr) + xlat;
--        if (!section->readonly) {
+-bool set_preferred_target_page_bits(int bits)
--            iotlb |= PHYS_SECTION_NOTDIRTY;
+-{
 -    /* The target page size is the lowest common denominator for all
 -     * the CPUs in the system, so we can only make it smaller, never
 -     * larger. And we can't make it smaller once we've committed to
 -     * a particular size.
 -     */
 -#ifdef TARGET_PAGE_BITS_VARY
 -    assert(bits >= TARGET_PAGE_BITS_MIN);
 -    if (target_page_bits == 0 || target_page_bits > bits) {
 -        if (target_page_bits_decided) {
 -            return false;
 -        }
-     } else {
+-        target_page_bits = bits;
-         AddressSpaceDispatch *d;
+-    }
+-#endif
-@@ -XXX,XX +XXX,XX @@ void memory_notdirty_write_complete(NotDirtyInfo *ndi)
+-    return true;
      }
  }
 -/* Called within RCU critical section.  */
 -static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
 -                               uint64_t val, unsigned size)
 -{
 -    NotDirtyInfo ndi;
 -
 -    memory_notdirty_write_prepare(&ndi, current_cpu, current_cpu->mem_io_vaddr,
 -                         ram_addr, size);
 -
 -    stn_p(qemu_map_ram_ptr(NULL, ram_addr), size, val);
 -    memory_notdirty_write_complete(&ndi);
 -}
 -
--static bool notdirty_mem_accepts(void *opaque, hwaddr addr,
+ #if !defined(CONFIG_USER_ONLY)
--                                 unsigned size, bool is_write,
--                                 MemTxAttrs attrs)
+-static void finalize_target_page_bits(void)
 -{
--    return is_write;
+-#ifdef TARGET_PAGE_BITS_VARY
 -    if (target_page_bits == 0) {
 -        target_page_bits = TARGET_PAGE_BITS_MIN;
 -    }
 -    target_page_bits_decided = true;
 -#endif
 -}
 -
--static const MemoryRegionOps notdirty_mem_ops = {
+ typedef struct PhysPageEntry PhysPageEntry;
--    .write = notdirty_mem_write,
--    .valid.accepts = notdirty_mem_accepts,
+ struct PhysPageEntry {
 -    .endianness = DEVICE_NATIVE_ENDIAN,
 -    .valid = {
 -        .min_access_size = 1,
 -        .max_access_size = 8,
 -        .unaligned = false,
 -    },
 -    .impl = {
 -        .min_access_size = 1,
 -        .max_access_size = 8,
 -        .unaligned = false,
 -    },
 -};
 -
  /* Generate a debug exception if a watchpoint has been hit.  */
  void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
                            MemTxAttrs attrs, int flags, uintptr_t ra)
@@ -XXX,XX +XXX,XX @@ static void io_mem_init(void)
  {
      memory_region_init_io(&io_mem_unassigned, NULL, &unassigned_mem_ops, NULL,
                            NULL, UINT64_MAX);
 -
 -    /* io_mem_notdirty calls tb_invalidate_phys_page_fast,
 -     * which can be called without the iothread mutex.
 -     */
 -    memory_region_init_io(&io_mem_notdirty, NULL, &notdirty_mem_ops, NULL,
 -                          NULL, UINT64_MAX);
 -    memory_region_clear_global_locking(&io_mem_notdirty);
  }
  AddressSpaceDispatch *address_space_dispatch_new(FlatView *fv)
@@ -XXX,XX +XXX,XX @@ AddressSpaceDispatch *address_space_dispatch_new(FlatView *fv)
      n = dummy_section(&d->map, fv, &io_mem_unassigned);
      assert(n == PHYS_SECTION_UNASSIGNED);
 -    n = dummy_section(&d->map, fv, &io_mem_notdirty);
 -    assert(n == PHYS_SECTION_NOTDIRTY);
      d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .skip = 1 };
 diff --git a/memory.c b/memory.c
 index XXXXXXX..XXXXXXX 100644
 --- a/memory.c
 +++ b/memory.c
@@ -XXX,XX +XXX,XX @@ static MemTxResult  memory_region_read_accessor(MemoryRegion *mr,
      tmp = mr->ops->read(mr->opaque, addr, size);
      if (mr->subpage) {
          trace_memory_region_subpage_read(get_cpu_index(), mr, addr, tmp, size);
 -    } else if (mr == &io_mem_notdirty) {
 -        /* Accesses to code which has previously been translated into a TB show
 -         * up in the MMIO path, as accesses to the io_mem_notdirty
 -         * MemoryRegion. */
      } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) {
          hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
          trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_read_with_attrs_accessor(MemoryRegion *mr,
      r = mr->ops->read_with_attrs(mr->opaque, addr, &tmp, size, attrs);
      if (mr->subpage) {
          trace_memory_region_subpage_read(get_cpu_index(), mr, addr, tmp, size);
 -    } else if (mr == &io_mem_notdirty) {
 -        /* Accesses to code which has previously been translated into a TB show
 -         * up in the MMIO path, as accesses to the io_mem_notdirty
 -         * MemoryRegion. */
      } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) {
          hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
          trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_write_accessor(MemoryRegion *mr,
      if (mr->subpage) {
          trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, size);
 -    } else if (mr == &io_mem_notdirty) {
 -        /* Accesses to code which has previously been translated into a TB show
 -         * up in the MMIO path, as accesses to the io_mem_notdirty
 -         * MemoryRegion. */
      } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) {
          hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
          trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_write_with_attrs_accessor(MemoryRegion *mr,
      if (mr->subpage) {
          trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, size);
 -    } else if (mr == &io_mem_notdirty) {
 -        /* Accesses to code which has previously been translated into a TB show
 -         * up in the MMIO path, as accesses to the io_mem_notdirty
 -         * MemoryRegion. */
      } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) {
          hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
          trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size);
 --
 .17.1

-[PULL 10/16] cputlb: Partially inline memory_region_section_get_iotlb
+[PULL v2 05/12] configure: Detect compiler support for __attribute__((alias))
-There is only one caller, tlb_set_page_with_attrs.  We cannot
+Such support is present almost everywhere, except for Xcode 9.
-inline the entire function because the AddressSpaceDispatch
+It is added in Xcode 10, but travis uses xcode9 by default,
-structure is private to exec.c, and cannot easily be moved to
+so we should support it for a while yet.
 include/exec/memory-internal.h.
 Compute is_ram and is_romd once within tlb_set_page_with_attrs.
 Fold the number of tests against these predicates.  Compute
 cpu_physical_memory_is_clean outside of the tlb lock region.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/exec-all.h |  6 +---
+ configure | 19 +++++++++++++++++++
- accel/tcg/cputlb.c      | 68 ++++++++++++++++++++++++++---------------
+file changed, 19 insertions(+)
  exec.c                  | 22 ++-----------
 files changed, 47 insertions(+), 49 deletions(-)
-diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
+diff --git a/configure b/configure
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/include/exec/exec-all.h
+--- a/configure
-+++ b/include/exec/exec-all.h
++++ b/configure
-@@ -XXX,XX +XXX,XX @@ address_space_translate_for_iotlb(CPUState *cpu, int asidx, hwaddr addr,
+@@ -XXX,XX +XXX,XX @@ if compile_prog "" "" ; then
-                                   hwaddr *xlat, hwaddr *plen,
+   vector16=yes
-                                   MemTxAttrs attrs, int *prot);
+ fi
- hwaddr memory_region_section_get_iotlb(CPUState *cpu,
--                                       MemoryRegionSection *section,
++########################################
--                                       target_ulong vaddr,
++# See if __attribute__((alias)) is supported.
--                                       hwaddr paddr, hwaddr xlat,
++# This false for Xcode 9, but has been remedied for Xcode 10.
--                                       int prot,
++# Unfortunately, travis uses Xcode 9 by default.
 -                                       target_ulong *address);
 +                                       MemoryRegionSection *section);
  #endif
  /* vl.c */
 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cputlb.c
 +++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
      MemoryRegionSection *section;
      unsigned int index;
      target_ulong address;
 -    target_ulong code_address;
 +    target_ulong write_address;
      uintptr_t addend;
      CPUTLBEntry *te, tn;
      hwaddr iotlb, xlat, sz, paddr_page;
      target_ulong vaddr_page;
      int asidx = cpu_asidx_from_attrs(cpu, attrs);
      int wp_flags;
 +    bool is_ram, is_romd;
      assert_cpu_is_self(cpu);
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
      if (attrs.byte_swap) {
          address |= TLB_BSWAP;
      }
 -    if (!memory_region_is_ram(section->mr) &&
 -        !memory_region_is_romd(section->mr)) {
 -        /* IO memory case */
 -        address |= TLB_MMIO;
 -        addend = 0;
 -    } else {
 +
-+    is_ram = memory_region_is_ram(section->mr);
++attralias=no
-+    is_romd = memory_region_is_romd(section->mr);
++cat > $TMPC << EOF
 +int x = 1;
 +extern const int y __attribute__((alias("x")));
 +int main(void) { return 0; }
 +EOF
 +if compile_prog "" "" ; then
 +    attralias=yes
 +fi
 +
-+    if (is_ram || is_romd) {
+ ########################################
-+        /* RAM and ROMD both have associated host memory. */
+ # check if getauxval is available.
-         addend = (uintptr_t)memory_region_get_ram_ptr(section->mr) + xlat;
-+    } else {
+@@ -XXX,XX +XXX,XX @@ if test "$vector16" = "yes" ; then
-+        /* I/O does not; force the host address to NULL. */
+   echo "CONFIG_VECTOR16=y" >> $config_host_mak
-+        addend = 0;
+ fi
-+    }
 +if test "$attralias" = "yes" ; then
 +  echo "CONFIG_ATTRIBUTE_ALIAS=y" >> $config_host_mak
 +fi
 +
-+    write_address = address;
+ if test "$getauxval" = "yes" ; then
-+    if (is_ram) {
+   echo "CONFIG_GETAUXVAL=y" >> $config_host_mak
-+        iotlb = memory_region_get_ram_addr(section->mr) + xlat;
+ fi
 +        /*
 +         * Computing is_clean is expensive; avoid all that unless
 +         * the page is actually writable.
 +         */
 +        if (prot & PAGE_WRITE) {
 +            if (section->readonly) {
 +                write_address |= TLB_DISCARD_WRITE;
 +            } else if (cpu_physical_memory_is_clean(iotlb)) {
 +                write_address |= TLB_NOTDIRTY;
 +            }
 +        }
 +    } else {
 +        /* I/O or ROMD */
 +        iotlb = memory_region_section_get_iotlb(cpu, section) + xlat;
 +        /*
 +         * Writes to romd devices must go through MMIO to enable write.
 +         * Reads to romd devices go through the ram_ptr found above,
 +         * but of course reads to I/O must go through MMIO.
 +         */
 +        write_address |= TLB_MMIO;
 +        if (!is_romd) {
 +            address = write_address;
 +        }
      }
 -    code_address = address;
 -    iotlb = memory_region_section_get_iotlb(cpu, section, vaddr_page,
 -                                            paddr_page, xlat, prot, &address);
      wp_flags = cpu_watchpoint_address_matches(cpu, vaddr_page,
                                                TARGET_PAGE_SIZE);
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
      /*
       * At this point iotlb contains a physical section number in the lower
       * TARGET_PAGE_BITS, and either
 -     *  + the ram_addr_t of the page base of the target RAM (if NOTDIRTY or ROM)
 -     *  + the offset within section->mr of the page base (otherwise)
 +     *  + the ram_addr_t of the page base of the target RAM (RAM)
 +     *  + the offset within section->mr of the page base (I/O, ROMD)
       * We subtract the vaddr_page (which is page aligned and thus won't
       * disturb the low bits) to give an offset which can be added to the
       * (non-page-aligned) vaddr of the eventual memory access to get
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
      }
      if (prot & PAGE_EXEC) {
 -        tn.addr_code = code_address;
 +        tn.addr_code = address;
      } else {
          tn.addr_code = -1;
      }
      tn.addr_write = -1;
      if (prot & PAGE_WRITE) {
 -        tn.addr_write = address;
 -        if (memory_region_is_romd(section->mr)) {
 -            /* Use the MMIO path so that the device can switch states. */
 -            tn.addr_write |= TLB_MMIO;
 -        } else if (memory_region_is_ram(section->mr)) {
 -            if (section->readonly) {
 -                tn.addr_write |= TLB_DISCARD_WRITE;
 -            } else if (cpu_physical_memory_is_clean(
 -                        memory_region_get_ram_addr(section->mr) + xlat)) {
 -                tn.addr_write |= TLB_NOTDIRTY;
 -            }
 -        }
 +        tn.addr_write = write_address;
          if (prot & PAGE_WRITE_INV) {
              tn.addr_write |= TLB_INVALID_MASK;
          }
 diff --git a/exec.c b/exec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/exec.c
 +++ b/exec.c
@@ -XXX,XX +XXX,XX @@ bool cpu_physical_memory_snapshot_get_dirty(DirtyBitmapSnapshot *snap,
  /* Called from RCU critical section */
  hwaddr memory_region_section_get_iotlb(CPUState *cpu,
 -                                       MemoryRegionSection *section,
 -                                       target_ulong vaddr,
 -                                       hwaddr paddr, hwaddr xlat,
 -                                       int prot,
 -                                       target_ulong *address)
 +                                       MemoryRegionSection *section)
  {
 -    hwaddr iotlb;
 -
 -    if (memory_region_is_ram(section->mr)) {
 -        /* Normal RAM.  */
 -        iotlb = memory_region_get_ram_addr(section->mr) + xlat;
 -    } else {
 -        AddressSpaceDispatch *d;
 -
 -        d = flatview_to_dispatch(section->fv);
 -        iotlb = section - d->map.sections;
 -        iotlb += xlat;
 -    }
 -
 -    return iotlb;
 +    AddressSpaceDispatch *d = flatview_to_dispatch(section->fv);
 +    return section - d->map.sections;
  }
  #endif /* defined(CONFIG_USER_ONLY) */
 --
 .17.1

-[PULL 06/16] cputlb: Introduce TLB_BSWAP
+[PULL v2 06/12] exec: Use const alias for TARGET_PAGE_BITS_VARY
-Handle bswap on ram directly in load/store_helper.  This fixes a
+Using a variable that is declared "const" for this tells the
-bug with the previous implementation in that one cannot use the
+compiler that it may read the value once and assume that it
-I/O path for RAM.
+does not change across function calls.
-Fixes: a26fc6f5152b47f1
+For target_page_size, this means we have only one assert per
 function, and one read of the variable.
 This reduces the size of qemu-system-aarch64 by 8k.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Reviewed-by: David Hildenbrand <david@redhat.com>
+Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/cpu-all.h |  4 ++-
+ include/exec/cpu-all.h | 14 ++++++---
- accel/tcg/cputlb.c     | 72 +++++++++++++++++++++++++-----------------
+ exec-vary.c            | 66 +++++++++++++++++++++++++++++++++++++-----
-files changed, 46 insertions(+), 30 deletions(-)
+files changed, 68 insertions(+), 12 deletions(-)
 diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/exec/cpu-all.h
 +++ b/include/exec/cpu-all.h
-@@ -XXX,XX +XXX,XX @@ CPUArchState *cpu_copy(CPUArchState *env);
+@@ -XXX,XX +XXX,XX @@ static inline void stl_phys_notdirty(AddressSpace *as, hwaddr addr, uint32_t val
- #define TLB_MMIO            (1 << (TARGET_PAGE_BITS_MIN - 3))
+ /* page related stuff */
- /* Set if TLB entry contains a watchpoint.  */
- #define TLB_WATCHPOINT      (1 << (TARGET_PAGE_BITS_MIN - 4))
+ #ifdef TARGET_PAGE_BITS_VARY
-+/* Set if TLB entry requires byte swap.  */
+-extern bool target_page_bits_decided;
-+#define TLB_BSWAP           (1 << (TARGET_PAGE_BITS_MIN - 5))
+-extern int target_page_bits;
+-#define TARGET_PAGE_BITS ({ assert(target_page_bits_decided); \
- /* Use this mask to check interception with an alignment mask
+-                            target_page_bits; })
-  * in a TCG backend.
++typedef struct {
-  */
++    bool decided;
- #define TLB_FLAGS_MASK \
++    int bits;
--    (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO | TLB_WATCHPOINT)
++} TargetPageBits;
-+    (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO | TLB_WATCHPOINT | TLB_BSWAP)
++#if defined(CONFIG_ATTRIBUTE_ALIAS) || !defined(IN_EXEC_VARY)
++extern const TargetPageBits target_page;
- /**
++#else
-  * tlb_hit_page: return true if page aligned @addr is a hit against the
++extern TargetPageBits target_page;
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
++#endif
 +#define TARGET_PAGE_BITS   ({ assert(target_page.decided); target_page.bits; })
  #else
  #define TARGET_PAGE_BITS_MIN TARGET_PAGE_BITS
  #endif
 diff --git a/exec-vary.c b/exec-vary.c
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
+--- a/exec-vary.c
-+++ b/accel/tcg/cputlb.c
++++ b/exec-vary.c
-@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
+@@ -XXX,XX +XXX,XX @@
-         address |= TLB_INVALID_MASK;
  #include "qemu/osdep.h"
  #include "qemu-common.h"
 +
 +#define IN_EXEC_VARY 1
 +
  #include "exec/exec-all.h"
  #ifdef TARGET_PAGE_BITS_VARY
 -int target_page_bits;
 -bool target_page_bits_decided;
 +# ifdef CONFIG_ATTRIBUTE_ALIAS
 +/*
 + * We want to declare the "target_page" variable as const, which tells
 + * the compiler that it can cache any value that it reads across calls.
 + * This avoids multiple assertions and multiple reads within any one user.
 + *
 + * This works because we finish initializing the data before we ever read
 + * from the "target_page" symbol.
 + *
 + * This also requires that we have a non-constant symbol by which we can
 + * perform the actual initialization, and which forces the data to be
 + * allocated within writable memory.  Thus "init_target_page", and we use
 + * that symbol exclusively in the two functions that initialize this value.
 + *
 + * The "target_page" symbol is created as an alias of "init_target_page".
 + */
 +static TargetPageBits init_target_page;
 +
 +/*
 + * Note that this is *not* a redundant decl, this is the definition of
 + * the "target_page" symbol.  The syntax for this definition requires
 + * the use of the extern keyword.  This seems to be a GCC bug in
 + * either the syntax for the alias attribute or in -Wredundant-decls.
 + *
 + * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91765
 + */
 +#  pragma GCC diagnostic push
 +#  pragma GCC diagnostic ignored "-Wredundant-decls"
 +
 +extern const TargetPageBits target_page
 +    __attribute__((alias("init_target_page")));
 +
 +#  pragma GCC diagnostic pop
 +# else
 +/*
 + * When aliases are not supported then we force two different declarations,
 + * by way of suppressing the header declaration with IN_EXEC_VARY.
 + * We assume that on such an old compiler, LTO cannot be used, and so the
 + * compiler cannot not detect the mismatched declarations, and all is well.
 + */
 +TargetPageBits target_page;
 +#  define init_target_page target_page
 +# endif
  #endif
  bool set_preferred_target_page_bits(int bits)
@@ -XXX,XX +XXX,XX @@ bool set_preferred_target_page_bits(int bits)
       */
  #ifdef TARGET_PAGE_BITS_VARY
      assert(bits >= TARGET_PAGE_BITS_MIN);
 -    if (target_page_bits == 0 || target_page_bits > bits) {
 -        if (target_page_bits_decided) {
 +    if (init_target_page.bits == 0 || init_target_page.bits > bits) {
 +        if (init_target_page.decided) {
              return false;
          }
 -        target_page_bits = bits;
 +        init_target_page.bits = bits;
      }
-     if (attrs.byte_swap) {
+ #endif
--        /* Force the access through the I/O slow path.  */
+     return true;
--        address |= TLB_MMIO;
+@@ -XXX,XX +XXX,XX @@ bool set_preferred_target_page_bits(int bits)
-+        address |= TLB_BSWAP;
+ void finalize_target_page_bits(void)
  {
  #ifdef TARGET_PAGE_BITS_VARY
 -    if (target_page_bits == 0) {
 -        target_page_bits = TARGET_PAGE_BITS_MIN;
 +    if (init_target_page.bits == 0) {
 +        init_target_page.bits = TARGET_PAGE_BITS_MIN;
      }
-     if (!memory_region_is_ram(section->mr) &&
+-    target_page_bits_decided = true;
-         !memory_region_is_romd(section->mr)) {
++    init_target_page.decided = true;
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
      bool locked = false;
      MemTxResult r;
 -    if (iotlbentry->attrs.byte_swap) {
 -        op ^= MO_BSWAP;
 -    }
 -
      section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
      mr = section->mr;
      mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
      bool locked = false;
      MemTxResult r;
 -    if (iotlbentry->attrs.byte_swap) {
 -        op ^= MO_BSWAP;
 -    }
 -
      section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
      mr = section->mr;
      mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
                               wp_access, retaddr);
      }
 -    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO)) {
 -        /* I/O access */
 +    /* Reject I/O access, or other required slow-path.  */
 +    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO | TLB_BSWAP)) {
          return NULL;
      }
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
      /* Handle anything that isn't just a straight memory access.  */
      if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
          CPUIOTLBEntry *iotlbentry;
 +        bool need_swap;
          /* For anything that is unaligned, recurse through full_load.  */
          if ((addr & (size - 1)) != 0) {
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
              /* On watchpoint hit, this will longjmp out.  */
              cpu_check_watchpoint(env_cpu(env), addr, size,
                                   iotlbentry->attrs, BP_MEM_READ, retaddr);
 -
 -            /* The backing page may or may not require I/O.  */
 -            tlb_addr &= ~TLB_WATCHPOINT;
 -            if ((tlb_addr & ~TARGET_PAGE_MASK) == 0) {
 -                goto do_aligned_access;
 -            }
          }
 +        need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
 +
-         /* Handle I/O access.  */
++    /*
--        return io_readx(env, iotlbentry, mmu_idx, addr,
++     * For the benefit of an -flto build, prevent the compiler from
--                        retaddr, access_type, op);
++     * hoisting a read from target_page before we finish initializing.
-+        if (likely(tlb_addr & TLB_MMIO)) {
++     */
-+            return io_readx(env, iotlbentry, mmu_idx, addr, retaddr,
++    barrier();
-+                            access_type, op ^ (need_swap * MO_BSWAP));
+ #endif
 +        }
 +
 +        haddr = (void *)((uintptr_t)addr + entry->addend);
 +
 +        /*
 +         * Keep these two load_memop separate to ensure that the compiler
 +         * is able to fold the entire function to a single instruction.
 +         * There is a build-time assert inside to remind you of this.  ;-)
 +         */
 +        if (unlikely(need_swap)) {
 +            return load_memop(haddr, op ^ MO_BSWAP);
 +        }
 +        return load_memop(haddr, op);
      }
      /* Handle slow unaligned access (it spans two pages or IO).  */
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
          return res & MAKE_64BIT_MASK(0, size * 8);
      }
 - do_aligned_access:
      haddr = (void *)((uintptr_t)addr + entry->addend);
      return load_memop(haddr, op);
  }
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
      /* Handle anything that isn't just a straight memory access.  */
      if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
          CPUIOTLBEntry *iotlbentry;
 +        bool need_swap;
          /* For anything that is unaligned, recurse through byte stores.  */
          if ((addr & (size - 1)) != 0) {
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
              /* On watchpoint hit, this will longjmp out.  */
              cpu_check_watchpoint(env_cpu(env), addr, size,
                                   iotlbentry->attrs, BP_MEM_WRITE, retaddr);
 -
 -            /* The backing page may or may not require I/O.  */
 -            tlb_addr &= ~TLB_WATCHPOINT;
 -            if ((tlb_addr & ~TARGET_PAGE_MASK) == 0) {
 -                goto do_aligned_access;
 -            }
          }
 +        need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
 +
          /* Handle I/O access.  */
 -        io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, op);
 +        if (likely(tlb_addr & (TLB_MMIO | TLB_NOTDIRTY))) {
 +            io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr,
 +                      op ^ (need_swap * MO_BSWAP));
 +            return;
 +        }
 +
 +        haddr = (void *)((uintptr_t)addr + entry->addend);
 +
 +        /*
 +         * Keep these two store_memop separate to ensure that the compiler
 +         * is able to fold the entire function to a single instruction.
 +         * There is a build-time assert inside to remind you of this.  ;-)
 +         */
 +        if (unlikely(need_swap)) {
 +            store_memop(haddr, val, op ^ MO_BSWAP);
 +        } else {
 +            store_memop(haddr, val, op);
 +        }
          return;
      }
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
          return;
      }
 - do_aligned_access:
      haddr = (void *)((uintptr_t)addr + entry->addend);
      store_memop(haddr, val, op);
  }
 --
 .17.1

-[PULL 07/16] exec: Adjust notdirty tracing
+[PULL v2 07/12] exec: Restrict TARGET_PAGE_BITS_VARY assert to CONFIG_DEBUG_TCG
-The memory_region_tb_read tracepoint is unreachable, since notdirty
+This reduces the size of a release build by about 10k.
-is supposed to apply only to writes.  The memory_region_tb_write
+Noticably, within the tlb miss helpers.
 tracepoint is mis-named, because notdirty is not only used for TB
 invalidation.  It is also used for e.g. VGA RAM updates and migration.
 Replace memory_region_tb_write with memory_notdirty_write_access,
 and place it in memory_notdirty_write_prepare where it can catch
 all of the instances.  Add memory_notdirty_set_dirty to log when
 we no longer intercept writes to a page.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
-Reviewed-by: David Hildenbrand <david@redhat.com>
+Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- exec.c       | 3 +++
+ include/exec/cpu-all.h | 4 ++++
- memory.c     | 4 ----
+file changed, 4 insertions(+)
  trace-events | 4 ++--
 files changed, 5 insertions(+), 6 deletions(-)
-diff --git a/exec.c b/exec.c
+diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
 index XXXXXXX..XXXXXXX 100644
---- a/exec.c
+--- a/include/exec/cpu-all.h
-+++ b/exec.c
++++ b/include/exec/cpu-all.h
-@@ -XXX,XX +XXX,XX @@ void memory_notdirty_write_prepare(NotDirtyInfo *ndi,
+@@ -XXX,XX +XXX,XX @@ extern const TargetPageBits target_page;
-     ndi->size = size;
+ #else
-     ndi->pages = NULL;
+ extern TargetPageBits target_page;
+ #endif
-+    trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size);
++#ifdef CONFIG_DEBUG_TCG
-+
+ #define TARGET_PAGE_BITS   ({ assert(target_page.decided); target_page.bits; })
-     assert(tcg_enabled());
+ #else
-     if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
++#define TARGET_PAGE_BITS   target_page.bits
-         ndi->pages = page_collection_lock(ram_addr, ram_addr + size);
++#endif
-@@ -XXX,XX +XXX,XX @@ void memory_notdirty_write_complete(NotDirtyInfo *ndi)
++#else
-     /* we remove the notdirty callback only if the code has been
+ #define TARGET_PAGE_BITS_MIN TARGET_PAGE_BITS
-        flushed */
+ #endif
-     if (!cpu_physical_memory_is_clean(ndi->ram_addr)) {
 +        trace_memory_notdirty_set_dirty(ndi->mem_vaddr);
          tlb_set_dirty(ndi->cpu, ndi->mem_vaddr);
      }
  }
 diff --git a/memory.c b/memory.c
 index XXXXXXX..XXXXXXX 100644
 --- a/memory.c
 +++ b/memory.c
@@ -XXX,XX +XXX,XX @@ static MemTxResult  memory_region_read_accessor(MemoryRegion *mr,
          /* Accesses to code which has previously been translated into a TB show
           * up in the MMIO path, as accesses to the io_mem_notdirty
           * MemoryRegion. */
 -        trace_memory_region_tb_read(get_cpu_index(), addr, tmp, size);
      } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) {
          hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
          trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_read_with_attrs_accessor(MemoryRegion *mr,
          /* Accesses to code which has previously been translated into a TB show
           * up in the MMIO path, as accesses to the io_mem_notdirty
           * MemoryRegion. */
 -        trace_memory_region_tb_read(get_cpu_index(), addr, tmp, size);
      } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) {
          hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
          trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_write_accessor(MemoryRegion *mr,
          /* Accesses to code which has previously been translated into a TB show
           * up in the MMIO path, as accesses to the io_mem_notdirty
           * MemoryRegion. */
 -        trace_memory_region_tb_write(get_cpu_index(), addr, tmp, size);
      } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) {
          hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
          trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_write_with_attrs_accessor(MemoryRegion *mr,
          /* Accesses to code which has previously been translated into a TB show
           * up in the MMIO path, as accesses to the io_mem_notdirty
           * MemoryRegion. */
 -        trace_memory_region_tb_write(get_cpu_index(), addr, tmp, size);
      } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) {
          hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
          trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size);
 diff --git a/trace-events b/trace-events
 index XXXXXXX..XXXXXXX 100644
 --- a/trace-events
 +++ b/trace-events
@@ -XXX,XX +XXX,XX @@ dma_map_wait(void *dbs) "dbs=%p"
  find_ram_offset(uint64_t size, uint64_t offset) "size: 0x%" PRIx64 " @ 0x%" PRIx64
  find_ram_offset_loop(uint64_t size, uint64_t candidate, uint64_t offset, uint64_t next, uint64_t mingap) "trying size: 0x%" PRIx64 " @ 0x%" PRIx64 ", offset: 0x%" PRIx64" next: 0x%" PRIx64 " mingap: 0x%" PRIx64
  ram_block_discard_range(const char *rbname, void *hva, size_t length, bool need_madvise, bool need_fallocate, int ret) "%s@%p + 0x%zx: madvise: %d fallocate: %d ret: %d"
 +memory_notdirty_write_access(uint64_t vaddr, uint64_t ram_addr, unsigned size) "0x%" PRIx64 " ram_addr 0x%" PRIx64 " size %u"
 +memory_notdirty_set_dirty(uint64_t vaddr) "0x%" PRIx64
  # memory.c
  memory_region_ops_read(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
  memory_region_ops_write(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
  memory_region_subpage_read(int cpu_index, void *mr, uint64_t offset, uint64_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx64" size %u"
  memory_region_subpage_write(int cpu_index, void *mr, uint64_t offset, uint64_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx64" size %u"
 -memory_region_tb_read(int cpu_index, uint64_t addr, uint64_t value, unsigned size) "cpu %d addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
 -memory_region_tb_write(int cpu_index, uint64_t addr, uint64_t value, unsigned size) "cpu %d addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
  memory_region_ram_device_read(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
  memory_region_ram_device_write(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
  flatview_new(void *view, void *root) "%p (root %p)"
 --
 .17.1

-[PULL 08/16] cputlb: Move ROM handling from I/O path to TLB path
+[PULL v2 08/12] exec: Promote TARGET_PAGE_MASK to target_long
-It does not require going through the whole I/O path
+There are some uint64_t uses that expect TARGET_PAGE_MASK to
-in order to discard a write.
+extend for a 32-bit, so this must continue to be a signed type.
 Define based on TARGET_PAGE_BITS not TARGET_PAGE_SIZE; this
 will make a following patch more clear.
-Reviewed-by: David Hildenbrand <david@redhat.com>
+This should not have a functional effect so far.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/cpu-all.h    |  5 ++++-
+ include/exec/cpu-all.h | 2 +-
- include/exec/cpu-common.h |  1 -
+file changed, 1 insertion(+), 1 deletion(-)
  accel/tcg/cputlb.c        | 36 ++++++++++++++++++++--------------
  exec.c                    | 41 +--------------------------------------
 files changed, 26 insertions(+), 57 deletions(-)
 diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/exec/cpu-all.h
 +++ b/include/exec/cpu-all.h
-@@ -XXX,XX +XXX,XX @@ CPUArchState *cpu_copy(CPUArchState *env);
+@@ -XXX,XX +XXX,XX @@ extern TargetPageBits target_page;
  #define TLB_WATCHPOINT      (1 << (TARGET_PAGE_BITS_MIN - 4))
  /* Set if TLB entry requires byte swap.  */
  #define TLB_BSWAP           (1 << (TARGET_PAGE_BITS_MIN - 5))
 +/* Set if TLB entry writes ignored.  */
 +#define TLB_DISCARD_WRITE   (1 << (TARGET_PAGE_BITS_MIN - 6))
  /* Use this mask to check interception with an alignment mask
   * in a TCG backend.
   */
  #define TLB_FLAGS_MASK \
 -    (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO | TLB_WATCHPOINT | TLB_BSWAP)
 +    (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO \
 +    | TLB_WATCHPOINT | TLB_BSWAP | TLB_DISCARD_WRITE)
  /**
   * tlb_hit_page: return true if page aligned @addr is a hit against the
 diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/exec/cpu-common.h
 +++ b/include/exec/cpu-common.h
@@ -XXX,XX +XXX,XX @@ void qemu_flush_coalesced_mmio_buffer(void);
  void cpu_flush_icache_range(hwaddr start, hwaddr len);
 -extern struct MemoryRegion io_mem_rom;
  extern struct MemoryRegion io_mem_notdirty;
  typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);
 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cputlb.c
 +++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tlb_reset_dirty_range_locked(CPUTLBEntry *tlb_entry,
  {
      uintptr_t addr = tlb_entry->addr_write;
 -    if ((addr & (TLB_INVALID_MASK | TLB_MMIO | TLB_NOTDIRTY)) == 0) {
 +    if ((addr & (TLB_INVALID_MASK | TLB_MMIO |
 +                 TLB_DISCARD_WRITE | TLB_NOTDIRTY)) == 0) {
          addr &= TARGET_PAGE_MASK;
          addr += tlb_entry->addend;
          if ((addr - start) < length) {
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
          address |= TLB_MMIO;
          addend = 0;
      } else {
 -        /* TLB_MMIO for rom/romd handled below */
          addend = (uintptr_t)memory_region_get_ram_ptr(section->mr) + xlat;
      }
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
      tn.addr_write = -1;
      if (prot & PAGE_WRITE) {
 -        if ((memory_region_is_ram(section->mr) && section->readonly)
 -            || memory_region_is_romd(section->mr)) {
 -            /* Write access calls the I/O callback.  */
 -            tn.addr_write = address | TLB_MMIO;
 -        } else if (memory_region_is_ram(section->mr)
 -                   && cpu_physical_memory_is_clean(
 -                       memory_region_get_ram_addr(section->mr) + xlat)) {
 -            tn.addr_write = address | TLB_NOTDIRTY;
 -        } else {
 -            tn.addr_write = address;
 +        tn.addr_write = address;
 +        if (memory_region_is_romd(section->mr)) {
 +            /* Use the MMIO path so that the device can switch states. */
 +            tn.addr_write |= TLB_MMIO;
 +        } else if (memory_region_is_ram(section->mr)) {
 +            if (section->readonly) {
 +                tn.addr_write |= TLB_DISCARD_WRITE;
 +            } else if (cpu_physical_memory_is_clean(
 +                        memory_region_get_ram_addr(section->mr) + xlat)) {
 +                tn.addr_write |= TLB_NOTDIRTY;
 +            }
          }
          if (prot & PAGE_WRITE_INV) {
              tn.addr_write |= TLB_INVALID_MASK;
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
      mr = section->mr;
      mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
      cpu->mem_io_pc = retaddr;
 -    if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) {
 +    if (mr != &io_mem_notdirty && !cpu->can_do_io) {
          cpu_io_recompile(cpu, retaddr);
      }
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
      section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
      mr = section->mr;
      mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
 -    if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) {
 +    if (mr != &io_mem_notdirty && !cpu->can_do_io) {
          cpu_io_recompile(cpu, retaddr);
      }
      cpu->mem_io_vaddr = addr;
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
      }
      /* Reject I/O access, or other required slow-path.  */
 -    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO | TLB_BSWAP)) {
 +    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO | TLB_BSWAP | TLB_DISCARD_WRITE)) {
          return NULL;
      }
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
              return;
          }
 +        /* Ignore writes to ROM.  */
 +        if (unlikely(tlb_addr & TLB_DISCARD_WRITE)) {
 +            return;
 +        }
 +
          haddr = (void *)((uintptr_t)addr + entry->addend);
          /*
 diff --git a/exec.c b/exec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/exec.c
 +++ b/exec.c
@@ -XXX,XX +XXX,XX @@ static MemoryRegion *system_io;
  AddressSpace address_space_io;
  AddressSpace address_space_memory;
 -MemoryRegion io_mem_rom, io_mem_notdirty;
 +MemoryRegion io_mem_notdirty;
  static MemoryRegion io_mem_unassigned;
  #endif
-@@ -XXX,XX +XXX,XX @@ typedef struct subpage_t {
+ #define TARGET_PAGE_SIZE (1 << TARGET_PAGE_BITS)
+-#define TARGET_PAGE_MASK ~(TARGET_PAGE_SIZE - 1)
- #define PHYS_SECTION_UNASSIGNED 0
++#define TARGET_PAGE_MASK ((target_long)-1 << TARGET_PAGE_BITS)
- #define PHYS_SECTION_NOTDIRTY 1
+ #define TARGET_PAGE_ALIGN(addr) ROUND_UP((addr), TARGET_PAGE_SIZE)
--#define PHYS_SECTION_ROM 2
+ /* Using intptr_t ensures that qemu_*_page_mask is sign-extended even
  static void io_mem_init(void);
  static void memory_map_init(void);
@@ -XXX,XX +XXX,XX @@ hwaddr memory_region_section_get_iotlb(CPUState *cpu,
          iotlb = memory_region_get_ram_addr(section->mr) + xlat;
          if (!section->readonly) {
              iotlb |= PHYS_SECTION_NOTDIRTY;
 -        } else {
 -            iotlb |= PHYS_SECTION_ROM;
          }
      } else {
          AddressSpaceDispatch *d;
@@ -XXX,XX +XXX,XX @@ static uint16_t dummy_section(PhysPageMap *map, FlatView *fv, MemoryRegion *mr)
      return phys_section_add(map, &section);
  }
 -static void readonly_mem_write(void *opaque, hwaddr addr,
 -                               uint64_t val, unsigned size)
 -{
 -    /* Ignore any write to ROM. */
 -}
 -
 -static bool readonly_mem_accepts(void *opaque, hwaddr addr,
 -                                 unsigned size, bool is_write,
 -                                 MemTxAttrs attrs)
 -{
 -    return is_write;
 -}
 -
 -/* This will only be used for writes, because reads are special cased
 - * to directly access the underlying host ram.
 - */
 -static const MemoryRegionOps readonly_mem_ops = {
 -    .write = readonly_mem_write,
 -    .valid.accepts = readonly_mem_accepts,
 -    .endianness = DEVICE_NATIVE_ENDIAN,
 -    .valid = {
 -        .min_access_size = 1,
 -        .max_access_size = 8,
 -        .unaligned = false,
 -    },
 -    .impl = {
 -        .min_access_size = 1,
 -        .max_access_size = 8,
 -        .unaligned = false,
 -    },
 -};
 -
  MemoryRegionSection *iotlb_to_section(CPUState *cpu,
                                        hwaddr index, MemTxAttrs attrs)
  {
@@ -XXX,XX +XXX,XX @@ MemoryRegionSection *iotlb_to_section(CPUState *cpu,
  static void io_mem_init(void)
  {
 -    memory_region_init_io(&io_mem_rom, NULL, &readonly_mem_ops,
 -                          NULL, NULL, UINT64_MAX);
      memory_region_init_io(&io_mem_unassigned, NULL, &unassigned_mem_ops, NULL,
                            NULL, UINT64_MAX);
@@ -XXX,XX +XXX,XX @@ AddressSpaceDispatch *address_space_dispatch_new(FlatView *fv)
      assert(n == PHYS_SECTION_UNASSIGNED);
      n = dummy_section(&d->map, fv, &io_mem_notdirty);
      assert(n == PHYS_SECTION_NOTDIRTY);
 -    n = dummy_section(&d->map, fv, &io_mem_rom);
 -    assert(n == PHYS_SECTION_ROM);
      d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .skip = 1 };
 --
 .17.1

-[PULL 01/16] exec: Use TARGET_PAGE_BITS_MIN for TLB flags
+[PULL v2 09/12] exec: Cache TARGET_PAGE_MASK for TARGET_PAGE_BITS_VARY
-These bits do not need to vary with the actual page size
+This eliminates a set of runtime shifts.  It turns out that we
-used by the guest.
+require TARGET_PAGE_MASK more often than TARGET_PAGE_SIZE, so
 redefine TARGET_PAGE_SIZE based on TARGET_PAGE_MASK instead of
 the other way around.
 Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
 Reviewed-by: David Hildenbrand <david@redhat.com>
 Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/exec/cpu-all.h | 16 ++++++++++------
+ include/exec/cpu-all.h | 8 ++++++--
-file changed, 10 insertions(+), 6 deletions(-)
+ exec-vary.c            | 1 +
 files changed, 7 insertions(+), 2 deletions(-)
 diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/exec/cpu-all.h
 +++ b/include/exec/cpu-all.h
-@@ -XXX,XX +XXX,XX @@ CPUArchState *cpu_copy(CPUArchState *env);
+@@ -XXX,XX +XXX,XX @@ static inline void stl_phys_notdirty(AddressSpace *as, hwaddr addr, uint32_t val
+ typedef struct {
- #if !defined(CONFIG_USER_ONLY)
+     bool decided;
+     int bits;
--/* Flags stored in the low bits of the TLB virtual address.  These are
++    target_long mask;
-- * defined so that fast path ram access is all zeros.
+ } TargetPageBits;
-+/*
+ #if defined(CONFIG_ATTRIBUTE_ALIAS) || !defined(IN_EXEC_VARY)
-+ * Flags stored in the low bits of the TLB virtual address.
+ extern const TargetPageBits target_page;
-+ * These are defined so that fast path ram access is all zeros.
+@@ -XXX,XX +XXX,XX @@ extern TargetPageBits target_page;
-  * The flags all must be between TARGET_PAGE_BITS and
+ #endif
-  * maximum address alignment bit.
+ #ifdef CONFIG_DEBUG_TCG
-+ *
+ #define TARGET_PAGE_BITS   ({ assert(target_page.decided); target_page.bits; })
-+ * Use TARGET_PAGE_BITS_MIN so that these bits are constant
++#define TARGET_PAGE_MASK   ({ assert(target_page.decided); target_page.mask; })
-+ * when TARGET_PAGE_BITS_VARY is in effect.
+ #else
-  */
+ #define TARGET_PAGE_BITS   target_page.bits
- /* Zero if TLB entry is valid.  */
++#define TARGET_PAGE_MASK   target_page.mask
--#define TLB_INVALID_MASK    (1 << (TARGET_PAGE_BITS - 1))
+ #endif
-+#define TLB_INVALID_MASK    (1 << (TARGET_PAGE_BITS_MIN - 1))
++#define TARGET_PAGE_SIZE   (-(int)TARGET_PAGE_MASK)
- /* Set if TLB entry references a clean RAM page.  The iotlb entry will
+ #else
-    contain the page physical address.  */
+ #define TARGET_PAGE_BITS_MIN TARGET_PAGE_BITS
--#define TLB_NOTDIRTY        (1 << (TARGET_PAGE_BITS - 2))
++#define TARGET_PAGE_SIZE   (1 << TARGET_PAGE_BITS)
-+#define TLB_NOTDIRTY        (1 << (TARGET_PAGE_BITS_MIN - 2))
++#define TARGET_PAGE_MASK   ((target_long)-1 << TARGET_PAGE_BITS)
- /* Set if TLB entry is an IO callback.  */
+ #endif
--#define TLB_MMIO            (1 << (TARGET_PAGE_BITS - 3))
-+#define TLB_MMIO            (1 << (TARGET_PAGE_BITS_MIN - 3))
+-#define TARGET_PAGE_SIZE (1 << TARGET_PAGE_BITS)
- /* Set if TLB entry contains a watchpoint.  */
+-#define TARGET_PAGE_MASK ((target_long)-1 << TARGET_PAGE_BITS)
--#define TLB_WATCHPOINT      (1 << (TARGET_PAGE_BITS - 4))
+ #define TARGET_PAGE_ALIGN(addr) ROUND_UP((addr), TARGET_PAGE_SIZE)
-+#define TLB_WATCHPOINT      (1 << (TARGET_PAGE_BITS_MIN - 4))
+ /* Using intptr_t ensures that qemu_*_page_mask is sign-extended even
- /* Use this mask to check interception with an alignment mask
+diff --git a/exec-vary.c b/exec-vary.c
-  * in a TCG backend.
+index XXXXXXX..XXXXXXX 100644
 --- a/exec-vary.c
 +++ b/exec-vary.c
@@ -XXX,XX +XXX,XX @@ void finalize_target_page_bits(void)
      if (init_target_page.bits == 0) {
          init_target_page.bits = TARGET_PAGE_BITS_MIN;
      }
 +    init_target_page.mask = (target_long)-1 << init_target_page.bits;
      init_target_page.decided = true;
      /*
 --
 .17.1

-[PULL 02/16] cputlb: Disable __always_inline__ without optimization
+Deleted patch
-This forced inlining can result in missing symbols,
-which makes a debugging build harder to follow.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
-Reviewed-by: David Hildenbrand <david@redhat.com>
-Reported-by: Peter Maydell <peter.maydell@linaro.org>
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
----
- include/qemu/compiler.h | 11 +++++++++++
- accel/tcg/cputlb.c      |  4 ++--
-files changed, 13 insertions(+), 2 deletions(-)
-diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/qemu/compiler.h
-+++ b/include/qemu/compiler.h
-@@ -XXX,XX +XXX,XX @@
- # define QEMU_NONSTRING
- #endif
-+/*
-+ * Forced inlining may be desired to encourage constant propagation
-+ * of function parameters.  However, it can also make debugging harder,
-+ * so disable it for a non-optimizing build.
-+ */
-+#if defined(__OPTIMIZE__)
-+#define QEMU_ALWAYS_INLINE  __attribute__((always_inline))
-+#else
-+#define QEMU_ALWAYS_INLINE
-+#endif
-+
- /* Implement C11 _Generic via GCC builtins.  Example:
-  *
-  *    QEMU_GENERIC(x, (float, sinf), (long double, sinl), sin) (x)
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
-index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
-+++ b/accel/tcg/cputlb.c
-@@ -XXX,XX +XXX,XX @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
- typedef uint64_t FullLoadHelper(CPUArchState *env, target_ulong addr,
-                                 TCGMemOpIdx oi, uintptr_t retaddr);
--static inline uint64_t __attribute__((always_inline))
-+static inline uint64_t QEMU_ALWAYS_INLINE
- load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
-             uintptr_t retaddr, MemOp op, bool code_read,
-             FullLoadHelper *full_load)
-@@ -XXX,XX +XXX,XX @@ tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *env, target_ulong addr,
-  * Store Helpers
-  */
--static inline void __attribute__((always_inline))
-+static inline void QEMU_ALWAYS_INLINE
- store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
-              TCGMemOpIdx oi, uintptr_t retaddr, MemOp op)
- {
---
-.17.1

-[PULL 03/16] qemu/compiler.h: Add qemu_build_not_reached
+Deleted patch
-Use this as a compile-time assert that a particular
-code path is not reachable.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
----
- include/qemu/compiler.h | 15 +++++++++++++++
-file changed, 15 insertions(+)
-diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/qemu/compiler.h
-+++ b/include/qemu/compiler.h
-@@ -XXX,XX +XXX,XX @@
- #define QEMU_GENERIC9(x, a0, ...) QEMU_GENERIC_IF(x, a0, QEMU_GENERIC8(x, __VA_ARGS__))
- #define QEMU_GENERIC10(x, a0, ...) QEMU_GENERIC_IF(x, a0, QEMU_GENERIC9(x, __VA_ARGS__))
-+/**
-+ * qemu_build_not_reached()
-+ *
-+ * The compiler, during optimization, is expected to prove that a call
-+ * to this function cannot be reached and remove it.  If the compiler
-+ * supports QEMU_ERROR, this will be reported at compile time; otherwise
-+ * this will be reported at link time due to the missing symbol.
-+ */
-+#ifdef __OPTIMIZE__
-+extern void QEMU_NORETURN QEMU_ERROR("code path is reachable")
-+    qemu_build_not_reached(void);
-+#else
-+#define qemu_build_not_reached()  g_assert_not_reached()
-+#endif
-+
- #endif /* COMPILER_H */
---
-.17.1

-[PULL 04/16] cputlb: Use qemu_build_not_reached in load/store_helpers
+Deleted patch
-Increase the current runtime assert to a compile-time assert.
-Reviewed-by: David Hildenbrand <david@redhat.com>
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
----
- accel/tcg/cputlb.c | 5 ++---
-file changed, 2 insertions(+), 3 deletions(-)
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
-index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
-+++ b/accel/tcg/cputlb.c
-@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
-         res = ldq_le_p(haddr);
-         break;
-     default:
--        g_assert_not_reached();
-+        qemu_build_not_reached();
-     }
-     return res;
-@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
-         stq_le_p(haddr, val);
-         break;
-     default:
--        g_assert_not_reached();
--        break;
-+        qemu_build_not_reached();
-     }
- }
---
-.17.1

-[PULL 12/16] cputlb: Handle TLB_NOTDIRTY in probe_access
+Deleted patch
-We can use notdirty_write for the write and return a valid host
-pointer for this case.
-Reviewed-by: David Hildenbrand <david@redhat.com>
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
----
- accel/tcg/cputlb.c | 26 +++++++++++++++++---------
-file changed, 17 insertions(+), 9 deletions(-)
-diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
-index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/cputlb.c
-+++ b/accel/tcg/cputlb.c
-@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
-         return NULL;
-     }
--    /* Handle watchpoints.  */
--    if (tlb_addr & TLB_WATCHPOINT) {
--        cpu_check_watchpoint(env_cpu(env), addr, size,
--                             env_tlb(env)->d[mmu_idx].iotlb[index].attrs,
--                             wp_access, retaddr);
--    }
-+    if (unlikely(tlb_addr & TLB_FLAGS_MASK)) {
-+        CPUIOTLBEntry *iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
--    /* Reject I/O access, or other required slow-path.  */
--    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO | TLB_BSWAP | TLB_DISCARD_WRITE)) {
--        return NULL;
-+        /* Reject I/O access, or other required slow-path.  */
-+        if (tlb_addr & (TLB_MMIO | TLB_BSWAP | TLB_DISCARD_WRITE)) {
-+            return NULL;
-+        }
-+
-+        /* Handle watchpoints.  */
-+        if (tlb_addr & TLB_WATCHPOINT) {
-+            cpu_check_watchpoint(env_cpu(env), addr, size,
-+                                 iotlbentry->attrs, wp_access, retaddr);
-+        }
-+
-+        /* Handle clean RAM pages.  */
-+        if (tlb_addr & TLB_NOTDIRTY) {
-+            notdirty_write(env_cpu(env), addr, size, iotlbentry, retaddr);
-+        }
-     }
-     return (void *)((uintptr_t)addr + entry->addend);
---
-.17.1

-[PULL 13/16] cputlb: Remove cpu->mem_io_vaddr
+[PULL v2 10/12] cputlb: Fix tlb_vaddr_to_host
-With the merge of notdirty handling into store_helper,
+Using uintptr_t instead of target_ulong meant that, for 64-bit guest
-the last user of cpu->mem_io_vaddr was removed.
+and 32-bit host, we truncated the guest address comparator and so may
 not hit the tlb when we should.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Fixes: 4811e9095c0
 Reviewed-by: David Hildenbrand <david@redhat.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- include/hw/core/cpu.h | 2 --
+ accel/tcg/cputlb.c | 2 +-
- accel/tcg/cputlb.c    | 2 --
+file changed, 1 insertion(+), 1 deletion(-)
  hw/core/cpu.c         | 1 -
 files changed, 5 deletions(-)
-diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/hw/core/cpu.h
-+++ b/include/hw/core/cpu.h
-@@ -XXX,XX +XXX,XX @@ struct qemu_work_item;
-  * @next_cpu: Next CPU sharing TB cache.
-  * @opaque: User data.
-  * @mem_io_pc: Host Program Counter at which the memory was accessed.
-- * @mem_io_vaddr: Target virtual address at which the memory was accessed.
-  * @kvm_fd: vCPU file descriptor for KVM.
-  * @work_mutex: Lock to prevent multiple access to queued_work_*.
-  * @queued_work_first: First asynchronous work pending.
-@@ -XXX,XX +XXX,XX @@ struct CPUState {
-      * we store some rarely used information in the CPU context.
-      */
-     uintptr_t mem_io_pc;
--    vaddr mem_io_vaddr;
-     /*
-      * This is only needed for the legacy cpu_unassigned_access() hook;
-      * when all targets using it have been converted to use
 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/cputlb.c
 +++ b/accel/tcg/cputlb.c
-@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
+@@ -XXX,XX +XXX,XX @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr,
-         cpu_io_recompile(cpu, retaddr);
+                         MMUAccessType access_type, int mmu_idx)
-     }
+ {
+     CPUTLBEntry *entry = tlb_entry(env, mmu_idx, addr);
--    cpu->mem_io_vaddr = addr;
+-    uintptr_t tlb_addr, page;
-     cpu->mem_io_access_type = access_type;
++    target_ulong tlb_addr, page;
+     size_t elt_ofs;
-     if (mr->global_locking && !qemu_mutex_iothread_locked()) {
-@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
+     switch (access_type) {
      if (!cpu->can_do_io) {
          cpu_io_recompile(cpu, retaddr);
      }
 -    cpu->mem_io_vaddr = addr;
      cpu->mem_io_pc = retaddr;
      if (mr->global_locking && !qemu_mutex_iothread_locked()) {
 diff --git a/hw/core/cpu.c b/hw/core/cpu.c
 index XXXXXXX..XXXXXXX 100644
 --- a/hw/core/cpu.c
 +++ b/hw/core/cpu.c
@@ -XXX,XX +XXX,XX @@ static void cpu_common_reset(CPUState *cpu)
      cpu->interrupt_request = 0;
      cpu->halted = 0;
      cpu->mem_io_pc = 0;
 -    cpu->mem_io_vaddr = 0;
      cpu->icount_extra = 0;
      atomic_set(&cpu->icount_decr_ptr->u32, 0);
      cpu->can_do_io = 1;
 --
 .17.1

-[PULL 14/16] cputlb: Remove tb_invalidate_phys_page_range is_cpu_write_access
+[PULL v2 11/12] translate-all: fix uninitialized tb->orig_tb
-All callers pass false to this argument.  Remove it and pass the
+From: Clement Deschamps <clement.deschamps@greensocs.com>
 constant on to tb_invalidate_phys_page_range__locked.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+This fixes a segmentation fault in icount mode when executing
-Reviewed-by: David Hildenbrand <david@redhat.com>
+from an IO region.
 TB is marked as CF_NOCACHE but tb->orig_tb is not initialized
 (equals previous value in code_gen_buffer).
 The issue happens in cpu_io_recompile() when it tries to invalidate orig_tb.
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Signed-off-by: Clement Deschamps <clement.deschamps@greensocs.com>
 Message-Id: <20191022140016.918371-1-clement.deschamps@greensocs.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/translate-all.h | 3 +--
+ accel/tcg/translate-all.c | 1 +
- accel/tcg/translate-all.c | 6 ++----
+file changed, 1 insertion(+)
  exec.c                    | 4 ++--
 files changed, 5 insertions(+), 8 deletions(-)
-diff --git a/accel/tcg/translate-all.h b/accel/tcg/translate-all.h
-index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/translate-all.h
-+++ b/accel/tcg/translate-all.h
-@@ -XXX,XX +XXX,XX @@ struct page_collection *page_collection_lock(tb_page_addr_t start,
- void page_collection_unlock(struct page_collection *set);
- void tb_invalidate_phys_page_fast(struct page_collection *pages,
-                                   tb_page_addr_t start, int len);
--void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end,
--                                   int is_cpu_write_access);
-+void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end);
- void tb_check_watchpoint(CPUState *cpu);
- #ifdef CONFIG_USER_ONLY
 diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/translate-all.c
 +++ b/accel/tcg/translate-all.c
-@@ -XXX,XX +XXX,XX @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
+@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
-  *
+     tb->cs_base = cs_base;
-  * Called with mmap_lock held for user-mode emulation
+     tb->flags = flags;
-  */
+     tb->cflags = cflags;
--void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end,
++    tb->orig_tb = NULL;
--                                   int is_cpu_write_access)
+     tb->trace_vcpu_dstate = *cpu->trace_dstate;
-+void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end)
+     tcg_ctx->tb_cflags = cflags;
- {
+  tb_overflow:
      struct page_collection *pages;
      PageDesc *p;
@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end,
          return;
      }
      pages = page_collection_lock(start, end);
 -    tb_invalidate_phys_page_range__locked(pages, p, start, end,
 -                                          is_cpu_write_access);
 +    tb_invalidate_phys_page_range__locked(pages, p, start, end, 0);
      page_collection_unlock(pages);
  }
 diff --git a/exec.c b/exec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/exec.c
 +++ b/exec.c
@@ -XXX,XX +XXX,XX @@ const char *parse_cpu_option(const char *cpu_option)
  void tb_invalidate_phys_addr(target_ulong addr)
  {
      mmap_lock();
 -    tb_invalidate_phys_page_range(addr, addr + 1, 0);
 +    tb_invalidate_phys_page_range(addr, addr + 1);
      mmap_unlock();
  }
@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr, MemTxAttrs attrs)
          return;
      }
      ram_addr = memory_region_get_ram_addr(mr) + addr;
 -    tb_invalidate_phys_page_range(ram_addr, ram_addr + 1, 0);
 +    tb_invalidate_phys_page_range(ram_addr, ram_addr + 1);
      rcu_read_unlock();
  }
 --
 .17.1

-[PULL 16/16] cputlb: Pass retaddr to tb_check_watchpoint
+[PULL v2 12/12] translate-all: Remove tb_alloc
-Fixes the previous TLB_WATCHPOINT patches because we are currently
+Since 2ac01d6dafab, this function does only two things: assert a
-failing to set cpu->mem_io_pc with the call to cpu_check_watchpoint.
+lock is held, and call tcg_tb_alloc.  It is used exactly once,
-Pass down the retaddr directly because it's readily available.
+and its user has already done the assert.
-Fixes: 50b107c5d61
+Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+Reviewed-by: Clement Deschamps <clement.deschamps@greensocs.com>
 Reviewed-by: David Hildenbrand <david@redhat.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/translate-all.h | 2 +-
+ accel/tcg/translate-all.c | 20 ++------------------
- accel/tcg/translate-all.c | 6 +++---
+file changed, 2 insertions(+), 18 deletions(-)
  exec.c                    | 2 +-
 files changed, 5 insertions(+), 5 deletions(-)
-diff --git a/accel/tcg/translate-all.h b/accel/tcg/translate-all.h
-index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/translate-all.h
-+++ b/accel/tcg/translate-all.h
-@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_page_fast(struct page_collection *pages,
-                                   tb_page_addr_t start, int len,
-                                   uintptr_t retaddr);
- void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end);
--void tb_check_watchpoint(CPUState *cpu);
-+void tb_check_watchpoint(CPUState *cpu, uintptr_t retaddr);
- #ifdef CONFIG_USER_ONLY
- int page_unprotect(target_ulong address, uintptr_t pc);
 diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/translate-all.c
 +++ b/accel/tcg/translate-all.c
-@@ -XXX,XX +XXX,XX @@ static bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc)
+@@ -XXX,XX +XXX,XX @@ void tcg_exec_init(unsigned long tb_size)
  #endif
+ }
- /* user-mode: call with mmap_lock held */
--void tb_check_watchpoint(CPUState *cpu)
+-/*
-+void tb_check_watchpoint(CPUState *cpu, uintptr_t retaddr)
+- * Allocate a new translation block. Flush the translation buffer if
 - * too many translation blocks or too much generated code.
 - */
 -static TranslationBlock *tb_alloc(target_ulong pc)
 -{
 -    TranslationBlock *tb;
 -
 -    assert_memory_lock();
 -
 -    tb = tcg_tb_alloc(tcg_ctx);
 -    if (unlikely(tb == NULL)) {
 -        return NULL;
 -    }
 -    return tb;
 -}
 -
  /* call with @p->lock held */
  static inline void invalidate_page_bitmap(PageDesc *p)
  {
-     TranslationBlock *tb;
+@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
+     TCGProfile *prof = &tcg_ctx->prof;
      int64_t ti;
  #endif
 +
      assert_memory_lock();
--    tb = tcg_tb_lookup(cpu->mem_io_pc);
+     phys_pc = get_page_addr_code(env, pc);
-+    tb = tcg_tb_lookup(retaddr);
+@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
-     if (tb) {
+     }
-         /* We can use retranslation to find the PC.  */
--        cpu_restore_state_from_tb(cpu, tb, cpu->mem_io_pc, true);
+  buffer_overflow:
-+        cpu_restore_state_from_tb(cpu, tb, retaddr, true);
+-    tb = tb_alloc(pc);
-         tb_phys_invalidate(tb, -1);
++    tb = tcg_tb_alloc(tcg_ctx);
-     } else {
+     if (unlikely(!tb)) {
-         /* The exception probably happened in a helper.  The CPU state should
+         /* flush must be done */
-diff --git a/exec.c b/exec.c
+         tb_flush(cpu);
 index XXXXXXX..XXXXXXX 100644
 --- a/exec.c
 +++ b/exec.c
@@ -XXX,XX +XXX,XX @@ void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
                  cpu->watchpoint_hit = wp;
                  mmap_lock();
 -                tb_check_watchpoint(cpu);
 +                tb_check_watchpoint(cpu, ra);
                  if (wp->flags & BP_STOP_BEFORE_ACCESS) {
                      cpu->exception_index = EXCP_DEBUG;
                      mmap_unlock();
 --
 .17.1

This is v4 of my notdirty + rom patch set with two suggested name
changes (qemu_build_not_reached, TLB_DISCARD_WRITE) from David and Alex.

The following changes since commit 240ab11fb72049d6373cbbec8d788f8e411a00bc:

Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20190924' into staging (2019-09-24 15:36:31 +0100)

are available in the Git repository at:

https://github.com/rth7680/qemu.git tags/pull-tcg-20190925

for you to fetch changes up to ae57db63acf5a0399232f852acc5c1d83ef63400:

cputlb: Pass retaddr to tb_check_watchpoint (2019-09-25 10:56:28 -0700)

----------------------------------------------------------------
Fixes for TLB_BSWAP
Coversion of NOTDIRTY and ROM handling to cputlb
Followup cleanups to cputlb

----------------------------------------------------------------
Richard Henderson (16):
      exec: Use TARGET_PAGE_BITS_MIN for TLB flags
      cputlb: Disable __always_inline__ without optimization
      qemu/compiler.h: Add qemu_build_not_reached
      cputlb: Use qemu_build_not_reached in load/store_helpers
      cputlb: Split out load/store_memop
      cputlb: Introduce TLB_BSWAP
      exec: Adjust notdirty tracing
      cputlb: Move ROM handling from I/O path to TLB path
      cputlb: Move NOTDIRTY handling from I/O path to TLB path
      cputlb: Partially inline memory_region_section_get_iotlb
      cputlb: Merge and move memory_notdirty_write_{prepare,complete}
      cputlb: Handle TLB_NOTDIRTY in probe_access
      cputlb: Remove cpu->mem_io_vaddr
      cputlb: Remove tb_invalidate_phys_page_range is_cpu_write_access
      cputlb: Pass retaddr to tb_invalidate_phys_page_fast
      cputlb: Pass retaddr to tb_check_watchpoint

These bits do not need to vary with the actual page size
used by the guest.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-all.h | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -XXX,XX +XXX,XX @@ CPUArchState *cpu_copy(CPUArchState *env);
 
 #if !defined(CONFIG_USER_ONLY)
 
-/* Flags stored in the low bits of the TLB virtual address.  These are
- * defined so that fast path ram access is all zeros.
+/*
+ * Flags stored in the low bits of the TLB virtual address.
+ * These are defined so that fast path ram access is all zeros.
  * The flags all must be between TARGET_PAGE_BITS and
  * maximum address alignment bit.
+ *
+ * Use TARGET_PAGE_BITS_MIN so that these bits are constant
+ * when TARGET_PAGE_BITS_VARY is in effect.
  */
 /* Zero if TLB entry is valid.  */
-#define TLB_INVALID_MASK    (1 << (TARGET_PAGE_BITS - 1))
+#define TLB_INVALID_MASK    (1 << (TARGET_PAGE_BITS_MIN - 1))
 /* Set if TLB entry references a clean RAM page.  The iotlb entry will
    contain the page physical address.  */
-#define TLB_NOTDIRTY        (1 << (TARGET_PAGE_BITS - 2))
+#define TLB_NOTDIRTY        (1 << (TARGET_PAGE_BITS_MIN - 2))
 /* Set if TLB entry is an IO callback.  */
-#define TLB_MMIO            (1 << (TARGET_PAGE_BITS - 3))
+#define TLB_MMIO            (1 << (TARGET_PAGE_BITS_MIN - 3))
 /* Set if TLB entry contains a watchpoint.  */
-#define TLB_WATCHPOINT      (1 << (TARGET_PAGE_BITS - 4))
+#define TLB_WATCHPOINT      (1 << (TARGET_PAGE_BITS_MIN - 4))
 
 /* Use this mask to check interception with an alignment mask
  * in a TCG backend.
-- 
2.17.1

This forced inlining can result in missing symbols,
which makes a debugging build harder to follow.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/compiler.h | 11 +++++++++++
 accel/tcg/cputlb.c      |  4 ++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/compiler.h
+++ b/include/qemu/compiler.h
@@ -XXX,XX +XXX,XX @@
 # define QEMU_NONSTRING
 #endif
 
+/*
+ * Forced inlining may be desired to encourage constant propagation
+ * of function parameters.  However, it can also make debugging harder,
+ * so disable it for a non-optimizing build.
+ */
+#if defined(__OPTIMIZE__)
+#define QEMU_ALWAYS_INLINE  __attribute__((always_inline))
+#else
+#define QEMU_ALWAYS_INLINE
+#endif
+
 /* Implement C11 _Generic via GCC builtins.  Example:
  *
  *    QEMU_GENERIC(x, (float, sinf), (long double, sinl), sin) (x)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
 typedef uint64_t FullLoadHelper(CPUArchState *env, target_ulong addr,
                                 TCGMemOpIdx oi, uintptr_t retaddr);
 
-static inline uint64_t __attribute__((always_inline))
+static inline uint64_t QEMU_ALWAYS_INLINE
 load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
             uintptr_t retaddr, MemOp op, bool code_read,
             FullLoadHelper *full_load)
@@ -XXX,XX +XXX,XX @@ tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *env, target_ulong addr,
  * Store Helpers
  */
 
-static inline void __attribute__((always_inline))
+static inline void QEMU_ALWAYS_INLINE
 store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
              TCGMemOpIdx oi, uintptr_t retaddr, MemOp op)
 {
-- 
2.17.1

Use this as a compile-time assert that a particular
code path is not reachable.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/qemu/compiler.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/compiler.h
+++ b/include/qemu/compiler.h
@@ -XXX,XX +XXX,XX @@
 #define QEMU_GENERIC9(x, a0, ...) QEMU_GENERIC_IF(x, a0, QEMU_GENERIC8(x, __VA_ARGS__))
 #define QEMU_GENERIC10(x, a0, ...) QEMU_GENERIC_IF(x, a0, QEMU_GENERIC9(x, __VA_ARGS__))
 
+/**
+ * qemu_build_not_reached()
+ *
+ * The compiler, during optimization, is expected to prove that a call
+ * to this function cannot be reached and remove it.  If the compiler
+ * supports QEMU_ERROR, this will be reported at compile time; otherwise
+ * this will be reported at link time due to the missing symbol.
+ */
+#ifdef __OPTIMIZE__
+extern void QEMU_NORETURN QEMU_ERROR("code path is reachable")
+    qemu_build_not_reached(void);
+#else
+#define qemu_build_not_reached()  g_assert_not_reached()
+#endif
+
 #endif /* COMPILER_H */
-- 
2.17.1

We will shortly be using these more than once.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/cputlb.c | 107 +++++++++++++++++++++++----------------------
 1 file changed, 55 insertions(+), 52 deletions(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
 typedef uint64_t FullLoadHelper(CPUArchState *env, target_ulong addr,
                                 TCGMemOpIdx oi, uintptr_t retaddr);
 
+static inline uint64_t QEMU_ALWAYS_INLINE
+load_memop(const void *haddr, MemOp op)
+{
+    switch (op) {
+    case MO_UB:
+        return ldub_p(haddr);
+    case MO_BEUW:
+        return lduw_be_p(haddr);
+    case MO_LEUW:
+        return lduw_le_p(haddr);
+    case MO_BEUL:
+        return (uint32_t)ldl_be_p(haddr);
+    case MO_LEUL:
+        return (uint32_t)ldl_le_p(haddr);
+    case MO_BEQ:
+        return ldq_be_p(haddr);
+    case MO_LEQ:
+        return ldq_le_p(haddr);
+    default:
+        qemu_build_not_reached();
+    }
+}
+
 static inline uint64_t QEMU_ALWAYS_INLINE
 load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
             uintptr_t retaddr, MemOp op, bool code_read,
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
 
  do_aligned_access:
     haddr = (void *)((uintptr_t)addr + entry->addend);
-    switch (op) {
-    case MO_UB:
-        res = ldub_p(haddr);
-        break;
-    case MO_BEUW:
-        res = lduw_be_p(haddr);
-        break;
-    case MO_LEUW:
-        res = lduw_le_p(haddr);
-        break;
-    case MO_BEUL:
-        res = (uint32_t)ldl_be_p(haddr);
-        break;
-    case MO_LEUL:
-        res = (uint32_t)ldl_le_p(haddr);
-        break;
-    case MO_BEQ:
-        res = ldq_be_p(haddr);
-        break;
-    case MO_LEQ:
-        res = ldq_le_p(haddr);
-        break;
-    default:
-        qemu_build_not_reached();
-    }
-
-    return res;
+    return load_memop(haddr, op);
 }
 
 /*
@@ -XXX,XX +XXX,XX @@ tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *env, target_ulong addr,
  * Store Helpers
  */
 
+static inline void QEMU_ALWAYS_INLINE
+store_memop(void *haddr, uint64_t val, MemOp op)
+{
+    switch (op) {
+    case MO_UB:
+        stb_p(haddr, val);
+        break;
+    case MO_BEUW:
+        stw_be_p(haddr, val);
+        break;
+    case MO_LEUW:
+        stw_le_p(haddr, val);
+        break;
+    case MO_BEUL:
+        stl_be_p(haddr, val);
+        break;
+    case MO_LEUL:
+        stl_le_p(haddr, val);
+        break;
+    case MO_BEQ:
+        stq_be_p(haddr, val);
+        break;
+    case MO_LEQ:
+        stq_le_p(haddr, val);
+        break;
+    default:
+        qemu_build_not_reached();
+    }
+}
+
 static inline void QEMU_ALWAYS_INLINE
 store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
              TCGMemOpIdx oi, uintptr_t retaddr, MemOp op)
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
 
  do_aligned_access:
     haddr = (void *)((uintptr_t)addr + entry->addend);
-    switch (op) {
-    case MO_UB:
-        stb_p(haddr, val);
-        break;
-    case MO_BEUW:
-        stw_be_p(haddr, val);
-        break;
-    case MO_LEUW:
-        stw_le_p(haddr, val);
-        break;
-    case MO_BEUL:
-        stl_be_p(haddr, val);
-        break;
-    case MO_LEUL:
-        stl_le_p(haddr, val);
-        break;
-    case MO_BEQ:
-        stq_be_p(haddr, val);
-        break;
-    case MO_LEQ:
-        stq_le_p(haddr, val);
-        break;
-    default:
-        qemu_build_not_reached();
-    }
+    store_memop(haddr, val, op);
 }
 
 void helper_ret_stb_mmu(CPUArchState *env, target_ulong addr, uint8_t val,
-- 
2.17.1

Handle bswap on ram directly in load/store_helper.  This fixes a
bug with the previous implementation in that one cannot use the
I/O path for RAM.

Fixes: a26fc6f5152b47f1
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-all.h |  4 ++-
 accel/tcg/cputlb.c     | 72 +++++++++++++++++++++++++-----------------
 2 files changed, 46 insertions(+), 30 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -XXX,XX +XXX,XX @@ CPUArchState *cpu_copy(CPUArchState *env);
 #define TLB_MMIO            (1 << (TARGET_PAGE_BITS_MIN - 3))
 /* Set if TLB entry contains a watchpoint.  */
 #define TLB_WATCHPOINT      (1 << (TARGET_PAGE_BITS_MIN - 4))
+/* Set if TLB entry requires byte swap.  */
+#define TLB_BSWAP           (1 << (TARGET_PAGE_BITS_MIN - 5))
 
 /* Use this mask to check interception with an alignment mask
  * in a TCG backend.
  */
 #define TLB_FLAGS_MASK \
-    (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO | TLB_WATCHPOINT)
+    (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO | TLB_WATCHPOINT | TLB_BSWAP)
 
 /**
  * tlb_hit_page: return true if page aligned @addr is a hit against the
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
         address |= TLB_INVALID_MASK;
     }
     if (attrs.byte_swap) {
-        /* Force the access through the I/O slow path.  */
-        address |= TLB_MMIO;
+        address |= TLB_BSWAP;
     }
     if (!memory_region_is_ram(section->mr) &&
         !memory_region_is_romd(section->mr)) {
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     bool locked = false;
     MemTxResult r;
 
-    if (iotlbentry->attrs.byte_swap) {
-        op ^= MO_BSWAP;
-    }
-
     section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
     mr = section->mr;
     mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     bool locked = false;
     MemTxResult r;
 
-    if (iotlbentry->attrs.byte_swap) {
-        op ^= MO_BSWAP;
-    }
-
     section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
     mr = section->mr;
     mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
                              wp_access, retaddr);
     }
 
-    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO)) {
-        /* I/O access */
+    /* Reject I/O access, or other required slow-path.  */
+    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO | TLB_BSWAP)) {
         return NULL;
     }
 
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
     /* Handle anything that isn't just a straight memory access.  */
     if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
         CPUIOTLBEntry *iotlbentry;
+        bool need_swap;
 
         /* For anything that is unaligned, recurse through full_load.  */
         if ((addr & (size - 1)) != 0) {
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
             /* On watchpoint hit, this will longjmp out.  */
             cpu_check_watchpoint(env_cpu(env), addr, size,
                                  iotlbentry->attrs, BP_MEM_READ, retaddr);
-
-            /* The backing page may or may not require I/O.  */
-            tlb_addr &= ~TLB_WATCHPOINT;
-            if ((tlb_addr & ~TARGET_PAGE_MASK) == 0) {
-                goto do_aligned_access;
-            }
         }
 
+        need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
+
         /* Handle I/O access.  */
-        return io_readx(env, iotlbentry, mmu_idx, addr,
-                        retaddr, access_type, op);
+        if (likely(tlb_addr & TLB_MMIO)) {
+            return io_readx(env, iotlbentry, mmu_idx, addr, retaddr,
+                            access_type, op ^ (need_swap * MO_BSWAP));
+        }
+
+        haddr = (void *)((uintptr_t)addr + entry->addend);
+
+        /*
+         * Keep these two load_memop separate to ensure that the compiler
+         * is able to fold the entire function to a single instruction.
+         * There is a build-time assert inside to remind you of this.  ;-)
+         */
+        if (unlikely(need_swap)) {
+            return load_memop(haddr, op ^ MO_BSWAP);
+        }
+        return load_memop(haddr, op);
     }
 
     /* Handle slow unaligned access (it spans two pages or IO).  */
@@ -XXX,XX +XXX,XX @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi,
         return res & MAKE_64BIT_MASK(0, size * 8);
     }
 
- do_aligned_access:
     haddr = (void *)((uintptr_t)addr + entry->addend);
     return load_memop(haddr, op);
 }
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
     /* Handle anything that isn't just a straight memory access.  */
     if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
         CPUIOTLBEntry *iotlbentry;
+        bool need_swap;
 
         /* For anything that is unaligned, recurse through byte stores.  */
         if ((addr & (size - 1)) != 0) {
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
             /* On watchpoint hit, this will longjmp out.  */
             cpu_check_watchpoint(env_cpu(env), addr, size,
                                  iotlbentry->attrs, BP_MEM_WRITE, retaddr);
-
-            /* The backing page may or may not require I/O.  */
-            tlb_addr &= ~TLB_WATCHPOINT;
-            if ((tlb_addr & ~TARGET_PAGE_MASK) == 0) {
-                goto do_aligned_access;
-            }
         }
 
+        need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
+
         /* Handle I/O access.  */
-        io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, op);
+        if (likely(tlb_addr & (TLB_MMIO | TLB_NOTDIRTY))) {
+            io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr,
+                      op ^ (need_swap * MO_BSWAP));
+            return;
+        }
+
+        haddr = (void *)((uintptr_t)addr + entry->addend);
+
+        /*
+         * Keep these two store_memop separate to ensure that the compiler
+         * is able to fold the entire function to a single instruction.
+         * There is a build-time assert inside to remind you of this.  ;-)
+         */
+        if (unlikely(need_swap)) {
+            store_memop(haddr, val, op ^ MO_BSWAP);
+        } else {
+            store_memop(haddr, val, op);
+        }
         return;
     }
 
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
         return;
     }
 
- do_aligned_access:
     haddr = (void *)((uintptr_t)addr + entry->addend);
     store_memop(haddr, val, op);
 }
-- 
2.17.1

The memory_region_tb_read tracepoint is unreachable, since notdirty
is supposed to apply only to writes.  The memory_region_tb_write
tracepoint is mis-named, because notdirty is not only used for TB
invalidation.  It is also used for e.g. VGA RAM updates and migration.

Replace memory_region_tb_write with memory_notdirty_write_access,
and place it in memory_notdirty_write_prepare where it can catch
all of the instances.  Add memory_notdirty_set_dirty to log when
we no longer intercept writes to a page.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 exec.c       | 3 +++
 memory.c     | 4 ----
 trace-events | 4 ++--
 3 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/exec.c b/exec.c
index XXXXXXX..XXXXXXX 100644
--- a/exec.c
+++ b/exec.c
@@ -XXX,XX +XXX,XX @@ void memory_notdirty_write_prepare(NotDirtyInfo *ndi,
     ndi->size = size;
     ndi->pages = NULL;
 
+    trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size);
+
     assert(tcg_enabled());
     if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
         ndi->pages = page_collection_lock(ram_addr, ram_addr + size);
@@ -XXX,XX +XXX,XX @@ void memory_notdirty_write_complete(NotDirtyInfo *ndi)
     /* we remove the notdirty callback only if the code has been
        flushed */
     if (!cpu_physical_memory_is_clean(ndi->ram_addr)) {
+        trace_memory_notdirty_set_dirty(ndi->mem_vaddr);
         tlb_set_dirty(ndi->cpu, ndi->mem_vaddr);
     }
 }
diff --git a/memory.c b/memory.c
index XXXXXXX..XXXXXXX 100644
--- a/memory.c
+++ b/memory.c
@@ -XXX,XX +XXX,XX @@ static MemTxResult  memory_region_read_accessor(MemoryRegion *mr,
         /* Accesses to code which has previously been translated into a TB show
          * up in the MMIO path, as accesses to the io_mem_notdirty
          * MemoryRegion. */
-        trace_memory_region_tb_read(get_cpu_index(), addr, tmp, size);
     } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) {
         hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
         trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_read_with_attrs_accessor(MemoryRegion *mr,
         /* Accesses to code which has previously been translated into a TB show
          * up in the MMIO path, as accesses to the io_mem_notdirty
          * MemoryRegion. */
-        trace_memory_region_tb_read(get_cpu_index(), addr, tmp, size);
     } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) {
         hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
         trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_write_accessor(MemoryRegion *mr,
         /* Accesses to code which has previously been translated into a TB show
          * up in the MMIO path, as accesses to the io_mem_notdirty
          * MemoryRegion. */
-        trace_memory_region_tb_write(get_cpu_index(), addr, tmp, size);
     } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) {
         hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
         trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_write_with_attrs_accessor(MemoryRegion *mr,
         /* Accesses to code which has previously been translated into a TB show
          * up in the MMIO path, as accesses to the io_mem_notdirty
          * MemoryRegion. */
-        trace_memory_region_tb_write(get_cpu_index(), addr, tmp, size);
     } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) {
         hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
         trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size);
diff --git a/trace-events b/trace-events
index XXXXXXX..XXXXXXX 100644
--- a/trace-events
+++ b/trace-events
@@ -XXX,XX +XXX,XX @@ dma_map_wait(void *dbs) "dbs=%p"
 find_ram_offset(uint64_t size, uint64_t offset) "size: 0x%" PRIx64 " @ 0x%" PRIx64
 find_ram_offset_loop(uint64_t size, uint64_t candidate, uint64_t offset, uint64_t next, uint64_t mingap) "trying size: 0x%" PRIx64 " @ 0x%" PRIx64 ", offset: 0x%" PRIx64" next: 0x%" PRIx64 " mingap: 0x%" PRIx64
 ram_block_discard_range(const char *rbname, void *hva, size_t length, bool need_madvise, bool need_fallocate, int ret) "%s@%p + 0x%zx: madvise: %d fallocate: %d ret: %d"
+memory_notdirty_write_access(uint64_t vaddr, uint64_t ram_addr, unsigned size) "0x%" PRIx64 " ram_addr 0x%" PRIx64 " size %u"
+memory_notdirty_set_dirty(uint64_t vaddr) "0x%" PRIx64
 
 # memory.c
 memory_region_ops_read(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
 memory_region_ops_write(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
 memory_region_subpage_read(int cpu_index, void *mr, uint64_t offset, uint64_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx64" size %u"
 memory_region_subpage_write(int cpu_index, void *mr, uint64_t offset, uint64_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx64" size %u"
-memory_region_tb_read(int cpu_index, uint64_t addr, uint64_t value, unsigned size) "cpu %d addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
-memory_region_tb_write(int cpu_index, uint64_t addr, uint64_t value, unsigned size) "cpu %d addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
 memory_region_ram_device_read(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
 memory_region_ram_device_write(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u"
 flatview_new(void *view, void *root) "%p (root %p)"
-- 
2.17.1

It does not require going through the whole I/O path
in order to discard a write.

Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-all.h    |  5 ++++-
 include/exec/cpu-common.h |  1 -
 accel/tcg/cputlb.c        | 36 ++++++++++++++++++++--------------
 exec.c                    | 41 +--------------------------------------
 4 files changed, 26 insertions(+), 57 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -XXX,XX +XXX,XX @@ CPUArchState *cpu_copy(CPUArchState *env);
 #define TLB_WATCHPOINT      (1 << (TARGET_PAGE_BITS_MIN - 4))
 /* Set if TLB entry requires byte swap.  */
 #define TLB_BSWAP           (1 << (TARGET_PAGE_BITS_MIN - 5))
+/* Set if TLB entry writes ignored.  */
+#define TLB_DISCARD_WRITE   (1 << (TARGET_PAGE_BITS_MIN - 6))
 
 /* Use this mask to check interception with an alignment mask
  * in a TCG backend.
  */
 #define TLB_FLAGS_MASK \
-    (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO | TLB_WATCHPOINT | TLB_BSWAP)
+    (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO \
+    | TLB_WATCHPOINT | TLB_BSWAP | TLB_DISCARD_WRITE)
 
 /**
  * tlb_hit_page: return true if page aligned @addr is a hit against the
diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -XXX,XX +XXX,XX @@ void qemu_flush_coalesced_mmio_buffer(void);
 
 void cpu_flush_icache_range(hwaddr start, hwaddr len);
 
-extern struct MemoryRegion io_mem_rom;
 extern struct MemoryRegion io_mem_notdirty;
 
 typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tlb_reset_dirty_range_locked(CPUTLBEntry *tlb_entry,
 {
     uintptr_t addr = tlb_entry->addr_write;
 
-    if ((addr & (TLB_INVALID_MASK | TLB_MMIO | TLB_NOTDIRTY)) == 0) {
+    if ((addr & (TLB_INVALID_MASK | TLB_MMIO |
+                 TLB_DISCARD_WRITE | TLB_NOTDIRTY)) == 0) {
         addr &= TARGET_PAGE_MASK;
         addr += tlb_entry->addend;
         if ((addr - start) < length) {
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
         address |= TLB_MMIO;
         addend = 0;
     } else {
-        /* TLB_MMIO for rom/romd handled below */
         addend = (uintptr_t)memory_region_get_ram_ptr(section->mr) + xlat;
     }
 
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
 
     tn.addr_write = -1;
     if (prot & PAGE_WRITE) {
-        if ((memory_region_is_ram(section->mr) && section->readonly)
-            || memory_region_is_romd(section->mr)) {
-            /* Write access calls the I/O callback.  */
-            tn.addr_write = address | TLB_MMIO;
-        } else if (memory_region_is_ram(section->mr)
-                   && cpu_physical_memory_is_clean(
-                       memory_region_get_ram_addr(section->mr) + xlat)) {
-            tn.addr_write = address | TLB_NOTDIRTY;
-        } else {
-            tn.addr_write = address;
+        tn.addr_write = address;
+        if (memory_region_is_romd(section->mr)) {
+            /* Use the MMIO path so that the device can switch states. */
+            tn.addr_write |= TLB_MMIO;
+        } else if (memory_region_is_ram(section->mr)) {
+            if (section->readonly) {
+                tn.addr_write |= TLB_DISCARD_WRITE;
+            } else if (cpu_physical_memory_is_clean(
+                        memory_region_get_ram_addr(section->mr) + xlat)) {
+                tn.addr_write |= TLB_NOTDIRTY;
+            }
         }
         if (prot & PAGE_WRITE_INV) {
             tn.addr_write |= TLB_INVALID_MASK;
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     mr = section->mr;
     mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
     cpu->mem_io_pc = retaddr;
-    if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) {
+    if (mr != &io_mem_notdirty && !cpu->can_do_io) {
         cpu_io_recompile(cpu, retaddr);
     }
 
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
     mr = section->mr;
     mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
-    if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) {
+    if (mr != &io_mem_notdirty && !cpu->can_do_io) {
         cpu_io_recompile(cpu, retaddr);
     }
     cpu->mem_io_vaddr = addr;
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
     }
 
     /* Reject I/O access, or other required slow-path.  */
-    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO | TLB_BSWAP)) {
+    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO | TLB_BSWAP | TLB_DISCARD_WRITE)) {
         return NULL;
     }
 
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
             return;
         }
 
+        /* Ignore writes to ROM.  */
+        if (unlikely(tlb_addr & TLB_DISCARD_WRITE)) {
+            return;
+        }
+
         haddr = (void *)((uintptr_t)addr + entry->addend);
 
         /*
diff --git a/exec.c b/exec.c
index XXXXXXX..XXXXXXX 100644
--- a/exec.c
+++ b/exec.c
@@ -XXX,XX +XXX,XX @@ static MemoryRegion *system_io;
 AddressSpace address_space_io;
 AddressSpace address_space_memory;
 
-MemoryRegion io_mem_rom, io_mem_notdirty;
+MemoryRegion io_mem_notdirty;
 static MemoryRegion io_mem_unassigned;
 #endif
 
@@ -XXX,XX +XXX,XX @@ typedef struct subpage_t {
 
 #define PHYS_SECTION_UNASSIGNED 0
 #define PHYS_SECTION_NOTDIRTY 1
-#define PHYS_SECTION_ROM 2
 
 static void io_mem_init(void);
 static void memory_map_init(void);
@@ -XXX,XX +XXX,XX @@ hwaddr memory_region_section_get_iotlb(CPUState *cpu,
         iotlb = memory_region_get_ram_addr(section->mr) + xlat;
         if (!section->readonly) {
             iotlb |= PHYS_SECTION_NOTDIRTY;
-        } else {
-            iotlb |= PHYS_SECTION_ROM;
         }
     } else {
         AddressSpaceDispatch *d;
@@ -XXX,XX +XXX,XX @@ static uint16_t dummy_section(PhysPageMap *map, FlatView *fv, MemoryRegion *mr)
     return phys_section_add(map, &section);
 }
 
-static void readonly_mem_write(void *opaque, hwaddr addr,
-                               uint64_t val, unsigned size)
-{
-    /* Ignore any write to ROM. */
-}
-
-static bool readonly_mem_accepts(void *opaque, hwaddr addr,
-                                 unsigned size, bool is_write,
-                                 MemTxAttrs attrs)
-{
-    return is_write;
-}
-
-/* This will only be used for writes, because reads are special cased
- * to directly access the underlying host ram.
- */
-static const MemoryRegionOps readonly_mem_ops = {
-    .write = readonly_mem_write,
-    .valid.accepts = readonly_mem_accepts,
-    .endianness = DEVICE_NATIVE_ENDIAN,
-    .valid = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-        .unaligned = false,
-    },
-    .impl = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-        .unaligned = false,
-    },
-};
-
 MemoryRegionSection *iotlb_to_section(CPUState *cpu,
                                       hwaddr index, MemTxAttrs attrs)
 {
@@ -XXX,XX +XXX,XX @@ MemoryRegionSection *iotlb_to_section(CPUState *cpu,
 
 static void io_mem_init(void)
 {
-    memory_region_init_io(&io_mem_rom, NULL, &readonly_mem_ops,
-                          NULL, NULL, UINT64_MAX);
     memory_region_init_io(&io_mem_unassigned, NULL, &unassigned_mem_ops, NULL,
                           NULL, UINT64_MAX);
 
@@ -XXX,XX +XXX,XX @@ AddressSpaceDispatch *address_space_dispatch_new(FlatView *fv)
     assert(n == PHYS_SECTION_UNASSIGNED);
     n = dummy_section(&d->map, fv, &io_mem_notdirty);
     assert(n == PHYS_SECTION_NOTDIRTY);
-    n = dummy_section(&d->map, fv, &io_mem_rom);
-    assert(n == PHYS_SECTION_ROM);
 
     d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .skip = 1 };
 
-- 
2.17.1

Pages that we want to track for NOTDIRTY are RAM.  We do not
really need to go through the I/O path to handle them.

Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-common.h |  2 --
 accel/tcg/cputlb.c        | 26 +++++++++++++++++---
 exec.c                    | 50 ---------------------------------------
 memory.c                  | 16 -------------
 4 files changed, 23 insertions(+), 71 deletions(-)

diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -XXX,XX +XXX,XX @@ void qemu_flush_coalesced_mmio_buffer(void);
 
 void cpu_flush_icache_range(hwaddr start, hwaddr len);
 
-extern struct MemoryRegion io_mem_notdirty;
-
 typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);
 
 int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque);
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     mr = section->mr;
     mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
     cpu->mem_io_pc = retaddr;
-    if (mr != &io_mem_notdirty && !cpu->can_do_io) {
+    if (!cpu->can_do_io) {
         cpu_io_recompile(cpu, retaddr);
     }
 
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
     mr = section->mr;
     mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
-    if (mr != &io_mem_notdirty && !cpu->can_do_io) {
+    if (!cpu->can_do_io) {
         cpu_io_recompile(cpu, retaddr);
     }
     cpu->mem_io_vaddr = addr;
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
         need_swap = size > 1 && (tlb_addr & TLB_BSWAP);
 
         /* Handle I/O access.  */
-        if (likely(tlb_addr & (TLB_MMIO | TLB_NOTDIRTY))) {
+        if (tlb_addr & TLB_MMIO) {
             io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr,
                       op ^ (need_swap * MO_BSWAP));
             return;
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
 
         haddr = (void *)((uintptr_t)addr + entry->addend);
 
+        /* Handle clean RAM pages.  */
+        if (tlb_addr & TLB_NOTDIRTY) {
+            NotDirtyInfo ndi;
+
+            /* We require mem_io_pc in tb_invalidate_phys_page_range.  */
+            env_cpu(env)->mem_io_pc = retaddr;
+
+            memory_notdirty_write_prepare(&ndi, env_cpu(env), addr,
+                                          addr + iotlbentry->addr, size);
+
+            if (unlikely(need_swap)) {
+                store_memop(haddr, val, op ^ MO_BSWAP);
+            } else {
+                store_memop(haddr, val, op);
+            }
+
+            memory_notdirty_write_complete(&ndi);
+            return;
+        }
+
         /*
          * Keep these two store_memop separate to ensure that the compiler
          * is able to fold the entire function to a single instruction.
diff --git a/exec.c b/exec.c
index XXXXXXX..XXXXXXX 100644
--- a/exec.c
+++ b/exec.c
@@ -XXX,XX +XXX,XX @@ static MemoryRegion *system_io;
 AddressSpace address_space_io;
 AddressSpace address_space_memory;
 
-MemoryRegion io_mem_notdirty;
 static MemoryRegion io_mem_unassigned;
 #endif
 
@@ -XXX,XX +XXX,XX @@ typedef struct subpage_t {
 } subpage_t;
 
 #define PHYS_SECTION_UNASSIGNED 0
-#define PHYS_SECTION_NOTDIRTY 1
 
 static void io_mem_init(void);
 static void memory_map_init(void);
@@ -XXX,XX +XXX,XX @@ hwaddr memory_region_section_get_iotlb(CPUState *cpu,
     if (memory_region_is_ram(section->mr)) {
         /* Normal RAM.  */
         iotlb = memory_region_get_ram_addr(section->mr) + xlat;
-        if (!section->readonly) {
-            iotlb |= PHYS_SECTION_NOTDIRTY;
-        }
     } else {
         AddressSpaceDispatch *d;
 
@@ -XXX,XX +XXX,XX @@ void memory_notdirty_write_complete(NotDirtyInfo *ndi)
     }
 }
 
-/* Called within RCU critical section.  */
-static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
-                               uint64_t val, unsigned size)
-{
-    NotDirtyInfo ndi;
-
-    memory_notdirty_write_prepare(&ndi, current_cpu, current_cpu->mem_io_vaddr,
-                         ram_addr, size);
-
-    stn_p(qemu_map_ram_ptr(NULL, ram_addr), size, val);
-    memory_notdirty_write_complete(&ndi);
-}
-
-static bool notdirty_mem_accepts(void *opaque, hwaddr addr,
-                                 unsigned size, bool is_write,
-                                 MemTxAttrs attrs)
-{
-    return is_write;
-}
-
-static const MemoryRegionOps notdirty_mem_ops = {
-    .write = notdirty_mem_write,
-    .valid.accepts = notdirty_mem_accepts,
-    .endianness = DEVICE_NATIVE_ENDIAN,
-    .valid = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-        .unaligned = false,
-    },
-    .impl = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-        .unaligned = false,
-    },
-};
-
 /* Generate a debug exception if a watchpoint has been hit.  */
 void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
                           MemTxAttrs attrs, int flags, uintptr_t ra)
@@ -XXX,XX +XXX,XX @@ static void io_mem_init(void)
 {
     memory_region_init_io(&io_mem_unassigned, NULL, &unassigned_mem_ops, NULL,
                           NULL, UINT64_MAX);
-
-    /* io_mem_notdirty calls tb_invalidate_phys_page_fast,
-     * which can be called without the iothread mutex.
-     */
-    memory_region_init_io(&io_mem_notdirty, NULL, &notdirty_mem_ops, NULL,
-                          NULL, UINT64_MAX);
-    memory_region_clear_global_locking(&io_mem_notdirty);
 }
 
 AddressSpaceDispatch *address_space_dispatch_new(FlatView *fv)
@@ -XXX,XX +XXX,XX @@ AddressSpaceDispatch *address_space_dispatch_new(FlatView *fv)
 
     n = dummy_section(&d->map, fv, &io_mem_unassigned);
     assert(n == PHYS_SECTION_UNASSIGNED);
-    n = dummy_section(&d->map, fv, &io_mem_notdirty);
-    assert(n == PHYS_SECTION_NOTDIRTY);
 
     d->phys_map  = (PhysPageEntry) { .ptr = PHYS_MAP_NODE_NIL, .skip = 1 };
 
diff --git a/memory.c b/memory.c
index XXXXXXX..XXXXXXX 100644
--- a/memory.c
+++ b/memory.c
@@ -XXX,XX +XXX,XX @@ static MemTxResult  memory_region_read_accessor(MemoryRegion *mr,
     tmp = mr->ops->read(mr->opaque, addr, size);
     if (mr->subpage) {
         trace_memory_region_subpage_read(get_cpu_index(), mr, addr, tmp, size);
-    } else if (mr == &io_mem_notdirty) {
-        /* Accesses to code which has previously been translated into a TB show
-         * up in the MMIO path, as accesses to the io_mem_notdirty
-         * MemoryRegion. */
     } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) {
         hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
         trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_read_with_attrs_accessor(MemoryRegion *mr,
     r = mr->ops->read_with_attrs(mr->opaque, addr, &tmp, size, attrs);
     if (mr->subpage) {
         trace_memory_region_subpage_read(get_cpu_index(), mr, addr, tmp, size);
-    } else if (mr == &io_mem_notdirty) {
-        /* Accesses to code which has previously been translated into a TB show
-         * up in the MMIO path, as accesses to the io_mem_notdirty
-         * MemoryRegion. */
     } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) {
         hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
         trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_write_accessor(MemoryRegion *mr,
 
     if (mr->subpage) {
         trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, size);
-    } else if (mr == &io_mem_notdirty) {
-        /* Accesses to code which has previously been translated into a TB show
-         * up in the MMIO path, as accesses to the io_mem_notdirty
-         * MemoryRegion. */
     } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) {
         hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
         trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size);
@@ -XXX,XX +XXX,XX @@ static MemTxResult memory_region_write_with_attrs_accessor(MemoryRegion *mr,
 
     if (mr->subpage) {
         trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, size);
-    } else if (mr == &io_mem_notdirty) {
-        /* Accesses to code which has previously been translated into a TB show
-         * up in the MMIO path, as accesses to the io_mem_notdirty
-         * MemoryRegion. */
     } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) {
         hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
         trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size);
-- 
2.17.1

There is only one caller, tlb_set_page_with_attrs.  We cannot
inline the entire function because the AddressSpaceDispatch
structure is private to exec.c, and cannot easily be moved to
include/exec/memory-internal.h.

Compute is_ram and is_romd once within tlb_set_page_with_attrs.
Fold the number of tests against these predicates.  Compute
cpu_physical_memory_is_clean outside of the tlb lock region.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/exec-all.h |  6 +---
 accel/tcg/cputlb.c      | 68 ++++++++++++++++++++++++++---------------
 exec.c                  | 22 ++-----------
 3 files changed, 47 insertions(+), 49 deletions(-)

diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -XXX,XX +XXX,XX @@ address_space_translate_for_iotlb(CPUState *cpu, int asidx, hwaddr addr,
                                   hwaddr *xlat, hwaddr *plen,
                                   MemTxAttrs attrs, int *prot);
 hwaddr memory_region_section_get_iotlb(CPUState *cpu,
-                                       MemoryRegionSection *section,
-                                       target_ulong vaddr,
-                                       hwaddr paddr, hwaddr xlat,
-                                       int prot,
-                                       target_ulong *address);
+                                       MemoryRegionSection *section);
 #endif
 
 /* vl.c */
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
     MemoryRegionSection *section;
     unsigned int index;
     target_ulong address;
-    target_ulong code_address;
+    target_ulong write_address;
     uintptr_t addend;
     CPUTLBEntry *te, tn;
     hwaddr iotlb, xlat, sz, paddr_page;
     target_ulong vaddr_page;
     int asidx = cpu_asidx_from_attrs(cpu, attrs);
     int wp_flags;
+    bool is_ram, is_romd;
 
     assert_cpu_is_self(cpu);
 
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
     if (attrs.byte_swap) {
         address |= TLB_BSWAP;
     }
-    if (!memory_region_is_ram(section->mr) &&
-        !memory_region_is_romd(section->mr)) {
-        /* IO memory case */
-        address |= TLB_MMIO;
-        addend = 0;
-    } else {
+
+    is_ram = memory_region_is_ram(section->mr);
+    is_romd = memory_region_is_romd(section->mr);
+
+    if (is_ram || is_romd) {
+        /* RAM and ROMD both have associated host memory. */
         addend = (uintptr_t)memory_region_get_ram_ptr(section->mr) + xlat;
+    } else {
+        /* I/O does not; force the host address to NULL. */
+        addend = 0;
+    }
+
+    write_address = address;
+    if (is_ram) {
+        iotlb = memory_region_get_ram_addr(section->mr) + xlat;
+        /*
+         * Computing is_clean is expensive; avoid all that unless
+         * the page is actually writable.
+         */
+        if (prot & PAGE_WRITE) {
+            if (section->readonly) {
+                write_address |= TLB_DISCARD_WRITE;
+            } else if (cpu_physical_memory_is_clean(iotlb)) {
+                write_address |= TLB_NOTDIRTY;
+            }
+        }
+    } else {
+        /* I/O or ROMD */
+        iotlb = memory_region_section_get_iotlb(cpu, section) + xlat;
+        /*
+         * Writes to romd devices must go through MMIO to enable write.
+         * Reads to romd devices go through the ram_ptr found above,
+         * but of course reads to I/O must go through MMIO.
+         */
+        write_address |= TLB_MMIO;
+        if (!is_romd) {
+            address = write_address;
+        }
     }
 
-    code_address = address;
-    iotlb = memory_region_section_get_iotlb(cpu, section, vaddr_page,
-                                            paddr_page, xlat, prot, &address);
     wp_flags = cpu_watchpoint_address_matches(cpu, vaddr_page,
                                               TARGET_PAGE_SIZE);
 
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
     /*
      * At this point iotlb contains a physical section number in the lower
      * TARGET_PAGE_BITS, and either
-     *  + the ram_addr_t of the page base of the target RAM (if NOTDIRTY or ROM)
-     *  + the offset within section->mr of the page base (otherwise)
+     *  + the ram_addr_t of the page base of the target RAM (RAM)
+     *  + the offset within section->mr of the page base (I/O, ROMD)
      * We subtract the vaddr_page (which is page aligned and thus won't
      * disturb the low bits) to give an offset which can be added to the
      * (non-page-aligned) vaddr of the eventual memory access to get
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr,
     }
 
     if (prot & PAGE_EXEC) {
-        tn.addr_code = code_address;
+        tn.addr_code = address;
     } else {
         tn.addr_code = -1;
     }
 
     tn.addr_write = -1;
     if (prot & PAGE_WRITE) {
-        tn.addr_write = address;
-        if (memory_region_is_romd(section->mr)) {
-            /* Use the MMIO path so that the device can switch states. */
-            tn.addr_write |= TLB_MMIO;
-        } else if (memory_region_is_ram(section->mr)) {
-            if (section->readonly) {
-                tn.addr_write |= TLB_DISCARD_WRITE;
-            } else if (cpu_physical_memory_is_clean(
-                        memory_region_get_ram_addr(section->mr) + xlat)) {
-                tn.addr_write |= TLB_NOTDIRTY;
-            }
-        }
+        tn.addr_write = write_address;
         if (prot & PAGE_WRITE_INV) {
             tn.addr_write |= TLB_INVALID_MASK;
         }
diff --git a/exec.c b/exec.c
index XXXXXXX..XXXXXXX 100644
--- a/exec.c
+++ b/exec.c
@@ -XXX,XX +XXX,XX @@ bool cpu_physical_memory_snapshot_get_dirty(DirtyBitmapSnapshot *snap,
 
 /* Called from RCU critical section */
 hwaddr memory_region_section_get_iotlb(CPUState *cpu,
-                                       MemoryRegionSection *section,
-                                       target_ulong vaddr,
-                                       hwaddr paddr, hwaddr xlat,
-                                       int prot,
-                                       target_ulong *address)
+                                       MemoryRegionSection *section)
 {
-    hwaddr iotlb;
-
-    if (memory_region_is_ram(section->mr)) {
-        /* Normal RAM.  */
-        iotlb = memory_region_get_ram_addr(section->mr) + xlat;
-    } else {
-        AddressSpaceDispatch *d;
-
-        d = flatview_to_dispatch(section->fv);
-        iotlb = section - d->map.sections;
-        iotlb += xlat;
-    }
-
-    return iotlb;
+    AddressSpaceDispatch *d = flatview_to_dispatch(section->fv);
+    return section - d->map.sections;
 }
 #endif /* defined(CONFIG_USER_ONLY) */
 
-- 
2.17.1

Since 9458a9a1df1a, all readers of the dirty bitmaps wait
for the rcu lock, which means that they wait until the end
of any executing TranslationBlock.

As a consequence, there is no need for the actual access
to happen in between the _prepare and _complete.  Therefore,
we can improve things by merging the two functions into
notdirty_write and dropping the NotDirtyInfo structure.

In addition, the only users of notdirty_write are in cputlb.c,
so move the merged function there.  Pass in the CPUIOTLBEntry
from which the ram_addr_t may be computed.

Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/memory-internal.h | 65 -----------------------------
 accel/tcg/cputlb.c             | 76 +++++++++++++++++++---------------
 exec.c                         | 44 --------------------
 3 files changed, 42 insertions(+), 143 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -XXX,XX +XXX,XX @@ void address_space_dispatch_free(AddressSpaceDispatch *d);
 
 void mtree_print_dispatch(struct AddressSpaceDispatch *d,
                           MemoryRegion *root);
-
-struct page_collection;
-
-/* Opaque struct for passing info from memory_notdirty_write_prepare()
- * to memory_notdirty_write_complete(). Callers should treat all fields
- * as private, with the exception of @active.
- *
- * @active is a field which is not touched by either the prepare or
- * complete functions, but which the caller can use if it wishes to
- * track whether it has called prepare for this struct and so needs
- * to later call the complete function.
- */
-typedef struct {
-    CPUState *cpu;
-    struct page_collection *pages;
-    ram_addr_t ram_addr;
-    vaddr mem_vaddr;
-    unsigned size;
-    bool active;
-} NotDirtyInfo;
-
-/**
- * memory_notdirty_write_prepare: call before writing to non-dirty memory
- * @ndi: pointer to opaque NotDirtyInfo struct
- * @cpu: CPU doing the write
- * @mem_vaddr: virtual address of write
- * @ram_addr: the ram address of the write
- * @size: size of write in bytes
- *
- * Any code which writes to the host memory corresponding to
- * guest RAM which has been marked as NOTDIRTY must wrap those
- * writes in calls to memory_notdirty_write_prepare() and
- * memory_notdirty_write_complete():
- *
- *  NotDirtyInfo ndi;
- *  memory_notdirty_write_prepare(&ndi, ....);
- *  ... perform write here ...
- *  memory_notdirty_write_complete(&ndi);
- *
- * These calls will ensure that we flush any TCG translated code for
- * the memory being written, update the dirty bits and (if possible)
- * remove the slowpath callback for writing to the memory.
- *
- * This must only be called if we are using TCG; it will assert otherwise.
- *
- * We may take locks in the prepare call, so callers must ensure that
- * they don't exit (via longjump or otherwise) without calling complete.
- *
- * This call must only be made inside an RCU critical section.
- * (Note that while we're executing a TCG TB we're always in an
- * RCU critical section, which is likely to be the case for callers
- * of these functions.)
- */
-void memory_notdirty_write_prepare(NotDirtyInfo *ndi,
-                                   CPUState *cpu,
-                                   vaddr mem_vaddr,
-                                   ram_addr_t ram_addr,
-                                   unsigned size);
-/**
- * memory_notdirty_write_complete: finish write to non-dirty memory
- * @ndi: pointer to the opaque NotDirtyInfo struct which was initialized
- * by memory_not_dirty_write_prepare().
- */
-void memory_notdirty_write_complete(NotDirtyInfo *ndi);
-
 #endif
 #endif
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@
 #include "exec/helper-proto.h"
 #include "qemu/atomic.h"
 #include "qemu/atomic128.h"
+#include "translate-all.h"
 
 /* DEBUG defines, enable DEBUG_TLB_LOG to log to the CPU_LOG_MMU target */
 /* #define DEBUG_TLB */
@@ -XXX,XX +XXX,XX @@ tb_page_addr_t get_page_addr_code(CPUArchState *env, target_ulong addr)
     return qemu_ram_addr_from_host_nofail(p);
 }
 
+static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
+                           CPUIOTLBEntry *iotlbentry, uintptr_t retaddr)
+{
+    ram_addr_t ram_addr = mem_vaddr + iotlbentry->addr;
+
+    trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size);
+
+    if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
+        struct page_collection *pages
+            = page_collection_lock(ram_addr, ram_addr + size);
+
+        /* We require mem_io_pc in tb_invalidate_phys_page_range.  */
+        cpu->mem_io_pc = retaddr;
+
+        tb_invalidate_phys_page_fast(pages, ram_addr, size);
+        page_collection_unlock(pages);
+    }
+
+    /*
+     * Set both VGA and migration bits for simplicity and to remove
+     * the notdirty callback faster.
+     */
+    cpu_physical_memory_set_dirty_range(ram_addr, size, DIRTY_CLIENTS_NOCODE);
+
+    /* We remove the notdirty callback only if the code has been flushed. */
+    if (!cpu_physical_memory_is_clean(ram_addr)) {
+        trace_memory_notdirty_set_dirty(mem_vaddr);
+        tlb_set_dirty(cpu, mem_vaddr);
+    }
+}
+
 /*
  * Probe for whether the specified guest access is permitted. If it is not
  * permitted then an exception will be taken in the same way as if this
@@ -XXX,XX +XXX,XX @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr,
 /* Probe for a read-modify-write atomic operation.  Do not allow unaligned
  * operations, or io operations to proceed.  Return the host address.  */
 static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
-                               TCGMemOpIdx oi, uintptr_t retaddr,
-                               NotDirtyInfo *ndi)
+                               TCGMemOpIdx oi, uintptr_t retaddr)
 {
     size_t mmu_idx = get_mmuidx(oi);
     uintptr_t index = tlb_index(env, mmu_idx, addr);
@@ -XXX,XX +XXX,XX @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr,
 
     hostaddr = (void *)((uintptr_t)addr + tlbe->addend);
 
-    ndi->active = false;
     if (unlikely(tlb_addr & TLB_NOTDIRTY)) {
-        ndi->active = true;
-        memory_notdirty_write_prepare(ndi, env_cpu(env), addr,
-                                      qemu_ram_addr_from_host_nofail(hostaddr),
-                                      1 << s_bits);
+        notdirty_write(env_cpu(env), addr, 1 << s_bits,
+                       &env_tlb(env)->d[mmu_idx].iotlb[index], retaddr);
     }
 
     return hostaddr;
@@ -XXX,XX +XXX,XX @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val,
             return;
         }
 
-        haddr = (void *)((uintptr_t)addr + entry->addend);
-
         /* Handle clean RAM pages.  */
         if (tlb_addr & TLB_NOTDIRTY) {
-            NotDirtyInfo ndi;
-
-            /* We require mem_io_pc in tb_invalidate_phys_page_range.  */
-            env_cpu(env)->mem_io_pc = retaddr;
-
-            memory_notdirty_write_prepare(&ndi, env_cpu(env), addr,
-                                          addr + iotlbentry->addr, size);
-
-            if (unlikely(need_swap)) {
-                store_memop(haddr, val, op ^ MO_BSWAP);
-            } else {
-                store_memop(haddr, val, op);
-            }
-
-            memory_notdirty_write_complete(&ndi);
-            return;
+            notdirty_write(env_cpu(env), addr, size, iotlbentry, retaddr);
         }
 
+        haddr = (void *)((uintptr_t)addr + entry->addend);
+
         /*
          * Keep these two store_memop separate to ensure that the compiler
          * is able to fold the entire function to a single instruction.
@@ -XXX,XX +XXX,XX @@ void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val,
 #define EXTRA_ARGS     , TCGMemOpIdx oi, uintptr_t retaddr
 #define ATOMIC_NAME(X) \
     HELPER(glue(glue(glue(atomic_ ## X, SUFFIX), END), _mmu))
-#define ATOMIC_MMU_DECLS NotDirtyInfo ndi
-#define ATOMIC_MMU_LOOKUP atomic_mmu_lookup(env, addr, oi, retaddr, &ndi)
-#define ATOMIC_MMU_CLEANUP                              \
-    do {                                                \
-        if (unlikely(ndi.active)) {                     \
-            memory_notdirty_write_complete(&ndi);       \
-        }                                               \
-    } while (0)
+#define ATOMIC_MMU_DECLS
+#define ATOMIC_MMU_LOOKUP atomic_mmu_lookup(env, addr, oi, retaddr)
+#define ATOMIC_MMU_CLEANUP
 
 #define DATA_SIZE 1
 #include "atomic_template.h"
@@ -XXX,XX +XXX,XX @@ void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val,
 #undef ATOMIC_MMU_LOOKUP
 #define EXTRA_ARGS         , TCGMemOpIdx oi
 #define ATOMIC_NAME(X)     HELPER(glue(glue(atomic_ ## X, SUFFIX), END))
-#define ATOMIC_MMU_LOOKUP  atomic_mmu_lookup(env, addr, oi, GETPC(), &ndi)
+#define ATOMIC_MMU_LOOKUP  atomic_mmu_lookup(env, addr, oi, GETPC())
 
 #define DATA_SIZE 1
 #include "atomic_template.h"
diff --git a/exec.c b/exec.c
index XXXXXXX..XXXXXXX 100644
--- a/exec.c
+++ b/exec.c
@@ -XXX,XX +XXX,XX @@ ram_addr_t qemu_ram_addr_from_host(void *ptr)
     return block->offset + offset;
 }
 
-/* Called within RCU critical section. */
-void memory_notdirty_write_prepare(NotDirtyInfo *ndi,
-                          CPUState *cpu,
-                          vaddr mem_vaddr,
-                          ram_addr_t ram_addr,
-                          unsigned size)
-{
-    ndi->cpu = cpu;
-    ndi->ram_addr = ram_addr;
-    ndi->mem_vaddr = mem_vaddr;
-    ndi->size = size;
-    ndi->pages = NULL;
-
-    trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size);
-
-    assert(tcg_enabled());
-    if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
-        ndi->pages = page_collection_lock(ram_addr, ram_addr + size);
-        tb_invalidate_phys_page_fast(ndi->pages, ram_addr, size);
-    }
-}
-
-/* Called within RCU critical section. */
-void memory_notdirty_write_complete(NotDirtyInfo *ndi)
-{
-    if (ndi->pages) {
-        assert(tcg_enabled());
-        page_collection_unlock(ndi->pages);
-        ndi->pages = NULL;
-    }
-
-    /* Set both VGA and migration bits for simplicity and to remove
-     * the notdirty callback faster.
-     */
-    cpu_physical_memory_set_dirty_range(ndi->ram_addr, ndi->size,
-                                        DIRTY_CLIENTS_NOCODE);
-    /* we remove the notdirty callback only if the code has been
-       flushed */
-    if (!cpu_physical_memory_is_clean(ndi->ram_addr)) {
-        trace_memory_notdirty_set_dirty(ndi->mem_vaddr);
-        tlb_set_dirty(ndi->cpu, ndi->mem_vaddr);
-    }
-}
-
 /* Generate a debug exception if a watchpoint has been hit.  */
 void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
                           MemTxAttrs attrs, int flags, uintptr_t ra)
-- 
2.17.1

We can use notdirty_write for the write and return a valid host
pointer for this case.

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ void *probe_access(CPUArchState *env, target_ulong addr, int size,
         return NULL;
     }
 
-    /* Handle watchpoints.  */
-    if (tlb_addr & TLB_WATCHPOINT) {
-        cpu_check_watchpoint(env_cpu(env), addr, size,
-                             env_tlb(env)->d[mmu_idx].iotlb[index].attrs,
-                             wp_access, retaddr);
-    }
+    if (unlikely(tlb_addr & TLB_FLAGS_MASK)) {
+        CPUIOTLBEntry *iotlbentry = &env_tlb(env)->d[mmu_idx].iotlb[index];
 
-    /* Reject I/O access, or other required slow-path.  */
-    if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO | TLB_BSWAP | TLB_DISCARD_WRITE)) {
-        return NULL;
+        /* Reject I/O access, or other required slow-path.  */
+        if (tlb_addr & (TLB_MMIO | TLB_BSWAP | TLB_DISCARD_WRITE)) {
+            return NULL;
+        }
+
+        /* Handle watchpoints.  */
+        if (tlb_addr & TLB_WATCHPOINT) {
+            cpu_check_watchpoint(env_cpu(env), addr, size,
+                                 iotlbentry->attrs, wp_access, retaddr);
+        }
+
+        /* Handle clean RAM pages.  */
+        if (tlb_addr & TLB_NOTDIRTY) {
+            notdirty_write(env_cpu(env), addr, size, iotlbentry, retaddr);
+        }
     }
 
     return (void *)((uintptr_t)addr + entry->addend);
-- 
2.17.1

With the merge of notdirty handling into store_helper,
the last user of cpu->mem_io_vaddr was removed.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/hw/core/cpu.h | 2 --
 accel/tcg/cputlb.c    | 2 --
 hw/core/cpu.c         | 1 -
 3 files changed, 5 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index XXXXXXX..XXXXXXX 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -XXX,XX +XXX,XX @@ struct qemu_work_item;
  * @next_cpu: Next CPU sharing TB cache.
  * @opaque: User data.
  * @mem_io_pc: Host Program Counter at which the memory was accessed.
- * @mem_io_vaddr: Target virtual address at which the memory was accessed.
  * @kvm_fd: vCPU file descriptor for KVM.
  * @work_mutex: Lock to prevent multiple access to queued_work_*.
  * @queued_work_first: First asynchronous work pending.
@@ -XXX,XX +XXX,XX @@ struct CPUState {
      * we store some rarely used information in the CPU context.
      */
     uintptr_t mem_io_pc;
-    vaddr mem_io_vaddr;
     /*
      * This is only needed for the legacy cpu_unassigned_access() hook;
      * when all targets using it have been converted to use
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
         cpu_io_recompile(cpu, retaddr);
     }
 
-    cpu->mem_io_vaddr = addr;
     cpu->mem_io_access_type = access_type;
 
     if (mr->global_locking && !qemu_mutex_iothread_locked()) {
@@ -XXX,XX +XXX,XX @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry,
     if (!cpu->can_do_io) {
         cpu_io_recompile(cpu, retaddr);
     }
-    cpu->mem_io_vaddr = addr;
     cpu->mem_io_pc = retaddr;
 
     if (mr->global_locking && !qemu_mutex_iothread_locked()) {
diff --git a/hw/core/cpu.c b/hw/core/cpu.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/core/cpu.c
+++ b/hw/core/cpu.c
@@ -XXX,XX +XXX,XX @@ static void cpu_common_reset(CPUState *cpu)
     cpu->interrupt_request = 0;
     cpu->halted = 0;
     cpu->mem_io_pc = 0;
-    cpu->mem_io_vaddr = 0;
     cpu->icount_extra = 0;
     atomic_set(&cpu->icount_decr_ptr->u32, 0);
     cpu->can_do_io = 1;
-- 
2.17.1

All callers pass false to this argument.  Remove it and pass the
constant on to tb_invalidate_phys_page_range__locked.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/translate-all.h | 3 +--
 accel/tcg/translate-all.c | 6 ++----
 exec.c                    | 4 ++--
 3 files changed, 5 insertions(+), 8 deletions(-)

Rather than rely on cpu->mem_io_pc, pass retaddr down directly.

Within tb_invalidate_phys_page_range__locked, the is_cpu_write_access
parameter is non-zero exactly when retaddr would be non-zero, so that
is a simple replacement.

Recognize that current_tb_not_found is true only when mem_io_pc
(and now retaddr) are also non-zero, so remove a redundant test.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/translate-all.h |  3 ++-
 accel/tcg/cputlb.c        |  6 +-----
 accel/tcg/translate-all.c | 39 +++++++++++++++++++--------------------
 3 files changed, 22 insertions(+), 26 deletions(-)

diff --git a/accel/tcg/translate-all.h b/accel/tcg/translate-all.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.h
+++ b/accel/tcg/translate-all.h
@@ -XXX,XX +XXX,XX @@ struct page_collection *page_collection_lock(tb_page_addr_t start,
                                              tb_page_addr_t end);
 void page_collection_unlock(struct page_collection *set);
 void tb_invalidate_phys_page_fast(struct page_collection *pages,
-                                  tb_page_addr_t start, int len);
+                                  tb_page_addr_t start, int len,
+                                  uintptr_t retaddr);
 void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end);
 void tb_check_watchpoint(CPUState *cpu);
 
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
     if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
         struct page_collection *pages
             = page_collection_lock(ram_addr, ram_addr + size);
-
-        /* We require mem_io_pc in tb_invalidate_phys_page_range.  */
-        cpu->mem_io_pc = retaddr;
-
-        tb_invalidate_phys_page_fast(pages, ram_addr, size);
+        tb_invalidate_phys_page_fast(pages, ram_addr, size, retaddr);
         page_collection_unlock(pages);
     }
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static void
 tb_invalidate_phys_page_range__locked(struct page_collection *pages,
                                       PageDesc *p, tb_page_addr_t start,
                                       tb_page_addr_t end,
-                                      int is_cpu_write_access)
+                                      uintptr_t retaddr)
 {
     TranslationBlock *tb;
     tb_page_addr_t tb_start, tb_end;
@@ -XXX,XX +XXX,XX @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
 #ifdef TARGET_HAS_PRECISE_SMC
     CPUState *cpu = current_cpu;
     CPUArchState *env = NULL;
-    int current_tb_not_found = is_cpu_write_access;
+    bool current_tb_not_found = retaddr != 0;
+    bool current_tb_modified = false;
     TranslationBlock *current_tb = NULL;
-    int current_tb_modified = 0;
     target_ulong current_pc = 0;
     target_ulong current_cs_base = 0;
     uint32_t current_flags = 0;
@@ -XXX,XX +XXX,XX @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
         if (!(tb_end <= start || tb_start >= end)) {
 #ifdef TARGET_HAS_PRECISE_SMC
             if (current_tb_not_found) {
-                current_tb_not_found = 0;
-                current_tb = NULL;
-                if (cpu->mem_io_pc) {
-                    /* now we have a real cpu fault */
-                    current_tb = tcg_tb_lookup(cpu->mem_io_pc);
-                }
+                current_tb_not_found = false;
+                /* now we have a real cpu fault */
+                current_tb = tcg_tb_lookup(retaddr);
             }
             if (current_tb == tb &&
                 (tb_cflags(current_tb) & CF_COUNT_MASK) != 1) {
-                /* If we are modifying the current TB, we must stop
-                its execution. We could be more precise by checking
-                that the modification is after the current PC, but it
-                would require a specialized function to partially
-                restore the CPU state */
-
-                current_tb_modified = 1;
-                cpu_restore_state_from_tb(cpu, current_tb,
-                                          cpu->mem_io_pc, true);
+                /*
+                 * If we are modifying the current TB, we must stop
+                 * its execution. We could be more precise by checking
+                 * that the modification is after the current PC, but it
+                 * would require a specialized function to partially
+                 * restore the CPU state.
+                 */
+                current_tb_modified = true;
+                cpu_restore_state_from_tb(cpu, current_tb, retaddr, true);
                 cpu_get_tb_cpu_state(env, &current_pc, &current_cs_base,
                                      &current_flags);
             }
@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_range(target_ulong start, target_ulong end)
  * Call with all @pages in the range [@start, @start + len[ locked.
  */
 void tb_invalidate_phys_page_fast(struct page_collection *pages,
-                                  tb_page_addr_t start, int len)
+                                  tb_page_addr_t start, int len,
+                                  uintptr_t retaddr)
 {
     PageDesc *p;
 
@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_page_fast(struct page_collection *pages,
         }
     } else {
     do_invalidate:
-        tb_invalidate_phys_page_range__locked(pages, p, start, start + len, 1);
+        tb_invalidate_phys_page_range__locked(pages, p, start, start + len,
+                                              retaddr);
     }
 }
 #else
-- 
2.17.1

Fixes the previous TLB_WATCHPOINT patches because we are currently
failing to set cpu->mem_io_pc with the call to cpu_check_watchpoint.
Pass down the retaddr directly because it's readily available.

Fixes: 50b107c5d61
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/translate-all.h | 2 +-
 accel/tcg/translate-all.c | 6 +++---
 exec.c                    | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/accel/tcg/translate-all.h b/accel/tcg/translate-all.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.h
+++ b/accel/tcg/translate-all.h
@@ -XXX,XX +XXX,XX @@ void tb_invalidate_phys_page_fast(struct page_collection *pages,
                                   tb_page_addr_t start, int len,
                                   uintptr_t retaddr);
 void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end);
-void tb_check_watchpoint(CPUState *cpu);
+void tb_check_watchpoint(CPUState *cpu, uintptr_t retaddr);
 
 #ifdef CONFIG_USER_ONLY
 int page_unprotect(target_ulong address, uintptr_t pc);
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ static bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc)
 #endif
 
 /* user-mode: call with mmap_lock held */
-void tb_check_watchpoint(CPUState *cpu)
+void tb_check_watchpoint(CPUState *cpu, uintptr_t retaddr)
 {
     TranslationBlock *tb;
 
     assert_memory_lock();
 
-    tb = tcg_tb_lookup(cpu->mem_io_pc);
+    tb = tcg_tb_lookup(retaddr);
     if (tb) {
         /* We can use retranslation to find the PC.  */
-        cpu_restore_state_from_tb(cpu, tb, cpu->mem_io_pc, true);
+        cpu_restore_state_from_tb(cpu, tb, retaddr, true);
         tb_phys_invalidate(tb, -1);
     } else {
         /* The exception probably happened in a helper.  The CPU state should
diff --git a/exec.c b/exec.c
index XXXXXXX..XXXXXXX 100644
--- a/exec.c
+++ b/exec.c
@@ -XXX,XX +XXX,XX @@ void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
                 cpu->watchpoint_hit = wp;
 
                 mmap_lock();
-                tb_check_watchpoint(cpu);
+                tb_check_watchpoint(cpu, ra);
                 if (wp->flags & BP_STOP_BEFORE_ACCESS) {
                     cpu->exception_index = EXCP_DEBUG;
                     mmap_unlock();
-- 
2.17.1

V2 fixes a build problem that affected win32.

The following changes since commit 187f35512106501fe9a11057f4d8705431e0026d:

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-next-251019-3' into staging (2019-10-26 10:13:48 +0100)

are available in the Git repository at:

https://github.com/rth7680/qemu.git tags/pull-tcg-20191028

for you to fetch changes up to fe9b676fb3160496b4b2bf0c57d33be724bf04c3:

translate-all: Remove tb_alloc (2019-10-28 10:35:23 +0100)

----------------------------------------------------------------
Improvements for TARGET_PAGE_BITS_VARY
Fix for TCI ld16u_i64.
Fix for segv on icount execute from i/o memory.
Two misc cleanups.

----------------------------------------------------------------
Alex Bennée (1):
      cputlb: ensure _cmmu helper functions follow the naming standard

Clement Deschamps (1):
      translate-all: fix uninitialized tb->orig_tb

Richard Henderson (8):
      exec: Split out variable page size support to exec-vary.c
      configure: Detect compiler support for __attribute__((alias))
      exec: Use const alias for TARGET_PAGE_BITS_VARY
      exec: Restrict TARGET_PAGE_BITS_VARY assert to CONFIG_DEBUG_TCG
      exec: Promote TARGET_PAGE_MASK to target_long
      exec: Cache TARGET_PAGE_MASK for TARGET_PAGE_BITS_VARY
      cputlb: Fix tlb_vaddr_to_host
      translate-all: Remove tb_alloc

Stefan Weil (1):
      tci: Add implementation for INDEX_op_ld16u_i64

Wei Yang (1):
      cpu: use ROUND_UP() to define xxx_PAGE_ALIGN

From: Stefan Weil <sw@weilnetz.de>

This fixes "make check-tcg" on a Debian x86_64 host.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190410194838.10123-1-sw@weilnetz.de>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/tci.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/tcg/tci.c b/tcg/tci.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tci.c
+++ b/tcg/tci.c
@@ -XXX,XX +XXX,XX @@ static void tci_write_reg8(tcg_target_ulong *regs, TCGReg index, uint8_t value)
     tci_write_reg(regs, index, value);
 }
 
+static void
+tci_write_reg16(tcg_target_ulong *regs, TCGReg index, uint16_t value)
+{
+    tci_write_reg(regs, index, value);
+}
+
 static void
 tci_write_reg32(tcg_target_ulong *regs, TCGReg index, uint32_t value)
 {
@@ -XXX,XX +XXX,XX @@ uintptr_t tcg_qemu_tb_exec(CPUArchState *env, uint8_t *tb_ptr)
             tci_write_reg8(regs, t0, *(uint8_t *)(t1 + t2));
             break;
         case INDEX_op_ld8s_i32:
+            TODO();
+            break;
         case INDEX_op_ld16u_i32:
             TODO();
             break;
@@ -XXX,XX +XXX,XX @@ uintptr_t tcg_qemu_tb_exec(CPUArchState *env, uint8_t *tb_ptr)
             tci_write_reg8(regs, t0, *(uint8_t *)(t1 + t2));
             break;
         case INDEX_op_ld8s_i64:
+            TODO();
+            break;
         case INDEX_op_ld16u_i64:
+            t0 = *tb_ptr++;
+            t1 = tci_read_r(regs, &tb_ptr);
+            t2 = tci_read_s32(&tb_ptr);
+            tci_write_reg16(regs, t0, *(uint16_t *)(t1 + t2));
+            break;
         case INDEX_op_ld16s_i64:
             TODO();
             break;
-- 
2.17.1

From: Alex Bennée <alex.bennee@linaro.org>

We document this in docs/devel/load-stores.rst so lets follow it. The
32 bit and 64 bit access functions have historically not included the
sign so we leave those as is. We also introduce some signed helpers
which are used for loading immediate values in the translator.

Fixes: 282dffc8
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20191021150910.23216-1-alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu_ldst_template.h |  4 ++--
 tcg/tcg.h                        | 20 ++++++++++++++------
 accel/tcg/cputlb.c               | 24 +++++++++++++++++++++---
 target/cris/translate_v10.inc.c  |  3 +--
 4 files changed, 38 insertions(+), 13 deletions(-)

diff --git a/include/exec/cpu_ldst_template.h b/include/exec/cpu_ldst_template.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu_ldst_template.h
+++ b/include/exec/cpu_ldst_template.h
@@ -XXX,XX +XXX,XX @@
 #ifdef SOFTMMU_CODE_ACCESS
 #define ADDR_READ addr_code
 #define MMUSUFFIX _cmmu
-#define URETSUFFIX SUFFIX
-#define SRETSUFFIX SUFFIX
+#define URETSUFFIX USUFFIX
+#define SRETSUFFIX glue(s, SUFFIX)
 #else
 #define ADDR_READ addr_read
 #define MMUSUFFIX _mmu
diff --git a/tcg/tcg.h b/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.h
+++ b/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ void helper_be_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val,
 void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val,
                        TCGMemOpIdx oi, uintptr_t retaddr);
 
-uint8_t helper_ret_ldb_cmmu(CPUArchState *env, target_ulong addr,
+uint8_t helper_ret_ldub_cmmu(CPUArchState *env, target_ulong addr,
                             TCGMemOpIdx oi, uintptr_t retaddr);
-uint16_t helper_le_ldw_cmmu(CPUArchState *env, target_ulong addr,
+int8_t helper_ret_ldsb_cmmu(CPUArchState *env, target_ulong addr,
                             TCGMemOpIdx oi, uintptr_t retaddr);
+uint16_t helper_le_lduw_cmmu(CPUArchState *env, target_ulong addr,
+                             TCGMemOpIdx oi, uintptr_t retaddr);
+int16_t helper_le_ldsw_cmmu(CPUArchState *env, target_ulong addr,
+                             TCGMemOpIdx oi, uintptr_t retaddr);
 uint32_t helper_le_ldl_cmmu(CPUArchState *env, target_ulong addr,
                             TCGMemOpIdx oi, uintptr_t retaddr);
 uint64_t helper_le_ldq_cmmu(CPUArchState *env, target_ulong addr,
                             TCGMemOpIdx oi, uintptr_t retaddr);
-uint16_t helper_be_ldw_cmmu(CPUArchState *env, target_ulong addr,
-                            TCGMemOpIdx oi, uintptr_t retaddr);
+uint16_t helper_be_lduw_cmmu(CPUArchState *env, target_ulong addr,
+                             TCGMemOpIdx oi, uintptr_t retaddr);
+int16_t helper_be_ldsw_cmmu(CPUArchState *env, target_ulong addr,
+                             TCGMemOpIdx oi, uintptr_t retaddr);
 uint32_t helper_be_ldl_cmmu(CPUArchState *env, target_ulong addr,
                             TCGMemOpIdx oi, uintptr_t retaddr);
 uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr,
@@ -XXX,XX +XXX,XX @@ uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr,
 # define helper_ret_stw_mmu   helper_be_stw_mmu
 # define helper_ret_stl_mmu   helper_be_stl_mmu
 # define helper_ret_stq_mmu   helper_be_stq_mmu
-# define helper_ret_ldw_cmmu  helper_be_ldw_cmmu
+# define helper_ret_lduw_cmmu  helper_be_lduw_cmmu
+# define helper_ret_ldsw_cmmu  helper_be_ldsw_cmmu
 # define helper_ret_ldl_cmmu  helper_be_ldl_cmmu
 # define helper_ret_ldq_cmmu  helper_be_ldq_cmmu
 #else
@@ -XXX,XX +XXX,XX @@ uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr,
 # define helper_ret_stw_mmu   helper_le_stw_mmu
 # define helper_ret_stl_mmu   helper_le_stl_mmu
 # define helper_ret_stq_mmu   helper_le_stq_mmu
-# define helper_ret_ldw_cmmu  helper_le_ldw_cmmu
+# define helper_ret_lduw_cmmu  helper_le_lduw_cmmu
+# define helper_ret_ldsw_cmmu  helper_le_ldsw_cmmu
 # define helper_ret_ldl_cmmu  helper_le_ldl_cmmu
 # define helper_ret_ldq_cmmu  helper_le_ldq_cmmu
 #endif
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static uint64_t full_ldub_cmmu(CPUArchState *env, target_ulong addr,
     return load_helper(env, addr, oi, retaddr, MO_8, true, full_ldub_cmmu);
 }
 
-uint8_t helper_ret_ldb_cmmu(CPUArchState *env, target_ulong addr,
+uint8_t helper_ret_ldub_cmmu(CPUArchState *env, target_ulong addr,
                             TCGMemOpIdx oi, uintptr_t retaddr)
 {
     return full_ldub_cmmu(env, addr, oi, retaddr);
 }
 
+int8_t helper_ret_ldsb_cmmu(CPUArchState *env, target_ulong addr,
+                            TCGMemOpIdx oi, uintptr_t retaddr)
+{
+    return (int8_t) full_ldub_cmmu(env, addr, oi, retaddr);
+}
+
 static uint64_t full_le_lduw_cmmu(CPUArchState *env, target_ulong addr,
                                   TCGMemOpIdx oi, uintptr_t retaddr)
 {
@@ -XXX,XX +XXX,XX @@ static uint64_t full_le_lduw_cmmu(CPUArchState *env, target_ulong addr,
                        full_le_lduw_cmmu);
 }
 
-uint16_t helper_le_ldw_cmmu(CPUArchState *env, target_ulong addr,
+uint16_t helper_le_lduw_cmmu(CPUArchState *env, target_ulong addr,
                             TCGMemOpIdx oi, uintptr_t retaddr)
 {
     return full_le_lduw_cmmu(env, addr, oi, retaddr);
 }
 
+int16_t helper_le_ldsw_cmmu(CPUArchState *env, target_ulong addr,
+                            TCGMemOpIdx oi, uintptr_t retaddr)
+{
+    return (int16_t) full_le_lduw_cmmu(env, addr, oi, retaddr);
+}
+
 static uint64_t full_be_lduw_cmmu(CPUArchState *env, target_ulong addr,
                                   TCGMemOpIdx oi, uintptr_t retaddr)
 {
@@ -XXX,XX +XXX,XX @@ static uint64_t full_be_lduw_cmmu(CPUArchState *env, target_ulong addr,
                        full_be_lduw_cmmu);
 }
 
-uint16_t helper_be_ldw_cmmu(CPUArchState *env, target_ulong addr,
+uint16_t helper_be_lduw_cmmu(CPUArchState *env, target_ulong addr,
                             TCGMemOpIdx oi, uintptr_t retaddr)
 {
     return full_be_lduw_cmmu(env, addr, oi, retaddr);
 }
 
+int16_t helper_be_ldsw_cmmu(CPUArchState *env, target_ulong addr,
+                            TCGMemOpIdx oi, uintptr_t retaddr)
+{
+    return (int16_t) full_be_lduw_cmmu(env, addr, oi, retaddr);
+}
+
 static uint64_t full_le_ldul_cmmu(CPUArchState *env, target_ulong addr,
                                   TCGMemOpIdx oi, uintptr_t retaddr)
 {
diff --git a/target/cris/translate_v10.inc.c b/target/cris/translate_v10.inc.c
index XXXXXXX..XXXXXXX 100644
--- a/target/cris/translate_v10.inc.c
+++ b/target/cris/translate_v10.inc.c
@@ -XXX,XX +XXX,XX @@ static unsigned int dec10_ind(CPUCRISState *env, DisasContext *dc)
         case CRISV10_IND_BCC_M:
 
             cris_cc_mask(dc, 0);
-            imm = cpu_ldsw_code(env, dc->pc + 2);
-            simm = (int16_t)imm;
+            simm = cpu_ldsw_code(env, dc->pc + 2);
             simm += 4;
 
             LOG_DIS("bcc_m: b%s %x\n", cc_name(dc->cond), dc->pc + simm);
-- 
2.17.1

From: Wei Yang <richardw.yang@linux.intel.com>

Use ROUND_UP() to define, which is a little bit easy to read.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Message-Id: <20191013021145.16011-2-richardw.yang@linux.intel.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-all.h | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -XXX,XX +XXX,XX @@ extern int target_page_bits;
 
 #define TARGET_PAGE_SIZE (1 << TARGET_PAGE_BITS)
 #define TARGET_PAGE_MASK ~(TARGET_PAGE_SIZE - 1)
-#define TARGET_PAGE_ALIGN(addr) (((addr) + TARGET_PAGE_SIZE - 1) & TARGET_PAGE_MASK)
+#define TARGET_PAGE_ALIGN(addr) ROUND_UP((addr), TARGET_PAGE_SIZE)
 
 /* Using intptr_t ensures that qemu_*_page_mask is sign-extended even
  * when intptr_t is 32-bit and we are aligning a long long.
@@ -XXX,XX +XXX,XX @@ extern int target_page_bits;
 extern uintptr_t qemu_host_page_size;
 extern intptr_t qemu_host_page_mask;
 
-#define HOST_PAGE_ALIGN(addr) (((addr) + qemu_host_page_size - 1) & qemu_host_page_mask)
-#define REAL_HOST_PAGE_ALIGN(addr) (((addr) + qemu_real_host_page_size - 1) & \
-                                    qemu_real_host_page_mask)
+#define HOST_PAGE_ALIGN(addr) ROUND_UP((addr), qemu_host_page_size)
+#define REAL_HOST_PAGE_ALIGN(addr) ROUND_UP((addr), qemu_real_host_page_size)
 
 /* same as PROT_xxx */
 #define PAGE_READ      0x0001
-- 
2.17.1

The next patch will play a trick with "const" that will
confuse the compiler about the uses of target_page_bits
within exec.c.  Moving everything to a new file prevents
this confusion.

No functional change so far.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 Makefile.target       |  2 +-
 include/qemu-common.h |  6 +++++
 exec-vary.c           | 57 +++++++++++++++++++++++++++++++++++++++++++
 exec.c                | 34 --------------------------
 4 files changed, 64 insertions(+), 35 deletions(-)
 create mode 100644 exec-vary.c

diff --git a/Makefile.target b/Makefile.target
index XXXXXXX..XXXXXXX 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -XXX,XX +XXX,XX @@ obj-y += trace/
 
 #########################################################
 # cpu emulator library
-obj-y += exec.o
+obj-y += exec.o exec-vary.o
 obj-y += accel/
 obj-$(CONFIG_TCG) += tcg/tcg.o tcg/tcg-op.o tcg/tcg-op-vec.o tcg/tcg-op-gvec.o
 obj-$(CONFIG_TCG) += tcg/tcg-common.o tcg/optimize.o
diff --git a/include/qemu-common.h b/include/qemu-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu-common.h
+++ b/include/qemu-common.h
@@ -XXX,XX +XXX,XX @@ void cpu_exec_step_atomic(CPUState *cpu);
  */
 bool set_preferred_target_page_bits(int bits);
 
+/**
+ * finalize_target_page_bits:
+ * Commit the final value set by set_preferred_target_page_bits.
+ */
+void finalize_target_page_bits(void);
+
 /**
  * Sends a (part of) iovec down a socket, yielding when the socket is full, or
  * Receives data into a (part of) iovec from a socket,
diff --git a/exec-vary.c b/exec-vary.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/exec-vary.c
@@ -XXX,XX +XXX,XX @@
+/*
+ * Variable page size handling
+ *
+ *  Copyright (c) 2003 Fabrice Bellard
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "exec/exec-all.h"
+
+#ifdef TARGET_PAGE_BITS_VARY
+int target_page_bits;
+bool target_page_bits_decided;
+#endif
+
+bool set_preferred_target_page_bits(int bits)
+{
+    /*
+     * The target page size is the lowest common denominator for all
+     * the CPUs in the system, so we can only make it smaller, never
+     * larger. And we can't make it smaller once we've committed to
+     * a particular size.
+     */
+#ifdef TARGET_PAGE_BITS_VARY
+    assert(bits >= TARGET_PAGE_BITS_MIN);
+    if (target_page_bits == 0 || target_page_bits > bits) {
+        if (target_page_bits_decided) {
+            return false;
+        }
+        target_page_bits = bits;
+    }
+#endif
+    return true;
+}
+
+void finalize_target_page_bits(void)
+{
+#ifdef TARGET_PAGE_BITS_VARY
+    if (target_page_bits == 0) {
+        target_page_bits = TARGET_PAGE_BITS_MIN;
+    }
+    target_page_bits_decided = true;
+#endif
+}
diff --git a/exec.c b/exec.c
index XXXXXXX..XXXXXXX 100644
--- a/exec.c
+++ b/exec.c
@@ -XXX,XX +XXX,XX @@ AddressSpace address_space_memory;
 static MemoryRegion io_mem_unassigned;
 #endif
 
-#ifdef TARGET_PAGE_BITS_VARY
-int target_page_bits;
-bool target_page_bits_decided;
-#endif
-
 CPUTailQ cpus = QTAILQ_HEAD_INITIALIZER(cpus);
 
 /* current CPU in the current thread. It is only valid inside
@@ -XXX,XX +XXX,XX @@ int use_icount;
 uintptr_t qemu_host_page_size;
 intptr_t qemu_host_page_mask;
 
-bool set_preferred_target_page_bits(int bits)
-{
-    /* The target page size is the lowest common denominator for all
-     * the CPUs in the system, so we can only make it smaller, never
-     * larger. And we can't make it smaller once we've committed to
-     * a particular size.
-     */
-#ifdef TARGET_PAGE_BITS_VARY
-    assert(bits >= TARGET_PAGE_BITS_MIN);
-    if (target_page_bits == 0 || target_page_bits > bits) {
-        if (target_page_bits_decided) {
-            return false;
-        }
-        target_page_bits = bits;
-    }
-#endif
-    return true;
-}
-
 #if !defined(CONFIG_USER_ONLY)
 
-static void finalize_target_page_bits(void)
-{
-#ifdef TARGET_PAGE_BITS_VARY
-    if (target_page_bits == 0) {
-        target_page_bits = TARGET_PAGE_BITS_MIN;
-    }
-    target_page_bits_decided = true;
-#endif
-}
-
 typedef struct PhysPageEntry PhysPageEntry;
 
 struct PhysPageEntry {
-- 
2.17.1

Such support is present almost everywhere, except for Xcode 9.
It is added in Xcode 10, but travis uses xcode9 by default,
so we should support it for a while yet.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 configure | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/configure b/configure
index XXXXXXX..XXXXXXX 100755
--- a/configure
+++ b/configure
@@ -XXX,XX +XXX,XX @@ if compile_prog "" "" ; then
   vector16=yes
 fi
 
+########################################
+# See if __attribute__((alias)) is supported.
+# This false for Xcode 9, but has been remedied for Xcode 10.
+# Unfortunately, travis uses Xcode 9 by default.
+
+attralias=no
+cat > $TMPC << EOF
+int x = 1;
+extern const int y __attribute__((alias("x")));
+int main(void) { return 0; }
+EOF
+if compile_prog "" "" ; then
+    attralias=yes
+fi
+
 ########################################
 # check if getauxval is available.
 
@@ -XXX,XX +XXX,XX @@ if test "$vector16" = "yes" ; then
   echo "CONFIG_VECTOR16=y" >> $config_host_mak
 fi
 
+if test "$attralias" = "yes" ; then
+  echo "CONFIG_ATTRIBUTE_ALIAS=y" >> $config_host_mak
+fi
+
 if test "$getauxval" = "yes" ; then
   echo "CONFIG_GETAUXVAL=y" >> $config_host_mak
 fi
-- 
2.17.1

Using a variable that is declared "const" for this tells the
compiler that it may read the value once and assume that it
does not change across function calls.

For target_page_size, this means we have only one assert per
function, and one read of the variable.

This reduces the size of qemu-system-aarch64 by 8k.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-all.h | 14 ++++++---
 exec-vary.c            | 66 +++++++++++++++++++++++++++++++++++++-----
 2 files changed, 68 insertions(+), 12 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -XXX,XX +XXX,XX @@ static inline void stl_phys_notdirty(AddressSpace *as, hwaddr addr, uint32_t val
 /* page related stuff */
 
 #ifdef TARGET_PAGE_BITS_VARY
-extern bool target_page_bits_decided;
-extern int target_page_bits;
-#define TARGET_PAGE_BITS ({ assert(target_page_bits_decided); \
-                            target_page_bits; })
+typedef struct {
+    bool decided;
+    int bits;
+} TargetPageBits;
+#if defined(CONFIG_ATTRIBUTE_ALIAS) || !defined(IN_EXEC_VARY)
+extern const TargetPageBits target_page;
+#else
+extern TargetPageBits target_page;
+#endif
+#define TARGET_PAGE_BITS   ({ assert(target_page.decided); target_page.bits; })
 #else
 #define TARGET_PAGE_BITS_MIN TARGET_PAGE_BITS
 #endif
diff --git a/exec-vary.c b/exec-vary.c
index XXXXXXX..XXXXXXX 100644
--- a/exec-vary.c
+++ b/exec-vary.c
@@ -XXX,XX +XXX,XX @@
 
 #include "qemu/osdep.h"
 #include "qemu-common.h"
+
+#define IN_EXEC_VARY 1
+
 #include "exec/exec-all.h"
 
 #ifdef TARGET_PAGE_BITS_VARY
-int target_page_bits;
-bool target_page_bits_decided;
+# ifdef CONFIG_ATTRIBUTE_ALIAS
+/*
+ * We want to declare the "target_page" variable as const, which tells
+ * the compiler that it can cache any value that it reads across calls.
+ * This avoids multiple assertions and multiple reads within any one user.
+ *
+ * This works because we finish initializing the data before we ever read
+ * from the "target_page" symbol.
+ *
+ * This also requires that we have a non-constant symbol by which we can
+ * perform the actual initialization, and which forces the data to be
+ * allocated within writable memory.  Thus "init_target_page", and we use
+ * that symbol exclusively in the two functions that initialize this value.
+ *
+ * The "target_page" symbol is created as an alias of "init_target_page".
+ */
+static TargetPageBits init_target_page;
+
+/*
+ * Note that this is *not* a redundant decl, this is the definition of
+ * the "target_page" symbol.  The syntax for this definition requires
+ * the use of the extern keyword.  This seems to be a GCC bug in
+ * either the syntax for the alias attribute or in -Wredundant-decls.
+ *
+ * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91765
+ */
+#  pragma GCC diagnostic push
+#  pragma GCC diagnostic ignored "-Wredundant-decls"
+
+extern const TargetPageBits target_page
+    __attribute__((alias("init_target_page")));
+
+#  pragma GCC diagnostic pop
+# else
+/*
+ * When aliases are not supported then we force two different declarations,
+ * by way of suppressing the header declaration with IN_EXEC_VARY.
+ * We assume that on such an old compiler, LTO cannot be used, and so the
+ * compiler cannot not detect the mismatched declarations, and all is well.
+ */
+TargetPageBits target_page;
+#  define init_target_page target_page
+# endif
 #endif
 
 bool set_preferred_target_page_bits(int bits)
@@ -XXX,XX +XXX,XX @@ bool set_preferred_target_page_bits(int bits)
      */
 #ifdef TARGET_PAGE_BITS_VARY
     assert(bits >= TARGET_PAGE_BITS_MIN);
-    if (target_page_bits == 0 || target_page_bits > bits) {
-        if (target_page_bits_decided) {
+    if (init_target_page.bits == 0 || init_target_page.bits > bits) {
+        if (init_target_page.decided) {
             return false;
         }
-        target_page_bits = bits;
+        init_target_page.bits = bits;
     }
 #endif
     return true;
@@ -XXX,XX +XXX,XX @@ bool set_preferred_target_page_bits(int bits)
 void finalize_target_page_bits(void)
 {
 #ifdef TARGET_PAGE_BITS_VARY
-    if (target_page_bits == 0) {
-        target_page_bits = TARGET_PAGE_BITS_MIN;
+    if (init_target_page.bits == 0) {
+        init_target_page.bits = TARGET_PAGE_BITS_MIN;
     }
-    target_page_bits_decided = true;
+    init_target_page.decided = true;
+
+    /*
+     * For the benefit of an -flto build, prevent the compiler from
+     * hoisting a read from target_page before we finish initializing.
+     */
+    barrier();
 #endif
 }
-- 
2.17.1

There are some uint64_t uses that expect TARGET_PAGE_MASK to
extend for a 32-bit, so this must continue to be a signed type.
Define based on TARGET_PAGE_BITS not TARGET_PAGE_SIZE; this
will make a following patch more clear.

This should not have a functional effect so far.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-all.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -XXX,XX +XXX,XX @@ extern TargetPageBits target_page;
 #endif
 
 #define TARGET_PAGE_SIZE (1 << TARGET_PAGE_BITS)
-#define TARGET_PAGE_MASK ~(TARGET_PAGE_SIZE - 1)
+#define TARGET_PAGE_MASK ((target_long)-1 << TARGET_PAGE_BITS)
 #define TARGET_PAGE_ALIGN(addr) ROUND_UP((addr), TARGET_PAGE_SIZE)
 
 /* Using intptr_t ensures that qemu_*_page_mask is sign-extended even
-- 
2.17.1

This eliminates a set of runtime shifts.  It turns out that we
require TARGET_PAGE_MASK more often than TARGET_PAGE_SIZE, so
redefine TARGET_PAGE_SIZE based on TARGET_PAGE_MASK instead of
the other way around.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/cpu-all.h | 8 ++++++--
 exec-vary.c            | 1 +
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -XXX,XX +XXX,XX @@ static inline void stl_phys_notdirty(AddressSpace *as, hwaddr addr, uint32_t val
 typedef struct {
     bool decided;
     int bits;
+    target_long mask;
 } TargetPageBits;
 #if defined(CONFIG_ATTRIBUTE_ALIAS) || !defined(IN_EXEC_VARY)
 extern const TargetPageBits target_page;
@@ -XXX,XX +XXX,XX @@ extern TargetPageBits target_page;
 #endif
 #ifdef CONFIG_DEBUG_TCG
 #define TARGET_PAGE_BITS   ({ assert(target_page.decided); target_page.bits; })
+#define TARGET_PAGE_MASK   ({ assert(target_page.decided); target_page.mask; })
 #else
 #define TARGET_PAGE_BITS   target_page.bits
+#define TARGET_PAGE_MASK   target_page.mask
 #endif
+#define TARGET_PAGE_SIZE   (-(int)TARGET_PAGE_MASK)
 #else
 #define TARGET_PAGE_BITS_MIN TARGET_PAGE_BITS
+#define TARGET_PAGE_SIZE   (1 << TARGET_PAGE_BITS)
+#define TARGET_PAGE_MASK   ((target_long)-1 << TARGET_PAGE_BITS)
 #endif
 
-#define TARGET_PAGE_SIZE (1 << TARGET_PAGE_BITS)
-#define TARGET_PAGE_MASK ((target_long)-1 << TARGET_PAGE_BITS)
 #define TARGET_PAGE_ALIGN(addr) ROUND_UP((addr), TARGET_PAGE_SIZE)
 
 /* Using intptr_t ensures that qemu_*_page_mask is sign-extended even
diff --git a/exec-vary.c b/exec-vary.c
index XXXXXXX..XXXXXXX 100644
--- a/exec-vary.c
+++ b/exec-vary.c
@@ -XXX,XX +XXX,XX @@ void finalize_target_page_bits(void)
     if (init_target_page.bits == 0) {
         init_target_page.bits = TARGET_PAGE_BITS_MIN;
     }
+    init_target_page.mask = (target_long)-1 << init_target_page.bits;
     init_target_page.decided = true;
 
     /*
-- 
2.17.1

From: Clement Deschamps <clement.deschamps@greensocs.com>

This fixes a segmentation fault in icount mode when executing
from an IO region.

TB is marked as CF_NOCACHE but tb->orig_tb is not initialized
(equals previous value in code_gen_buffer).

The issue happens in cpu_io_recompile() when it tries to invalidate orig_tb.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Clement Deschamps <clement.deschamps@greensocs.com>
Message-Id: <20191022140016.918371-1-clement.deschamps@greensocs.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/translate-all.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
     tb->cs_base = cs_base;
     tb->flags = flags;
     tb->cflags = cflags;
+    tb->orig_tb = NULL;
     tb->trace_vcpu_dstate = *cpu->trace_dstate;
     tcg_ctx->tb_cflags = cflags;
  tb_overflow:
-- 
2.17.1

Since 2ac01d6dafab, this function does only two things: assert a
lock is held, and call tcg_tb_alloc.  It is used exactly once,
and its user has already done the assert.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Clement Deschamps <clement.deschamps@greensocs.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/translate-all.c | 20 ++------------------
 1 file changed, 2 insertions(+), 18 deletions(-)

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -XXX,XX +XXX,XX @@ void tcg_exec_init(unsigned long tb_size)
 #endif
 }
 
-/*
- * Allocate a new translation block. Flush the translation buffer if
- * too many translation blocks or too much generated code.
- */
-static TranslationBlock *tb_alloc(target_ulong pc)
-{
-    TranslationBlock *tb;
-
-    assert_memory_lock();
-
-    tb = tcg_tb_alloc(tcg_ctx);
-    if (unlikely(tb == NULL)) {
-        return NULL;
-    }
-    return tb;
-}
-
 /* call with @p->lock held */
 static inline void invalidate_page_bitmap(PageDesc *p)
 {
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
     TCGProfile *prof = &tcg_ctx->prof;
     int64_t ti;
 #endif
+
     assert_memory_lock();
 
     phys_pc = get_page_addr_code(env, pc);
@@ -XXX,XX +XXX,XX @@ TranslationBlock *tb_gen_code(CPUState *cpu,
     }
 
  buffer_overflow:
-    tb = tb_alloc(pc);
+    tb = tcg_tb_alloc(tcg_ctx);
     if (unlikely(!tb)) {
         /* flush must be done */
         tb_flush(cpu);
-- 
2.17.1