Series comparison

-[PULL 00/12] tcg patch queue
+[PULL for-8.1-rc1 0/7] tcg patch queue
-I have not been able to prod reviews of all of the rotate patches
+The following changes since commit d1181d29370a4318a9f11ea92065bea6bb159f83:
 in 4 weeks, but let's not let that block ARM work forever.
+  Merge tag 'pull-nbd-2023-07-19' of https://repo.or.cz/qemu/ericb into staging (2023-07-20 09:54:07 +0100)
 r~
 The following changes since commit cccdd8c7971896c339d59c9c5d4647d4ffd9568a:
   Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging (2020-06-02 10:25:55 +0100)
 are available in the Git repository at:
-  https://github.com/rth7680/qemu.git tags/pull-tcg-20200602
+  https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20230724
-for you to fetch changes up to 71b04329c4f7d5824a289ca5225e1883a278cf3b:
+for you to fetch changes up to 32b120394c578bc824f1db4835b3bffbeca88fae:
-  accel/tcg: Provide a NetBSD specific aarch64 cpu_signal_handler (2020-06-02 08:42:37 -0700)
+  accel/tcg: Fix type of 'last' for pageflags_{find,next} (2023-07-24 09:48:49 +0100)
 ----------------------------------------------------------------
-Vector rotate support
+accel/tcg: Zero-pad vaddr in tlb debug output
-Signal handling support for NetBSD arm/aarch64
+accel/tcg: Fix type of 'last' for pageflags_{find,next}
 accel/tcg: Fix sense of read-only probes in ldst_atomicity
 accel/tcg: Take mmap_lock in load_atomic*_or_exit
 tcg: Add earlyclobber to op_add2 for x86 and s390x
 tcg/ppc: Fix race in goto_tb implementation
 ----------------------------------------------------------------
-Nick Hudson (2):
+Anton Johansson (1):
-      accel/tcg: Adjust cpu_signal_handler for NetBSD/arm
+      accel/tcg: Zero-pad vaddr in tlb_debug output
       accel/tcg: Provide a NetBSD specific aarch64 cpu_signal_handler
-Richard Henderson (10):
+Ilya Leoshkevich (1):
-      tcg: Implement gvec support for rotate by immediate
+      tcg/{i386, s390x}: Add earlyclobber to the op_add2's first output
       tcg: Implement gvec support for rotate by vector
       tcg: Remove expansion to shift by vector from do_shifts
       tcg: Implement gvec support for rotate by scalar
       tcg/i386: Implement INDEX_op_rotl{i,s,v}_vec
       tcg/aarch64: Implement INDEX_op_rotl{i,v}_vec
       tcg/ppc: Implement INDEX_op_rot[lr]v_vec
       target/ppc: Use tcg_gen_gvec_rotlv
       target/s390x: Use tcg_gen_gvec_rotl{i,s,v}
       tcg: Improve move ops in liveness_pass_2
- accel/tcg/tcg-runtime.h             |  15 +++
+Jordan Niethe (1):
- include/tcg/tcg-op-gvec.h           |  12 ++
+      tcg/ppc: Fix race in goto_tb implementation
  include/tcg/tcg-op.h                |   5 +
  include/tcg/tcg-opc.h               |   4 +
  include/tcg/tcg.h                   |   3 +
  target/ppc/helper.h                 |   4 -
  target/s390x/helper.h               |   4 -
  tcg/aarch64/tcg-target.h            |   3 +
  tcg/aarch64/tcg-target.opc.h        |   1 +
  tcg/i386/tcg-target.h               |   3 +
  tcg/ppc/tcg-target.h                |   3 +
  tcg/ppc/tcg-target.opc.h            |   1 -
  accel/tcg/tcg-runtime-gvec.c        | 144 ++++++++++++++++++++++++
  accel/tcg/user-exec.c               |  43 +++++++-
  target/ppc/int_helper.c             |  17 ---
  target/ppc/translate/vmx-impl.inc.c |   8 +-
  target/s390x/translate_vx.inc.c     |  66 ++---------
  target/s390x/vec_int_helper.c       |  31 ------
  tcg/aarch64/tcg-target.inc.c        |  53 ++++++++-
  tcg/i386/tcg-target.inc.c           | 116 +++++++++++++++++---
  tcg/ppc/tcg-target.inc.c            |  23 +++-
  tcg/tcg-op-gvec.c                   | 212 ++++++++++++++++++++++++++++++++++++
  tcg/tcg-op-vec.c                    |  62 +++++++----
  tcg/tcg.c                           |  85 +++++++++++----
  target/s390x/insn-data.def          |   4 +-
  tcg/README                          |   7 +-
 files changed, 736 insertions(+), 193 deletions(-)
+Luca Bonissi (1):
+      accel/tcg: Fix type of 'last' for pageflags_{find,next}
+Richard Henderson (3):
+      include/exec: Add WITH_MMAP_LOCK_GUARD
+      accel/tcg: Fix sense of read-only probes in ldst_atomicity
+      accel/tcg: Take mmap_lock in load_atomic*_or_exit
+ include/exec/exec-all.h        | 10 ++++++++++
+ tcg/i386/tcg-target-con-set.h  |  5 ++++-
+ tcg/s390x/tcg-target-con-set.h |  8 +++++---
+ accel/tcg/cputlb.c             | 20 ++++++++++----------
+ accel/tcg/user-exec.c          |  4 ++--
+ bsd-user/mmap.c                |  1 +
+ linux-user/mmap.c              |  1 +
+ tcg/tcg.c                      |  8 +++++++-
+ accel/tcg/ldst_atomicity.c.inc | 32 ++++++++++++++++++--------------
+ tcg/i386/tcg-target.c.inc      |  2 +-
+ tcg/ppc/tcg-target.c.inc       |  9 +++++----
+ tcg/s390x/tcg-target.c.inc     |  4 ++--
+files changed, 66 insertions(+), 38 deletions(-)

-[PULL 05/12] tcg/i386: Implement INDEX_op_rotl{i,s,v}_vec
+[PULL 1/7] tcg/ppc: Fix race in goto_tb implementation
-For immediates, we must continue the special casing of 8-bit
+From: Jordan Niethe <jniethe5@gmail.com>
 elements.  The other element sizes and shift types are trivially
 implemented with shifts.
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
+Commit 20b6643324 ("tcg/ppc: Reorg goto_tb implementation") modified
 goto_tb to ensure only a single instruction was patched to prevent
 incorrect behavior if a thread was in the middle of multiple
 instructions when they were replaced. However this introduced a race
 between loading the jmp target into TCG_REG_TB and patching and
 executing the direct branch.
 The relevant part of the goto_tb implementation:
     ld TCG_REG_TB, TARGET_ADDR_LOCATION(TCG_REG_TB)
   patch_location:
     mtctr TCG_REG_TB
     bctr
 tb_target_set_jmp_target() will replace 'patch_location' with a direct
 branch if the target is in range. The direct branch now relies on
 TCG_REG_TB being set up correctly by the ld. Prior to this commit
 multiple instructions were patched in for the direct branch case; these
 instructions would initialize TCG_REG_TB to the same value as the branch
 target.
 Imagine the following sequence:
 ) Thread A is executing the goto_tb sequence and loads the jmp
    target into TCG_REG_TB.
 ) Thread B updates the jmp target address and calls
    tb_target_set_jmp_target(). This patches a new direct branch into the
    goto_tb sequence.
 ) Thread A executes the newly patched direct branch. The value in
    TCG_REG_TB still contains the old jmp target.
 TCG_REG_TB MUST contain the translation block's tc.ptr. Execution will
 eventually crash after performing memory accesses generated from a
 faulty value in TCG_REG_TB.
 This presents as segfaults or illegal instruction exceptions.
 Do not revert commit 20b6643324 as it did fix a different race
 condition. Instead remove the direct branch optimization and always use
 indirect branches.
 The direct branch optimization can be re-added later with a race free
 sequence.
 Fixes: 20b6643324 ("tcg/ppc: Reorg goto_tb implementation")
 Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1726
 Reported-by: Anushree Mathur <anushree.mathur@linux.vnet.ibm.com>
 Tested-by: Anushree Mathur <anushree.mathur@linux.vnet.ibm.com>
 Tested-by: Michael Tokarev <mjt@tls.msk.ru>
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Co-developed-by: Benjamin Gray <bgray@linux.ibm.com>
 Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
 Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
 Message-Id: <20230717093001.13167-1-jniethe5@gmail.com>
 ---
- tcg/i386/tcg-target.inc.c | 116 ++++++++++++++++++++++++++++++++------
+ tcg/ppc/tcg-target.c.inc | 9 +++++----
-file changed, 100 insertions(+), 16 deletions(-)
+file changed, 5 insertions(+), 4 deletions(-)
-diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c
+diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc
 index XXXXXXX..XXXXXXX 100644
---- a/tcg/i386/tcg-target.inc.c
+--- a/tcg/ppc/tcg-target.c.inc
-+++ b/tcg/i386/tcg-target.inc.c
++++ b/tcg/ppc/tcg-target.c.inc
-@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op)
+@@ -XXX,XX +XXX,XX @@ static void tcg_out_goto_tb(TCGContext *s, int which)
-     case INDEX_op_shls_vec:
+         ptrdiff_t offset = tcg_tbrel_diff(s, (void *)ptr);
-     case INDEX_op_shrs_vec:
+         tcg_out_mem_long(s, LD, LDX, TCG_REG_TB, TCG_REG_TB, offset);
-     case INDEX_op_sars_vec:
-+    case INDEX_op_rotls_vec:
+-        /* Direct branch will be patched by tb_target_set_jmp_target. */
-     case INDEX_op_cmp_vec:
++        /* TODO: Use direct branches when possible. */
-     case INDEX_op_x86_shufps_vec:
+         set_jmp_insn_offset(s, which);
-     case INDEX_op_x86_blend_vec:
+         tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR);
-@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
-     case INDEX_op_xor_vec:
+-        /* When branch is out of range, fall through to indirect. */
-     case INDEX_op_andc_vec:
+         tcg_out32(s, BCCTR | BO_ALWAYS);
-         return 1;
-+    case INDEX_op_rotli_vec:
+         /* For the unlinked case, need to reset TCG_REG_TB.  */
-     case INDEX_op_cmp_vec:
+@@ -XXX,XX +XXX,XX @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n,
-     case INDEX_op_cmpsel_vec:
+     intptr_t diff = addr - jmp_rx;
-         return -1;
+     tcg_insn_unit insn;
-@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
-         return vece >= MO_16;
++    if (USE_REG_TB) {
      case INDEX_op_sars_vec:
          return vece >= MO_16 && vece <= MO_32;
 +    case INDEX_op_rotls_vec:
 +        return vece >= MO_16 ? -1 : 0;
      case INDEX_op_shlv_vec:
      case INDEX_op_shrv_vec:
          return have_avx2 && vece >= MO_32;
      case INDEX_op_sarv_vec:
          return have_avx2 && vece == MO_32;
 +    case INDEX_op_rotlv_vec:
 +    case INDEX_op_rotrv_vec:
 +        return have_avx2 && vece >= MO_32 ? -1 : 0;
      case INDEX_op_mul_vec:
          if (vece == MO_8) {
@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
      }
  }
 -static void expand_vec_shi(TCGType type, unsigned vece, bool shr,
 +static void expand_vec_shi(TCGType type, unsigned vece, TCGOpcode opc,
                             TCGv_vec v0, TCGv_vec v1, TCGArg imm)
  {
      TCGv_vec t1, t2;
@@ -XXX,XX +XXX,XX @@ static void expand_vec_shi(TCGType type, unsigned vece, bool shr,
      t1 = tcg_temp_new_vec(type);
      t2 = tcg_temp_new_vec(type);
 -    /* Unpack to W, shift, and repack.  Tricky bits:
 -       (1) Use punpck*bw x,x to produce DDCCBBAA,
 -           i.e. duplicate in other half of the 16-bit lane.
 -       (2) For right-shift, add 8 so that the high half of
 -           the lane becomes zero.  For left-shift, we must
 -           shift up and down again.
 -       (3) Step 2 leaves high half zero such that PACKUSWB
 -           (pack with unsigned saturation) does not modify
 -           the quantity.  */
 +    /*
 +     * Unpack to W, shift, and repack.  Tricky bits:
 +     * (1) Use punpck*bw x,x to produce DDCCBBAA,
 +     *     i.e. duplicate in other half of the 16-bit lane.
 +     * (2) For right-shift, add 8 so that the high half of the lane
 +     *     becomes zero.  For left-shift, and left-rotate, we must
 +     *     shift up and down again.
 +     * (3) Step 2 leaves high half zero such that PACKUSWB
 +     *     (pack with unsigned saturation) does not modify
 +     *     the quantity.
 +     */
      vec_gen_3(INDEX_op_x86_punpckl_vec, type, MO_8,
                tcgv_vec_arg(t1), tcgv_vec_arg(v1), tcgv_vec_arg(v1));
      vec_gen_3(INDEX_op_x86_punpckh_vec, type, MO_8,
                tcgv_vec_arg(t2), tcgv_vec_arg(v1), tcgv_vec_arg(v1));
 -    if (shr) {
 -        tcg_gen_shri_vec(MO_16, t1, t1, imm + 8);
 -        tcg_gen_shri_vec(MO_16, t2, t2, imm + 8);
 +    if (opc != INDEX_op_rotli_vec) {
 +        imm += 8;
 +    }
 +    if (opc == INDEX_op_shri_vec) {
 +        tcg_gen_shri_vec(MO_16, t1, t1, imm);
 +        tcg_gen_shri_vec(MO_16, t2, t2, imm);
      } else {
 -        tcg_gen_shli_vec(MO_16, t1, t1, imm + 8);
 -        tcg_gen_shli_vec(MO_16, t2, t2, imm + 8);
 +        tcg_gen_shli_vec(MO_16, t1, t1, imm);
 +        tcg_gen_shli_vec(MO_16, t2, t2, imm);
          tcg_gen_shri_vec(MO_16, t1, t1, 8);
          tcg_gen_shri_vec(MO_16, t2, t2, 8);
      }
@@ -XXX,XX +XXX,XX @@ static void expand_vec_sari(TCGType type, unsigned vece,
      }
  }
 +static void expand_vec_rotli(TCGType type, unsigned vece,
 +                             TCGv_vec v0, TCGv_vec v1, TCGArg imm)
 +{
 +    TCGv_vec t;
 +
 +    if (vece == MO_8) {
 +        expand_vec_shi(type, vece, INDEX_op_rotli_vec, v0, v1, imm);
 +        return;
 +    }
 +
-+    t = tcg_temp_new_vec(type);
+     if (in_range_b(diff)) {
-+    tcg_gen_shli_vec(vece, t, v1, imm);
+         insn = B | (diff & 0x3fffffc);
-+    tcg_gen_shri_vec(vece, v0, v1, (8 << vece) - imm);
+-    } else if (USE_REG_TB) {
-+    tcg_gen_or_vec(vece, v0, v0, t);
+-        insn = MTSPR | RS(TCG_REG_TB) | CTR;
-+    tcg_temp_free_vec(t);
+     } else {
-+}
+         insn = NOP;
-+
+     }
 +static void expand_vec_rotls(TCGType type, unsigned vece,
 +                             TCGv_vec v0, TCGv_vec v1, TCGv_i32 lsh)
 +{
 +    TCGv_i32 rsh;
 +    TCGv_vec t;
 +
 +    tcg_debug_assert(vece != MO_8);
 +
 +    t = tcg_temp_new_vec(type);
 +    rsh = tcg_temp_new_i32();
 +
 +    tcg_gen_neg_i32(rsh, lsh);
 +    tcg_gen_andi_i32(rsh, rsh, (8 << vece) - 1);
 +    tcg_gen_shls_vec(vece, t, v1, lsh);
 +    tcg_gen_shrs_vec(vece, v0, v1, rsh);
 +    tcg_gen_or_vec(vece, v0, v0, t);
 +    tcg_temp_free_vec(t);
 +    tcg_temp_free_i32(rsh);
 +}
 +
 +static void expand_vec_rotv(TCGType type, unsigned vece, TCGv_vec v0,
 +                            TCGv_vec v1, TCGv_vec sh, bool right)
 +{
 +    TCGv_vec t = tcg_temp_new_vec(type);
 +
 +    tcg_gen_dupi_vec(vece, t, 8 << vece);
 +    tcg_gen_sub_vec(vece, t, t, sh);
 +    if (right) {
 +        tcg_gen_shlv_vec(vece, t, v1, t);
 +        tcg_gen_shrv_vec(vece, v0, v1, sh);
 +    } else {
 +        tcg_gen_shrv_vec(vece, t, v1, t);
 +        tcg_gen_shlv_vec(vece, v0, v1, sh);
 +    }
 +    tcg_gen_or_vec(vece, v0, v0, t);
 +    tcg_temp_free_vec(t);
 +}
 +
  static void expand_vec_mul(TCGType type, unsigned vece,
                             TCGv_vec v0, TCGv_vec v1, TCGv_vec v2)
  {
@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
      switch (opc) {
      case INDEX_op_shli_vec:
      case INDEX_op_shri_vec:
 -        expand_vec_shi(type, vece, opc == INDEX_op_shri_vec, v0, v1, a2);
 +        expand_vec_shi(type, vece, opc, v0, v1, a2);
          break;
      case INDEX_op_sari_vec:
          expand_vec_sari(type, vece, v0, v1, a2);
          break;
 +    case INDEX_op_rotli_vec:
 +        expand_vec_rotli(type, vece, v0, v1, a2);
 +        break;
 +
 +    case INDEX_op_rotls_vec:
 +        expand_vec_rotls(type, vece, v0, v1, temp_tcgv_i32(arg_temp(a2)));
 +        break;
 +
 +    case INDEX_op_rotlv_vec:
 +        v2 = temp_tcgv_vec(arg_temp(a2));
 +        expand_vec_rotv(type, vece, v0, v1, v2, false);
 +        break;
 +    case INDEX_op_rotrv_vec:
 +        v2 = temp_tcgv_vec(arg_temp(a2));
 +        expand_vec_rotv(type, vece, v0, v1, v2, true);
 +        break;
 +
      case INDEX_op_mul_vec:
          v2 = temp_tcgv_vec(arg_temp(a2));
          expand_vec_mul(type, vece, v0, v1, v2);
 --
-.25.1
+.34.1

-[PULL 10/12] tcg: Improve move ops in liveness_pass_2
+[PULL 2/7] include/exec: Add WITH_MMAP_LOCK_GUARD
-If the output of the move is dead, then the last use is in
+Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
 the store.  If we propagate the input to the store, then we
 can remove the move opcode entirely.
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- tcg/tcg.c | 78 +++++++++++++++++++++++++++++++++++++++----------------
+ include/exec/exec-all.h | 10 ++++++++++
-file changed, 56 insertions(+), 22 deletions(-)
+ bsd-user/mmap.c         |  1 +
  linux-user/mmap.c       |  1 +
 files changed, 12 insertions(+)
-diff --git a/tcg/tcg.c b/tcg/tcg.c
+diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
 index XXXXXXX..XXXXXXX 100644
---- a/tcg/tcg.c
+--- a/include/exec/exec-all.h
-+++ b/tcg/tcg.c
++++ b/include/exec/exec-all.h
-@@ -XXX,XX +XXX,XX @@ static bool liveness_pass_2(TCGContext *s)
+@@ -XXX,XX +XXX,XX @@ void TSA_NO_TSA mmap_lock(void);
-         }
+ void TSA_NO_TSA mmap_unlock(void);
+ bool have_mmap_lock(void);
-         /* Outputs become available.  */
--        for (i = 0; i < nb_oargs; i++) {
++static inline void mmap_unlock_guard(void *unused)
--            arg_ts = arg_temp(op->args[i]);
++{
-+        if (opc == INDEX_op_mov_i32 || opc == INDEX_op_mov_i64) {
++    mmap_unlock();
-+            arg_ts = arg_temp(op->args[0]);
++}
              dir_ts = arg_ts->state_ptr;
 -            if (!dir_ts) {
 -                continue;
 +            if (dir_ts) {
 +                op->args[0] = temp_arg(dir_ts);
 +                changes = true;
 +
-+                /* The output is now live and modified.  */
++#define WITH_MMAP_LOCK_GUARD()                                            \
-+                arg_ts->state = 0;
++    for (int _mmap_lock_iter __attribute__((cleanup(mmap_unlock_guard)))  \
 +         = (mmap_lock(), 0); _mmap_lock_iter == 0; _mmap_lock_iter = 1)
 +
-+                if (NEED_SYNC_ARG(0)) {
+ /**
-+                    TCGOpcode sopc = (arg_ts->type == TCG_TYPE_I32
+  * adjust_signal_pc:
-+                                      ? INDEX_op_st_i32
+  * @pc: raw pc from the host signal ucontext_t.
-+                                      : INDEX_op_st_i64);
+@@ -XXX,XX +XXX,XX @@ G_NORETURN void cpu_loop_exit_sigbus(CPUState *cpu, target_ulong addr,
-+                    TCGOp *sop = tcg_op_insert_after(s, op, sopc);
+ #else
-+                    TCGTemp *out_ts = dir_ts;
+ static inline void mmap_lock(void) {}
-+
+ static inline void mmap_unlock(void) {}
-+                    if (IS_DEAD_ARG(0)) {
++#define WITH_MMAP_LOCK_GUARD()
-+                        out_ts = arg_temp(op->args[1]);
-+                        arg_ts->state = TS_DEAD;
+ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length);
-+                        tcg_op_remove(s, op);
+ void tlb_set_dirty(CPUState *cpu, vaddr addr);
-+                    } else {
+diff --git a/bsd-user/mmap.c b/bsd-user/mmap.c
-+                        arg_ts->state = TS_MEM;
+index XXXXXXX..XXXXXXX 100644
-+                    }
+--- a/bsd-user/mmap.c
-+
++++ b/bsd-user/mmap.c
-+                    sop->args[0] = temp_arg(out_ts);
+@@ -XXX,XX +XXX,XX @@ void mmap_lock(void)
-+                    sop->args[1] = temp_arg(arg_ts->mem_base);
-+                    sop->args[2] = arg_ts->mem_offset;
+ void mmap_unlock(void)
-+                } else {
+ {
-+                    tcg_debug_assert(!IS_DEAD_ARG(0));
++    assert(mmap_lock_count > 0);
-+                }
+     if (--mmap_lock_count == 0) {
-             }
+         pthread_mutex_unlock(&mmap_mutex);
--            op->args[i] = temp_arg(dir_ts);
+     }
--            changes = true;
+diff --git a/linux-user/mmap.c b/linux-user/mmap.c
-+        } else {
+index XXXXXXX..XXXXXXX 100644
-+            for (i = 0; i < nb_oargs; i++) {
+--- a/linux-user/mmap.c
-+                arg_ts = arg_temp(op->args[i]);
++++ b/linux-user/mmap.c
-+                dir_ts = arg_ts->state_ptr;
+@@ -XXX,XX +XXX,XX @@ void mmap_lock(void)
-+                if (!dir_ts) {
-+                    continue;
+ void mmap_unlock(void)
-+                }
+ {
-+                op->args[i] = temp_arg(dir_ts);
++    assert(mmap_lock_count > 0);
-+                changes = true;
+     if (--mmap_lock_count == 0) {
+         pthread_mutex_unlock(&mmap_mutex);
 -            /* The output is now live and modified.  */
 -            arg_ts->state = 0;
 +                /* The output is now live and modified.  */
 +                arg_ts->state = 0;
 -            /* Sync outputs upon their last write.  */
 -            if (NEED_SYNC_ARG(i)) {
 -                TCGOpcode sopc = (arg_ts->type == TCG_TYPE_I32
 -                                  ? INDEX_op_st_i32
 -                                  : INDEX_op_st_i64);
 -                TCGOp *sop = tcg_op_insert_after(s, op, sopc);
 +                /* Sync outputs upon their last write.  */
 +                if (NEED_SYNC_ARG(i)) {
 +                    TCGOpcode sopc = (arg_ts->type == TCG_TYPE_I32
 +                                      ? INDEX_op_st_i32
 +                                      : INDEX_op_st_i64);
 +                    TCGOp *sop = tcg_op_insert_after(s, op, sopc);
 -                sop->args[0] = temp_arg(dir_ts);
 -                sop->args[1] = temp_arg(arg_ts->mem_base);
 -                sop->args[2] = arg_ts->mem_offset;
 +                    sop->args[0] = temp_arg(dir_ts);
 +                    sop->args[1] = temp_arg(arg_ts->mem_base);
 +                    sop->args[2] = arg_ts->mem_offset;
 -                arg_ts->state = TS_MEM;
 -            }
 -            /* Drop outputs that are dead.  */
 -            if (IS_DEAD_ARG(i)) {
 -                arg_ts->state = TS_DEAD;
 +                    arg_ts->state = TS_MEM;
 +                }
 +                /* Drop outputs that are dead.  */
 +                if (IS_DEAD_ARG(i)) {
 +                    arg_ts->state = TS_DEAD;
 +                }
              }
          }
      }
 --
-.25.1
+.34.1

-[PULL 09/12] target/s390x: Use tcg_gen_gvec_rotl{i,s,v}
+[PULL 3/7] accel/tcg: Fix sense of read-only probes in ldst_atomicity
-Merge VERLL and VERLLV into op_vesv and op_ves, alongside
+In the initial commit, cdfac37be0d, the sense of the test is incorrect,
-all of the other vector shift operations.
+as the -1/0 return was confusing.  In bef6f008b981, we mechanically
 invert all callers while changing to false/true return, preserving the
 incorrectness of the test.
-Reviewed-by: David Hildenbrand <david@redhat.com>
+Now that the return sense is sane, it's easy to see that if !write,
 then the page is not modifiable (i.e. most likely read-only, with
 PROT_NONE handled via SIGSEGV).
 Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- target/s390x/helper.h           |  4 --
+ accel/tcg/ldst_atomicity.c.inc | 4 ++--
- target/s390x/translate_vx.inc.c | 66 +++++----------------------------
+file changed, 2 insertions(+), 2 deletions(-)
  target/s390x/vec_int_helper.c   | 31 ----------------
  target/s390x/insn-data.def      |  4 +-
 files changed, 11 insertions(+), 94 deletions(-)
-diff --git a/target/s390x/helper.h b/target/s390x/helper.h
+diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc
 index XXXXXXX..XXXXXXX 100644
---- a/target/s390x/helper.h
+--- a/accel/tcg/ldst_atomicity.c.inc
-+++ b/target/s390x/helper.h
++++ b/accel/tcg/ldst_atomicity.c.inc
-@@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vmlo16, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
+@@ -XXX,XX +XXX,XX @@ static uint64_t load_atomic8_or_exit(CPUArchState *env, uintptr_t ra, void *pv)
- DEF_HELPER_FLAGS_4(gvec_vmlo32, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
+      * another process, because the fallback start_exclusive solution
- DEF_HELPER_FLAGS_3(gvec_vpopct8, TCG_CALL_NO_RWG, void, ptr, cptr, i32)
+      * provides no protection across processes.
- DEF_HELPER_FLAGS_3(gvec_vpopct16, TCG_CALL_NO_RWG, void, ptr, cptr, i32)
+      */
--DEF_HELPER_FLAGS_4(gvec_verllv8, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
+-    if (page_check_range(h2g(pv), 8, PAGE_WRITE_ORG)) {
--DEF_HELPER_FLAGS_4(gvec_verllv16, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
++    if (!page_check_range(h2g(pv), 8, PAGE_WRITE_ORG)) {
--DEF_HELPER_FLAGS_4(gvec_verll8, TCG_CALL_NO_RWG, void, ptr, cptr, i64, i32)
+         uint64_t *p = __builtin_assume_aligned(pv, 8);
--DEF_HELPER_FLAGS_4(gvec_verll16, TCG_CALL_NO_RWG, void, ptr, cptr, i64, i32)
+         return *p;
- DEF_HELPER_FLAGS_4(gvec_verim8, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
+     }
- DEF_HELPER_FLAGS_4(gvec_verim16, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
+@@ -XXX,XX +XXX,XX @@ static Int128 load_atomic16_or_exit(CPUArchState *env, uintptr_t ra, void *pv)
- DEF_HELPER_FLAGS_4(gvec_vsl, TCG_CALL_NO_RWG, void, ptr, cptr, i64, i32)
+      * another process, because the fallback start_exclusive solution
-diff --git a/target/s390x/translate_vx.inc.c b/target/s390x/translate_vx.inc.c
+      * provides no protection across processes.
-index XXXXXXX..XXXXXXX 100644
+      */
---- a/target/s390x/translate_vx.inc.c
+-    if (page_check_range(h2g(p), 16, PAGE_WRITE_ORG)) {
-+++ b/target/s390x/translate_vx.inc.c
++    if (!page_check_range(h2g(p), 16, PAGE_WRITE_ORG)) {
-@@ -XXX,XX +XXX,XX @@ static DisasJumpType op_vpopct(DisasContext *s, DisasOps *o)
+         return *p;
-     return DISAS_NEXT;
+     }
- }
+ #endif
 -static void gen_rll_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
 -{
 -    TCGv_i32 t0 = tcg_temp_new_i32();
 -
 -    tcg_gen_andi_i32(t0, b, 31);
 -    tcg_gen_rotl_i32(d, a, t0);
 -    tcg_temp_free_i32(t0);
 -}
 -
 -static void gen_rll_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
 -{
 -    TCGv_i64 t0 = tcg_temp_new_i64();
 -
 -    tcg_gen_andi_i64(t0, b, 63);
 -    tcg_gen_rotl_i64(d, a, t0);
 -    tcg_temp_free_i64(t0);
 -}
 -
 -static DisasJumpType op_verllv(DisasContext *s, DisasOps *o)
 -{
 -    const uint8_t es = get_field(s, m4);
 -    static const GVecGen3 g[4] = {
 -        { .fno = gen_helper_gvec_verllv8, },
 -        { .fno = gen_helper_gvec_verllv16, },
 -        { .fni4 = gen_rll_i32, },
 -        { .fni8 = gen_rll_i64, },
 -    };
 -
 -    if (es > ES_64) {
 -        gen_program_exception(s, PGM_SPECIFICATION);
 -        return DISAS_NORETURN;
 -    }
 -
 -    gen_gvec_3(get_field(s, v1), get_field(s, v2),
 -               get_field(s, v3), &g[es]);
 -    return DISAS_NEXT;
 -}
 -
 -static DisasJumpType op_verll(DisasContext *s, DisasOps *o)
 -{
 -    const uint8_t es = get_field(s, m4);
 -    static const GVecGen2s g[4] = {
 -        { .fno = gen_helper_gvec_verll8, },
 -        { .fno = gen_helper_gvec_verll16, },
 -        { .fni4 = gen_rll_i32, },
 -        { .fni8 = gen_rll_i64, },
 -    };
 -
 -    if (es > ES_64) {
 -        gen_program_exception(s, PGM_SPECIFICATION);
 -        return DISAS_NORETURN;
 -    }
 -    gen_gvec_2s(get_field(s, v1), get_field(s, v3), o->addr1,
 -                &g[es]);
 -    return DISAS_NEXT;
 -}
 -
  static void gen_rim_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b, int32_t c)
  {
      TCGv_i32 t = tcg_temp_new_i32();
@@ -XXX,XX +XXX,XX @@ static DisasJumpType op_vesv(DisasContext *s, DisasOps *o)
      case 0x70:
          gen_gvec_fn_3(shlv, es, v1, v2, v3);
          break;
 +    case 0x73:
 +        gen_gvec_fn_3(rotlv, es, v1, v2, v3);
 +        break;
      case 0x7a:
          gen_gvec_fn_3(sarv, es, v1, v2, v3);
          break;
@@ -XXX,XX +XXX,XX @@ static DisasJumpType op_ves(DisasContext *s, DisasOps *o)
          case 0x30:
              gen_gvec_fn_2i(shli, es, v1, v3, d2);
              break;
 +        case 0x33:
 +            gen_gvec_fn_2i(rotli, es, v1, v3, d2);
 +            break;
          case 0x3a:
              gen_gvec_fn_2i(sari, es, v1, v3, d2);
              break;
@@ -XXX,XX +XXX,XX @@ static DisasJumpType op_ves(DisasContext *s, DisasOps *o)
          case 0x30:
              gen_gvec_fn_2s(shls, es, v1, v3, shift);
              break;
 +        case 0x33:
 +            gen_gvec_fn_2s(rotls, es, v1, v3, shift);
 +            break;
          case 0x3a:
              gen_gvec_fn_2s(sars, es, v1, v3, shift);
              break;
 diff --git a/target/s390x/vec_int_helper.c b/target/s390x/vec_int_helper.c
 index XXXXXXX..XXXXXXX 100644
 --- a/target/s390x/vec_int_helper.c
 +++ b/target/s390x/vec_int_helper.c
@@ -XXX,XX +XXX,XX @@ void HELPER(gvec_vpopct##BITS)(void *v1, const void *v2, uint32_t desc)        \
  DEF_VPOPCT(8)
  DEF_VPOPCT(16)
 -#define DEF_VERLLV(BITS)                                                       \
 -void HELPER(gvec_verllv##BITS)(void *v1, const void *v2, const void *v3,       \
 -                               uint32_t desc)                                  \
 -{                                                                              \
 -    int i;                                                                     \
 -                                                                               \
 -    for (i = 0; i < (128 / BITS); i++) {                                       \
 -        const uint##BITS##_t a = s390_vec_read_element##BITS(v2, i);           \
 -        const uint##BITS##_t b = s390_vec_read_element##BITS(v3, i);           \
 -                                                                               \
 -        s390_vec_write_element##BITS(v1, i, rol##BITS(a, b));                  \
 -    }                                                                          \
 -}
 -DEF_VERLLV(8)
 -DEF_VERLLV(16)
 -
 -#define DEF_VERLL(BITS)                                                        \
 -void HELPER(gvec_verll##BITS)(void *v1, const void *v2, uint64_t count,        \
 -                              uint32_t desc)                                   \
 -{                                                                              \
 -    int i;                                                                     \
 -                                                                               \
 -    for (i = 0; i < (128 / BITS); i++) {                                       \
 -        const uint##BITS##_t a = s390_vec_read_element##BITS(v2, i);           \
 -                                                                               \
 -        s390_vec_write_element##BITS(v1, i, rol##BITS(a, count));              \
 -    }                                                                          \
 -}
 -DEF_VERLL(8)
 -DEF_VERLL(16)
 -
  #define DEF_VERIM(BITS)                                                        \
  void HELPER(gvec_verim##BITS)(void *v1, const void *v2, const void *v3,        \
                                uint32_t desc)                                   \
 diff --git a/target/s390x/insn-data.def b/target/s390x/insn-data.def
 index XXXXXXX..XXXXXXX 100644
 --- a/target/s390x/insn-data.def
 +++ b/target/s390x/insn-data.def
@@ -XXX,XX +XXX,XX @@
  /* VECTOR POPULATION COUNT */
      F(0xe750, VPOPCT,  VRR_a, V,   0, 0, 0, 0, vpopct, 0, IF_VEC)
  /* VECTOR ELEMENT ROTATE LEFT LOGICAL */
 -    F(0xe773, VERLLV,  VRR_c, V,   0, 0, 0, 0, verllv, 0, IF_VEC)
 -    F(0xe733, VERLL,   VRS_a, V,   la2, 0, 0, 0, verll, 0, IF_VEC)
 +    F(0xe773, VERLLV,  VRR_c, V,   0, 0, 0, 0, vesv, 0, IF_VEC)
 +    F(0xe733, VERLL,   VRS_a, V,   la2, 0, 0, 0, ves, 0, IF_VEC)
  /* VECTOR ELEMENT ROTATE AND INSERT UNDER MASK */
      F(0xe772, VERIM,   VRI_d, V,   0, 0, 0, 0, verim, 0, IF_VEC)
  /* VECTOR ELEMENT SHIFT LEFT */
 --
-.25.1
+.34.1

-[PULL 11/12] accel/tcg: Adjust cpu_signal_handler for NetBSD/arm
+[PULL 4/7] accel/tcg: Take mmap_lock in load_atomic*_or_exit
-From: Nick Hudson <skrll@netbsd.org>
+For user-only, the probe for page writability may race with another
 thread's mprotect.  Take the mmap_lock around the operation.  This
 is still faster than the start/end_exclusive fallback.
-Fix building on NetBSD/arm by extracting the FSR value from the
+Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
 correct siginfo_t field.
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Signed-off-by: Nick Hudson <skrll@netbsd.org>
 Message-Id: <20200516154147.24842-1-skrll@netbsd.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/user-exec.c | 16 +++++++++++++---
+ accel/tcg/ldst_atomicity.c.inc | 32 ++++++++++++++++++--------------
-file changed, 13 insertions(+), 3 deletions(-)
+file changed, 18 insertions(+), 14 deletions(-)
-diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
+diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/user-exec.c
+--- a/accel/tcg/ldst_atomicity.c.inc
-+++ b/accel/tcg/user-exec.c
++++ b/accel/tcg/ldst_atomicity.c.inc
-@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo,
+@@ -XXX,XX +XXX,XX @@ static uint64_t load_atomic8_or_exit(CPUArchState *env, uintptr_t ra, void *pv)
+      * another process, because the fallback start_exclusive solution
- #if defined(__NetBSD__)
+      * provides no protection across processes.
- #include <ucontext.h>
+      */
-+#include <sys/siginfo.h>
+-    if (!page_check_range(h2g(pv), 8, PAGE_WRITE_ORG)) {
 -        uint64_t *p = __builtin_assume_aligned(pv, 8);
 -        return *p;
 +    WITH_MMAP_LOCK_GUARD() {
 +        if (!page_check_range(h2g(pv), 8, PAGE_WRITE_ORG)) {
 +            uint64_t *p = __builtin_assume_aligned(pv, 8);
 +            return *p;
 +        }
      }
  #endif
- int cpu_signal_handler(int host_signum, void *pinfo,
+@@ -XXX,XX +XXX,XX @@ static Int128 load_atomic16_or_exit(CPUArchState *env, uintptr_t ra, void *pv)
-@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo,
+         return atomic16_read_ro(p);
-     siginfo_t *info = pinfo;
+     }
- #if defined(__NetBSD__)
-     ucontext_t *uc = puc;
+-#ifdef CONFIG_USER_ONLY
-+    siginfo_t *si = pinfo;
+     /*
- #else
+      * We can only use cmpxchg to emulate a load if the page is writable.
-     ucontext_t *uc = puc;
+      * If the page is not writable, then assume the value is immutable
       * and requires no locking.  This ignores the case of MAP_SHARED with
       * another process, because the fallback start_exclusive solution
       * provides no protection across processes.
 +     *
 +     * In system mode all guest pages are writable.  For user mode,
 +     * we must take mmap_lock so that the query remains valid until
 +     * the write is complete -- tests/tcg/multiarch/munmap-pthread.c
 +     * is an example that can race.
       */
 -    if (!page_check_range(h2g(p), 16, PAGE_WRITE_ORG)) {
 -        return *p;
 -    }
 +    WITH_MMAP_LOCK_GUARD() {
 +#ifdef CONFIG_USER_ONLY
 +        if (!page_check_range(h2g(p), 16, PAGE_WRITE_ORG)) {
 +            return *p;
 +        }
  #endif
-     unsigned long pc;
+-
-+    uint32_t fsr;
+-    /*
-     int is_write;
+-     * In system mode all guest pages are writable, and for user-only
+-     * we have just checked writability.  Try cmpxchg.
- #if defined(__NetBSD__)
+-     */
-@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo,
+-    if (HAVE_ATOMIC128_RW) {
-     pc = uc->uc_mcontext.arm_pc;
+-        return atomic16_read_rw(p);
- #endif
++        if (HAVE_ATOMIC128_RW) {
++            return atomic16_read_rw(p);
--    /* error_code is the FSR value, in which bit 11 is WnR (assuming a v6 or
++        }
--     * later processor; on v5 we will always report this as a read).
+     }
-+#ifdef __NetBSD__
-+    fsr = si->si_trap;
+     /* Ultimate fallback: re-execute in serial context. */
 +#else
 +    fsr = uc->uc_mcontext.error_code;
 +#endif
 +    /*
 +     * In the FSR, bit 11 is WnR, assuming a v6 or
 +     * later processor.  On v5 we will always report
 +     * this as a read, which will fail later.
       */
 -    is_write = extract32(uc->uc_mcontext.error_code, 11, 1);
 +    is_write = extract32(fsr, 11, 1);
      return handle_cpu_signal(pc, info, is_write, &uc->uc_sigmask);
  }
 --
-.25.1
+.34.1

-[PULL 02/12] tcg: Implement gvec support for rotate by vector
+[PULL 5/7] tcg/{i386, s390x}: Add earlyclobber to the op_add2's first output
-No host backend support yet, but the interfaces for rotlv
+From: Ilya Leoshkevich <iii@linux.ibm.com>
 and rotrv are in place.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+i386 and s390x implementations of op_add2 require an earlyclobber,
 which is currently missing. This breaks VCKSM in s390x guests. E.g., on
 x86_64 the following op:
     add2_i32 tmp2,tmp3,tmp2,tmp3,tmp3,tmp2   dead: 0 2 3 4 5  pref=none,0xffff
 is translated to:
     addl     %ebx, %r12d
     adcl     %r12d, %ebx
 Introduce a new C_N1_O1_I4 constraint, and make sure that earlyclobber
 of aliased outputs is honored.
 Cc: qemu-stable@nongnu.org
 Fixes: 82790a870992 ("tcg: Add markup for output requires new register")
 Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Message-Id: <20230719221310.1968845-7-iii@linux.ibm.com>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
-v3: Drop the generic expansion from rot to shift; we can do better
+ tcg/i386/tcg-target-con-set.h  | 5 ++++-
-    for each backend, and then this code becomes unused.
+ tcg/s390x/tcg-target-con-set.h | 8 +++++---
----
+ tcg/tcg.c                      | 8 +++++++-
- accel/tcg/tcg-runtime.h      |  10 +++
+ tcg/i386/tcg-target.c.inc      | 2 +-
- include/tcg/tcg-op-gvec.h    |   4 ++
+ tcg/s390x/tcg-target.c.inc     | 4 ++--
- include/tcg/tcg-op.h         |   2 +
+files changed, 19 insertions(+), 8 deletions(-)
  include/tcg/tcg-opc.h        |   2 +
  include/tcg/tcg.h            |   1 +
  tcg/aarch64/tcg-target.h     |   1 +
  tcg/i386/tcg-target.h        |   1 +
  tcg/ppc/tcg-target.h         |   1 +
  accel/tcg/tcg-runtime-gvec.c |  96 +++++++++++++++++++++++++++
  tcg/tcg-op-gvec.c            | 122 +++++++++++++++++++++++++++++++++++
  tcg/tcg-op-vec.c             |  10 +++
  tcg/tcg.c                    |   3 +
  tcg/README                   |   4 +-
 files changed, 256 insertions(+), 1 deletion(-)
-diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h
+diff --git a/tcg/i386/tcg-target-con-set.h b/tcg/i386/tcg-target-con-set.h
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/tcg-runtime.h
+--- a/tcg/i386/tcg-target-con-set.h
-+++ b/accel/tcg/tcg-runtime.h
++++ b/tcg/i386/tcg-target-con-set.h
-@@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_sar16v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+@@ -XXX,XX +XXX,XX @@
- DEF_HELPER_FLAGS_4(gvec_sar32v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+  *
- DEF_HELPER_FLAGS_4(gvec_sar64v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+  * C_N1_Im(...) defines a constraint set with 1 output and <m> inputs,
+  * except that the output must use a new register.
-+DEF_HELPER_FLAGS_4(gvec_rotl8v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
++ *
-+DEF_HELPER_FLAGS_4(gvec_rotl16v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
++ * C_Nn_Om_Ik(...) defines a constraint set with <n + m> outputs and <k>
-+DEF_HELPER_FLAGS_4(gvec_rotl32v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
++ * inputs, except that the first <n> outputs must use new registers.
-+DEF_HELPER_FLAGS_4(gvec_rotl64v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+  */
-+
+ C_O0_I1(r)
-+DEF_HELPER_FLAGS_4(gvec_rotr8v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+ C_O0_I2(L, L)
-+DEF_HELPER_FLAGS_4(gvec_rotr16v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+@@ -XXX,XX +XXX,XX @@ C_O2_I1(r, r, L)
-+DEF_HELPER_FLAGS_4(gvec_rotr32v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+ C_O2_I2(a, d, a, r)
-+DEF_HELPER_FLAGS_4(gvec_rotr64v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+ C_O2_I2(r, r, L, L)
-+
+ C_O2_I3(a, d, 0, 1, r)
- DEF_HELPER_FLAGS_4(gvec_eq8, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+-C_O2_I4(r, r, 0, 1, re, re)
- DEF_HELPER_FLAGS_4(gvec_eq16, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
++C_N1_O1_I4(r, r, 0, 1, re, re)
- DEF_HELPER_FLAGS_4(gvec_eq32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+diff --git a/tcg/s390x/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h
 diff --git a/include/tcg/tcg-op-gvec.h b/include/tcg/tcg-op-gvec.h
 index XXXXXXX..XXXXXXX 100644
---- a/include/tcg/tcg-op-gvec.h
+--- a/tcg/s390x/tcg-target-con-set.h
-+++ b/include/tcg/tcg-op-gvec.h
++++ b/tcg/s390x/tcg-target-con-set.h
-@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_shrv(unsigned vece, uint32_t dofs, uint32_t aofs,
+@@ -XXX,XX +XXX,XX @@
-                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz);
+  * C_On_Im(...) defines a constraint set with <n> outputs and <m> inputs.
- void tcg_gen_gvec_sarv(unsigned vece, uint32_t dofs, uint32_t aofs,
+  * Each operand should be a sequence of constraint letters as defined by
-                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz);
+  * tcg-target-con-str.h; the constraint combination is inclusive or.
-+void tcg_gen_gvec_rotlv(unsigned vece, uint32_t dofs, uint32_t aofs,
++ *
-+                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz);
++ * C_Nn_Om_Ik(...) defines a constraint set with <n + m> outputs and <k>
-+void tcg_gen_gvec_rotrv(unsigned vece, uint32_t dofs, uint32_t aofs,
++ * inputs, except that the first <n> outputs must use new registers.
-+                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz);
+  */
+ C_O0_I1(r)
- void tcg_gen_gvec_cmp(TCGCond cond, unsigned vece, uint32_t dofs,
+ C_O0_I2(r, r)
-                       uint32_t aofs, uint32_t bofs,
+@@ -XXX,XX +XXX,XX @@ C_O2_I1(o, m, r)
-diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h
+ C_O2_I2(o, m, 0, r)
-index XXXXXXX..XXXXXXX 100644
+ C_O2_I2(o, m, r, r)
---- a/include/tcg/tcg-op.h
+ C_O2_I3(o, m, 0, 1, r)
-+++ b/include/tcg/tcg-op.h
+-C_O2_I4(r, r, 0, 1, rA, r)
-@@ -XXX,XX +XXX,XX @@ void tcg_gen_sars_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
+-C_O2_I4(r, r, 0, 1, ri, r)
- void tcg_gen_shlv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
+-C_O2_I4(r, r, 0, 1, r, r)
- void tcg_gen_shrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
++C_N1_O1_I4(r, r, 0, 1, ri, r)
- void tcg_gen_sarv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
++C_N1_O1_I4(r, r, 0, 1, rA, r)
 +void tcg_gen_rotlv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
 +void tcg_gen_rotrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
  void tcg_gen_cmp_vec(TCGCond cond, unsigned vece, TCGv_vec r,
                       TCGv_vec a, TCGv_vec b);
 diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg-opc.h
 +++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(sars_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
  DEF(shlv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
  DEF(shrv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
  DEF(sarv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
 +DEF(rotlv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_rotv_vec))
 +DEF(rotrv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_rotv_vec))
  DEF(cmp_vec, 1, 2, 1, IMPLVEC)
 diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg.h
 +++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ typedef uint64_t TCGRegSet;
  #define TCG_TARGET_HAS_andc_vec         0
  #define TCG_TARGET_HAS_orc_vec          0
  #define TCG_TARGET_HAS_roti_vec         0
 +#define TCG_TARGET_HAS_rotv_vec         0
  #define TCG_TARGET_HAS_shi_vec          0
  #define TCG_TARGET_HAS_shs_vec          0
  #define TCG_TARGET_HAS_shv_vec          0
 diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/aarch64/tcg-target.h
 +++ b/tcg/aarch64/tcg-target.h
@@ -XXX,XX +XXX,XX @@ typedef enum {
  #define TCG_TARGET_HAS_neg_vec          1
  #define TCG_TARGET_HAS_abs_vec          1
  #define TCG_TARGET_HAS_roti_vec         0
 +#define TCG_TARGET_HAS_rotv_vec         0
  #define TCG_TARGET_HAS_shi_vec          1
  #define TCG_TARGET_HAS_shs_vec          0
  #define TCG_TARGET_HAS_shv_vec          1
 diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/i386/tcg-target.h
 +++ b/tcg/i386/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_avx2;
  #define TCG_TARGET_HAS_neg_vec          0
  #define TCG_TARGET_HAS_abs_vec          1
  #define TCG_TARGET_HAS_roti_vec         0
 +#define TCG_TARGET_HAS_rotv_vec         0
  #define TCG_TARGET_HAS_shi_vec          1
  #define TCG_TARGET_HAS_shs_vec          1
  #define TCG_TARGET_HAS_shv_vec          have_avx2
 diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/ppc/tcg-target.h
 +++ b/tcg/ppc/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_vsx;
  #define TCG_TARGET_HAS_neg_vec          have_isa_3_00
  #define TCG_TARGET_HAS_abs_vec          0
  #define TCG_TARGET_HAS_roti_vec         0
 +#define TCG_TARGET_HAS_rotv_vec         0
  #define TCG_TARGET_HAS_shi_vec          0
  #define TCG_TARGET_HAS_shs_vec          0
  #define TCG_TARGET_HAS_shv_vec          1
 diff --git a/accel/tcg/tcg-runtime-gvec.c b/accel/tcg/tcg-runtime-gvec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-runtime-gvec.c
 +++ b/accel/tcg/tcg-runtime-gvec.c
@@ -XXX,XX +XXX,XX @@ void HELPER(gvec_sar64v)(void *d, void *a, void *b, uint32_t desc)
      clear_high(d, oprsz, desc);
  }
 +void HELPER(gvec_rotl8v)(void *d, void *a, void *b, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint8_t)) {
 +        uint8_t sh = *(uint8_t *)(b + i) & 7;
 +        *(uint8_t *)(d + i) = rol8(*(uint8_t *)(a + i), sh);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotl16v)(void *d, void *a, void *b, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint16_t)) {
 +        uint8_t sh = *(uint16_t *)(b + i) & 15;
 +        *(uint16_t *)(d + i) = rol16(*(uint16_t *)(a + i), sh);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotl32v)(void *d, void *a, void *b, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint32_t)) {
 +        uint8_t sh = *(uint32_t *)(b + i) & 31;
 +        *(uint32_t *)(d + i) = rol32(*(uint32_t *)(a + i), sh);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotl64v)(void *d, void *a, void *b, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint64_t)) {
 +        uint8_t sh = *(uint64_t *)(b + i) & 63;
 +        *(uint64_t *)(d + i) = rol64(*(uint64_t *)(a + i), sh);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotr8v)(void *d, void *a, void *b, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint8_t)) {
 +        uint8_t sh = *(uint8_t *)(b + i) & 7;
 +        *(uint8_t *)(d + i) = ror8(*(uint8_t *)(a + i), sh);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotr16v)(void *d, void *a, void *b, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint16_t)) {
 +        uint8_t sh = *(uint16_t *)(b + i) & 15;
 +        *(uint16_t *)(d + i) = ror16(*(uint16_t *)(a + i), sh);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotr32v)(void *d, void *a, void *b, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint32_t)) {
 +        uint8_t sh = *(uint32_t *)(b + i) & 31;
 +        *(uint32_t *)(d + i) = ror32(*(uint32_t *)(a + i), sh);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotr64v)(void *d, void *a, void *b, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint64_t)) {
 +        uint8_t sh = *(uint64_t *)(b + i) & 63;
 +        *(uint64_t *)(d + i) = ror64(*(uint64_t *)(a + i), sh);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
  #define DO_CMP1(NAME, TYPE, OP)                                            \
  void HELPER(NAME)(void *d, void *a, void *b, uint32_t desc)                \
  {                                                                          \
 diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg-op-gvec.c
 +++ b/tcg/tcg-op-gvec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_sarv(unsigned vece, uint32_t dofs, uint32_t aofs,
      tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]);
  }
 +/*
 + * Similarly for rotates.
 + */
 +
 +static void tcg_gen_rotlv_mod_vec(unsigned vece, TCGv_vec d,
 +                                  TCGv_vec a, TCGv_vec b)
 +{
 +    TCGv_vec t = tcg_temp_new_vec_matching(d);
 +
 +    tcg_gen_dupi_vec(vece, t, (8 << vece) - 1);
 +    tcg_gen_and_vec(vece, t, t, b);
 +    tcg_gen_rotlv_vec(vece, d, a, t);
 +    tcg_temp_free_vec(t);
 +}
 +
 +static void tcg_gen_rotl_mod_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
 +{
 +    TCGv_i32 t = tcg_temp_new_i32();
 +
 +    tcg_gen_andi_i32(t, b, 31);
 +    tcg_gen_rotl_i32(d, a, t);
 +    tcg_temp_free_i32(t);
 +}
 +
 +static void tcg_gen_rotl_mod_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
 +{
 +    TCGv_i64 t = tcg_temp_new_i64();
 +
 +    tcg_gen_andi_i64(t, b, 63);
 +    tcg_gen_rotl_i64(d, a, t);
 +    tcg_temp_free_i64(t);
 +}
 +
 +void tcg_gen_gvec_rotlv(unsigned vece, uint32_t dofs, uint32_t aofs,
 +                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz)
 +{
 +    static const TCGOpcode vecop_list[] = { INDEX_op_rotlv_vec, 0 };
 +    static const GVecGen3 g[4] = {
 +        { .fniv = tcg_gen_rotlv_mod_vec,
 +          .fno = gen_helper_gvec_rotl8v,
 +          .opt_opc = vecop_list,
 +          .vece = MO_8 },
 +        { .fniv = tcg_gen_rotlv_mod_vec,
 +          .fno = gen_helper_gvec_rotl16v,
 +          .opt_opc = vecop_list,
 +          .vece = MO_16 },
 +        { .fni4 = tcg_gen_rotl_mod_i32,
 +          .fniv = tcg_gen_rotlv_mod_vec,
 +          .fno = gen_helper_gvec_rotl32v,
 +          .opt_opc = vecop_list,
 +          .vece = MO_32 },
 +        { .fni8 = tcg_gen_rotl_mod_i64,
 +          .fniv = tcg_gen_rotlv_mod_vec,
 +          .fno = gen_helper_gvec_rotl64v,
 +          .opt_opc = vecop_list,
 +          .prefer_i64 = TCG_TARGET_REG_BITS == 64,
 +          .vece = MO_64 },
 +    };
 +
 +    tcg_debug_assert(vece <= MO_64);
 +    tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]);
 +}
 +
 +static void tcg_gen_rotrv_mod_vec(unsigned vece, TCGv_vec d,
 +                                  TCGv_vec a, TCGv_vec b)
 +{
 +    TCGv_vec t = tcg_temp_new_vec_matching(d);
 +
 +    tcg_gen_dupi_vec(vece, t, (8 << vece) - 1);
 +    tcg_gen_and_vec(vece, t, t, b);
 +    tcg_gen_rotrv_vec(vece, d, a, t);
 +    tcg_temp_free_vec(t);
 +}
 +
 +static void tcg_gen_rotr_mod_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
 +{
 +    TCGv_i32 t = tcg_temp_new_i32();
 +
 +    tcg_gen_andi_i32(t, b, 31);
 +    tcg_gen_rotr_i32(d, a, t);
 +    tcg_temp_free_i32(t);
 +}
 +
 +static void tcg_gen_rotr_mod_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
 +{
 +    TCGv_i64 t = tcg_temp_new_i64();
 +
 +    tcg_gen_andi_i64(t, b, 63);
 +    tcg_gen_rotr_i64(d, a, t);
 +    tcg_temp_free_i64(t);
 +}
 +
 +void tcg_gen_gvec_rotrv(unsigned vece, uint32_t dofs, uint32_t aofs,
 +                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz)
 +{
 +    static const TCGOpcode vecop_list[] = { INDEX_op_rotrv_vec, 0 };
 +    static const GVecGen3 g[4] = {
 +        { .fniv = tcg_gen_rotrv_mod_vec,
 +          .fno = gen_helper_gvec_rotr8v,
 +          .opt_opc = vecop_list,
 +          .vece = MO_8 },
 +        { .fniv = tcg_gen_rotrv_mod_vec,
 +          .fno = gen_helper_gvec_rotr16v,
 +          .opt_opc = vecop_list,
 +          .vece = MO_16 },
 +        { .fni4 = tcg_gen_rotr_mod_i32,
 +          .fniv = tcg_gen_rotrv_mod_vec,
 +          .fno = gen_helper_gvec_rotr32v,
 +          .opt_opc = vecop_list,
 +          .vece = MO_32 },
 +        { .fni8 = tcg_gen_rotr_mod_i64,
 +          .fniv = tcg_gen_rotrv_mod_vec,
 +          .fno = gen_helper_gvec_rotr64v,
 +          .opt_opc = vecop_list,
 +          .prefer_i64 = TCG_TARGET_REG_BITS == 64,
 +          .vece = MO_64 },
 +    };
 +
 +    tcg_debug_assert(vece <= MO_64);
 +    tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]);
 +}
 +
  /* Expand OPSZ bytes worth of three-operand operations using i32 elements.  */
  static void expand_cmp_i32(uint32_t dofs, uint32_t aofs, uint32_t bofs,
                             uint32_t oprsz, TCGCond cond)
 diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg-op-vec.c
 +++ b/tcg/tcg-op-vec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_sarv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
      do_op3_nofail(vece, r, a, b, INDEX_op_sarv_vec);
  }
 +void tcg_gen_rotlv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
 +{
 +    do_op3_nofail(vece, r, a, b, INDEX_op_rotlv_vec);
 +}
 +
 +void tcg_gen_rotrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
 +{
 +    do_op3_nofail(vece, r, a, b, INDEX_op_rotrv_vec);
 +}
 +
  static void do_shifts(unsigned vece, TCGv_vec r, TCGv_vec a,
                        TCGv_i32 s, TCGOpcode opc_s, TCGOpcode opc_v)
  {
 diff --git a/tcg/tcg.c b/tcg/tcg.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg.c
 +++ b/tcg/tcg.c
-@@ -XXX,XX +XXX,XX @@ bool tcg_op_supported(TCGOpcode op)
+@@ -XXX,XX +XXX,XX @@ static void tcg_out_movext3(TCGContext *s, const TCGMovExtend *i1,
-         return have_vec && TCG_TARGET_HAS_shv_vec;
+ #define C_O2_I2(O1, O2, I1, I2)         C_PFX4(c_o2_i2_, O1, O2, I1, I2),
-     case INDEX_op_rotli_vec:
+ #define C_O2_I3(O1, O2, I1, I2, I3)     C_PFX5(c_o2_i3_, O1, O2, I1, I2, I3),
-         return have_vec && TCG_TARGET_HAS_roti_vec;
+ #define C_O2_I4(O1, O2, I1, I2, I3, I4) C_PFX6(c_o2_i4_, O1, O2, I1, I2, I3, I4),
-+    case INDEX_op_rotlv_vec:
++#define C_N1_O1_I4(O1, O2, I1, I2, I3, I4) C_PFX6(c_n1_o1_i4_, O1, O2, I1, I2, I3, I4),
-+    case INDEX_op_rotrv_vec:
-+        return have_vec && TCG_TARGET_HAS_rotv_vec;
+ typedef enum {
-     case INDEX_op_ssadd_vec:
+ #include "tcg-target-con-set.h"
-     case INDEX_op_usadd_vec:
+@@ -XXX,XX +XXX,XX @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode);
-     case INDEX_op_sssub_vec:
+ #undef C_O2_I2
-diff --git a/tcg/README b/tcg/README
+ #undef C_O2_I3
  #undef C_O2_I4
 +#undef C_N1_O1_I4
  /* Put all of the constraint sets into an array, indexed by the enum. */
@@ -XXX,XX +XXX,XX @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode);
  #define C_O2_I2(O1, O2, I1, I2)         { .args_ct_str = { #O1, #O2, #I1, #I2 } },
  #define C_O2_I3(O1, O2, I1, I2, I3)     { .args_ct_str = { #O1, #O2, #I1, #I2, #I3 } },
  #define C_O2_I4(O1, O2, I1, I2, I3, I4) { .args_ct_str = { #O1, #O2, #I1, #I2, #I3, #I4 } },
 +#define C_N1_O1_I4(O1, O2, I1, I2, I3, I4) { .args_ct_str = { "&" #O1, #O2, #I1, #I2, #I3, #I4 } },
  static const TCGTargetOpDef constraint_sets[] = {
  #include "tcg-target-con-set.h"
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef constraint_sets[] = {
  #undef C_O2_I2
  #undef C_O2_I3
  #undef C_O2_I4
 +#undef C_N1_O1_I4
  /* Expand the enumerator to be returned from tcg_target_op_def(). */
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef constraint_sets[] = {
  #define C_O2_I2(O1, O2, I1, I2)         C_PFX4(c_o2_i2_, O1, O2, I1, I2)
  #define C_O2_I3(O1, O2, I1, I2, I3)     C_PFX5(c_o2_i3_, O1, O2, I1, I2, I3)
  #define C_O2_I4(O1, O2, I1, I2, I3, I4) C_PFX6(c_o2_i4_, O1, O2, I1, I2, I3, I4)
 +#define C_N1_O1_I4(O1, O2, I1, I2, I3, I4) C_PFX6(c_n1_o1_i4_, O1, O2, I1, I2, I3, I4)
  #include "tcg-target.c.inc"
@@ -XXX,XX +XXX,XX @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)
                   * dead after the instruction, we must allocate a new
                   * register and move it.
                   */
 -                if (temp_readonly(ts) || !IS_DEAD_ARG(i)) {
 +                if (temp_readonly(ts) || !IS_DEAD_ARG(i)
 +                    || def->args_ct[arg_ct->alias_index].newreg) {
                      allocate_new_reg = true;
                  } else if (ts->val_type == TEMP_VAL_REG) {
                      /*
 diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc
 index XXXXXXX..XXXXXXX 100644
---- a/tcg/README
+--- a/tcg/i386/tcg-target.c.inc
-+++ b/tcg/README
++++ b/tcg/i386/tcg-target.c.inc
-@@ -XXX,XX +XXX,XX @@ E.g. VECL=1 -> 64 << 1 -> v128, and VECE=2 -> 1 << 2 -> i32.
+@@ -XXX,XX +XXX,XX @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op)
+     case INDEX_op_add2_i64:
- * shrv_vec   v0, v1, v2
+     case INDEX_op_sub2_i32:
- * sarv_vec   v0, v1, v2
+     case INDEX_op_sub2_i64:
-+* rotlv_vec  v0, v1, v2
+-        return C_O2_I4(r, r, 0, 1, re, re);
-+* rotrv_vec  v0, v1, v2
++        return C_N1_O1_I4(r, r, 0, 1, re, re);
--  Similarly for logical and arithmetic right shift.
+     case INDEX_op_ctz_i32:
-+  Similarly for logical and arithmetic right shift, and rotates.
+     case INDEX_op_ctz_i64:
+diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
- * cmp_vec  v0, v1, v2, cond
+index XXXXXXX..XXXXXXX 100644
+--- a/tcg/s390x/tcg-target.c.inc
 +++ b/tcg/s390x/tcg-target.c.inc
@@ -XXX,XX +XXX,XX @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op)
      case INDEX_op_add2_i32:
      case INDEX_op_sub2_i32:
 -        return C_O2_I4(r, r, 0, 1, ri, r);
 +        return C_N1_O1_I4(r, r, 0, 1, ri, r);
      case INDEX_op_add2_i64:
      case INDEX_op_sub2_i64:
 -        return C_O2_I4(r, r, 0, 1, rA, r);
 +        return C_N1_O1_I4(r, r, 0, 1, rA, r);
      case INDEX_op_st_vec:
          return C_O0_I2(v, r);
 --
-.25.1
+.34.1

-[PULL 01/12] tcg: Implement gvec support for rotate by immediate
+[PULL 6/7] accel/tcg: Zero-pad vaddr in tlb_debug output
-No host backend support yet, but the interfaces for rotli
+From: Anton Johansson <anjo@rev.ng>
 are in place.  Canonicalize immediate rotate to the left,
 based on a survey of architectures, but provide both left
 and right shift interfaces to the translators.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
+In replacing target_ulong with vaddr and TARGET_FMT_lx with VADDR_PRIx,
 the zero-padding of TARGET_FMT_lx got lost.  Readd 16-wide zero-padding
 for logging consistency.
 Suggested-by: Peter Maydell <peter.maydell@linaro.org>
 Signed-off-by: Anton Johansson <anjo@rev.ng>
 Message-Id: <20230713120746.26897-1-anjo@rev.ng>
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/tcg-runtime.h      |  5 +++
+ accel/tcg/cputlb.c | 20 ++++++++++----------
- include/tcg/tcg-op-gvec.h    |  6 ++++
+file changed, 10 insertions(+), 10 deletions(-)
  include/tcg/tcg-op.h         |  2 ++
  include/tcg/tcg-opc.h        |  1 +
  include/tcg/tcg.h            |  1 +
  tcg/aarch64/tcg-target.h     |  1 +
  tcg/i386/tcg-target.h        |  1 +
  tcg/ppc/tcg-target.h         |  1 +
  accel/tcg/tcg-runtime-gvec.c | 48 +++++++++++++++++++++++++
  tcg/tcg-op-gvec.c            | 68 ++++++++++++++++++++++++++++++++++++
  tcg/tcg-op-vec.c             | 12 +++++++
  tcg/tcg.c                    |  2 ++
  tcg/README                   |  3 +-
 files changed, 150 insertions(+), 1 deletion(-)
-diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h
+diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
 index XXXXXXX..XXXXXXX 100644
---- a/accel/tcg/tcg-runtime.h
+--- a/accel/tcg/cputlb.c
-+++ b/accel/tcg/tcg-runtime.h
++++ b/accel/tcg/cputlb.c
-@@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(gvec_sar16i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+@@ -XXX,XX +XXX,XX @@ static void tlb_flush_page_locked(CPUArchState *env, int midx, vaddr page)
- DEF_HELPER_FLAGS_3(gvec_sar32i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
- DEF_HELPER_FLAGS_3(gvec_sar64i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+     /* Check if we need to flush due to large pages.  */
+     if ((page & lp_mask) == lp_addr) {
-+DEF_HELPER_FLAGS_3(gvec_rotl8i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+-        tlb_debug("forcing full flush midx %d (%"
-+DEF_HELPER_FLAGS_3(gvec_rotl16i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+-                  VADDR_PRIx "/%" VADDR_PRIx ")\n",
-+DEF_HELPER_FLAGS_3(gvec_rotl32i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
++        tlb_debug("forcing full flush midx %d (%016"
-+DEF_HELPER_FLAGS_3(gvec_rotl64i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
++                  VADDR_PRIx "/%016" VADDR_PRIx ")\n",
-+
+                   midx, lp_addr, lp_mask);
- DEF_HELPER_FLAGS_4(gvec_shl8v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+         tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
- DEF_HELPER_FLAGS_4(gvec_shl16v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+     } else {
- DEF_HELPER_FLAGS_4(gvec_shl32v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+@@ -XXX,XX +XXX,XX @@ static void tlb_flush_page_by_mmuidx_async_0(CPUState *cpu,
-diff --git a/include/tcg/tcg-op-gvec.h b/include/tcg/tcg-op-gvec.h
-index XXXXXXX..XXXXXXX 100644
+     assert_cpu_is_self(cpu);
---- a/include/tcg/tcg-op-gvec.h
-+++ b/include/tcg/tcg-op-gvec.h
+-    tlb_debug("page addr: %" VADDR_PRIx " mmu_map:0x%x\n", addr, idxmap);
-@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_shri(unsigned vece, uint32_t dofs, uint32_t aofs,
++    tlb_debug("page addr: %016" VADDR_PRIx " mmu_map:0x%x\n", addr, idxmap);
-                        int64_t shift, uint32_t oprsz, uint32_t maxsz);
- void tcg_gen_gvec_sari(unsigned vece, uint32_t dofs, uint32_t aofs,
+     qemu_spin_lock(&env_tlb(env)->c.lock);
-                        int64_t shift, uint32_t oprsz, uint32_t maxsz);
+     for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) {
-+void tcg_gen_gvec_rotli(unsigned vece, uint32_t dofs, uint32_t aofs,
+@@ -XXX,XX +XXX,XX @@ static void tlb_flush_page_by_mmuidx_async_2(CPUState *cpu,
-+                        int64_t shift, uint32_t oprsz, uint32_t maxsz);
-+void tcg_gen_gvec_rotri(unsigned vece, uint32_t dofs, uint32_t aofs,
+ void tlb_flush_page_by_mmuidx(CPUState *cpu, vaddr addr, uint16_t idxmap)
 +                        int64_t shift, uint32_t oprsz, uint32_t maxsz);
  void tcg_gen_gvec_shls(unsigned vece, uint32_t dofs, uint32_t aofs,
                         TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz);
@@ -XXX,XX +XXX,XX @@ void tcg_gen_vec_shr8i_i64(TCGv_i64 d, TCGv_i64 a, int64_t);
  void tcg_gen_vec_shr16i_i64(TCGv_i64 d, TCGv_i64 a, int64_t);
  void tcg_gen_vec_sar8i_i64(TCGv_i64 d, TCGv_i64 a, int64_t);
  void tcg_gen_vec_sar16i_i64(TCGv_i64 d, TCGv_i64 a, int64_t);
 +void tcg_gen_vec_rotl8i_i64(TCGv_i64 d, TCGv_i64 a, int64_t c);
 +void tcg_gen_vec_rotl16i_i64(TCGv_i64 d, TCGv_i64 a, int64_t c);
  #endif
 diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg-op.h
 +++ b/include/tcg/tcg-op.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_umax_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b);
  void tcg_gen_shli_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
  void tcg_gen_shri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
  void tcg_gen_sari_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
 +void tcg_gen_rotli_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
 +void tcg_gen_rotri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
  void tcg_gen_shls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
  void tcg_gen_shrs_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
 diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg-opc.h
 +++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(not_vec, 1, 1, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_not_vec))
  DEF(shli_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_shi_vec))
  DEF(shri_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_shi_vec))
  DEF(sari_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_shi_vec))
 +DEF(rotli_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_roti_vec))
  DEF(shls_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
  DEF(shrs_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
 diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/tcg/tcg.h
 +++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ typedef uint64_t TCGRegSet;
  #define TCG_TARGET_HAS_not_vec          0
  #define TCG_TARGET_HAS_andc_vec         0
  #define TCG_TARGET_HAS_orc_vec          0
 +#define TCG_TARGET_HAS_roti_vec         0
  #define TCG_TARGET_HAS_shi_vec          0
  #define TCG_TARGET_HAS_shs_vec          0
  #define TCG_TARGET_HAS_shv_vec          0
 diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/aarch64/tcg-target.h
 +++ b/tcg/aarch64/tcg-target.h
@@ -XXX,XX +XXX,XX @@ typedef enum {
  #define TCG_TARGET_HAS_not_vec          1
  #define TCG_TARGET_HAS_neg_vec          1
  #define TCG_TARGET_HAS_abs_vec          1
 +#define TCG_TARGET_HAS_roti_vec         0
  #define TCG_TARGET_HAS_shi_vec          1
  #define TCG_TARGET_HAS_shs_vec          0
  #define TCG_TARGET_HAS_shv_vec          1
 diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/i386/tcg-target.h
 +++ b/tcg/i386/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_avx2;
  #define TCG_TARGET_HAS_not_vec          0
  #define TCG_TARGET_HAS_neg_vec          0
  #define TCG_TARGET_HAS_abs_vec          1
 +#define TCG_TARGET_HAS_roti_vec         0
  #define TCG_TARGET_HAS_shi_vec          1
  #define TCG_TARGET_HAS_shs_vec          1
  #define TCG_TARGET_HAS_shv_vec          have_avx2
 diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/ppc/tcg-target.h
 +++ b/tcg/ppc/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_vsx;
  #define TCG_TARGET_HAS_not_vec          1
  #define TCG_TARGET_HAS_neg_vec          have_isa_3_00
  #define TCG_TARGET_HAS_abs_vec          0
 +#define TCG_TARGET_HAS_roti_vec         0
  #define TCG_TARGET_HAS_shi_vec          0
  #define TCG_TARGET_HAS_shs_vec          0
  #define TCG_TARGET_HAS_shv_vec          1
 diff --git a/accel/tcg/tcg-runtime-gvec.c b/accel/tcg/tcg-runtime-gvec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/tcg-runtime-gvec.c
 +++ b/accel/tcg/tcg-runtime-gvec.c
@@ -XXX,XX +XXX,XX @@ void HELPER(gvec_sar64i)(void *d, void *a, uint32_t desc)
      clear_high(d, oprsz, desc);
  }
 +void HELPER(gvec_rotl8i)(void *d, void *a, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    int shift = simd_data(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint8_t)) {
 +        *(uint8_t *)(d + i) = rol8(*(uint8_t *)(a + i), shift);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotl16i)(void *d, void *a, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    int shift = simd_data(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint16_t)) {
 +        *(uint16_t *)(d + i) = rol16(*(uint16_t *)(a + i), shift);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotl32i)(void *d, void *a, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    int shift = simd_data(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint32_t)) {
 +        *(uint32_t *)(d + i) = rol32(*(uint32_t *)(a + i), shift);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
 +void HELPER(gvec_rotl64i)(void *d, void *a, uint32_t desc)
 +{
 +    intptr_t oprsz = simd_oprsz(desc);
 +    int shift = simd_data(desc);
 +    intptr_t i;
 +
 +    for (i = 0; i < oprsz; i += sizeof(uint64_t)) {
 +        *(uint64_t *)(d + i) = rol64(*(uint64_t *)(a + i), shift);
 +    }
 +    clear_high(d, oprsz, desc);
 +}
 +
  void HELPER(gvec_shl8v)(void *d, void *a, void *b, uint32_t desc)
  {
-     intptr_t oprsz = simd_oprsz(desc);
+-    tlb_debug("addr: %" VADDR_PRIx " mmu_idx:%" PRIx16 "\n", addr, idxmap);
-diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c
++    tlb_debug("addr: %016" VADDR_PRIx " mmu_idx:%" PRIx16 "\n", addr, idxmap);
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/tcg-op-gvec.c
+     /* This should already be page aligned */
-+++ b/tcg/tcg-op-gvec.c
+     addr &= TARGET_PAGE_MASK;
-@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_sari(unsigned vece, uint32_t dofs, uint32_t aofs,
+@@ -XXX,XX +XXX,XX @@ void tlb_flush_page(CPUState *cpu, vaddr addr)
-     }
+ void tlb_flush_page_by_mmuidx_all_cpus(CPUState *src_cpu, vaddr addr,
- }
+                                        uint16_t idxmap)
 +void tcg_gen_vec_rotl8i_i64(TCGv_i64 d, TCGv_i64 a, int64_t c)
 +{
 +    uint64_t mask = dup_const(MO_8, 0xff << c);
 +
 +    tcg_gen_shli_i64(d, a, c);
 +    tcg_gen_shri_i64(a, a, 8 - c);
 +    tcg_gen_andi_i64(d, d, mask);
 +    tcg_gen_andi_i64(a, a, ~mask);
 +    tcg_gen_or_i64(d, d, a);
 +}
 +
 +void tcg_gen_vec_rotl16i_i64(TCGv_i64 d, TCGv_i64 a, int64_t c)
 +{
 +    uint64_t mask = dup_const(MO_16, 0xffff << c);
 +
 +    tcg_gen_shli_i64(d, a, c);
 +    tcg_gen_shri_i64(a, a, 16 - c);
 +    tcg_gen_andi_i64(d, d, mask);
 +    tcg_gen_andi_i64(a, a, ~mask);
 +    tcg_gen_or_i64(d, d, a);
 +}
 +
 +void tcg_gen_gvec_rotli(unsigned vece, uint32_t dofs, uint32_t aofs,
 +                        int64_t shift, uint32_t oprsz, uint32_t maxsz)
 +{
 +    static const TCGOpcode vecop_list[] = { INDEX_op_rotli_vec, 0 };
 +    static const GVecGen2i g[4] = {
 +        { .fni8 = tcg_gen_vec_rotl8i_i64,
 +          .fniv = tcg_gen_rotli_vec,
 +          .fno = gen_helper_gvec_rotl8i,
 +          .opt_opc = vecop_list,
 +          .vece = MO_8 },
 +        { .fni8 = tcg_gen_vec_rotl16i_i64,
 +          .fniv = tcg_gen_rotli_vec,
 +          .fno = gen_helper_gvec_rotl16i,
 +          .opt_opc = vecop_list,
 +          .vece = MO_16 },
 +        { .fni4 = tcg_gen_rotli_i32,
 +          .fniv = tcg_gen_rotli_vec,
 +          .fno = gen_helper_gvec_rotl32i,
 +          .opt_opc = vecop_list,
 +          .vece = MO_32 },
 +        { .fni8 = tcg_gen_rotli_i64,
 +          .fniv = tcg_gen_rotli_vec,
 +          .fno = gen_helper_gvec_rotl64i,
 +          .opt_opc = vecop_list,
 +          .prefer_i64 = TCG_TARGET_REG_BITS == 64,
 +          .vece = MO_64 },
 +    };
 +
 +    tcg_debug_assert(vece <= MO_64);
 +    tcg_debug_assert(shift >= 0 && shift < (8 << vece));
 +    if (shift == 0) {
 +        tcg_gen_gvec_mov(vece, dofs, aofs, oprsz, maxsz);
 +    } else {
 +        tcg_gen_gvec_2i(dofs, aofs, oprsz, maxsz, shift, &g[vece]);
 +    }
 +}
 +
 +void tcg_gen_gvec_rotri(unsigned vece, uint32_t dofs, uint32_t aofs,
 +                        int64_t shift, uint32_t oprsz, uint32_t maxsz)
 +{
 +    tcg_debug_assert(vece <= MO_64);
 +    tcg_debug_assert(shift >= 0 && shift < (8 << vece));
 +    tcg_gen_gvec_rotli(vece, dofs, aofs, -shift & ((8 << vece) - 1),
 +                       oprsz, maxsz);
 +}
 +
  /*
   * Specialized generation vector shifts by a non-constant scalar.
   */
 diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tcg/tcg-op-vec.c
 +++ b/tcg/tcg-op-vec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_sari_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i)
      do_shifti(INDEX_op_sari_vec, vece, r, a, i);
  }
 +void tcg_gen_rotli_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i)
 +{
 +    do_shifti(INDEX_op_rotli_vec, vece, r, a, i);
 +}
 +
 +void tcg_gen_rotri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i)
 +{
 +    int bits = 8 << vece;
 +    tcg_debug_assert(i >= 0 && i < bits);
 +    do_shifti(INDEX_op_rotli_vec, vece, r, a, -i & (bits - 1));
 +}
 +
  void tcg_gen_cmp_vec(TCGCond cond, unsigned vece,
                       TCGv_vec r, TCGv_vec a, TCGv_vec b)
  {
-diff --git a/tcg/tcg.c b/tcg/tcg.c
+-    tlb_debug("addr: %" VADDR_PRIx " mmu_idx:%"PRIx16"\n", addr, idxmap);
-index XXXXXXX..XXXXXXX 100644
++    tlb_debug("addr: %016" VADDR_PRIx " mmu_idx:%"PRIx16"\n", addr, idxmap);
---- a/tcg/tcg.c
-+++ b/tcg/tcg.c
+     /* This should already be page aligned */
-@@ -XXX,XX +XXX,XX @@ bool tcg_op_supported(TCGOpcode op)
+     addr &= TARGET_PAGE_MASK;
-     case INDEX_op_shrv_vec:
+@@ -XXX,XX +XXX,XX @@ void tlb_flush_page_by_mmuidx_all_cpus_synced(CPUState *src_cpu,
-     case INDEX_op_sarv_vec:
+                                               vaddr addr,
-         return have_vec && TCG_TARGET_HAS_shv_vec;
+                                               uint16_t idxmap)
-+    case INDEX_op_rotli_vec:
+ {
-+        return have_vec && TCG_TARGET_HAS_roti_vec;
+-    tlb_debug("addr: %" VADDR_PRIx " mmu_idx:%"PRIx16"\n", addr, idxmap);
-     case INDEX_op_ssadd_vec:
++    tlb_debug("addr: %016" VADDR_PRIx " mmu_idx:%"PRIx16"\n", addr, idxmap);
-     case INDEX_op_usadd_vec:
-     case INDEX_op_sssub_vec:
+     /* This should already be page aligned */
-diff --git a/tcg/README b/tcg/README
+     addr &= TARGET_PAGE_MASK;
-index XXXXXXX..XXXXXXX 100644
+@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_locked(CPUArchState *env, int midx,
---- a/tcg/README
+      */
-+++ b/tcg/README
+     if (mask < f->mask || len > f->mask) {
-@@ -XXX,XX +XXX,XX @@ E.g. VECL=1 -> 64 << 1 -> v128, and VECE=2 -> 1 << 2 -> i32.
+         tlb_debug("forcing full flush midx %d ("
+-                  "%" VADDR_PRIx "/%" VADDR_PRIx "+%" VADDR_PRIx ")\n",
- * shri_vec   v0, v1, i2
++                  "%016" VADDR_PRIx "/%016" VADDR_PRIx "+%016" VADDR_PRIx ")\n",
- * sari_vec   v0, v1, i2
+                   midx, addr, mask, len);
-+* rotli_vec  v0, v1, i2
+         tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
- * shrs_vec   v0, v1, s2
+         return;
- * sars_vec   v0, v1, s2
+@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_locked(CPUArchState *env, int midx,
+      */
--  Similarly for logical and arithmetic right shift.
+     if (((addr + len - 1) & d->large_page_mask) == d->large_page_addr) {
-+  Similarly for logical and arithmetic right shift, and left rotate.
+         tlb_debug("forcing full flush midx %d ("
+-                  "%" VADDR_PRIx "/%" VADDR_PRIx ")\n",
- * shlv_vec   v0, v1, v2
++                  "%016" VADDR_PRIx "/%016" VADDR_PRIx ")\n",
                    midx, d->large_page_addr, d->large_page_mask);
          tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
          return;
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_by_mmuidx_async_0(CPUState *cpu,
      assert_cpu_is_self(cpu);
 -    tlb_debug("range: %" VADDR_PRIx "/%u+%" VADDR_PRIx " mmu_map:0x%x\n",
 +    tlb_debug("range: %016" VADDR_PRIx "/%u+%016" VADDR_PRIx " mmu_map:0x%x\n",
                d.addr, d.bits, d.len, d.idxmap);
      qemu_spin_lock(&env_tlb(env)->c.lock);
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx,
                                                  &xlat, &sz, full->attrs, &prot);
      assert(sz >= TARGET_PAGE_SIZE);
 -    tlb_debug("vaddr=%" VADDR_PRIx " paddr=0x" HWADDR_FMT_plx
 +    tlb_debug("vaddr=%016" VADDR_PRIx " paddr=0x" HWADDR_FMT_plx
                " prot=%x idx=%d\n",
                addr, full->phys_addr, prot, mmu_idx);
 --
-.25.1
+.34.1

-[PULL 03/12] tcg: Remove expansion to shift by vector from do_shifts
+Deleted patch
-We do not reflect this expansion in tcg_can_emit_vecop_list,
-so it is unused and unusable.  However, we actually perform
-the same expansion in do_gvec_shifts, so it is also unneeded.
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
----
- tcg/tcg-op-vec.c | 35 +++++++++++------------------------
-file changed, 11 insertions(+), 24 deletions(-)
-diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/tcg-op-vec.c
-+++ b/tcg/tcg-op-vec.c
-@@ -XXX,XX +XXX,XX @@ void tcg_gen_rotrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
- }
- static void do_shifts(unsigned vece, TCGv_vec r, TCGv_vec a,
--                      TCGv_i32 s, TCGOpcode opc_s, TCGOpcode opc_v)
-+                      TCGv_i32 s, TCGOpcode opc)
- {
-     TCGTemp *rt = tcgv_vec_temp(r);
-     TCGTemp *at = tcgv_vec_temp(a);
-@@ -XXX,XX +XXX,XX @@ static void do_shifts(unsigned vece, TCGv_vec r, TCGv_vec a,
-     TCGArg ai = temp_arg(at);
-     TCGArg si = temp_arg(st);
-     TCGType type = rt->base_type;
--    const TCGOpcode *hold_list;
-     int can;
-     tcg_debug_assert(at->base_type >= type);
--    tcg_assert_listed_vecop(opc_s);
--    hold_list = tcg_swap_vecop_list(NULL);
--
--    can = tcg_can_emit_vec_op(opc_s, type, vece);
-+    tcg_assert_listed_vecop(opc);
-+    can = tcg_can_emit_vec_op(opc, type, vece);
-     if (can > 0) {
--        vec_gen_3(opc_s, type, vece, ri, ai, si);
-+        vec_gen_3(opc, type, vece, ri, ai, si);
-     } else if (can < 0) {
--        tcg_expand_vec_op(opc_s, type, vece, ri, ai, si);
-+        const TCGOpcode *hold_list = tcg_swap_vecop_list(NULL);
-+        tcg_expand_vec_op(opc, type, vece, ri, ai, si);
-+        tcg_swap_vecop_list(hold_list);
-     } else {
--        TCGv_vec vec_s = tcg_temp_new_vec(type);
--
--        if (vece == MO_64) {
--            TCGv_i64 s64 = tcg_temp_new_i64();
--            tcg_gen_extu_i32_i64(s64, s);
--            tcg_gen_dup_i64_vec(MO_64, vec_s, s64);
--            tcg_temp_free_i64(s64);
--        } else {
--            tcg_gen_dup_i32_vec(vece, vec_s, s);
--        }
--        do_op3_nofail(vece, r, a, vec_s, opc_v);
--        tcg_temp_free_vec(vec_s);
-+        g_assert_not_reached();
-     }
--    tcg_swap_vecop_list(hold_list);
- }
- void tcg_gen_shls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 b)
- {
--    do_shifts(vece, r, a, b, INDEX_op_shls_vec, INDEX_op_shlv_vec);
-+    do_shifts(vece, r, a, b, INDEX_op_shls_vec);
- }
- void tcg_gen_shrs_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 b)
- {
--    do_shifts(vece, r, a, b, INDEX_op_shrs_vec, INDEX_op_shrv_vec);
-+    do_shifts(vece, r, a, b, INDEX_op_shrs_vec);
- }
- void tcg_gen_sars_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 b)
- {
--    do_shifts(vece, r, a, b, INDEX_op_sars_vec, INDEX_op_sarv_vec);
-+    do_shifts(vece, r, a, b, INDEX_op_sars_vec);
- }
- void tcg_gen_bitsel_vec(unsigned vece, TCGv_vec r, TCGv_vec a,
---
-.25.1

-[PULL 04/12] tcg: Implement gvec support for rotate by scalar
+Deleted patch
-No host backend support yet, but the interfaces for rotls
-are in place.  Only implement left-rotate for now, as the
-only known use of vector rotate by scalar is s390x, so any
-right-rotate would be unused and untestable.
-Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
----
- include/tcg/tcg-op-gvec.h |  2 ++
- include/tcg/tcg-op.h      |  1 +
- include/tcg/tcg-opc.h     |  1 +
- include/tcg/tcg.h         |  1 +
- tcg/aarch64/tcg-target.h  |  1 +
- tcg/i386/tcg-target.h     |  1 +
- tcg/ppc/tcg-target.h      |  1 +
- tcg/tcg-op-gvec.c         | 22 ++++++++++++++++++++++
- tcg/tcg-op-vec.c          |  5 +++++
- tcg/tcg.c                 |  2 ++
-files changed, 37 insertions(+)
-diff --git a/include/tcg/tcg-op-gvec.h b/include/tcg/tcg-op-gvec.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/tcg/tcg-op-gvec.h
-+++ b/include/tcg/tcg-op-gvec.h
-@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_shrs(unsigned vece, uint32_t dofs, uint32_t aofs,
-                        TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz);
- void tcg_gen_gvec_sars(unsigned vece, uint32_t dofs, uint32_t aofs,
-                        TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz);
-+void tcg_gen_gvec_rotls(unsigned vece, uint32_t dofs, uint32_t aofs,
-+                        TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz);
- /*
-  * Perform vector shift by vector element, modulo the element size.
-diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/tcg/tcg-op.h
-+++ b/include/tcg/tcg-op.h
-@@ -XXX,XX +XXX,XX @@ void tcg_gen_rotri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
- void tcg_gen_shls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
- void tcg_gen_shrs_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
- void tcg_gen_sars_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
-+void tcg_gen_rotls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
- void tcg_gen_shlv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
- void tcg_gen_shrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
-diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/tcg/tcg-opc.h
-+++ b/include/tcg/tcg-opc.h
-@@ -XXX,XX +XXX,XX @@ DEF(rotli_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_roti_vec))
- DEF(shls_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
- DEF(shrs_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
- DEF(sars_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
-+DEF(rotls_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_rots_vec))
- DEF(shlv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
- DEF(shrv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
-diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/tcg/tcg.h
-+++ b/include/tcg/tcg.h
-@@ -XXX,XX +XXX,XX @@ typedef uint64_t TCGRegSet;
- #define TCG_TARGET_HAS_andc_vec         0
- #define TCG_TARGET_HAS_orc_vec          0
- #define TCG_TARGET_HAS_roti_vec         0
-+#define TCG_TARGET_HAS_rots_vec         0
- #define TCG_TARGET_HAS_rotv_vec         0
- #define TCG_TARGET_HAS_shi_vec          0
- #define TCG_TARGET_HAS_shs_vec          0
-diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/aarch64/tcg-target.h
-+++ b/tcg/aarch64/tcg-target.h
-@@ -XXX,XX +XXX,XX @@ typedef enum {
- #define TCG_TARGET_HAS_neg_vec          1
- #define TCG_TARGET_HAS_abs_vec          1
- #define TCG_TARGET_HAS_roti_vec         0
-+#define TCG_TARGET_HAS_rots_vec         0
- #define TCG_TARGET_HAS_rotv_vec         0
- #define TCG_TARGET_HAS_shi_vec          1
- #define TCG_TARGET_HAS_shs_vec          0
-diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/i386/tcg-target.h
-+++ b/tcg/i386/tcg-target.h
-@@ -XXX,XX +XXX,XX @@ extern bool have_avx2;
- #define TCG_TARGET_HAS_neg_vec          0
- #define TCG_TARGET_HAS_abs_vec          1
- #define TCG_TARGET_HAS_roti_vec         0
-+#define TCG_TARGET_HAS_rots_vec         0
- #define TCG_TARGET_HAS_rotv_vec         0
- #define TCG_TARGET_HAS_shi_vec          1
- #define TCG_TARGET_HAS_shs_vec          1
-diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/ppc/tcg-target.h
-+++ b/tcg/ppc/tcg-target.h
-@@ -XXX,XX +XXX,XX @@ extern bool have_vsx;
- #define TCG_TARGET_HAS_neg_vec          have_isa_3_00
- #define TCG_TARGET_HAS_abs_vec          0
- #define TCG_TARGET_HAS_roti_vec         0
-+#define TCG_TARGET_HAS_rots_vec         0
- #define TCG_TARGET_HAS_rotv_vec         0
- #define TCG_TARGET_HAS_shi_vec          0
- #define TCG_TARGET_HAS_shs_vec          0
-diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/tcg-op-gvec.c
-+++ b/tcg/tcg-op-gvec.c
-@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_sars(unsigned vece, uint32_t dofs, uint32_t aofs,
-     do_gvec_shifts(vece, dofs, aofs, shift, oprsz, maxsz, &g);
- }
-+void tcg_gen_gvec_rotls(unsigned vece, uint32_t dofs, uint32_t aofs,
-+                        TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz)
-+{
-+    static const GVecGen2sh g = {
-+        .fni4 = tcg_gen_rotl_i32,
-+        .fni8 = tcg_gen_rotl_i64,
-+        .fniv_s = tcg_gen_rotls_vec,
-+        .fniv_v = tcg_gen_rotlv_vec,
-+        .fno = {
-+            gen_helper_gvec_rotl8i,
-+            gen_helper_gvec_rotl16i,
-+            gen_helper_gvec_rotl32i,
-+            gen_helper_gvec_rotl64i,
-+        },
-+        .s_list = { INDEX_op_rotls_vec, 0 },
-+        .v_list = { INDEX_op_rotlv_vec, 0 },
-+    };
-+
-+    tcg_debug_assert(vece <= MO_64);
-+    do_gvec_shifts(vece, dofs, aofs, shift, oprsz, maxsz, &g);
-+}
-+
- /*
-  * Expand D = A << (B % element bits)
-  *
-diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/tcg-op-vec.c
-+++ b/tcg/tcg-op-vec.c
-@@ -XXX,XX +XXX,XX @@ void tcg_gen_sars_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 b)
-     do_shifts(vece, r, a, b, INDEX_op_sars_vec);
- }
-+void tcg_gen_rotls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s)
-+{
-+    do_shifts(vece, r, a, s, INDEX_op_rotls_vec);
-+}
-+
- void tcg_gen_bitsel_vec(unsigned vece, TCGv_vec r, TCGv_vec a,
-                         TCGv_vec b, TCGv_vec c)
- {
-diff --git a/tcg/tcg.c b/tcg/tcg.c
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/tcg.c
-+++ b/tcg/tcg.c
-@@ -XXX,XX +XXX,XX @@ bool tcg_op_supported(TCGOpcode op)
-         return have_vec && TCG_TARGET_HAS_shv_vec;
-     case INDEX_op_rotli_vec:
-         return have_vec && TCG_TARGET_HAS_roti_vec;
-+    case INDEX_op_rotls_vec:
-+        return have_vec && TCG_TARGET_HAS_rots_vec;
-     case INDEX_op_rotlv_vec:
-     case INDEX_op_rotrv_vec:
-         return have_vec && TCG_TARGET_HAS_rotv_vec;
---
-.25.1

-[PULL 06/12] tcg/aarch64: Implement INDEX_op_rotl{i,v}_vec
+Deleted patch
-For immediate rotate , we can implement this in two instructions,
-using SLI.  For variable rotate, the oddness of aarch64 right-shift-
-as-negative-left-shift means a backend-specific expansion works best.
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
----
- tcg/aarch64/tcg-target.opc.h |  1 +
- tcg/aarch64/tcg-target.inc.c | 53 ++++++++++++++++++++++++++++++++++--
-files changed, 52 insertions(+), 2 deletions(-)
-diff --git a/tcg/aarch64/tcg-target.opc.h b/tcg/aarch64/tcg-target.opc.h
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/aarch64/tcg-target.opc.h
-+++ b/tcg/aarch64/tcg-target.opc.h
-@@ -XXX,XX +XXX,XX @@
-  */
- DEF(aa64_sshl_vec, 1, 2, 0, IMPLVEC)
-+DEF(aa64_sli_vec, 1, 2, 1, IMPLVEC)
-diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/aarch64/tcg-target.inc.c
-+++ b/tcg/aarch64/tcg-target.inc.c
-@@ -XXX,XX +XXX,XX @@ typedef enum {
-     I3614_SSHR      = 0x0f000400,
-     I3614_SSRA      = 0x0f001400,
-     I3614_SHL       = 0x0f005400,
-+    I3614_SLI       = 0x2f005400,
-     I3614_USHR      = 0x2f000400,
-     I3614_USRA      = 0x2f001400,
-@@ -XXX,XX +XXX,XX @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
-     case INDEX_op_sari_vec:
-         tcg_out_insn(s, 3614, SSHR, is_q, a0, a1, (16 << vece) - a2);
-         break;
-+    case INDEX_op_aa64_sli_vec:
-+        tcg_out_insn(s, 3614, SLI, is_q, a0, a2, args[3] + (8 << vece));
-+        break;
-     case INDEX_op_shlv_vec:
-         tcg_out_insn(s, 3616, USHL, is_q, vece, a0, a1, a2);
-         break;
-@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
-     case INDEX_op_shlv_vec:
-     case INDEX_op_bitsel_vec:
-         return 1;
-+    case INDEX_op_rotli_vec:
-     case INDEX_op_shrv_vec:
-     case INDEX_op_sarv_vec:
-+    case INDEX_op_rotlv_vec:
-+    case INDEX_op_rotrv_vec:
-         return -1;
-     case INDEX_op_mul_vec:
-     case INDEX_op_smax_vec:
-@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
-                        TCGArg a0, ...)
- {
-     va_list va;
--    TCGv_vec v0, v1, v2, t1;
-+    TCGv_vec v0, v1, v2, t1, t2;
-+    TCGArg a2;
-     va_start(va, a0);
-     v0 = temp_tcgv_vec(arg_temp(a0));
-     v1 = temp_tcgv_vec(arg_temp(va_arg(va, TCGArg)));
--    v2 = temp_tcgv_vec(arg_temp(va_arg(va, TCGArg)));
-+    a2 = va_arg(va, TCGArg);
-+    v2 = temp_tcgv_vec(arg_temp(a2));
-     switch (opc) {
-+    case INDEX_op_rotli_vec:
-+        t1 = tcg_temp_new_vec(type);
-+        tcg_gen_shri_vec(vece, t1, v1, -a2 & ((8 << vece) - 1));
-+        vec_gen_4(INDEX_op_aa64_sli_vec, type, vece,
-+                  tcgv_vec_arg(v0), tcgv_vec_arg(t1), tcgv_vec_arg(v1), a2);
-+        tcg_temp_free_vec(t1);
-+        break;
-+
-     case INDEX_op_shrv_vec:
-     case INDEX_op_sarv_vec:
-         /* Right shifts are negative left shifts for AArch64.  */
-@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
-         tcg_temp_free_vec(t1);
-         break;
-+    case INDEX_op_rotlv_vec:
-+        t1 = tcg_temp_new_vec(type);
-+        tcg_gen_dupi_vec(vece, t1, 8 << vece);
-+        tcg_gen_sub_vec(vece, t1, v2, t1);
-+        /* Right shifts are negative left shifts for AArch64.  */
-+        vec_gen_3(INDEX_op_shlv_vec, type, vece, tcgv_vec_arg(t1),
-+                  tcgv_vec_arg(v1), tcgv_vec_arg(t1));
-+        vec_gen_3(INDEX_op_shlv_vec, type, vece, tcgv_vec_arg(v0),
-+                  tcgv_vec_arg(v1), tcgv_vec_arg(v2));
-+        tcg_gen_or_vec(vece, v0, v0, t1);
-+        tcg_temp_free_vec(t1);
-+        break;
-+
-+    case INDEX_op_rotrv_vec:
-+        t1 = tcg_temp_new_vec(type);
-+        t2 = tcg_temp_new_vec(type);
-+        tcg_gen_neg_vec(vece, t1, v2);
-+        tcg_gen_dupi_vec(vece, t2, 8 << vece);
-+        tcg_gen_add_vec(vece, t2, t1, t2);
-+        /* Right shifts are negative left shifts for AArch64.  */
-+        vec_gen_3(INDEX_op_shlv_vec, type, vece, tcgv_vec_arg(t1),
-+                  tcgv_vec_arg(v1), tcgv_vec_arg(t1));
-+        vec_gen_3(INDEX_op_shlv_vec, type, vece, tcgv_vec_arg(t2),
-+                  tcgv_vec_arg(v1), tcgv_vec_arg(t2));
-+        tcg_gen_or_vec(vece, v0, t1, t2);
-+        tcg_temp_free_vec(t1);
-+        tcg_temp_free_vec(t2);
-+        break;
-+
-     default:
-         g_assert_not_reached();
-     }
-@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op)
-     static const TCGTargetOpDef lZ_l = { .args_ct_str = { "lZ", "l" } };
-     static const TCGTargetOpDef r_r_r = { .args_ct_str = { "r", "r", "r" } };
-     static const TCGTargetOpDef w_w_w = { .args_ct_str = { "w", "w", "w" } };
-+    static const TCGTargetOpDef w_0_w = { .args_ct_str = { "w", "0", "w" } };
-     static const TCGTargetOpDef w_w_wO = { .args_ct_str = { "w", "w", "wO" } };
-     static const TCGTargetOpDef w_w_wN = { .args_ct_str = { "w", "w", "wN" } };
-     static const TCGTargetOpDef w_w_wZ = { .args_ct_str = { "w", "w", "wZ" } };
-@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op)
-         return &w_w_wZ;
-     case INDEX_op_bitsel_vec:
-         return &w_w_w_w;
-+    case INDEX_op_aa64_sli_vec:
-+        return &w_0_w;
-     default:
-         return NULL;
---
-.25.1

-[PULL 07/12] tcg/ppc: Implement INDEX_op_rot[lr]v_vec
+Deleted patch
-We already had support for rotlv, using a target-specific opcode;
-convert to use the generic opcode.  Handle rotrv via simple negation.
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
----
- tcg/ppc/tcg-target.h     |  2 +-
- tcg/ppc/tcg-target.opc.h |  1 -
- tcg/ppc/tcg-target.inc.c | 23 +++++++++++++++++++----
-files changed, 20 insertions(+), 6 deletions(-)
-diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/ppc/tcg-target.h
-+++ b/tcg/ppc/tcg-target.h
-@@ -XXX,XX +XXX,XX @@ extern bool have_vsx;
- #define TCG_TARGET_HAS_abs_vec          0
- #define TCG_TARGET_HAS_roti_vec         0
- #define TCG_TARGET_HAS_rots_vec         0
--#define TCG_TARGET_HAS_rotv_vec         0
-+#define TCG_TARGET_HAS_rotv_vec         1
- #define TCG_TARGET_HAS_shi_vec          0
- #define TCG_TARGET_HAS_shs_vec          0
- #define TCG_TARGET_HAS_shv_vec          1
-diff --git a/tcg/ppc/tcg-target.opc.h b/tcg/ppc/tcg-target.opc.h
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/ppc/tcg-target.opc.h
-+++ b/tcg/ppc/tcg-target.opc.h
-@@ -XXX,XX +XXX,XX @@ DEF(ppc_msum_vec, 1, 3, 0, IMPLVEC)
- DEF(ppc_muleu_vec, 1, 2, 0, IMPLVEC)
- DEF(ppc_mulou_vec, 1, 2, 0, IMPLVEC)
- DEF(ppc_pkum_vec, 1, 2, 0, IMPLVEC)
--DEF(ppc_rotl_vec, 1, 2, 0, IMPLVEC)
-diff --git a/tcg/ppc/tcg-target.inc.c b/tcg/ppc/tcg-target.inc.c
-index XXXXXXX..XXXXXXX 100644
---- a/tcg/ppc/tcg-target.inc.c
-+++ b/tcg/ppc/tcg-target.inc.c
-@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
-     case INDEX_op_shlv_vec:
-     case INDEX_op_shrv_vec:
-     case INDEX_op_sarv_vec:
-+    case INDEX_op_rotlv_vec:
-         return vece <= MO_32 || have_isa_2_07;
-     case INDEX_op_ssadd_vec:
-     case INDEX_op_sssub_vec:
-@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
-     case INDEX_op_shli_vec:
-     case INDEX_op_shri_vec:
-     case INDEX_op_sari_vec:
-+    case INDEX_op_rotli_vec:
-         return vece <= MO_32 || have_isa_2_07 ? -1 : 0;
-     case INDEX_op_neg_vec:
-         return vece >= MO_32 && have_isa_3_00;
-@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
-         return 0;
-     case INDEX_op_bitsel_vec:
-         return have_vsx;
-+    case INDEX_op_rotrv_vec:
-+        return -1;
-     default:
-         return 0;
-     }
-@@ -XXX,XX +XXX,XX @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
-     case INDEX_op_ppc_pkum_vec:
-         insn = pkum_op[vece];
-         break;
--    case INDEX_op_ppc_rotl_vec:
-+    case INDEX_op_rotlv_vec:
-         insn = rotl_op[vece];
-         break;
-     case INDEX_op_ppc_msum_vec:
-@@ -XXX,XX +XXX,XX @@ static void expand_vec_mul(TCGType type, unsigned vece, TCGv_vec v0,
-         t3 = tcg_temp_new_vec(type);
-         t4 = tcg_temp_new_vec(type);
-         tcg_gen_dupi_vec(MO_8, t4, -16);
--        vec_gen_3(INDEX_op_ppc_rotl_vec, type, MO_32, tcgv_vec_arg(t1),
-+        vec_gen_3(INDEX_op_rotlv_vec, type, MO_32, tcgv_vec_arg(t1),
-                   tcgv_vec_arg(v2), tcgv_vec_arg(t4));
-         vec_gen_3(INDEX_op_ppc_mulou_vec, type, MO_16, tcgv_vec_arg(t2),
-                   tcgv_vec_arg(v1), tcgv_vec_arg(v2));
-@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
-                        TCGArg a0, ...)
- {
-     va_list va;
--    TCGv_vec v0, v1, v2;
-+    TCGv_vec v0, v1, v2, t0;
-     TCGArg a2;
-     va_start(va, a0);
-@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
-     case INDEX_op_sari_vec:
-         expand_vec_shi(type, vece, v0, v1, a2, INDEX_op_sarv_vec);
-         break;
-+    case INDEX_op_rotli_vec:
-+        expand_vec_shi(type, vece, v0, v1, a2, INDEX_op_rotlv_vec);
-+        break;
-     case INDEX_op_cmp_vec:
-         v2 = temp_tcgv_vec(arg_temp(a2));
-         expand_vec_cmp(type, vece, v0, v1, v2, va_arg(va, TCGArg));
-@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
-         v2 = temp_tcgv_vec(arg_temp(a2));
-         expand_vec_mul(type, vece, v0, v1, v2);
-         break;
-+    case INDEX_op_rotlv_vec:
-+        v2 = temp_tcgv_vec(arg_temp(a2));
-+        t0 = tcg_temp_new_vec(type);
-+        tcg_gen_neg_vec(vece, t0, v2);
-+        tcg_gen_rotlv_vec(vece, v0, v1, t0);
-+        tcg_temp_free_vec(t0);
-+        break;
-     default:
-         g_assert_not_reached();
-     }
-@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op)
-     case INDEX_op_shlv_vec:
-     case INDEX_op_shrv_vec:
-     case INDEX_op_sarv_vec:
-+    case INDEX_op_rotlv_vec:
-+    case INDEX_op_rotrv_vec:
-     case INDEX_op_ppc_mrgh_vec:
-     case INDEX_op_ppc_mrgl_vec:
-     case INDEX_op_ppc_muleu_vec:
-     case INDEX_op_ppc_mulou_vec:
-     case INDEX_op_ppc_pkum_vec:
--    case INDEX_op_ppc_rotl_vec:
-     case INDEX_op_dup2_vec:
-         return &v_v_v;
-     case INDEX_op_not_vec:
---
-.25.1

-[PULL 08/12] target/ppc: Use tcg_gen_gvec_rotlv
+Deleted patch
-Acked-by: David Gibson <david@gibson.dropbear.id.au>
-Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
----
- target/ppc/helper.h                 |  4 ----
- target/ppc/int_helper.c             | 17 -----------------
- target/ppc/translate/vmx-impl.inc.c |  8 ++++----
-files changed, 4 insertions(+), 25 deletions(-)
-diff --git a/target/ppc/helper.h b/target/ppc/helper.h
-index XXXXXXX..XXXXXXX 100644
---- a/target/ppc/helper.h
-+++ b/target/ppc/helper.h
-@@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vsubuqm, void, avr, avr, avr)
- DEF_HELPER_4(vsubecuq, void, avr, avr, avr, avr)
- DEF_HELPER_4(vsubeuqm, void, avr, avr, avr, avr)
- DEF_HELPER_3(vsubcuq, void, avr, avr, avr)
--DEF_HELPER_3(vrlb, void, avr, avr, avr)
--DEF_HELPER_3(vrlh, void, avr, avr, avr)
--DEF_HELPER_3(vrlw, void, avr, avr, avr)
--DEF_HELPER_3(vrld, void, avr, avr, avr)
- DEF_HELPER_4(vsldoi, void, avr, avr, avr, i32)
- DEF_HELPER_3(vextractub, void, avr, avr, i32)
- DEF_HELPER_3(vextractuh, void, avr, avr, i32)
-diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/ppc/int_helper.c
-+++ b/target/ppc/int_helper.c
-@@ -XXX,XX +XXX,XX @@ VRFI(p, float_round_up)
- VRFI(z, float_round_to_zero)
- #undef VRFI
--#define VROTATE(suffix, element, mask)                                  \
--    void helper_vrl##suffix(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b)   \
--    {                                                                   \
--        int i;                                                          \
--                                                                        \
--        for (i = 0; i < ARRAY_SIZE(r->element); i++) {                  \
--            unsigned int shift = b->element[i] & mask;                  \
--            r->element[i] = (a->element[i] << shift) |                  \
--                (a->element[i] >> (sizeof(a->element[0]) * 8 - shift)); \
--        }                                                               \
--    }
--VROTATE(b, u8, 0x7)
--VROTATE(h, u16, 0xF)
--VROTATE(w, u32, 0x1F)
--VROTATE(d, u64, 0x3F)
--#undef VROTATE
--
- void helper_vrsqrtefp(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *b)
- {
-     int i;
-diff --git a/target/ppc/translate/vmx-impl.inc.c b/target/ppc/translate/vmx-impl.inc.c
-index XXXXXXX..XXXXXXX 100644
---- a/target/ppc/translate/vmx-impl.inc.c
-+++ b/target/ppc/translate/vmx-impl.inc.c
-@@ -XXX,XX +XXX,XX @@ GEN_VXFORM3(vsubeuqm, 31, 0);
- GEN_VXFORM3(vsubecuq, 31, 0);
- GEN_VXFORM_DUAL(vsubeuqm, PPC_NONE, PPC2_ALTIVEC_207, \
-             vsubecuq, PPC_NONE, PPC2_ALTIVEC_207)
--GEN_VXFORM(vrlb, 2, 0);
--GEN_VXFORM(vrlh, 2, 1);
--GEN_VXFORM(vrlw, 2, 2);
-+GEN_VXFORM_V(vrlb, MO_8, tcg_gen_gvec_rotlv, 2, 0);
-+GEN_VXFORM_V(vrlh, MO_16, tcg_gen_gvec_rotlv, 2, 1);
-+GEN_VXFORM_V(vrlw, MO_32, tcg_gen_gvec_rotlv, 2, 2);
- GEN_VXFORM(vrlwmi, 2, 2);
- GEN_VXFORM_DUAL(vrlw, PPC_ALTIVEC, PPC_NONE, \
-                 vrlwmi, PPC_NONE, PPC2_ISA300)
--GEN_VXFORM(vrld, 2, 3);
-+GEN_VXFORM_V(vrld, MO_64, tcg_gen_gvec_rotlv, 2, 3);
- GEN_VXFORM(vrldmi, 2, 3);
- GEN_VXFORM_DUAL(vrld, PPC_NONE, PPC2_ALTIVEC_207, \
-                 vrldmi, PPC_NONE, PPC2_ISA300)
---
-.25.1

-[PULL 12/12] accel/tcg: Provide a NetBSD specific aarch64 cpu_signal_handler
+[PULL 7/7] accel/tcg: Fix type of 'last' for pageflags_{find,next}
-From: Nick Hudson <skrll@netbsd.org>
+From: Luca Bonissi <qemu@bonslack.org>
-Fix qemu build on NetBSD/evbarm-aarch64 by providing a NetBSD specific
+These should match 'start' as target_ulong, not target_long.
 cpu_signal_handler.
+On 32bit targets, the parameter was sign-extended to uint64_t,
+so only the first mmap within the upper 2GB memory can succeed.
+Signed-off-by: Luca Bonissi <qemu@bonslack.org>
+Message-Id: <327460e2-0ebd-9edb-426b-1df80d16c32a@bonslack.org>
 Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
-Signed-off-by: Nick Hudson <skrll@netbsd.org>
-Message-Id: <20200517101529.5367-1-skrll@netbsd.org>
 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
 ---
- accel/tcg/user-exec.c | 27 +++++++++++++++++++++++++++
+ accel/tcg/user-exec.c | 4 ++--
-file changed, 27 insertions(+)
+file changed, 2 insertions(+), 2 deletions(-)
 diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
 index XXXXXXX..XXXXXXX 100644
 --- a/accel/tcg/user-exec.c
 +++ b/accel/tcg/user-exec.c
-@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo,
+@@ -XXX,XX +XXX,XX @@ typedef struct PageFlagsNode {
- #elif defined(__aarch64__)
+ static IntervalTreeRoot pageflags_root;
-+#if defined(__NetBSD__)
+-static PageFlagsNode *pageflags_find(target_ulong start, target_long last)
-+
++static PageFlagsNode *pageflags_find(target_ulong start, target_ulong last)
-+#include <ucontext.h>
+ {
-+#include <sys/siginfo.h>
+     IntervalTreeNode *n;
-+
-+int cpu_signal_handler(int host_signum, void *pinfo, void *puc)
+@@ -XXX,XX +XXX,XX @@ static PageFlagsNode *pageflags_find(target_ulong start, target_long last)
 +{
 +    ucontext_t *uc = puc;
 +    siginfo_t *si = pinfo;
 +    unsigned long pc;
 +    int is_write;
 +    uint32_t esr;
 +
 +    pc = uc->uc_mcontext.__gregs[_REG_PC];
 +    esr = si->si_trap;
 +
 +    /*
 +     * siginfo_t::si_trap is the ESR value, for data aborts ESR.EC
 +     * is 0b10010x: then bit 6 is the WnR bit
 +     */
 +    is_write = extract32(esr, 27, 5) == 0x12 && extract32(esr, 6, 1) == 1;
 +    return handle_cpu_signal(pc, si, is_write, &uc->uc_sigmask);
 +}
 +
 +#else
 +
  #ifndef ESR_MAGIC
  /* Pre-3.16 kernel headers don't have these, so provide fallback definitions */
  #define ESR_MAGIC 0x45535201
@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo, void *puc)
      }
      return handle_cpu_signal(pc, info, is_write, &uc->uc_sigmask);
  }
-+#endif
+ static PageFlagsNode *pageflags_next(PageFlagsNode *p, target_ulong start,
- #elif defined(__s390__)
+-                                     target_long last)
 +                                     target_ulong last)
  {
      IntervalTreeNode *n;
 --
-.25.1
+.34.1

I have not been able to prod reviews of all of the rotate patches
in 4 weeks, but let's not let that block ARM work forever.

The following changes since commit cccdd8c7971896c339d59c9c5d4647d4ffd9568a:

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging (2020-06-02 10:25:55 +0100)

are available in the Git repository at:

https://github.com/rth7680/qemu.git tags/pull-tcg-20200602

for you to fetch changes up to 71b04329c4f7d5824a289ca5225e1883a278cf3b:

accel/tcg: Provide a NetBSD specific aarch64 cpu_signal_handler (2020-06-02 08:42:37 -0700)

----------------------------------------------------------------
Vector rotate support
Signal handling support for NetBSD arm/aarch64

----------------------------------------------------------------
Nick Hudson (2):
      accel/tcg: Adjust cpu_signal_handler for NetBSD/arm
      accel/tcg: Provide a NetBSD specific aarch64 cpu_signal_handler

Richard Henderson (10):
      tcg: Implement gvec support for rotate by immediate
      tcg: Implement gvec support for rotate by vector
      tcg: Remove expansion to shift by vector from do_shifts
      tcg: Implement gvec support for rotate by scalar
      tcg/i386: Implement INDEX_op_rotl{i,s,v}_vec
      tcg/aarch64: Implement INDEX_op_rotl{i,v}_vec
      tcg/ppc: Implement INDEX_op_rot[lr]v_vec
      target/ppc: Use tcg_gen_gvec_rotlv
      target/s390x: Use tcg_gen_gvec_rotl{i,s,v}
      tcg: Improve move ops in liveness_pass_2

No host backend support yet, but the interfaces for rotli
are in place.  Canonicalize immediate rotate to the left,
based on a survey of architectures, but provide both left
and right shift interfaces to the translators.

diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-runtime.h
+++ b/accel/tcg/tcg-runtime.h
@@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_3(gvec_sar16i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
 DEF_HELPER_FLAGS_3(gvec_sar32i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
 DEF_HELPER_FLAGS_3(gvec_sar64i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
 
+DEF_HELPER_FLAGS_3(gvec_rotl8i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_rotl16i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_rotl32i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+DEF_HELPER_FLAGS_3(gvec_rotl64i, TCG_CALL_NO_RWG, void, ptr, ptr, i32)
+
 DEF_HELPER_FLAGS_4(gvec_shl8v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_shl16v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_shl32v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
diff --git a/include/tcg/tcg-op-gvec.h b/include/tcg/tcg-op-gvec.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-op-gvec.h
+++ b/include/tcg/tcg-op-gvec.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_shri(unsigned vece, uint32_t dofs, uint32_t aofs,
                        int64_t shift, uint32_t oprsz, uint32_t maxsz);
 void tcg_gen_gvec_sari(unsigned vece, uint32_t dofs, uint32_t aofs,
                        int64_t shift, uint32_t oprsz, uint32_t maxsz);
+void tcg_gen_gvec_rotli(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        int64_t shift, uint32_t oprsz, uint32_t maxsz);
+void tcg_gen_gvec_rotri(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        int64_t shift, uint32_t oprsz, uint32_t maxsz);
 
 void tcg_gen_gvec_shls(unsigned vece, uint32_t dofs, uint32_t aofs,
                        TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz);
@@ -XXX,XX +XXX,XX @@ void tcg_gen_vec_shr8i_i64(TCGv_i64 d, TCGv_i64 a, int64_t);
 void tcg_gen_vec_shr16i_i64(TCGv_i64 d, TCGv_i64 a, int64_t);
 void tcg_gen_vec_sar8i_i64(TCGv_i64 d, TCGv_i64 a, int64_t);
 void tcg_gen_vec_sar16i_i64(TCGv_i64 d, TCGv_i64 a, int64_t);
+void tcg_gen_vec_rotl8i_i64(TCGv_i64 d, TCGv_i64 a, int64_t c);
+void tcg_gen_vec_rotl16i_i64(TCGv_i64 d, TCGv_i64 a, int64_t c);
 
 #endif
diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-op.h
+++ b/include/tcg/tcg-op.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_umax_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b);
 void tcg_gen_shli_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
 void tcg_gen_shri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
 void tcg_gen_sari_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
+void tcg_gen_rotli_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
+void tcg_gen_rotri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
 
 void tcg_gen_shls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
 void tcg_gen_shrs_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-opc.h
+++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(not_vec, 1, 1, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_not_vec))
 DEF(shli_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_shi_vec))
 DEF(shri_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_shi_vec))
 DEF(sari_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_shi_vec))
+DEF(rotli_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_roti_vec))
 
 DEF(shls_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
 DEF(shrs_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ typedef uint64_t TCGRegSet;
 #define TCG_TARGET_HAS_not_vec          0
 #define TCG_TARGET_HAS_andc_vec         0
 #define TCG_TARGET_HAS_orc_vec          0
+#define TCG_TARGET_HAS_roti_vec         0
 #define TCG_TARGET_HAS_shi_vec          0
 #define TCG_TARGET_HAS_shs_vec          0
 #define TCG_TARGET_HAS_shv_vec          0
diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/aarch64/tcg-target.h
+++ b/tcg/aarch64/tcg-target.h
@@ -XXX,XX +XXX,XX @@ typedef enum {
 #define TCG_TARGET_HAS_not_vec          1
 #define TCG_TARGET_HAS_neg_vec          1
 #define TCG_TARGET_HAS_abs_vec          1
+#define TCG_TARGET_HAS_roti_vec         0
 #define TCG_TARGET_HAS_shi_vec          1
 #define TCG_TARGET_HAS_shs_vec          0
 #define TCG_TARGET_HAS_shv_vec          1
diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/i386/tcg-target.h
+++ b/tcg/i386/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_avx2;
 #define TCG_TARGET_HAS_not_vec          0
 #define TCG_TARGET_HAS_neg_vec          0
 #define TCG_TARGET_HAS_abs_vec          1
+#define TCG_TARGET_HAS_roti_vec         0
 #define TCG_TARGET_HAS_shi_vec          1
 #define TCG_TARGET_HAS_shs_vec          1
 #define TCG_TARGET_HAS_shv_vec          have_avx2
diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/ppc/tcg-target.h
+++ b/tcg/ppc/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_vsx;
 #define TCG_TARGET_HAS_not_vec          1
 #define TCG_TARGET_HAS_neg_vec          have_isa_3_00
 #define TCG_TARGET_HAS_abs_vec          0
+#define TCG_TARGET_HAS_roti_vec         0
 #define TCG_TARGET_HAS_shi_vec          0
 #define TCG_TARGET_HAS_shs_vec          0
 #define TCG_TARGET_HAS_shv_vec          1
diff --git a/accel/tcg/tcg-runtime-gvec.c b/accel/tcg/tcg-runtime-gvec.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-runtime-gvec.c
+++ b/accel/tcg/tcg-runtime-gvec.c
@@ -XXX,XX +XXX,XX @@ void HELPER(gvec_sar64i)(void *d, void *a, uint32_t desc)
     clear_high(d, oprsz, desc);
 }
 
+void HELPER(gvec_rotl8i)(void *d, void *a, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    int shift = simd_data(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint8_t)) {
+        *(uint8_t *)(d + i) = rol8(*(uint8_t *)(a + i), shift);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotl16i)(void *d, void *a, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    int shift = simd_data(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint16_t)) {
+        *(uint16_t *)(d + i) = rol16(*(uint16_t *)(a + i), shift);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotl32i)(void *d, void *a, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    int shift = simd_data(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint32_t)) {
+        *(uint32_t *)(d + i) = rol32(*(uint32_t *)(a + i), shift);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotl64i)(void *d, void *a, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    int shift = simd_data(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint64_t)) {
+        *(uint64_t *)(d + i) = rol64(*(uint64_t *)(a + i), shift);
+    }
+    clear_high(d, oprsz, desc);
+}
+
 void HELPER(gvec_shl8v)(void *d, void *a, void *b, uint32_t desc)
 {
     intptr_t oprsz = simd_oprsz(desc);
diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op-gvec.c
+++ b/tcg/tcg-op-gvec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_sari(unsigned vece, uint32_t dofs, uint32_t aofs,
     }
 }
 
+void tcg_gen_vec_rotl8i_i64(TCGv_i64 d, TCGv_i64 a, int64_t c)
+{
+    uint64_t mask = dup_const(MO_8, 0xff << c);
+
+    tcg_gen_shli_i64(d, a, c);
+    tcg_gen_shri_i64(a, a, 8 - c);
+    tcg_gen_andi_i64(d, d, mask);
+    tcg_gen_andi_i64(a, a, ~mask);
+    tcg_gen_or_i64(d, d, a);
+}
+
+void tcg_gen_vec_rotl16i_i64(TCGv_i64 d, TCGv_i64 a, int64_t c)
+{
+    uint64_t mask = dup_const(MO_16, 0xffff << c);
+
+    tcg_gen_shli_i64(d, a, c);
+    tcg_gen_shri_i64(a, a, 16 - c);
+    tcg_gen_andi_i64(d, d, mask);
+    tcg_gen_andi_i64(a, a, ~mask);
+    tcg_gen_or_i64(d, d, a);
+}
+
+void tcg_gen_gvec_rotli(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        int64_t shift, uint32_t oprsz, uint32_t maxsz)
+{
+    static const TCGOpcode vecop_list[] = { INDEX_op_rotli_vec, 0 };
+    static const GVecGen2i g[4] = {
+        { .fni8 = tcg_gen_vec_rotl8i_i64,
+          .fniv = tcg_gen_rotli_vec,
+          .fno = gen_helper_gvec_rotl8i,
+          .opt_opc = vecop_list,
+          .vece = MO_8 },
+        { .fni8 = tcg_gen_vec_rotl16i_i64,
+          .fniv = tcg_gen_rotli_vec,
+          .fno = gen_helper_gvec_rotl16i,
+          .opt_opc = vecop_list,
+          .vece = MO_16 },
+        { .fni4 = tcg_gen_rotli_i32,
+          .fniv = tcg_gen_rotli_vec,
+          .fno = gen_helper_gvec_rotl32i,
+          .opt_opc = vecop_list,
+          .vece = MO_32 },
+        { .fni8 = tcg_gen_rotli_i64,
+          .fniv = tcg_gen_rotli_vec,
+          .fno = gen_helper_gvec_rotl64i,
+          .opt_opc = vecop_list,
+          .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+          .vece = MO_64 },
+    };
+
+    tcg_debug_assert(vece <= MO_64);
+    tcg_debug_assert(shift >= 0 && shift < (8 << vece));
+    if (shift == 0) {
+        tcg_gen_gvec_mov(vece, dofs, aofs, oprsz, maxsz);
+    } else {
+        tcg_gen_gvec_2i(dofs, aofs, oprsz, maxsz, shift, &g[vece]);
+    }
+}
+
+void tcg_gen_gvec_rotri(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        int64_t shift, uint32_t oprsz, uint32_t maxsz)
+{
+    tcg_debug_assert(vece <= MO_64);
+    tcg_debug_assert(shift >= 0 && shift < (8 << vece));
+    tcg_gen_gvec_rotli(vece, dofs, aofs, -shift & ((8 << vece) - 1),
+                       oprsz, maxsz);
+}
+
 /*
  * Specialized generation vector shifts by a non-constant scalar.
  */
diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op-vec.c
+++ b/tcg/tcg-op-vec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_sari_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i)
     do_shifti(INDEX_op_sari_vec, vece, r, a, i);
 }
 
+void tcg_gen_rotli_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i)
+{
+    do_shifti(INDEX_op_rotli_vec, vece, r, a, i);
+}
+
+void tcg_gen_rotri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i)
+{
+    int bits = 8 << vece;
+    tcg_debug_assert(i >= 0 && i < bits);
+    do_shifti(INDEX_op_rotli_vec, vece, r, a, -i & (bits - 1));
+}
+
 void tcg_gen_cmp_vec(TCGCond cond, unsigned vece,
                      TCGv_vec r, TCGv_vec a, TCGv_vec b)
 {
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ bool tcg_op_supported(TCGOpcode op)
     case INDEX_op_shrv_vec:
     case INDEX_op_sarv_vec:
         return have_vec && TCG_TARGET_HAS_shv_vec;
+    case INDEX_op_rotli_vec:
+        return have_vec && TCG_TARGET_HAS_roti_vec;
     case INDEX_op_ssadd_vec:
     case INDEX_op_usadd_vec:
     case INDEX_op_sssub_vec:
diff --git a/tcg/README b/tcg/README
index XXXXXXX..XXXXXXX 100644
--- a/tcg/README
+++ b/tcg/README
@@ -XXX,XX +XXX,XX @@ E.g. VECL=1 -> 64 << 1 -> v128, and VECE=2 -> 1 << 2 -> i32.
 
 * shri_vec   v0, v1, i2
 * sari_vec   v0, v1, i2
+* rotli_vec  v0, v1, i2
 * shrs_vec   v0, v1, s2
 * sars_vec   v0, v1, s2
 
-  Similarly for logical and arithmetic right shift.
+  Similarly for logical and arithmetic right shift, and left rotate.
 
 * shlv_vec   v0, v1, v2
 
-- 
2.25.1

No host backend support yet, but the interfaces for rotlv
and rotrv are in place.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
v3: Drop the generic expansion from rot to shift; we can do better
    for each backend, and then this code becomes unused.
---
 accel/tcg/tcg-runtime.h      |  10 +++
 include/tcg/tcg-op-gvec.h    |   4 ++
 include/tcg/tcg-op.h         |   2 +
 include/tcg/tcg-opc.h        |   2 +
 include/tcg/tcg.h            |   1 +
 tcg/aarch64/tcg-target.h     |   1 +
 tcg/i386/tcg-target.h        |   1 +
 tcg/ppc/tcg-target.h         |   1 +
 accel/tcg/tcg-runtime-gvec.c |  96 +++++++++++++++++++++++++++
 tcg/tcg-op-gvec.c            | 122 +++++++++++++++++++++++++++++++++++
 tcg/tcg-op-vec.c             |  10 +++
 tcg/tcg.c                    |   3 +
 tcg/README                   |   4 +-
 13 files changed, 256 insertions(+), 1 deletion(-)

diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-runtime.h
+++ b/accel/tcg/tcg-runtime.h
@@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_sar16v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_sar32v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_sar64v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 
+DEF_HELPER_FLAGS_4(gvec_rotl8v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_rotl16v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_rotl32v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_rotl64v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
+DEF_HELPER_FLAGS_4(gvec_rotr8v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_rotr16v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_rotr32v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+DEF_HELPER_FLAGS_4(gvec_rotr64v, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
+
 DEF_HELPER_FLAGS_4(gvec_eq8, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_eq16, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
 DEF_HELPER_FLAGS_4(gvec_eq32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32)
diff --git a/include/tcg/tcg-op-gvec.h b/include/tcg/tcg-op-gvec.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-op-gvec.h
+++ b/include/tcg/tcg-op-gvec.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_shrv(unsigned vece, uint32_t dofs, uint32_t aofs,
                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz);
 void tcg_gen_gvec_sarv(unsigned vece, uint32_t dofs, uint32_t aofs,
                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz);
+void tcg_gen_gvec_rotlv(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz);
+void tcg_gen_gvec_rotrv(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz);
 
 void tcg_gen_gvec_cmp(TCGCond cond, unsigned vece, uint32_t dofs,
                       uint32_t aofs, uint32_t bofs,
diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-op.h
+++ b/include/tcg/tcg-op.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_sars_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
 void tcg_gen_shlv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
 void tcg_gen_shrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
 void tcg_gen_sarv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
+void tcg_gen_rotlv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
+void tcg_gen_rotrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
 
 void tcg_gen_cmp_vec(TCGCond cond, unsigned vece, TCGv_vec r,
                      TCGv_vec a, TCGv_vec b);
diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-opc.h
+++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(sars_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
 DEF(shlv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
 DEF(shrv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
 DEF(sarv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
+DEF(rotlv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_rotv_vec))
+DEF(rotrv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_rotv_vec))
 
 DEF(cmp_vec, 1, 2, 1, IMPLVEC)
 
diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ typedef uint64_t TCGRegSet;
 #define TCG_TARGET_HAS_andc_vec         0
 #define TCG_TARGET_HAS_orc_vec          0
 #define TCG_TARGET_HAS_roti_vec         0
+#define TCG_TARGET_HAS_rotv_vec         0
 #define TCG_TARGET_HAS_shi_vec          0
 #define TCG_TARGET_HAS_shs_vec          0
 #define TCG_TARGET_HAS_shv_vec          0
diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/aarch64/tcg-target.h
+++ b/tcg/aarch64/tcg-target.h
@@ -XXX,XX +XXX,XX @@ typedef enum {
 #define TCG_TARGET_HAS_neg_vec          1
 #define TCG_TARGET_HAS_abs_vec          1
 #define TCG_TARGET_HAS_roti_vec         0
+#define TCG_TARGET_HAS_rotv_vec         0
 #define TCG_TARGET_HAS_shi_vec          1
 #define TCG_TARGET_HAS_shs_vec          0
 #define TCG_TARGET_HAS_shv_vec          1
diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/i386/tcg-target.h
+++ b/tcg/i386/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_avx2;
 #define TCG_TARGET_HAS_neg_vec          0
 #define TCG_TARGET_HAS_abs_vec          1
 #define TCG_TARGET_HAS_roti_vec         0
+#define TCG_TARGET_HAS_rotv_vec         0
 #define TCG_TARGET_HAS_shi_vec          1
 #define TCG_TARGET_HAS_shs_vec          1
 #define TCG_TARGET_HAS_shv_vec          have_avx2
diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/ppc/tcg-target.h
+++ b/tcg/ppc/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_vsx;
 #define TCG_TARGET_HAS_neg_vec          have_isa_3_00
 #define TCG_TARGET_HAS_abs_vec          0
 #define TCG_TARGET_HAS_roti_vec         0
+#define TCG_TARGET_HAS_rotv_vec         0
 #define TCG_TARGET_HAS_shi_vec          0
 #define TCG_TARGET_HAS_shs_vec          0
 #define TCG_TARGET_HAS_shv_vec          1
diff --git a/accel/tcg/tcg-runtime-gvec.c b/accel/tcg/tcg-runtime-gvec.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/tcg-runtime-gvec.c
+++ b/accel/tcg/tcg-runtime-gvec.c
@@ -XXX,XX +XXX,XX @@ void HELPER(gvec_sar64v)(void *d, void *a, void *b, uint32_t desc)
     clear_high(d, oprsz, desc);
 }
 
+void HELPER(gvec_rotl8v)(void *d, void *a, void *b, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint8_t)) {
+        uint8_t sh = *(uint8_t *)(b + i) & 7;
+        *(uint8_t *)(d + i) = rol8(*(uint8_t *)(a + i), sh);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotl16v)(void *d, void *a, void *b, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint16_t)) {
+        uint8_t sh = *(uint16_t *)(b + i) & 15;
+        *(uint16_t *)(d + i) = rol16(*(uint16_t *)(a + i), sh);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotl32v)(void *d, void *a, void *b, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint32_t)) {
+        uint8_t sh = *(uint32_t *)(b + i) & 31;
+        *(uint32_t *)(d + i) = rol32(*(uint32_t *)(a + i), sh);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotl64v)(void *d, void *a, void *b, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint64_t)) {
+        uint8_t sh = *(uint64_t *)(b + i) & 63;
+        *(uint64_t *)(d + i) = rol64(*(uint64_t *)(a + i), sh);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotr8v)(void *d, void *a, void *b, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint8_t)) {
+        uint8_t sh = *(uint8_t *)(b + i) & 7;
+        *(uint8_t *)(d + i) = ror8(*(uint8_t *)(a + i), sh);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotr16v)(void *d, void *a, void *b, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint16_t)) {
+        uint8_t sh = *(uint16_t *)(b + i) & 15;
+        *(uint16_t *)(d + i) = ror16(*(uint16_t *)(a + i), sh);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotr32v)(void *d, void *a, void *b, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint32_t)) {
+        uint8_t sh = *(uint32_t *)(b + i) & 31;
+        *(uint32_t *)(d + i) = ror32(*(uint32_t *)(a + i), sh);
+    }
+    clear_high(d, oprsz, desc);
+}
+
+void HELPER(gvec_rotr64v)(void *d, void *a, void *b, uint32_t desc)
+{
+    intptr_t oprsz = simd_oprsz(desc);
+    intptr_t i;
+
+    for (i = 0; i < oprsz; i += sizeof(uint64_t)) {
+        uint8_t sh = *(uint64_t *)(b + i) & 63;
+        *(uint64_t *)(d + i) = ror64(*(uint64_t *)(a + i), sh);
+    }
+    clear_high(d, oprsz, desc);
+}
+
 #define DO_CMP1(NAME, TYPE, OP)                                            \
 void HELPER(NAME)(void *d, void *a, void *b, uint32_t desc)                \
 {                                                                          \
diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op-gvec.c
+++ b/tcg/tcg-op-gvec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_sarv(unsigned vece, uint32_t dofs, uint32_t aofs,
     tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]);
 }
 
+/*
+ * Similarly for rotates.
+ */
+
+static void tcg_gen_rotlv_mod_vec(unsigned vece, TCGv_vec d,
+                                  TCGv_vec a, TCGv_vec b)
+{
+    TCGv_vec t = tcg_temp_new_vec_matching(d);
+
+    tcg_gen_dupi_vec(vece, t, (8 << vece) - 1);
+    tcg_gen_and_vec(vece, t, t, b);
+    tcg_gen_rotlv_vec(vece, d, a, t);
+    tcg_temp_free_vec(t);
+}
+
+static void tcg_gen_rotl_mod_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
+{
+    TCGv_i32 t = tcg_temp_new_i32();
+
+    tcg_gen_andi_i32(t, b, 31);
+    tcg_gen_rotl_i32(d, a, t);
+    tcg_temp_free_i32(t);
+}
+
+static void tcg_gen_rotl_mod_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+    TCGv_i64 t = tcg_temp_new_i64();
+
+    tcg_gen_andi_i64(t, b, 63);
+    tcg_gen_rotl_i64(d, a, t);
+    tcg_temp_free_i64(t);
+}
+
+void tcg_gen_gvec_rotlv(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz)
+{
+    static const TCGOpcode vecop_list[] = { INDEX_op_rotlv_vec, 0 };
+    static const GVecGen3 g[4] = {
+        { .fniv = tcg_gen_rotlv_mod_vec,
+          .fno = gen_helper_gvec_rotl8v,
+          .opt_opc = vecop_list,
+          .vece = MO_8 },
+        { .fniv = tcg_gen_rotlv_mod_vec,
+          .fno = gen_helper_gvec_rotl16v,
+          .opt_opc = vecop_list,
+          .vece = MO_16 },
+        { .fni4 = tcg_gen_rotl_mod_i32,
+          .fniv = tcg_gen_rotlv_mod_vec,
+          .fno = gen_helper_gvec_rotl32v,
+          .opt_opc = vecop_list,
+          .vece = MO_32 },
+        { .fni8 = tcg_gen_rotl_mod_i64,
+          .fniv = tcg_gen_rotlv_mod_vec,
+          .fno = gen_helper_gvec_rotl64v,
+          .opt_opc = vecop_list,
+          .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+          .vece = MO_64 },
+    };
+
+    tcg_debug_assert(vece <= MO_64);
+    tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]);
+}
+
+static void tcg_gen_rotrv_mod_vec(unsigned vece, TCGv_vec d,
+                                  TCGv_vec a, TCGv_vec b)
+{
+    TCGv_vec t = tcg_temp_new_vec_matching(d);
+
+    tcg_gen_dupi_vec(vece, t, (8 << vece) - 1);
+    tcg_gen_and_vec(vece, t, t, b);
+    tcg_gen_rotrv_vec(vece, d, a, t);
+    tcg_temp_free_vec(t);
+}
+
+static void tcg_gen_rotr_mod_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
+{
+    TCGv_i32 t = tcg_temp_new_i32();
+
+    tcg_gen_andi_i32(t, b, 31);
+    tcg_gen_rotr_i32(d, a, t);
+    tcg_temp_free_i32(t);
+}
+
+static void tcg_gen_rotr_mod_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
+{
+    TCGv_i64 t = tcg_temp_new_i64();
+
+    tcg_gen_andi_i64(t, b, 63);
+    tcg_gen_rotr_i64(d, a, t);
+    tcg_temp_free_i64(t);
+}
+
+void tcg_gen_gvec_rotrv(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        uint32_t bofs, uint32_t oprsz, uint32_t maxsz)
+{
+    static const TCGOpcode vecop_list[] = { INDEX_op_rotrv_vec, 0 };
+    static const GVecGen3 g[4] = {
+        { .fniv = tcg_gen_rotrv_mod_vec,
+          .fno = gen_helper_gvec_rotr8v,
+          .opt_opc = vecop_list,
+          .vece = MO_8 },
+        { .fniv = tcg_gen_rotrv_mod_vec,
+          .fno = gen_helper_gvec_rotr16v,
+          .opt_opc = vecop_list,
+          .vece = MO_16 },
+        { .fni4 = tcg_gen_rotr_mod_i32,
+          .fniv = tcg_gen_rotrv_mod_vec,
+          .fno = gen_helper_gvec_rotr32v,
+          .opt_opc = vecop_list,
+          .vece = MO_32 },
+        { .fni8 = tcg_gen_rotr_mod_i64,
+          .fniv = tcg_gen_rotrv_mod_vec,
+          .fno = gen_helper_gvec_rotr64v,
+          .opt_opc = vecop_list,
+          .prefer_i64 = TCG_TARGET_REG_BITS == 64,
+          .vece = MO_64 },
+    };
+
+    tcg_debug_assert(vece <= MO_64);
+    tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]);
+}
+
 /* Expand OPSZ bytes worth of three-operand operations using i32 elements.  */
 static void expand_cmp_i32(uint32_t dofs, uint32_t aofs, uint32_t bofs,
                            uint32_t oprsz, TCGCond cond)
diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op-vec.c
+++ b/tcg/tcg-op-vec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_sarv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
     do_op3_nofail(vece, r, a, b, INDEX_op_sarv_vec);
 }
 
+void tcg_gen_rotlv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
+{
+    do_op3_nofail(vece, r, a, b, INDEX_op_rotlv_vec);
+}
+
+void tcg_gen_rotrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
+{
+    do_op3_nofail(vece, r, a, b, INDEX_op_rotrv_vec);
+}
+
 static void do_shifts(unsigned vece, TCGv_vec r, TCGv_vec a,
                       TCGv_i32 s, TCGOpcode opc_s, TCGOpcode opc_v)
 {
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ bool tcg_op_supported(TCGOpcode op)
         return have_vec && TCG_TARGET_HAS_shv_vec;
     case INDEX_op_rotli_vec:
         return have_vec && TCG_TARGET_HAS_roti_vec;
+    case INDEX_op_rotlv_vec:
+    case INDEX_op_rotrv_vec:
+        return have_vec && TCG_TARGET_HAS_rotv_vec;
     case INDEX_op_ssadd_vec:
     case INDEX_op_usadd_vec:
     case INDEX_op_sssub_vec:
diff --git a/tcg/README b/tcg/README
index XXXXXXX..XXXXXXX 100644
--- a/tcg/README
+++ b/tcg/README
@@ -XXX,XX +XXX,XX @@ E.g. VECL=1 -> 64 << 1 -> v128, and VECE=2 -> 1 << 2 -> i32.
 
 * shrv_vec   v0, v1, v2
 * sarv_vec   v0, v1, v2
+* rotlv_vec  v0, v1, v2
+* rotrv_vec  v0, v1, v2
 
-  Similarly for logical and arithmetic right shift.
+  Similarly for logical and arithmetic right shift, and rotates.
 
 * cmp_vec  v0, v1, v2, cond
 
-- 
2.25.1

We do not reflect this expansion in tcg_can_emit_vecop_list,
so it is unused and unusable.  However, we actually perform
the same expansion in do_gvec_shifts, so it is also unneeded.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/tcg-op-vec.c | 35 +++++++++++------------------------
 1 file changed, 11 insertions(+), 24 deletions(-)

diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op-vec.c
+++ b/tcg/tcg-op-vec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_rotrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
 }
 
 static void do_shifts(unsigned vece, TCGv_vec r, TCGv_vec a,
-                      TCGv_i32 s, TCGOpcode opc_s, TCGOpcode opc_v)
+                      TCGv_i32 s, TCGOpcode opc)
 {
     TCGTemp *rt = tcgv_vec_temp(r);
     TCGTemp *at = tcgv_vec_temp(a);
@@ -XXX,XX +XXX,XX @@ static void do_shifts(unsigned vece, TCGv_vec r, TCGv_vec a,
     TCGArg ai = temp_arg(at);
     TCGArg si = temp_arg(st);
     TCGType type = rt->base_type;
-    const TCGOpcode *hold_list;
     int can;
 
     tcg_debug_assert(at->base_type >= type);
-    tcg_assert_listed_vecop(opc_s);
-    hold_list = tcg_swap_vecop_list(NULL);
-
-    can = tcg_can_emit_vec_op(opc_s, type, vece);
+    tcg_assert_listed_vecop(opc);
+    can = tcg_can_emit_vec_op(opc, type, vece);
     if (can > 0) {
-        vec_gen_3(opc_s, type, vece, ri, ai, si);
+        vec_gen_3(opc, type, vece, ri, ai, si);
     } else if (can < 0) {
-        tcg_expand_vec_op(opc_s, type, vece, ri, ai, si);
+        const TCGOpcode *hold_list = tcg_swap_vecop_list(NULL);
+        tcg_expand_vec_op(opc, type, vece, ri, ai, si);
+        tcg_swap_vecop_list(hold_list);
     } else {
-        TCGv_vec vec_s = tcg_temp_new_vec(type);
-
-        if (vece == MO_64) {
-            TCGv_i64 s64 = tcg_temp_new_i64();
-            tcg_gen_extu_i32_i64(s64, s);
-            tcg_gen_dup_i64_vec(MO_64, vec_s, s64);
-            tcg_temp_free_i64(s64);
-        } else {
-            tcg_gen_dup_i32_vec(vece, vec_s, s);
-        }
-        do_op3_nofail(vece, r, a, vec_s, opc_v);
-        tcg_temp_free_vec(vec_s);
+        g_assert_not_reached();
     }
-    tcg_swap_vecop_list(hold_list);
 }
 
 void tcg_gen_shls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 b)
 {
-    do_shifts(vece, r, a, b, INDEX_op_shls_vec, INDEX_op_shlv_vec);
+    do_shifts(vece, r, a, b, INDEX_op_shls_vec);
 }
 
 void tcg_gen_shrs_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 b)
 {
-    do_shifts(vece, r, a, b, INDEX_op_shrs_vec, INDEX_op_shrv_vec);
+    do_shifts(vece, r, a, b, INDEX_op_shrs_vec);
 }
 
 void tcg_gen_sars_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 b)
 {
-    do_shifts(vece, r, a, b, INDEX_op_sars_vec, INDEX_op_sarv_vec);
+    do_shifts(vece, r, a, b, INDEX_op_sars_vec);
 }
 
 void tcg_gen_bitsel_vec(unsigned vece, TCGv_vec r, TCGv_vec a,
-- 
2.25.1

No host backend support yet, but the interfaces for rotls
are in place.  Only implement left-rotate for now, as the
only known use of vector rotate by scalar is s390x, so any
right-rotate would be unused and untestable.

diff --git a/include/tcg/tcg-op-gvec.h b/include/tcg/tcg-op-gvec.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-op-gvec.h
+++ b/include/tcg/tcg-op-gvec.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_shrs(unsigned vece, uint32_t dofs, uint32_t aofs,
                        TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz);
 void tcg_gen_gvec_sars(unsigned vece, uint32_t dofs, uint32_t aofs,
                        TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz);
+void tcg_gen_gvec_rotls(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz);
 
 /*
  * Perform vector shift by vector element, modulo the element size.
diff --git a/include/tcg/tcg-op.h b/include/tcg/tcg-op.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-op.h
+++ b/include/tcg/tcg-op.h
@@ -XXX,XX +XXX,XX @@ void tcg_gen_rotri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i);
 void tcg_gen_shls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
 void tcg_gen_shrs_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
 void tcg_gen_sars_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
+void tcg_gen_rotls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s);
 
 void tcg_gen_shlv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
 void tcg_gen_shrv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec s);
diff --git a/include/tcg/tcg-opc.h b/include/tcg/tcg-opc.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg-opc.h
+++ b/include/tcg/tcg-opc.h
@@ -XXX,XX +XXX,XX @@ DEF(rotli_vec, 1, 1, 1, IMPLVEC | IMPL(TCG_TARGET_HAS_roti_vec))
 DEF(shls_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
 DEF(shrs_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
 DEF(sars_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shs_vec))
+DEF(rotls_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_rots_vec))
 
 DEF(shlv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
 DEF(shrv_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_shv_vec))
diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index XXXXXXX..XXXXXXX 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -XXX,XX +XXX,XX @@ typedef uint64_t TCGRegSet;
 #define TCG_TARGET_HAS_andc_vec         0
 #define TCG_TARGET_HAS_orc_vec          0
 #define TCG_TARGET_HAS_roti_vec         0
+#define TCG_TARGET_HAS_rots_vec         0
 #define TCG_TARGET_HAS_rotv_vec         0
 #define TCG_TARGET_HAS_shi_vec          0
 #define TCG_TARGET_HAS_shs_vec          0
diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/aarch64/tcg-target.h
+++ b/tcg/aarch64/tcg-target.h
@@ -XXX,XX +XXX,XX @@ typedef enum {
 #define TCG_TARGET_HAS_neg_vec          1
 #define TCG_TARGET_HAS_abs_vec          1
 #define TCG_TARGET_HAS_roti_vec         0
+#define TCG_TARGET_HAS_rots_vec         0
 #define TCG_TARGET_HAS_rotv_vec         0
 #define TCG_TARGET_HAS_shi_vec          1
 #define TCG_TARGET_HAS_shs_vec          0
diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/i386/tcg-target.h
+++ b/tcg/i386/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_avx2;
 #define TCG_TARGET_HAS_neg_vec          0
 #define TCG_TARGET_HAS_abs_vec          1
 #define TCG_TARGET_HAS_roti_vec         0
+#define TCG_TARGET_HAS_rots_vec         0
 #define TCG_TARGET_HAS_rotv_vec         0
 #define TCG_TARGET_HAS_shi_vec          1
 #define TCG_TARGET_HAS_shs_vec          1
diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/ppc/tcg-target.h
+++ b/tcg/ppc/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_vsx;
 #define TCG_TARGET_HAS_neg_vec          have_isa_3_00
 #define TCG_TARGET_HAS_abs_vec          0
 #define TCG_TARGET_HAS_roti_vec         0
+#define TCG_TARGET_HAS_rots_vec         0
 #define TCG_TARGET_HAS_rotv_vec         0
 #define TCG_TARGET_HAS_shi_vec          0
 #define TCG_TARGET_HAS_shs_vec          0
diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op-gvec.c
+++ b/tcg/tcg-op-gvec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_gvec_sars(unsigned vece, uint32_t dofs, uint32_t aofs,
     do_gvec_shifts(vece, dofs, aofs, shift, oprsz, maxsz, &g);
 }
 
+void tcg_gen_gvec_rotls(unsigned vece, uint32_t dofs, uint32_t aofs,
+                        TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz)
+{
+    static const GVecGen2sh g = {
+        .fni4 = tcg_gen_rotl_i32,
+        .fni8 = tcg_gen_rotl_i64,
+        .fniv_s = tcg_gen_rotls_vec,
+        .fniv_v = tcg_gen_rotlv_vec,
+        .fno = {
+            gen_helper_gvec_rotl8i,
+            gen_helper_gvec_rotl16i,
+            gen_helper_gvec_rotl32i,
+            gen_helper_gvec_rotl64i,
+        },
+        .s_list = { INDEX_op_rotls_vec, 0 },
+        .v_list = { INDEX_op_rotlv_vec, 0 },
+    };
+
+    tcg_debug_assert(vece <= MO_64);
+    do_gvec_shifts(vece, dofs, aofs, shift, oprsz, maxsz, &g);
+}
+
 /*
  * Expand D = A << (B % element bits)
  *
diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg-op-vec.c
+++ b/tcg/tcg-op-vec.c
@@ -XXX,XX +XXX,XX @@ void tcg_gen_sars_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 b)
     do_shifts(vece, r, a, b, INDEX_op_sars_vec);
 }
 
+void tcg_gen_rotls_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_i32 s)
+{
+    do_shifts(vece, r, a, s, INDEX_op_rotls_vec);
+}
+
 void tcg_gen_bitsel_vec(unsigned vece, TCGv_vec r, TCGv_vec a,
                         TCGv_vec b, TCGv_vec c)
 {
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ bool tcg_op_supported(TCGOpcode op)
         return have_vec && TCG_TARGET_HAS_shv_vec;
     case INDEX_op_rotli_vec:
         return have_vec && TCG_TARGET_HAS_roti_vec;
+    case INDEX_op_rotls_vec:
+        return have_vec && TCG_TARGET_HAS_rots_vec;
     case INDEX_op_rotlv_vec:
     case INDEX_op_rotrv_vec:
         return have_vec && TCG_TARGET_HAS_rotv_vec;
-- 
2.25.1

For immediates, we must continue the special casing of 8-bit
elements.  The other element sizes and shift types are trivially
implemented with shifts.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/i386/tcg-target.inc.c | 116 ++++++++++++++++++++++++++++++++------
 1 file changed, 100 insertions(+), 16 deletions(-)

diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/i386/tcg-target.inc.c
+++ b/tcg/i386/tcg-target.inc.c
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op)
     case INDEX_op_shls_vec:
     case INDEX_op_shrs_vec:
     case INDEX_op_sars_vec:
+    case INDEX_op_rotls_vec:
     case INDEX_op_cmp_vec:
     case INDEX_op_x86_shufps_vec:
     case INDEX_op_x86_blend_vec:
@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
     case INDEX_op_xor_vec:
     case INDEX_op_andc_vec:
         return 1;
+    case INDEX_op_rotli_vec:
     case INDEX_op_cmp_vec:
     case INDEX_op_cmpsel_vec:
         return -1;
@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
         return vece >= MO_16;
     case INDEX_op_sars_vec:
         return vece >= MO_16 && vece <= MO_32;
+    case INDEX_op_rotls_vec:
+        return vece >= MO_16 ? -1 : 0;
 
     case INDEX_op_shlv_vec:
     case INDEX_op_shrv_vec:
         return have_avx2 && vece >= MO_32;
     case INDEX_op_sarv_vec:
         return have_avx2 && vece == MO_32;
+    case INDEX_op_rotlv_vec:
+    case INDEX_op_rotrv_vec:
+        return have_avx2 && vece >= MO_32 ? -1 : 0;
 
     case INDEX_op_mul_vec:
         if (vece == MO_8) {
@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
     }
 }
 
-static void expand_vec_shi(TCGType type, unsigned vece, bool shr,
+static void expand_vec_shi(TCGType type, unsigned vece, TCGOpcode opc,
                            TCGv_vec v0, TCGv_vec v1, TCGArg imm)
 {
     TCGv_vec t1, t2;
@@ -XXX,XX +XXX,XX @@ static void expand_vec_shi(TCGType type, unsigned vece, bool shr,
     t1 = tcg_temp_new_vec(type);
     t2 = tcg_temp_new_vec(type);
 
-    /* Unpack to W, shift, and repack.  Tricky bits:
-       (1) Use punpck*bw x,x to produce DDCCBBAA,
-           i.e. duplicate in other half of the 16-bit lane.
-       (2) For right-shift, add 8 so that the high half of
-           the lane becomes zero.  For left-shift, we must
-           shift up and down again.
-       (3) Step 2 leaves high half zero such that PACKUSWB
-           (pack with unsigned saturation) does not modify
-           the quantity.  */
+    /*
+     * Unpack to W, shift, and repack.  Tricky bits:
+     * (1) Use punpck*bw x,x to produce DDCCBBAA,
+     *     i.e. duplicate in other half of the 16-bit lane.
+     * (2) For right-shift, add 8 so that the high half of the lane
+     *     becomes zero.  For left-shift, and left-rotate, we must
+     *     shift up and down again.
+     * (3) Step 2 leaves high half zero such that PACKUSWB
+     *     (pack with unsigned saturation) does not modify
+     *     the quantity.
+     */
     vec_gen_3(INDEX_op_x86_punpckl_vec, type, MO_8,
               tcgv_vec_arg(t1), tcgv_vec_arg(v1), tcgv_vec_arg(v1));
     vec_gen_3(INDEX_op_x86_punpckh_vec, type, MO_8,
               tcgv_vec_arg(t2), tcgv_vec_arg(v1), tcgv_vec_arg(v1));
 
-    if (shr) {
-        tcg_gen_shri_vec(MO_16, t1, t1, imm + 8);
-        tcg_gen_shri_vec(MO_16, t2, t2, imm + 8);
+    if (opc != INDEX_op_rotli_vec) {
+        imm += 8;
+    }
+    if (opc == INDEX_op_shri_vec) {
+        tcg_gen_shri_vec(MO_16, t1, t1, imm);
+        tcg_gen_shri_vec(MO_16, t2, t2, imm);
     } else {
-        tcg_gen_shli_vec(MO_16, t1, t1, imm + 8);
-        tcg_gen_shli_vec(MO_16, t2, t2, imm + 8);
+        tcg_gen_shli_vec(MO_16, t1, t1, imm);
+        tcg_gen_shli_vec(MO_16, t2, t2, imm);
         tcg_gen_shri_vec(MO_16, t1, t1, 8);
         tcg_gen_shri_vec(MO_16, t2, t2, 8);
     }
@@ -XXX,XX +XXX,XX @@ static void expand_vec_sari(TCGType type, unsigned vece,
     }
 }
 
+static void expand_vec_rotli(TCGType type, unsigned vece,
+                             TCGv_vec v0, TCGv_vec v1, TCGArg imm)
+{
+    TCGv_vec t;
+
+    if (vece == MO_8) {
+        expand_vec_shi(type, vece, INDEX_op_rotli_vec, v0, v1, imm);
+        return;
+    }
+
+    t = tcg_temp_new_vec(type);
+    tcg_gen_shli_vec(vece, t, v1, imm);
+    tcg_gen_shri_vec(vece, v0, v1, (8 << vece) - imm);
+    tcg_gen_or_vec(vece, v0, v0, t);
+    tcg_temp_free_vec(t);
+}
+
+static void expand_vec_rotls(TCGType type, unsigned vece,
+                             TCGv_vec v0, TCGv_vec v1, TCGv_i32 lsh)
+{
+    TCGv_i32 rsh;
+    TCGv_vec t;
+
+    tcg_debug_assert(vece != MO_8);
+
+    t = tcg_temp_new_vec(type);
+    rsh = tcg_temp_new_i32();
+
+    tcg_gen_neg_i32(rsh, lsh);
+    tcg_gen_andi_i32(rsh, rsh, (8 << vece) - 1);
+    tcg_gen_shls_vec(vece, t, v1, lsh);
+    tcg_gen_shrs_vec(vece, v0, v1, rsh);
+    tcg_gen_or_vec(vece, v0, v0, t);
+    tcg_temp_free_vec(t);
+    tcg_temp_free_i32(rsh);
+}
+
+static void expand_vec_rotv(TCGType type, unsigned vece, TCGv_vec v0,
+                            TCGv_vec v1, TCGv_vec sh, bool right)
+{
+    TCGv_vec t = tcg_temp_new_vec(type);
+
+    tcg_gen_dupi_vec(vece, t, 8 << vece);
+    tcg_gen_sub_vec(vece, t, t, sh);
+    if (right) {
+        tcg_gen_shlv_vec(vece, t, v1, t);
+        tcg_gen_shrv_vec(vece, v0, v1, sh);
+    } else {
+        tcg_gen_shrv_vec(vece, t, v1, t);
+        tcg_gen_shlv_vec(vece, v0, v1, sh);
+    }
+    tcg_gen_or_vec(vece, v0, v0, t);
+    tcg_temp_free_vec(t);
+}
+
 static void expand_vec_mul(TCGType type, unsigned vece,
                            TCGv_vec v0, TCGv_vec v1, TCGv_vec v2)
 {
@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
     switch (opc) {
     case INDEX_op_shli_vec:
     case INDEX_op_shri_vec:
-        expand_vec_shi(type, vece, opc == INDEX_op_shri_vec, v0, v1, a2);
+        expand_vec_shi(type, vece, opc, v0, v1, a2);
         break;
 
     case INDEX_op_sari_vec:
         expand_vec_sari(type, vece, v0, v1, a2);
         break;
 
+    case INDEX_op_rotli_vec:
+        expand_vec_rotli(type, vece, v0, v1, a2);
+        break;
+
+    case INDEX_op_rotls_vec:
+        expand_vec_rotls(type, vece, v0, v1, temp_tcgv_i32(arg_temp(a2)));
+        break;
+
+    case INDEX_op_rotlv_vec:
+        v2 = temp_tcgv_vec(arg_temp(a2));
+        expand_vec_rotv(type, vece, v0, v1, v2, false);
+        break;
+    case INDEX_op_rotrv_vec:
+        v2 = temp_tcgv_vec(arg_temp(a2));
+        expand_vec_rotv(type, vece, v0, v1, v2, true);
+        break;
+
     case INDEX_op_mul_vec:
         v2 = temp_tcgv_vec(arg_temp(a2));
         expand_vec_mul(type, vece, v0, v1, v2);
-- 
2.25.1

For immediate rotate , we can implement this in two instructions,
using SLI.  For variable rotate, the oddness of aarch64 right-shift-
as-negative-left-shift means a backend-specific expansion works best.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/aarch64/tcg-target.opc.h |  1 +
 tcg/aarch64/tcg-target.inc.c | 53 ++++++++++++++++++++++++++++++++++--
 2 files changed, 52 insertions(+), 2 deletions(-)

diff --git a/tcg/aarch64/tcg-target.opc.h b/tcg/aarch64/tcg-target.opc.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/aarch64/tcg-target.opc.h
+++ b/tcg/aarch64/tcg-target.opc.h
@@ -XXX,XX +XXX,XX @@
  */
 
 DEF(aa64_sshl_vec, 1, 2, 0, IMPLVEC)
+DEF(aa64_sli_vec, 1, 2, 1, IMPLVEC)
diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/aarch64/tcg-target.inc.c
+++ b/tcg/aarch64/tcg-target.inc.c
@@ -XXX,XX +XXX,XX @@ typedef enum {
     I3614_SSHR      = 0x0f000400,
     I3614_SSRA      = 0x0f001400,
     I3614_SHL       = 0x0f005400,
+    I3614_SLI       = 0x2f005400,
     I3614_USHR      = 0x2f000400,
     I3614_USRA      = 0x2f001400,
 
@@ -XXX,XX +XXX,XX @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
     case INDEX_op_sari_vec:
         tcg_out_insn(s, 3614, SSHR, is_q, a0, a1, (16 << vece) - a2);
         break;
+    case INDEX_op_aa64_sli_vec:
+        tcg_out_insn(s, 3614, SLI, is_q, a0, a2, args[3] + (8 << vece));
+        break;
     case INDEX_op_shlv_vec:
         tcg_out_insn(s, 3616, USHL, is_q, vece, a0, a1, a2);
         break;
@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
     case INDEX_op_shlv_vec:
     case INDEX_op_bitsel_vec:
         return 1;
+    case INDEX_op_rotli_vec:
     case INDEX_op_shrv_vec:
     case INDEX_op_sarv_vec:
+    case INDEX_op_rotlv_vec:
+    case INDEX_op_rotrv_vec:
         return -1;
     case INDEX_op_mul_vec:
     case INDEX_op_smax_vec:
@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
                        TCGArg a0, ...)
 {
     va_list va;
-    TCGv_vec v0, v1, v2, t1;
+    TCGv_vec v0, v1, v2, t1, t2;
+    TCGArg a2;
 
     va_start(va, a0);
     v0 = temp_tcgv_vec(arg_temp(a0));
     v1 = temp_tcgv_vec(arg_temp(va_arg(va, TCGArg)));
-    v2 = temp_tcgv_vec(arg_temp(va_arg(va, TCGArg)));
+    a2 = va_arg(va, TCGArg);
+    v2 = temp_tcgv_vec(arg_temp(a2));
 
     switch (opc) {
+    case INDEX_op_rotli_vec:
+        t1 = tcg_temp_new_vec(type);
+        tcg_gen_shri_vec(vece, t1, v1, -a2 & ((8 << vece) - 1));
+        vec_gen_4(INDEX_op_aa64_sli_vec, type, vece,
+                  tcgv_vec_arg(v0), tcgv_vec_arg(t1), tcgv_vec_arg(v1), a2);
+        tcg_temp_free_vec(t1);
+        break;
+
     case INDEX_op_shrv_vec:
     case INDEX_op_sarv_vec:
         /* Right shifts are negative left shifts for AArch64.  */
@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
         tcg_temp_free_vec(t1);
         break;
 
+    case INDEX_op_rotlv_vec:
+        t1 = tcg_temp_new_vec(type);
+        tcg_gen_dupi_vec(vece, t1, 8 << vece);
+        tcg_gen_sub_vec(vece, t1, v2, t1);
+        /* Right shifts are negative left shifts for AArch64.  */
+        vec_gen_3(INDEX_op_shlv_vec, type, vece, tcgv_vec_arg(t1),
+                  tcgv_vec_arg(v1), tcgv_vec_arg(t1));
+        vec_gen_3(INDEX_op_shlv_vec, type, vece, tcgv_vec_arg(v0),
+                  tcgv_vec_arg(v1), tcgv_vec_arg(v2));
+        tcg_gen_or_vec(vece, v0, v0, t1);
+        tcg_temp_free_vec(t1);
+        break;
+
+    case INDEX_op_rotrv_vec:
+        t1 = tcg_temp_new_vec(type);
+        t2 = tcg_temp_new_vec(type);
+        tcg_gen_neg_vec(vece, t1, v2);
+        tcg_gen_dupi_vec(vece, t2, 8 << vece);
+        tcg_gen_add_vec(vece, t2, t1, t2);
+        /* Right shifts are negative left shifts for AArch64.  */
+        vec_gen_3(INDEX_op_shlv_vec, type, vece, tcgv_vec_arg(t1),
+                  tcgv_vec_arg(v1), tcgv_vec_arg(t1));
+        vec_gen_3(INDEX_op_shlv_vec, type, vece, tcgv_vec_arg(t2),
+                  tcgv_vec_arg(v1), tcgv_vec_arg(t2));
+        tcg_gen_or_vec(vece, v0, t1, t2);
+        tcg_temp_free_vec(t1);
+        tcg_temp_free_vec(t2);
+        break;
+
     default:
         g_assert_not_reached();
     }
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op)
     static const TCGTargetOpDef lZ_l = { .args_ct_str = { "lZ", "l" } };
     static const TCGTargetOpDef r_r_r = { .args_ct_str = { "r", "r", "r" } };
     static const TCGTargetOpDef w_w_w = { .args_ct_str = { "w", "w", "w" } };
+    static const TCGTargetOpDef w_0_w = { .args_ct_str = { "w", "0", "w" } };
     static const TCGTargetOpDef w_w_wO = { .args_ct_str = { "w", "w", "wO" } };
     static const TCGTargetOpDef w_w_wN = { .args_ct_str = { "w", "w", "wN" } };
     static const TCGTargetOpDef w_w_wZ = { .args_ct_str = { "w", "w", "wZ" } };
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op)
         return &w_w_wZ;
     case INDEX_op_bitsel_vec:
         return &w_w_w_w;
+    case INDEX_op_aa64_sli_vec:
+        return &w_0_w;
 
     default:
         return NULL;
-- 
2.25.1

We already had support for rotlv, using a target-specific opcode;
convert to use the generic opcode.  Handle rotrv via simple negation.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/ppc/tcg-target.h     |  2 +-
 tcg/ppc/tcg-target.opc.h |  1 -
 tcg/ppc/tcg-target.inc.c | 23 +++++++++++++++++++----
 3 files changed, 20 insertions(+), 6 deletions(-)

diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/ppc/tcg-target.h
+++ b/tcg/ppc/tcg-target.h
@@ -XXX,XX +XXX,XX @@ extern bool have_vsx;
 #define TCG_TARGET_HAS_abs_vec          0
 #define TCG_TARGET_HAS_roti_vec         0
 #define TCG_TARGET_HAS_rots_vec         0
-#define TCG_TARGET_HAS_rotv_vec         0
+#define TCG_TARGET_HAS_rotv_vec         1
 #define TCG_TARGET_HAS_shi_vec          0
 #define TCG_TARGET_HAS_shs_vec          0
 #define TCG_TARGET_HAS_shv_vec          1
diff --git a/tcg/ppc/tcg-target.opc.h b/tcg/ppc/tcg-target.opc.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/ppc/tcg-target.opc.h
+++ b/tcg/ppc/tcg-target.opc.h
@@ -XXX,XX +XXX,XX @@ DEF(ppc_msum_vec, 1, 3, 0, IMPLVEC)
 DEF(ppc_muleu_vec, 1, 2, 0, IMPLVEC)
 DEF(ppc_mulou_vec, 1, 2, 0, IMPLVEC)
 DEF(ppc_pkum_vec, 1, 2, 0, IMPLVEC)
-DEF(ppc_rotl_vec, 1, 2, 0, IMPLVEC)
diff --git a/tcg/ppc/tcg-target.inc.c b/tcg/ppc/tcg-target.inc.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/ppc/tcg-target.inc.c
+++ b/tcg/ppc/tcg-target.inc.c
@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
     case INDEX_op_shlv_vec:
     case INDEX_op_shrv_vec:
     case INDEX_op_sarv_vec:
+    case INDEX_op_rotlv_vec:
         return vece <= MO_32 || have_isa_2_07;
     case INDEX_op_ssadd_vec:
     case INDEX_op_sssub_vec:
@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
     case INDEX_op_shli_vec:
     case INDEX_op_shri_vec:
     case INDEX_op_sari_vec:
+    case INDEX_op_rotli_vec:
         return vece <= MO_32 || have_isa_2_07 ? -1 : 0;
     case INDEX_op_neg_vec:
         return vece >= MO_32 && have_isa_3_00;
@@ -XXX,XX +XXX,XX @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
         return 0;
     case INDEX_op_bitsel_vec:
         return have_vsx;
+    case INDEX_op_rotrv_vec:
+        return -1;
     default:
         return 0;
     }
@@ -XXX,XX +XXX,XX @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
     case INDEX_op_ppc_pkum_vec:
         insn = pkum_op[vece];
         break;
-    case INDEX_op_ppc_rotl_vec:
+    case INDEX_op_rotlv_vec:
         insn = rotl_op[vece];
         break;
     case INDEX_op_ppc_msum_vec:
@@ -XXX,XX +XXX,XX @@ static void expand_vec_mul(TCGType type, unsigned vece, TCGv_vec v0,
         t3 = tcg_temp_new_vec(type);
         t4 = tcg_temp_new_vec(type);
         tcg_gen_dupi_vec(MO_8, t4, -16);
-        vec_gen_3(INDEX_op_ppc_rotl_vec, type, MO_32, tcgv_vec_arg(t1),
+        vec_gen_3(INDEX_op_rotlv_vec, type, MO_32, tcgv_vec_arg(t1),
                   tcgv_vec_arg(v2), tcgv_vec_arg(t4));
         vec_gen_3(INDEX_op_ppc_mulou_vec, type, MO_16, tcgv_vec_arg(t2),
                   tcgv_vec_arg(v1), tcgv_vec_arg(v2));
@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
                        TCGArg a0, ...)
 {
     va_list va;
-    TCGv_vec v0, v1, v2;
+    TCGv_vec v0, v1, v2, t0;
     TCGArg a2;
 
     va_start(va, a0);
@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
     case INDEX_op_sari_vec:
         expand_vec_shi(type, vece, v0, v1, a2, INDEX_op_sarv_vec);
         break;
+    case INDEX_op_rotli_vec:
+        expand_vec_shi(type, vece, v0, v1, a2, INDEX_op_rotlv_vec);
+        break;
     case INDEX_op_cmp_vec:
         v2 = temp_tcgv_vec(arg_temp(a2));
         expand_vec_cmp(type, vece, v0, v1, v2, va_arg(va, TCGArg));
@@ -XXX,XX +XXX,XX @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
         v2 = temp_tcgv_vec(arg_temp(a2));
         expand_vec_mul(type, vece, v0, v1, v2);
         break;
+    case INDEX_op_rotlv_vec:
+        v2 = temp_tcgv_vec(arg_temp(a2));
+        t0 = tcg_temp_new_vec(type);
+        tcg_gen_neg_vec(vece, t0, v2);
+        tcg_gen_rotlv_vec(vece, v0, v1, t0);
+        tcg_temp_free_vec(t0);
+        break;
     default:
         g_assert_not_reached();
     }
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op)
     case INDEX_op_shlv_vec:
     case INDEX_op_shrv_vec:
     case INDEX_op_sarv_vec:
+    case INDEX_op_rotlv_vec:
+    case INDEX_op_rotrv_vec:
     case INDEX_op_ppc_mrgh_vec:
     case INDEX_op_ppc_mrgl_vec:
     case INDEX_op_ppc_muleu_vec:
     case INDEX_op_ppc_mulou_vec:
     case INDEX_op_ppc_pkum_vec:
-    case INDEX_op_ppc_rotl_vec:
     case INDEX_op_dup2_vec:
         return &v_v_v;
     case INDEX_op_not_vec:
-- 
2.25.1

Acked-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/ppc/helper.h                 |  4 ----
 target/ppc/int_helper.c             | 17 -----------------
 target/ppc/translate/vmx-impl.inc.c |  8 ++++----
 3 files changed, 4 insertions(+), 25 deletions(-)

diff --git a/target/ppc/helper.h b/target/ppc/helper.h
index XXXXXXX..XXXXXXX 100644
--- a/target/ppc/helper.h
+++ b/target/ppc/helper.h
@@ -XXX,XX +XXX,XX @@ DEF_HELPER_3(vsubuqm, void, avr, avr, avr)
 DEF_HELPER_4(vsubecuq, void, avr, avr, avr, avr)
 DEF_HELPER_4(vsubeuqm, void, avr, avr, avr, avr)
 DEF_HELPER_3(vsubcuq, void, avr, avr, avr)
-DEF_HELPER_3(vrlb, void, avr, avr, avr)
-DEF_HELPER_3(vrlh, void, avr, avr, avr)
-DEF_HELPER_3(vrlw, void, avr, avr, avr)
-DEF_HELPER_3(vrld, void, avr, avr, avr)
 DEF_HELPER_4(vsldoi, void, avr, avr, avr, i32)
 DEF_HELPER_3(vextractub, void, avr, avr, i32)
 DEF_HELPER_3(vextractuh, void, avr, avr, i32)
diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c
index XXXXXXX..XXXXXXX 100644
--- a/target/ppc/int_helper.c
+++ b/target/ppc/int_helper.c
@@ -XXX,XX +XXX,XX @@ VRFI(p, float_round_up)
 VRFI(z, float_round_to_zero)
 #undef VRFI
 
-#define VROTATE(suffix, element, mask)                                  \
-    void helper_vrl##suffix(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b)   \
-    {                                                                   \
-        int i;                                                          \
-                                                                        \
-        for (i = 0; i < ARRAY_SIZE(r->element); i++) {                  \
-            unsigned int shift = b->element[i] & mask;                  \
-            r->element[i] = (a->element[i] << shift) |                  \
-                (a->element[i] >> (sizeof(a->element[0]) * 8 - shift)); \
-        }                                                               \
-    }
-VROTATE(b, u8, 0x7)
-VROTATE(h, u16, 0xF)
-VROTATE(w, u32, 0x1F)
-VROTATE(d, u64, 0x3F)
-#undef VROTATE
-
 void helper_vrsqrtefp(CPUPPCState *env, ppc_avr_t *r, ppc_avr_t *b)
 {
     int i;
diff --git a/target/ppc/translate/vmx-impl.inc.c b/target/ppc/translate/vmx-impl.inc.c
index XXXXXXX..XXXXXXX 100644
--- a/target/ppc/translate/vmx-impl.inc.c
+++ b/target/ppc/translate/vmx-impl.inc.c
@@ -XXX,XX +XXX,XX @@ GEN_VXFORM3(vsubeuqm, 31, 0);
 GEN_VXFORM3(vsubecuq, 31, 0);
 GEN_VXFORM_DUAL(vsubeuqm, PPC_NONE, PPC2_ALTIVEC_207, \
             vsubecuq, PPC_NONE, PPC2_ALTIVEC_207)
-GEN_VXFORM(vrlb, 2, 0);
-GEN_VXFORM(vrlh, 2, 1);
-GEN_VXFORM(vrlw, 2, 2);
+GEN_VXFORM_V(vrlb, MO_8, tcg_gen_gvec_rotlv, 2, 0);
+GEN_VXFORM_V(vrlh, MO_16, tcg_gen_gvec_rotlv, 2, 1);
+GEN_VXFORM_V(vrlw, MO_32, tcg_gen_gvec_rotlv, 2, 2);
 GEN_VXFORM(vrlwmi, 2, 2);
 GEN_VXFORM_DUAL(vrlw, PPC_ALTIVEC, PPC_NONE, \
                 vrlwmi, PPC_NONE, PPC2_ISA300)
-GEN_VXFORM(vrld, 2, 3);
+GEN_VXFORM_V(vrld, MO_64, tcg_gen_gvec_rotlv, 2, 3);
 GEN_VXFORM(vrldmi, 2, 3);
 GEN_VXFORM_DUAL(vrld, PPC_NONE, PPC2_ALTIVEC_207, \
                 vrldmi, PPC_NONE, PPC2_ISA300)
-- 
2.25.1

Merge VERLL and VERLLV into op_vesv and op_ves, alongside
all of the other vector shift operations.

Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 target/s390x/helper.h           |  4 --
 target/s390x/translate_vx.inc.c | 66 +++++----------------------------
 target/s390x/vec_int_helper.c   | 31 ----------------
 target/s390x/insn-data.def      |  4 +-
 4 files changed, 11 insertions(+), 94 deletions(-)

diff --git a/target/s390x/helper.h b/target/s390x/helper.h
index XXXXXXX..XXXXXXX 100644
--- a/target/s390x/helper.h
+++ b/target/s390x/helper.h
@@ -XXX,XX +XXX,XX @@ DEF_HELPER_FLAGS_4(gvec_vmlo16, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
 DEF_HELPER_FLAGS_4(gvec_vmlo32, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
 DEF_HELPER_FLAGS_3(gvec_vpopct8, TCG_CALL_NO_RWG, void, ptr, cptr, i32)
 DEF_HELPER_FLAGS_3(gvec_vpopct16, TCG_CALL_NO_RWG, void, ptr, cptr, i32)
-DEF_HELPER_FLAGS_4(gvec_verllv8, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
-DEF_HELPER_FLAGS_4(gvec_verllv16, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
-DEF_HELPER_FLAGS_4(gvec_verll8, TCG_CALL_NO_RWG, void, ptr, cptr, i64, i32)
-DEF_HELPER_FLAGS_4(gvec_verll16, TCG_CALL_NO_RWG, void, ptr, cptr, i64, i32)
 DEF_HELPER_FLAGS_4(gvec_verim8, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
 DEF_HELPER_FLAGS_4(gvec_verim16, TCG_CALL_NO_RWG, void, ptr, cptr, cptr, i32)
 DEF_HELPER_FLAGS_4(gvec_vsl, TCG_CALL_NO_RWG, void, ptr, cptr, i64, i32)
diff --git a/target/s390x/translate_vx.inc.c b/target/s390x/translate_vx.inc.c
index XXXXXXX..XXXXXXX 100644
--- a/target/s390x/translate_vx.inc.c
+++ b/target/s390x/translate_vx.inc.c
@@ -XXX,XX +XXX,XX @@ static DisasJumpType op_vpopct(DisasContext *s, DisasOps *o)
     return DISAS_NEXT;
 }
 
-static void gen_rll_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b)
-{
-    TCGv_i32 t0 = tcg_temp_new_i32();
-
-    tcg_gen_andi_i32(t0, b, 31);
-    tcg_gen_rotl_i32(d, a, t0);
-    tcg_temp_free_i32(t0);
-}
-
-static void gen_rll_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b)
-{
-    TCGv_i64 t0 = tcg_temp_new_i64();
-
-    tcg_gen_andi_i64(t0, b, 63);
-    tcg_gen_rotl_i64(d, a, t0);
-    tcg_temp_free_i64(t0);
-}
-
-static DisasJumpType op_verllv(DisasContext *s, DisasOps *o)
-{
-    const uint8_t es = get_field(s, m4);
-    static const GVecGen3 g[4] = {
-        { .fno = gen_helper_gvec_verllv8, },
-        { .fno = gen_helper_gvec_verllv16, },
-        { .fni4 = gen_rll_i32, },
-        { .fni8 = gen_rll_i64, },
-    };
-
-    if (es > ES_64) {
-        gen_program_exception(s, PGM_SPECIFICATION);
-        return DISAS_NORETURN;
-    }
-
-    gen_gvec_3(get_field(s, v1), get_field(s, v2),
-               get_field(s, v3), &g[es]);
-    return DISAS_NEXT;
-}
-
-static DisasJumpType op_verll(DisasContext *s, DisasOps *o)
-{
-    const uint8_t es = get_field(s, m4);
-    static const GVecGen2s g[4] = {
-        { .fno = gen_helper_gvec_verll8, },
-        { .fno = gen_helper_gvec_verll16, },
-        { .fni4 = gen_rll_i32, },
-        { .fni8 = gen_rll_i64, },
-    };
-
-    if (es > ES_64) {
-        gen_program_exception(s, PGM_SPECIFICATION);
-        return DISAS_NORETURN;
-    }
-    gen_gvec_2s(get_field(s, v1), get_field(s, v3), o->addr1,
-                &g[es]);
-    return DISAS_NEXT;
-}
-
 static void gen_rim_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b, int32_t c)
 {
     TCGv_i32 t = tcg_temp_new_i32();
@@ -XXX,XX +XXX,XX @@ static DisasJumpType op_vesv(DisasContext *s, DisasOps *o)
     case 0x70:
         gen_gvec_fn_3(shlv, es, v1, v2, v3);
         break;
+    case 0x73:
+        gen_gvec_fn_3(rotlv, es, v1, v2, v3);
+        break;
     case 0x7a:
         gen_gvec_fn_3(sarv, es, v1, v2, v3);
         break;
@@ -XXX,XX +XXX,XX @@ static DisasJumpType op_ves(DisasContext *s, DisasOps *o)
         case 0x30:
             gen_gvec_fn_2i(shli, es, v1, v3, d2);
             break;
+        case 0x33:
+            gen_gvec_fn_2i(rotli, es, v1, v3, d2);
+            break;
         case 0x3a:
             gen_gvec_fn_2i(sari, es, v1, v3, d2);
             break;
@@ -XXX,XX +XXX,XX @@ static DisasJumpType op_ves(DisasContext *s, DisasOps *o)
         case 0x30:
             gen_gvec_fn_2s(shls, es, v1, v3, shift);
             break;
+        case 0x33:
+            gen_gvec_fn_2s(rotls, es, v1, v3, shift);
+            break;
         case 0x3a:
             gen_gvec_fn_2s(sars, es, v1, v3, shift);
             break;
diff --git a/target/s390x/vec_int_helper.c b/target/s390x/vec_int_helper.c
index XXXXXXX..XXXXXXX 100644
--- a/target/s390x/vec_int_helper.c
+++ b/target/s390x/vec_int_helper.c
@@ -XXX,XX +XXX,XX @@ void HELPER(gvec_vpopct##BITS)(void *v1, const void *v2, uint32_t desc)        \
 DEF_VPOPCT(8)
 DEF_VPOPCT(16)
 
-#define DEF_VERLLV(BITS)                                                       \
-void HELPER(gvec_verllv##BITS)(void *v1, const void *v2, const void *v3,       \
-                               uint32_t desc)                                  \
-{                                                                              \
-    int i;                                                                     \
-                                                                               \
-    for (i = 0; i < (128 / BITS); i++) {                                       \
-        const uint##BITS##_t a = s390_vec_read_element##BITS(v2, i);           \
-        const uint##BITS##_t b = s390_vec_read_element##BITS(v3, i);           \
-                                                                               \
-        s390_vec_write_element##BITS(v1, i, rol##BITS(a, b));                  \
-    }                                                                          \
-}
-DEF_VERLLV(8)
-DEF_VERLLV(16)
-
-#define DEF_VERLL(BITS)                                                        \
-void HELPER(gvec_verll##BITS)(void *v1, const void *v2, uint64_t count,        \
-                              uint32_t desc)                                   \
-{                                                                              \
-    int i;                                                                     \
-                                                                               \
-    for (i = 0; i < (128 / BITS); i++) {                                       \
-        const uint##BITS##_t a = s390_vec_read_element##BITS(v2, i);           \
-                                                                               \
-        s390_vec_write_element##BITS(v1, i, rol##BITS(a, count));              \
-    }                                                                          \
-}
-DEF_VERLL(8)
-DEF_VERLL(16)
-
 #define DEF_VERIM(BITS)                                                        \
 void HELPER(gvec_verim##BITS)(void *v1, const void *v2, const void *v3,        \
                               uint32_t desc)                                   \
diff --git a/target/s390x/insn-data.def b/target/s390x/insn-data.def
index XXXXXXX..XXXXXXX 100644
--- a/target/s390x/insn-data.def
+++ b/target/s390x/insn-data.def
@@ -XXX,XX +XXX,XX @@
 /* VECTOR POPULATION COUNT */
     F(0xe750, VPOPCT,  VRR_a, V,   0, 0, 0, 0, vpopct, 0, IF_VEC)
 /* VECTOR ELEMENT ROTATE LEFT LOGICAL */
-    F(0xe773, VERLLV,  VRR_c, V,   0, 0, 0, 0, verllv, 0, IF_VEC)
-    F(0xe733, VERLL,   VRS_a, V,   la2, 0, 0, 0, verll, 0, IF_VEC)
+    F(0xe773, VERLLV,  VRR_c, V,   0, 0, 0, 0, vesv, 0, IF_VEC)
+    F(0xe733, VERLL,   VRS_a, V,   la2, 0, 0, 0, ves, 0, IF_VEC)
 /* VECTOR ELEMENT ROTATE AND INSERT UNDER MASK */
     F(0xe772, VERIM,   VRI_d, V,   0, 0, 0, 0, verim, 0, IF_VEC)
 /* VECTOR ELEMENT SHIFT LEFT */
-- 
2.25.1

If the output of the move is dead, then the last use is in
the store.  If we propagate the input to the store, then we
can remove the move opcode entirely.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/tcg.c | 78 +++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 56 insertions(+), 22 deletions(-)

diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static bool liveness_pass_2(TCGContext *s)
         }
 
         /* Outputs become available.  */
-        for (i = 0; i < nb_oargs; i++) {
-            arg_ts = arg_temp(op->args[i]);
+        if (opc == INDEX_op_mov_i32 || opc == INDEX_op_mov_i64) {
+            arg_ts = arg_temp(op->args[0]);
             dir_ts = arg_ts->state_ptr;
-            if (!dir_ts) {
-                continue;
+            if (dir_ts) {
+                op->args[0] = temp_arg(dir_ts);
+                changes = true;
+
+                /* The output is now live and modified.  */
+                arg_ts->state = 0;
+
+                if (NEED_SYNC_ARG(0)) {
+                    TCGOpcode sopc = (arg_ts->type == TCG_TYPE_I32
+                                      ? INDEX_op_st_i32
+                                      : INDEX_op_st_i64);
+                    TCGOp *sop = tcg_op_insert_after(s, op, sopc);
+                    TCGTemp *out_ts = dir_ts;
+
+                    if (IS_DEAD_ARG(0)) {
+                        out_ts = arg_temp(op->args[1]);
+                        arg_ts->state = TS_DEAD;
+                        tcg_op_remove(s, op);
+                    } else {
+                        arg_ts->state = TS_MEM;
+                    }
+
+                    sop->args[0] = temp_arg(out_ts);
+                    sop->args[1] = temp_arg(arg_ts->mem_base);
+                    sop->args[2] = arg_ts->mem_offset;
+                } else {
+                    tcg_debug_assert(!IS_DEAD_ARG(0));
+                }
             }
-            op->args[i] = temp_arg(dir_ts);
-            changes = true;
+        } else {
+            for (i = 0; i < nb_oargs; i++) {
+                arg_ts = arg_temp(op->args[i]);
+                dir_ts = arg_ts->state_ptr;
+                if (!dir_ts) {
+                    continue;
+                }
+                op->args[i] = temp_arg(dir_ts);
+                changes = true;
 
-            /* The output is now live and modified.  */
-            arg_ts->state = 0;
+                /* The output is now live and modified.  */
+                arg_ts->state = 0;
 
-            /* Sync outputs upon their last write.  */
-            if (NEED_SYNC_ARG(i)) {
-                TCGOpcode sopc = (arg_ts->type == TCG_TYPE_I32
-                                  ? INDEX_op_st_i32
-                                  : INDEX_op_st_i64);
-                TCGOp *sop = tcg_op_insert_after(s, op, sopc);
+                /* Sync outputs upon their last write.  */
+                if (NEED_SYNC_ARG(i)) {
+                    TCGOpcode sopc = (arg_ts->type == TCG_TYPE_I32
+                                      ? INDEX_op_st_i32
+                                      : INDEX_op_st_i64);
+                    TCGOp *sop = tcg_op_insert_after(s, op, sopc);
 
-                sop->args[0] = temp_arg(dir_ts);
-                sop->args[1] = temp_arg(arg_ts->mem_base);
-                sop->args[2] = arg_ts->mem_offset;
+                    sop->args[0] = temp_arg(dir_ts);
+                    sop->args[1] = temp_arg(arg_ts->mem_base);
+                    sop->args[2] = arg_ts->mem_offset;
 
-                arg_ts->state = TS_MEM;
-            }
-            /* Drop outputs that are dead.  */
-            if (IS_DEAD_ARG(i)) {
-                arg_ts->state = TS_DEAD;
+                    arg_ts->state = TS_MEM;
+                }
+                /* Drop outputs that are dead.  */
+                if (IS_DEAD_ARG(i)) {
+                    arg_ts->state = TS_DEAD;
+                }
             }
         }
     }
-- 
2.25.1

From: Nick Hudson <skrll@netbsd.org>

Fix building on NetBSD/arm by extracting the FSR value from the
correct siginfo_t field.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Nick Hudson <skrll@netbsd.org>
Message-Id: <20200516154147.24842-1-skrll@netbsd.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/user-exec.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo,
 
 #if defined(__NetBSD__)
 #include <ucontext.h>
+#include <sys/siginfo.h>
 #endif
 
 int cpu_signal_handler(int host_signum, void *pinfo,
@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo,
     siginfo_t *info = pinfo;
 #if defined(__NetBSD__)
     ucontext_t *uc = puc;
+    siginfo_t *si = pinfo;
 #else
     ucontext_t *uc = puc;
 #endif
     unsigned long pc;
+    uint32_t fsr;
     int is_write;
 
 #if defined(__NetBSD__)
@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo,
     pc = uc->uc_mcontext.arm_pc;
 #endif
 
-    /* error_code is the FSR value, in which bit 11 is WnR (assuming a v6 or
-     * later processor; on v5 we will always report this as a read).
+#ifdef __NetBSD__
+    fsr = si->si_trap;
+#else
+    fsr = uc->uc_mcontext.error_code;
+#endif
+    /*
+     * In the FSR, bit 11 is WnR, assuming a v6 or
+     * later processor.  On v5 we will always report
+     * this as a read, which will fail later.
      */
-    is_write = extract32(uc->uc_mcontext.error_code, 11, 1);
+    is_write = extract32(fsr, 11, 1);
     return handle_cpu_signal(pc, info, is_write, &uc->uc_sigmask);
 }
 
-- 
2.25.1

From: Nick Hudson <skrll@netbsd.org>

Fix qemu build on NetBSD/evbarm-aarch64 by providing a NetBSD specific
cpu_signal_handler.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Nick Hudson <skrll@netbsd.org>
Message-Id: <20200517101529.5367-1-skrll@netbsd.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/user-exec.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo,
 
 #elif defined(__aarch64__)
 
+#if defined(__NetBSD__)
+
+#include <ucontext.h>
+#include <sys/siginfo.h>
+
+int cpu_signal_handler(int host_signum, void *pinfo, void *puc)
+{
+    ucontext_t *uc = puc;
+    siginfo_t *si = pinfo;
+    unsigned long pc;
+    int is_write;
+    uint32_t esr;
+
+    pc = uc->uc_mcontext.__gregs[_REG_PC];
+    esr = si->si_trap;
+
+    /*
+     * siginfo_t::si_trap is the ESR value, for data aborts ESR.EC
+     * is 0b10010x: then bit 6 is the WnR bit
+     */
+    is_write = extract32(esr, 27, 5) == 0x12 && extract32(esr, 6, 1) == 1;
+    return handle_cpu_signal(pc, si, is_write, &uc->uc_sigmask);
+}
+
+#else
+
 #ifndef ESR_MAGIC
 /* Pre-3.16 kernel headers don't have these, so provide fallback definitions */
 #define ESR_MAGIC 0x45535201
@@ -XXX,XX +XXX,XX @@ int cpu_signal_handler(int host_signum, void *pinfo, void *puc)
     }
     return handle_cpu_signal(pc, info, is_write, &uc->uc_sigmask);
 }
+#endif
 
 #elif defined(__s390__)
 
-- 
2.25.1

The following changes since commit d1181d29370a4318a9f11ea92065bea6bb159f83:

Merge tag 'pull-nbd-2023-07-19' of https://repo.or.cz/qemu/ericb into staging (2023-07-20 09:54:07 +0100)

are available in the Git repository at:

https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20230724

for you to fetch changes up to 32b120394c578bc824f1db4835b3bffbeca88fae:

accel/tcg: Fix type of 'last' for pageflags_{find,next} (2023-07-24 09:48:49 +0100)

----------------------------------------------------------------
accel/tcg: Zero-pad vaddr in tlb debug output
accel/tcg: Fix type of 'last' for pageflags_{find,next}
accel/tcg: Fix sense of read-only probes in ldst_atomicity
accel/tcg: Take mmap_lock in load_atomic*_or_exit
tcg: Add earlyclobber to op_add2 for x86 and s390x
tcg/ppc: Fix race in goto_tb implementation

----------------------------------------------------------------
Anton Johansson (1):
      accel/tcg: Zero-pad vaddr in tlb_debug output

Ilya Leoshkevich (1):
      tcg/{i386, s390x}: Add earlyclobber to the op_add2's first output

Jordan Niethe (1):
      tcg/ppc: Fix race in goto_tb implementation

Luca Bonissi (1):
      accel/tcg: Fix type of 'last' for pageflags_{find,next}

Richard Henderson (3):
      include/exec: Add WITH_MMAP_LOCK_GUARD
      accel/tcg: Fix sense of read-only probes in ldst_atomicity
      accel/tcg: Take mmap_lock in load_atomic*_or_exit

From: Jordan Niethe <jniethe5@gmail.com>

Commit 20b6643324 ("tcg/ppc: Reorg goto_tb implementation") modified
goto_tb to ensure only a single instruction was patched to prevent
incorrect behavior if a thread was in the middle of multiple
instructions when they were replaced. However this introduced a race
between loading the jmp target into TCG_REG_TB and patching and
executing the direct branch.

The relevant part of the goto_tb implementation:

ld TCG_REG_TB, TARGET_ADDR_LOCATION(TCG_REG_TB)
  patch_location:
    mtctr TCG_REG_TB
    bctr

tb_target_set_jmp_target() will replace 'patch_location' with a direct
branch if the target is in range. The direct branch now relies on
TCG_REG_TB being set up correctly by the ld. Prior to this commit
multiple instructions were patched in for the direct branch case; these
instructions would initialize TCG_REG_TB to the same value as the branch
target.

Imagine the following sequence:

1) Thread A is executing the goto_tb sequence and loads the jmp
   target into TCG_REG_TB.

2) Thread B updates the jmp target address and calls
   tb_target_set_jmp_target(). This patches a new direct branch into the
   goto_tb sequence.

3) Thread A executes the newly patched direct branch. The value in
   TCG_REG_TB still contains the old jmp target.

TCG_REG_TB MUST contain the translation block's tc.ptr. Execution will
eventually crash after performing memory accesses generated from a
faulty value in TCG_REG_TB.

This presents as segfaults or illegal instruction exceptions.

Do not revert commit 20b6643324 as it did fix a different race
condition. Instead remove the direct branch optimization and always use
indirect branches.

The direct branch optimization can be re-added later with a race free
sequence.

Fixes: 20b6643324 ("tcg/ppc: Reorg goto_tb implementation")
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1726
Reported-by: Anushree Mathur <anushree.mathur@linux.vnet.ibm.com>
Tested-by: Anushree Mathur <anushree.mathur@linux.vnet.ibm.com>
Tested-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Co-developed-by: Benjamin Gray <bgray@linux.ibm.com>
Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
Message-Id: <20230717093001.13167-1-jniethe5@gmail.com>
---
 tcg/ppc/tcg-target.c.inc | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc
index XXXXXXX..XXXXXXX 100644
--- a/tcg/ppc/tcg-target.c.inc
+++ b/tcg/ppc/tcg-target.c.inc
@@ -XXX,XX +XXX,XX @@ static void tcg_out_goto_tb(TCGContext *s, int which)
         ptrdiff_t offset = tcg_tbrel_diff(s, (void *)ptr);
         tcg_out_mem_long(s, LD, LDX, TCG_REG_TB, TCG_REG_TB, offset);
     
-        /* Direct branch will be patched by tb_target_set_jmp_target. */
+        /* TODO: Use direct branches when possible. */
         set_jmp_insn_offset(s, which);
         tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR);
 
-        /* When branch is out of range, fall through to indirect. */
         tcg_out32(s, BCCTR | BO_ALWAYS);
 
         /* For the unlinked case, need to reset TCG_REG_TB.  */
@@ -XXX,XX +XXX,XX @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n,
     intptr_t diff = addr - jmp_rx;
     tcg_insn_unit insn;
 
+    if (USE_REG_TB) {
+        return;
+    }
+
     if (in_range_b(diff)) {
         insn = B | (diff & 0x3fffffc);
-    } else if (USE_REG_TB) {
-        insn = MTSPR | RS(TCG_REG_TB) | CTR;
     } else {
         insn = NOP;
     }
-- 
2.34.1

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 include/exec/exec-all.h | 10 ++++++++++
 bsd-user/mmap.c         |  1 +
 linux-user/mmap.c       |  1 +
 3 files changed, 12 insertions(+)

diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index XXXXXXX..XXXXXXX 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -XXX,XX +XXX,XX @@ void TSA_NO_TSA mmap_lock(void);
 void TSA_NO_TSA mmap_unlock(void);
 bool have_mmap_lock(void);
 
+static inline void mmap_unlock_guard(void *unused)
+{
+    mmap_unlock();
+}
+
+#define WITH_MMAP_LOCK_GUARD()                                            \
+    for (int _mmap_lock_iter __attribute__((cleanup(mmap_unlock_guard)))  \
+         = (mmap_lock(), 0); _mmap_lock_iter == 0; _mmap_lock_iter = 1)
+
 /**
  * adjust_signal_pc:
  * @pc: raw pc from the host signal ucontext_t.
@@ -XXX,XX +XXX,XX @@ G_NORETURN void cpu_loop_exit_sigbus(CPUState *cpu, target_ulong addr,
 #else
 static inline void mmap_lock(void) {}
 static inline void mmap_unlock(void) {}
+#define WITH_MMAP_LOCK_GUARD()
 
 void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length);
 void tlb_set_dirty(CPUState *cpu, vaddr addr);
diff --git a/bsd-user/mmap.c b/bsd-user/mmap.c
index XXXXXXX..XXXXXXX 100644
--- a/bsd-user/mmap.c
+++ b/bsd-user/mmap.c
@@ -XXX,XX +XXX,XX @@ void mmap_lock(void)
 
 void mmap_unlock(void)
 {
+    assert(mmap_lock_count > 0);
     if (--mmap_lock_count == 0) {
         pthread_mutex_unlock(&mmap_mutex);
     }
diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index XXXXXXX..XXXXXXX 100644
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -XXX,XX +XXX,XX @@ void mmap_lock(void)
 
 void mmap_unlock(void)
 {
+    assert(mmap_lock_count > 0);
     if (--mmap_lock_count == 0) {
         pthread_mutex_unlock(&mmap_mutex);
     }
-- 
2.34.1

In the initial commit, cdfac37be0d, the sense of the test is incorrect,
as the -1/0 return was confusing.  In bef6f008b981, we mechanically
invert all callers while changing to false/true return, preserving the
incorrectness of the test.

Now that the return sense is sane, it's easy to see that if !write,
then the page is not modifiable (i.e. most likely read-only, with
PROT_NONE handled via SIGSEGV).

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/ldst_atomicity.c.inc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

For user-only, the probe for page writability may race with another
thread's mprotect.  Take the mmap_lock around the operation.  This
is still faster than the start/end_exclusive fallback.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/ldst_atomicity.c.inc | 32 ++++++++++++++++++--------------
 1 file changed, 18 insertions(+), 14 deletions(-)

diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/ldst_atomicity.c.inc
+++ b/accel/tcg/ldst_atomicity.c.inc
@@ -XXX,XX +XXX,XX @@ static uint64_t load_atomic8_or_exit(CPUArchState *env, uintptr_t ra, void *pv)
      * another process, because the fallback start_exclusive solution
      * provides no protection across processes.
      */
-    if (!page_check_range(h2g(pv), 8, PAGE_WRITE_ORG)) {
-        uint64_t *p = __builtin_assume_aligned(pv, 8);
-        return *p;
+    WITH_MMAP_LOCK_GUARD() {
+        if (!page_check_range(h2g(pv), 8, PAGE_WRITE_ORG)) {
+            uint64_t *p = __builtin_assume_aligned(pv, 8);
+            return *p;
+        }
     }
 #endif
 
@@ -XXX,XX +XXX,XX @@ static Int128 load_atomic16_or_exit(CPUArchState *env, uintptr_t ra, void *pv)
         return atomic16_read_ro(p);
     }
 
-#ifdef CONFIG_USER_ONLY
     /*
      * We can only use cmpxchg to emulate a load if the page is writable.
      * If the page is not writable, then assume the value is immutable
      * and requires no locking.  This ignores the case of MAP_SHARED with
      * another process, because the fallback start_exclusive solution
      * provides no protection across processes.
+     *
+     * In system mode all guest pages are writable.  For user mode,
+     * we must take mmap_lock so that the query remains valid until
+     * the write is complete -- tests/tcg/multiarch/munmap-pthread.c
+     * is an example that can race.
      */
-    if (!page_check_range(h2g(p), 16, PAGE_WRITE_ORG)) {
-        return *p;
-    }
+    WITH_MMAP_LOCK_GUARD() {
+#ifdef CONFIG_USER_ONLY
+        if (!page_check_range(h2g(p), 16, PAGE_WRITE_ORG)) {
+            return *p;
+        }
 #endif
-
-    /*
-     * In system mode all guest pages are writable, and for user-only
-     * we have just checked writability.  Try cmpxchg.
-     */
-    if (HAVE_ATOMIC128_RW) {
-        return atomic16_read_rw(p);
+        if (HAVE_ATOMIC128_RW) {
+            return atomic16_read_rw(p);
+        }
     }
 
     /* Ultimate fallback: re-execute in serial context. */
-- 
2.34.1

From: Ilya Leoshkevich <iii@linux.ibm.com>

i386 and s390x implementations of op_add2 require an earlyclobber,
which is currently missing. This breaks VCKSM in s390x guests. E.g., on
x86_64 the following op:

add2_i32 tmp2,tmp3,tmp2,tmp3,tmp3,tmp2   dead: 0 2 3 4 5  pref=none,0xffff

is translated to:

addl     %ebx, %r12d
    adcl     %r12d, %ebx

Introduce a new C_N1_O1_I4 constraint, and make sure that earlyclobber
of aliased outputs is honored.

Cc: qemu-stable@nongnu.org
Fixes: 82790a870992 ("tcg: Add markup for output requires new register")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230719221310.1968845-7-iii@linux.ibm.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 tcg/i386/tcg-target-con-set.h  | 5 ++++-
 tcg/s390x/tcg-target-con-set.h | 8 +++++---
 tcg/tcg.c                      | 8 +++++++-
 tcg/i386/tcg-target.c.inc      | 2 +-
 tcg/s390x/tcg-target.c.inc     | 4 ++--
 5 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/tcg/i386/tcg-target-con-set.h b/tcg/i386/tcg-target-con-set.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/i386/tcg-target-con-set.h
+++ b/tcg/i386/tcg-target-con-set.h
@@ -XXX,XX +XXX,XX @@
  *
  * C_N1_Im(...) defines a constraint set with 1 output and <m> inputs,
  * except that the output must use a new register.
+ *
+ * C_Nn_Om_Ik(...) defines a constraint set with <n + m> outputs and <k>
+ * inputs, except that the first <n> outputs must use new registers.
  */
 C_O0_I1(r)
 C_O0_I2(L, L)
@@ -XXX,XX +XXX,XX @@ C_O2_I1(r, r, L)
 C_O2_I2(a, d, a, r)
 C_O2_I2(r, r, L, L)
 C_O2_I3(a, d, 0, 1, r)
-C_O2_I4(r, r, 0, 1, re, re)
+C_N1_O1_I4(r, r, 0, 1, re, re)
diff --git a/tcg/s390x/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h
index XXXXXXX..XXXXXXX 100644
--- a/tcg/s390x/tcg-target-con-set.h
+++ b/tcg/s390x/tcg-target-con-set.h
@@ -XXX,XX +XXX,XX @@
  * C_On_Im(...) defines a constraint set with <n> outputs and <m> inputs.
  * Each operand should be a sequence of constraint letters as defined by
  * tcg-target-con-str.h; the constraint combination is inclusive or.
+ *
+ * C_Nn_Om_Ik(...) defines a constraint set with <n + m> outputs and <k>
+ * inputs, except that the first <n> outputs must use new registers.
  */
 C_O0_I1(r)
 C_O0_I2(r, r)
@@ -XXX,XX +XXX,XX @@ C_O2_I1(o, m, r)
 C_O2_I2(o, m, 0, r)
 C_O2_I2(o, m, r, r)
 C_O2_I3(o, m, 0, 1, r)
-C_O2_I4(r, r, 0, 1, rA, r)
-C_O2_I4(r, r, 0, 1, ri, r)
-C_O2_I4(r, r, 0, 1, r, r)
+C_N1_O1_I4(r, r, 0, 1, ri, r)
+C_N1_O1_I4(r, r, 0, 1, rA, r)
diff --git a/tcg/tcg.c b/tcg/tcg.c
index XXXXXXX..XXXXXXX 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -XXX,XX +XXX,XX @@ static void tcg_out_movext3(TCGContext *s, const TCGMovExtend *i1,
 #define C_O2_I2(O1, O2, I1, I2)         C_PFX4(c_o2_i2_, O1, O2, I1, I2),
 #define C_O2_I3(O1, O2, I1, I2, I3)     C_PFX5(c_o2_i3_, O1, O2, I1, I2, I3),
 #define C_O2_I4(O1, O2, I1, I2, I3, I4) C_PFX6(c_o2_i4_, O1, O2, I1, I2, I3, I4),
+#define C_N1_O1_I4(O1, O2, I1, I2, I3, I4) C_PFX6(c_n1_o1_i4_, O1, O2, I1, I2, I3, I4),
 
 typedef enum {
 #include "tcg-target-con-set.h"
@@ -XXX,XX +XXX,XX @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode);
 #undef C_O2_I2
 #undef C_O2_I3
 #undef C_O2_I4
+#undef C_N1_O1_I4
 
 /* Put all of the constraint sets into an array, indexed by the enum. */
 
@@ -XXX,XX +XXX,XX @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode);
 #define C_O2_I2(O1, O2, I1, I2)         { .args_ct_str = { #O1, #O2, #I1, #I2 } },
 #define C_O2_I3(O1, O2, I1, I2, I3)     { .args_ct_str = { #O1, #O2, #I1, #I2, #I3 } },
 #define C_O2_I4(O1, O2, I1, I2, I3, I4) { .args_ct_str = { #O1, #O2, #I1, #I2, #I3, #I4 } },
+#define C_N1_O1_I4(O1, O2, I1, I2, I3, I4) { .args_ct_str = { "&" #O1, #O2, #I1, #I2, #I3, #I4 } },
 
 static const TCGTargetOpDef constraint_sets[] = {
 #include "tcg-target-con-set.h"
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef constraint_sets[] = {
 #undef C_O2_I2
 #undef C_O2_I3
 #undef C_O2_I4
+#undef C_N1_O1_I4
 
 /* Expand the enumerator to be returned from tcg_target_op_def(). */
 
@@ -XXX,XX +XXX,XX @@ static const TCGTargetOpDef constraint_sets[] = {
 #define C_O2_I2(O1, O2, I1, I2)         C_PFX4(c_o2_i2_, O1, O2, I1, I2)
 #define C_O2_I3(O1, O2, I1, I2, I3)     C_PFX5(c_o2_i3_, O1, O2, I1, I2, I3)
 #define C_O2_I4(O1, O2, I1, I2, I3, I4) C_PFX6(c_o2_i4_, O1, O2, I1, I2, I3, I4)
+#define C_N1_O1_I4(O1, O2, I1, I2, I3, I4) C_PFX6(c_n1_o1_i4_, O1, O2, I1, I2, I3, I4)
 
 #include "tcg-target.c.inc"
 
@@ -XXX,XX +XXX,XX @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)
                  * dead after the instruction, we must allocate a new
                  * register and move it.
                  */
-                if (temp_readonly(ts) || !IS_DEAD_ARG(i)) {
+                if (temp_readonly(ts) || !IS_DEAD_ARG(i)
+                    || def->args_ct[arg_ct->alias_index].newreg) {
                     allocate_new_reg = true;
                 } else if (ts->val_type == TEMP_VAL_REG) {
                     /*
diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc
index XXXXXXX..XXXXXXX 100644
--- a/tcg/i386/tcg-target.c.inc
+++ b/tcg/i386/tcg-target.c.inc
@@ -XXX,XX +XXX,XX @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op)
     case INDEX_op_add2_i64:
     case INDEX_op_sub2_i32:
     case INDEX_op_sub2_i64:
-        return C_O2_I4(r, r, 0, 1, re, re);
+        return C_N1_O1_I4(r, r, 0, 1, re, re);
 
     case INDEX_op_ctz_i32:
     case INDEX_op_ctz_i64:
diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index XXXXXXX..XXXXXXX 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -XXX,XX +XXX,XX @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op)
 
     case INDEX_op_add2_i32:
     case INDEX_op_sub2_i32:
-        return C_O2_I4(r, r, 0, 1, ri, r);
+        return C_N1_O1_I4(r, r, 0, 1, ri, r);
 
     case INDEX_op_add2_i64:
     case INDEX_op_sub2_i64:
-        return C_O2_I4(r, r, 0, 1, rA, r);
+        return C_N1_O1_I4(r, r, 0, 1, rA, r);
 
     case INDEX_op_st_vec:
         return C_O0_I2(v, r);
-- 
2.34.1

From: Anton Johansson <anjo@rev.ng>

In replacing target_ulong with vaddr and TARGET_FMT_lx with VADDR_PRIx,
the zero-padding of TARGET_FMT_lx got lost.  Readd 16-wide zero-padding
for logging consistency.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Anton Johansson <anjo@rev.ng>
Message-Id: <20230713120746.26897-1-anjo@rev.ng>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/cputlb.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_page_locked(CPUArchState *env, int midx, vaddr page)
 
     /* Check if we need to flush due to large pages.  */
     if ((page & lp_mask) == lp_addr) {
-        tlb_debug("forcing full flush midx %d (%"
-                  VADDR_PRIx "/%" VADDR_PRIx ")\n",
+        tlb_debug("forcing full flush midx %d (%016"
+                  VADDR_PRIx "/%016" VADDR_PRIx ")\n",
                   midx, lp_addr, lp_mask);
         tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
     } else {
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_page_by_mmuidx_async_0(CPUState *cpu,
 
     assert_cpu_is_self(cpu);
 
-    tlb_debug("page addr: %" VADDR_PRIx " mmu_map:0x%x\n", addr, idxmap);
+    tlb_debug("page addr: %016" VADDR_PRIx " mmu_map:0x%x\n", addr, idxmap);
 
     qemu_spin_lock(&env_tlb(env)->c.lock);
     for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) {
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_page_by_mmuidx_async_2(CPUState *cpu,
 
 void tlb_flush_page_by_mmuidx(CPUState *cpu, vaddr addr, uint16_t idxmap)
 {
-    tlb_debug("addr: %" VADDR_PRIx " mmu_idx:%" PRIx16 "\n", addr, idxmap);
+    tlb_debug("addr: %016" VADDR_PRIx " mmu_idx:%" PRIx16 "\n", addr, idxmap);
 
     /* This should already be page aligned */
     addr &= TARGET_PAGE_MASK;
@@ -XXX,XX +XXX,XX @@ void tlb_flush_page(CPUState *cpu, vaddr addr)
 void tlb_flush_page_by_mmuidx_all_cpus(CPUState *src_cpu, vaddr addr,
                                        uint16_t idxmap)
 {
-    tlb_debug("addr: %" VADDR_PRIx " mmu_idx:%"PRIx16"\n", addr, idxmap);
+    tlb_debug("addr: %016" VADDR_PRIx " mmu_idx:%"PRIx16"\n", addr, idxmap);
 
     /* This should already be page aligned */
     addr &= TARGET_PAGE_MASK;
@@ -XXX,XX +XXX,XX @@ void tlb_flush_page_by_mmuidx_all_cpus_synced(CPUState *src_cpu,
                                               vaddr addr,
                                               uint16_t idxmap)
 {
-    tlb_debug("addr: %" VADDR_PRIx " mmu_idx:%"PRIx16"\n", addr, idxmap);
+    tlb_debug("addr: %016" VADDR_PRIx " mmu_idx:%"PRIx16"\n", addr, idxmap);
 
     /* This should already be page aligned */
     addr &= TARGET_PAGE_MASK;
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_locked(CPUArchState *env, int midx,
      */
     if (mask < f->mask || len > f->mask) {
         tlb_debug("forcing full flush midx %d ("
-                  "%" VADDR_PRIx "/%" VADDR_PRIx "+%" VADDR_PRIx ")\n",
+                  "%016" VADDR_PRIx "/%016" VADDR_PRIx "+%016" VADDR_PRIx ")\n",
                   midx, addr, mask, len);
         tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
         return;
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_locked(CPUArchState *env, int midx,
      */
     if (((addr + len - 1) & d->large_page_mask) == d->large_page_addr) {
         tlb_debug("forcing full flush midx %d ("
-                  "%" VADDR_PRIx "/%" VADDR_PRIx ")\n",
+                  "%016" VADDR_PRIx "/%016" VADDR_PRIx ")\n",
                   midx, d->large_page_addr, d->large_page_mask);
         tlb_flush_one_mmuidx_locked(env, midx, get_clock_realtime());
         return;
@@ -XXX,XX +XXX,XX @@ static void tlb_flush_range_by_mmuidx_async_0(CPUState *cpu,
 
     assert_cpu_is_self(cpu);
 
-    tlb_debug("range: %" VADDR_PRIx "/%u+%" VADDR_PRIx " mmu_map:0x%x\n",
+    tlb_debug("range: %016" VADDR_PRIx "/%u+%016" VADDR_PRIx " mmu_map:0x%x\n",
               d.addr, d.bits, d.len, d.idxmap);
 
     qemu_spin_lock(&env_tlb(env)->c.lock);
@@ -XXX,XX +XXX,XX @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx,
                                                 &xlat, &sz, full->attrs, &prot);
     assert(sz >= TARGET_PAGE_SIZE);
 
-    tlb_debug("vaddr=%" VADDR_PRIx " paddr=0x" HWADDR_FMT_plx
+    tlb_debug("vaddr=%016" VADDR_PRIx " paddr=0x" HWADDR_FMT_plx
               " prot=%x idx=%d\n",
               addr, full->phys_addr, prot, mmu_idx);
 
-- 
2.34.1

From: Luca Bonissi <qemu@bonslack.org>

These should match 'start' as target_ulong, not target_long.

On 32bit targets, the parameter was sign-extended to uint64_t,
so only the first mmap within the upper 2GB memory can succeed.

Signed-off-by: Luca Bonissi <qemu@bonslack.org>
Message-Id: <327460e2-0ebd-9edb-426b-1df80d16c32a@bonslack.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 accel/tcg/user-exec.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index XXXXXXX..XXXXXXX 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -XXX,XX +XXX,XX @@ typedef struct PageFlagsNode {
 
 static IntervalTreeRoot pageflags_root;
 
-static PageFlagsNode *pageflags_find(target_ulong start, target_long last)
+static PageFlagsNode *pageflags_find(target_ulong start, target_ulong last)
 {
     IntervalTreeNode *n;
 
@@ -XXX,XX +XXX,XX @@ static PageFlagsNode *pageflags_find(target_ulong start, target_long last)
 }
 
 static PageFlagsNode *pageflags_next(PageFlagsNode *p, target_ulong start,
-                                     target_long last)
+                                     target_ulong last)
 {
     IntervalTreeNode *n;
 
-- 
2.34.1