[Qemu-devel] [RFC PATCH v3 29/30] replay: improve replay performance

Pavel Dovgalyuk posted 30 patches 7 years, 9 months ago
There is a newer version of this series
[Qemu-devel] [RFC PATCH v3 29/30] replay: improve replay performance
Posted by Pavel Dovgalyuk 7 years, 9 months ago
This patch skips cpu_exec when there is no CPU code to execute in replay mode.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
Signed-off-by: Maria Klimushenkova <maria.klimushenkova@ispras.ru>
---
 cpus.c                  |   50 +++++++++++++++++++++++++----------------------
 include/sysemu/replay.h |    3 +++
 replay/replay.c         |   13 ++++++++++++
 3 files changed, 43 insertions(+), 23 deletions(-)

diff --git a/cpus.c b/cpus.c
index bafedfc..ef4c5b9 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1449,39 +1449,43 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg)
             cpu = first_cpu;
         }
 
-        while (cpu && !cpu->queued_work_first && !cpu->exit_request) {
+        if (!replay_has_checkpoint()) {
+            while (cpu && !cpu->queued_work_first && !cpu->exit_request) {
 
-            atomic_mb_set(&tcg_current_rr_cpu, cpu);
-            current_cpu = cpu;
+                atomic_mb_set(&tcg_current_rr_cpu, cpu);
+                current_cpu = cpu;
 
-            qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
-                              (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
+                qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
+                                  (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
 
-            if (cpu_can_run(cpu)) {
-                int r;
+                if (cpu_can_run(cpu)) {
+                    int r;
 
-                prepare_icount_for_run(cpu);
+                    prepare_icount_for_run(cpu);
 
-                r = tcg_cpu_exec(cpu);
+                    r = tcg_cpu_exec(cpu);
 
-                process_icount_data(cpu);
+                    process_icount_data(cpu);
 
-                if (r == EXCP_DEBUG) {
-                    cpu_handle_guest_debug(cpu);
-                    break;
-                } else if (r == EXCP_ATOMIC) {
-                    cpu_exec_step_atomic(cpu);
+                    if (r == EXCP_DEBUG) {
+                        cpu_handle_guest_debug(cpu);
+                        break;
+                    } else if (r == EXCP_ATOMIC) {
+                        cpu_exec_step_atomic(cpu);
+                        break;
+                    }
+                } else if (cpu->stop) {
+                    if (cpu->unplug) {
+                        cpu = CPU_NEXT(cpu);
+                    }
                     break;
                 }
-            } else if (cpu->stop) {
-                if (cpu->unplug) {
-                    cpu = CPU_NEXT(cpu);
-                }
-                break;
-            }
 
-            cpu = CPU_NEXT(cpu);
-        } /* while (cpu && !cpu->exit_request).. */
+                cpu = CPU_NEXT(cpu);
+            } /* while (cpu && !cpu->exit_request).. */
+        } else {
+            qemu_notify_event();
+        }
 
         /* Does not need atomic_mb_set because a spurious wakeup is okay.  */
         atomic_set(&tcg_current_rr_cpu, NULL);
diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h
index d026b28..44c1ff7 100644
--- a/include/sysemu/replay.h
+++ b/include/sysemu/replay.h
@@ -122,6 +122,9 @@ void replay_shutdown_request(ShutdownCause cause);
     Returns 0 in PLAY mode if checkpoint was not found.
     Returns 1 in all other cases. */
 bool replay_checkpoint(ReplayCheckpoint checkpoint);
+/*! Used to determine that checkpoint is pending.
+    Does not proceed to the next event in the log. */
+bool replay_has_checkpoint(void);
 
 /* Asynchronous events queue */
 
diff --git a/replay/replay.c b/replay/replay.c
index c9fd984..b9c496a 100644
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -219,6 +219,19 @@ out:
     return res;
 }
 
+bool replay_has_checkpoint(void)
+{
+    bool res = false;
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        replay_mutex_lock();
+        replay_account_executed_instructions();
+        res = EVENT_CHECKPOINT <= replay_state.data_kind
+            && replay_state.data_kind <= EVENT_CHECKPOINT_LAST;
+        replay_mutex_unlock();
+    }
+    return res;
+}
+
 void replay_init_locks(void)
 {
     replay_mutex_init();


Re: [Qemu-devel] [RFC PATCH v3 29/30] replay: improve replay performance
Posted by Paolo Bonzini 7 years, 9 months ago
On 11/01/2018 09:27, Pavel Dovgalyuk wrote:
> +        } else {
> +            qemu_notify_event();
> +        }

Before this patch, what would do the qemu_notify_event?

Thanks,

Paolo

>          /* Does not need atomic_mb_set because a spurious wakeup is okay.  */


Re: [Qemu-devel] [RFC PATCH v3 29/30] replay: improve replay performance
Posted by Pavel Dovgalyuk 7 years, 9 months ago
> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> On 11/01/2018 09:27, Pavel Dovgalyuk wrote:
> > +        } else {
> > +            qemu_notify_event();
> > +        }
> 
> Before this patch, what would do the qemu_notify_event?

We tried using qemu_notify_event to wake up the iothread and speedup the replay,
when there is no work for vcpu thread to do.

Pavel Dovgalyuk


Re: [Qemu-devel] [RFC PATCH v3 29/30] replay: improve replay performance
Posted by Paolo Bonzini 7 years, 9 months ago
On 11/01/2018 14:12, Pavel Dovgalyuk wrote:
>> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
>> On 11/01/2018 09:27, Pavel Dovgalyuk wrote:
>>> +        } else {
>>> +            qemu_notify_event();
>>> +        }
>>
>> Before this patch, what would do the qemu_notify_event?
> 
> We tried using qemu_notify_event to wake up the iothread and speedup the replay,
> when there is no work for vcpu thread to do.

But why doesn't the iothread wake up on its own?

Paolo

Re: [Qemu-devel] [RFC PATCH v3 29/30] replay: improve replay performance
Posted by Pavel Dovgalyuk 7 years, 9 months ago
> From: Paolo Bonzini [mailto:pbonzini@redhat.com]
> On 11/01/2018 14:12, Pavel Dovgalyuk wrote:
> >> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> >> On 11/01/2018 09:27, Pavel Dovgalyuk wrote:
> >>> +        } else {
> >>> +            qemu_notify_event();
> >>> +        }
> >>
> >> Before this patch, what would do the qemu_notify_event?
> >
> > We tried using qemu_notify_event to wake up the iothread and speedup the replay,
> > when there is no work for vcpu thread to do.
> 
> But why doesn't the iothread wake up on its own?

It will wake up by timeout. But with this change this happens sooner and replay goes faster.

Pavel Dovgalyuk


Re: [Qemu-devel] [RFC PATCH v3 29/30] replay: improve replay performance
Posted by Paolo Bonzini 7 years, 9 months ago
On 12/01/2018 07:13, Pavel Dovgalyuk wrote:
>> From: Paolo Bonzini [mailto:pbonzini@redhat.com]
>> On 11/01/2018 14:12, Pavel Dovgalyuk wrote:
>>>> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
>>>> On 11/01/2018 09:27, Pavel Dovgalyuk wrote:
>>>>> +        } else {
>>>>> +            qemu_notify_event();
>>>>> +        }
>>>>
>>>> Before this patch, what would do the qemu_notify_event?
>>>
>>> We tried using qemu_notify_event to wake up the iothread and speedup the replay,
>>> when there is no work for vcpu thread to do.
>>
>> But why doesn't the iothread wake up on its own?
> 
> It will wake up by timeout. But with this change this happens sooner and replay goes faster.

If you want replays to ignore timeouts, that should be done by ensuring
that qemu-timer.c passes a zero timeout to main-loop.c.  (It probably
should be optional, too, maybe via -icount sleep).

Paolo

Re: [Qemu-devel] [RFC PATCH v3 29/30] replay: improve replay performance
Posted by Pavel Dovgalyuk 7 years, 9 months ago
> From: Paolo Bonzini [mailto:pbonzini@redhat.com]
> On 12/01/2018 07:13, Pavel Dovgalyuk wrote:
> >> From: Paolo Bonzini [mailto:pbonzini@redhat.com]
> >> On 11/01/2018 14:12, Pavel Dovgalyuk wrote:
> >>>> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> >>>> On 11/01/2018 09:27, Pavel Dovgalyuk wrote:
> >>>>> +        } else {
> >>>>> +            qemu_notify_event();
> >>>>> +        }
> >>>>
> >>>> Before this patch, what would do the qemu_notify_event?
> >>>
> >>> We tried using qemu_notify_event to wake up the iothread and speedup the replay,
> >>> when there is no work for vcpu thread to do.
> >>
> >> But why doesn't the iothread wake up on its own?
> >
> > It will wake up by timeout. But with this change this happens sooner and replay goes faster.
> 
> If you want replays to ignore timeouts, that should be done by ensuring
> that qemu-timer.c passes a zero timeout to main-loop.c.  (It probably
> should be optional, too, maybe via -icount sleep).

Making it optional is a good one.
I'll exclude this patch from the series for working on it later.

Pavel Dovgalyuk