[v2] migration/postcopy: ensure preempt channel is ready before loading states

[PATCH v2] migration/postcopy: ensure preempt channel is ready before loading states

Posted by Wei Wang 1 year, 10 months ago

Before loading the guest states, ensure that the preempt channel has been
ready to use, as some of the states (e.g. via virtio_load) might trigger
page faults that will be handled through the preempt channel. So yield to
the main thread in the case that the channel create event hasn't been
dispatched.

Originally-by: Lei Wang <lei4.wang@intel.com>
Link: https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@intel.com/T/
Suggested-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Lei Wang <lei4.wang@intel.com>
Signed-off-by: Wei Wang <wei.w.wang@intel.com>
---
 migration/savevm.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/migration/savevm.c b/migration/savevm.c
index 388d7af7cd..63f9991a8a 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -2342,6 +2342,23 @@ static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
 
     QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
 
+    /*
+     * Before loading the guest states, ensure that the preempt channel has
+     * been ready to use, as some of the states (e.g. via virtio_load) might
+     * trigger page faults that will be handled through the preempt channel.
+     * So yield to the main thread in the case that the channel create event
+     * hasn't been dispatched.
+     */
+    do {
+        if (!migrate_postcopy_preempt() || !qemu_in_coroutine() ||
+            mis->postcopy_qemufile_dst) {
+            break;
+        }
+
+        aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_self());
+        qemu_coroutine_yield();
+    } while (1);
+
     ret = qemu_loadvm_state_main(packf, mis);
     trace_loadvm_handle_cmd_packaged_main(ret);
     qemu_fclose(packf);
-- 
2.27.0

Re: [PATCH v2] migration/postcopy: ensure preempt channel is ready before loading states

Posted by Peter Xu 1 year, 10 months ago

On Fri, Apr 05, 2024 at 11:40:56AM +0800, Wei Wang wrote:
> Before loading the guest states, ensure that the preempt channel has been
> ready to use, as some of the states (e.g. via virtio_load) might trigger
> page faults that will be handled through the preempt channel. So yield to
> the main thread in the case that the channel create event hasn't been
> dispatched.
> 
> Originally-by: Lei Wang <lei4.wang@intel.com>
> Link: https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@intel.com/T/
> Suggested-by: Peter Xu <peterx@redhat.com>

The current version doesn't have any of my credits. :) Thanks, but I'll
just drop it to reflect reality, so we keep the credit to the right ones.

> Signed-off-by: Lei Wang <lei4.wang@intel.com>
> Signed-off-by: Wei Wang <wei.w.wang@intel.com>
> ---
>  migration/savevm.c | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
> 
> diff --git a/migration/savevm.c b/migration/savevm.c
> index 388d7af7cd..63f9991a8a 100644
> --- a/migration/savevm.c
> +++ b/migration/savevm.c
> @@ -2342,6 +2342,23 @@ static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
>  
>      QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
>  
> +    /*
> +     * Before loading the guest states, ensure that the preempt channel has
> +     * been ready to use, as some of the states (e.g. via virtio_load) might
> +     * trigger page faults that will be handled through the preempt channel.
> +     * So yield to the main thread in the case that the channel create event
> +     * hasn't been dispatched.

I'll squash below into it.  If any of you disagree please shoot: valid
until this Sunday.

+     * TODO: if we can move migration loadvm out of main thread, then we
+     * won't block main thread from polling the accept() fds.  We can drop
+     * this as a whole when that is done.

Huge thanks to both!

> +     */
> +    do {
> +        if (!migrate_postcopy_preempt() || !qemu_in_coroutine() ||
> +            mis->postcopy_qemufile_dst) {
> +            break;
> +        }
> +
> +        aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_self());
> +        qemu_coroutine_yield();
> +    } while (1);
> +
>      ret = qemu_loadvm_state_main(packf, mis);
>      trace_loadvm_handle_cmd_packaged_main(ret);
>      qemu_fclose(packf);
> -- 
> 2.27.0
> 

-- 
Peter Xu

Re: [PATCH v2] migration/postcopy: ensure preempt channel is ready before loading states

Posted by Wang, Lei 1 year, 10 months ago

On 4/6/2024 5:53, Peter Xu wrote:> On Fri, Apr 05, 2024 at 11:40:56AM +0800, Wei
Wang wrote:
>> Before loading the guest states, ensure that the preempt channel has been
>> ready to use, as some of the states (e.g. via virtio_load) might trigger
>> page faults that will be handled through the preempt channel. So yield to
>> the main thread in the case that the channel create event hasn't been
>> dispatched.
>>
>> Originally-by: Lei Wang <lei4.wang@intel.com>
>> Link: https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@intel.com/T/
>> Suggested-by: Peter Xu <peterx@redhat.com>
> 
> The current version doesn't have any of my credits. :) Thanks, but I'll
> just drop it to reflect reality, so we keep the credit to the right ones.
> 
>> Signed-off-by: Lei Wang <lei4.wang@intel.com>
>> Signed-off-by: Wei Wang <wei.w.wang@intel.com>
>> ---
>>  migration/savevm.c | 17 +++++++++++++++++
>>  1 file changed, 17 insertions(+)
>>
>> diff --git a/migration/savevm.c b/migration/savevm.c
>> index 388d7af7cd..63f9991a8a 100644
>> --- a/migration/savevm.c
>> +++ b/migration/savevm.c
>> @@ -2342,6 +2342,23 @@ static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
>>  
>>      QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
>>  
>> +    /*
>> +     * Before loading the guest states, ensure that the preempt channel has
>> +     * been ready to use, as some of the states (e.g. via virtio_load) might
>> +     * trigger page faults that will be handled through the preempt channel.
>> +     * So yield to the main thread in the case that the channel create event
>> +     * hasn't been dispatched.
> 
> I'll squash below into it.  If any of you disagree please shoot: valid
> until this Sunday.

I'm OK with it.

> 
> +     * TODO: if we can move migration loadvm out of main thread, then we
> +     * won't block main thread from polling the accept() fds.  We can drop
> +     * this as a whole when that is done.
> 
> Huge thanks to both!

Thank you Peter :)

> 
>> +     */
>> +    do {
>> +        if (!migrate_postcopy_preempt() || !qemu_in_coroutine() ||
>> +            mis->postcopy_qemufile_dst) {
>> +            break;
>> +        }
>> +
>> +        aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_self());
>> +        qemu_coroutine_yield();
>> +    } while (1);
>> +
>>      ret = qemu_loadvm_state_main(packf, mis);
>>      trace_loadvm_handle_cmd_packaged_main(ret);
>>      qemu_fclose(packf);
>> -- 
>> 2.27.0
>>
>

RE: [PATCH v2] migration/postcopy: ensure preempt channel is ready before loading states

Posted by Wang, Wei W 1 year, 10 months ago

On Saturday, April 6, 2024 5:53 AM, Peter Xu wrote:
> On Fri, Apr 05, 2024 at 11:40:56AM +0800, Wei Wang wrote:
> > Before loading the guest states, ensure that the preempt channel has
> > been ready to use, as some of the states (e.g. via virtio_load) might
> > trigger page faults that will be handled through the preempt channel.
> > So yield to the main thread in the case that the channel create event
> > hasn't been dispatched.
> >
> > Originally-by: Lei Wang <lei4.wang@intel.com>
> > Link:
> > https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@intel
> > .com/T/
> > Suggested-by: Peter Xu <peterx@redhat.com>
> 
> The current version doesn't have any of my credits. :) Thanks, but I'll just drop
> it to reflect reality, so we keep the credit to the right ones.
> 
> > Signed-off-by: Lei Wang <lei4.wang@intel.com>
> > Signed-off-by: Wei Wang <wei.w.wang@intel.com>
> > ---
> >  migration/savevm.c | 17 +++++++++++++++++
> >  1 file changed, 17 insertions(+)
> >
> > diff --git a/migration/savevm.c b/migration/savevm.c index
> > 388d7af7cd..63f9991a8a 100644
> > --- a/migration/savevm.c
> > +++ b/migration/savevm.c
> > @@ -2342,6 +2342,23 @@ static int
> > loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
> >
> >      QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
> >
> > +    /*
> > +     * Before loading the guest states, ensure that the preempt channel has
> > +     * been ready to use, as some of the states (e.g. via virtio_load) might
> > +     * trigger page faults that will be handled through the preempt channel.
> > +     * So yield to the main thread in the case that the channel create event
> > +     * hasn't been dispatched.
> 
> I'll squash below into it.  If any of you disagree please shoot: valid until this
> Sunday.
> 
> +     * TODO: if we can move migration loadvm out of main thread, then we
> +     * won't block main thread from polling the accept() fds.  We can drop
> +     * this as a whole when that is done.
> 
Looks good to me.

RE: [PATCH v2] migration/postcopy: ensure preempt channel is ready before loading states

Posted by Wang, Wei W 1 year, 10 months ago

On Friday, April 5, 2024 11:41 AM, Wang, Wei W wrote:
> 
> Before loading the guest states, ensure that the preempt channel has been
> ready to use, as some of the states (e.g. via virtio_load) might trigger page
> faults that will be handled through the preempt channel. So yield to the main
> thread in the case that the channel create event hasn't been dispatched.
> 
> Originally-by: Lei Wang <lei4.wang@intel.com>
> Link: https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-
> f7e041ced249@intel.com/T/
> Suggested-by: Peter Xu <peterx@redhat.com>
> Signed-off-by: Lei Wang <lei4.wang@intel.com>
> Signed-off-by: Wei Wang <wei.w.wang@intel.com>
> ---
>  migration/savevm.c | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
> 
> diff --git a/migration/savevm.c b/migration/savevm.c index
> 388d7af7cd..63f9991a8a 100644
> --- a/migration/savevm.c
> +++ b/migration/savevm.c
> @@ -2342,6 +2342,23 @@ static int
> loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
> 
>      QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
> 
> +    /*
> +     * Before loading the guest states, ensure that the preempt channel has
> +     * been ready to use, as some of the states (e.g. via virtio_load) might
> +     * trigger page faults that will be handled through the preempt channel.
> +     * So yield to the main thread in the case that the channel create event
> +     * hasn't been dispatched.
> +     */
> +    do {
> +        if (!migrate_postcopy_preempt() || !qemu_in_coroutine() ||
> +            mis->postcopy_qemufile_dst) {
> +            break;
> +        }
> +
> +        aio_co_schedule(qemu_get_current_aio_context(),
> qemu_coroutine_self());
> +        qemu_coroutine_yield();
> +    } while (1);
> +
>      ret = qemu_loadvm_state_main(packf, mis);
>      trace_loadvm_handle_cmd_packaged_main(ret);
>      qemu_fclose(packf);
> --
> 2.27.0

Main change from v1 is the drop of the wait on sem.
It's still patched to loadvm_handle_cmd_packaged, as the sem issue
(possible twice wait) isn't there now.