migration/qemu-file-channel.c | 12 ++-- migration/qemu-file.c | 13 +++- migration/qemu-file.h | 2 +- migration/rdma.c | 148 ++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 163 insertions(+), 12 deletions(-)
Current Qemu RDMA communication does not support send and receive data at the same time, so when RDMA live migration with postcopy enabled, the source qemu return path thread get qemu file error. Those patch add the postcopy support for RDMA live migration. Lidong Chen (5): migration: create a dedicated connection for rdma return path migration: add the interface to set get_return_path migration: implement the get_return_path for RDMA iochannel migration: fix qemu carsh when RDMA live migration migration: disable RDMA WRITR after postcopy started. migration/qemu-file-channel.c | 12 ++-- migration/qemu-file.c | 13 +++- migration/qemu-file.h | 2 +- migration/rdma.c | 148 ++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 163 insertions(+), 12 deletions(-) -- 1.8.3.1
ping. On Sat, Apr 7, 2018 at 4:26 PM, Lidong Chen <jemmy858585@gmail.com> wrote: > Current Qemu RDMA communication does not support send and receive > data at the same time, so when RDMA live migration with postcopy > enabled, the source qemu return path thread get qemu file error. > > Those patch add the postcopy support for RDMA live migration. > > Lidong Chen (5): > migration: create a dedicated connection for rdma return path > migration: add the interface to set get_return_path > migration: implement the get_return_path for RDMA iochannel > migration: fix qemu carsh when RDMA live migration > migration: disable RDMA WRITR after postcopy started. > > migration/qemu-file-channel.c | 12 ++-- > migration/qemu-file.c | 13 +++- > migration/qemu-file.h | 2 +- > migration/rdma.c | 148 ++++++++++++++++++++++++++++++++++++++++-- > 4 files changed, 163 insertions(+), 12 deletions(-) > > -- > 1.8.3.1 >
* Lidong Chen (jemmy858585@gmail.com) wrote: > Current Qemu RDMA communication does not support send and receive > data at the same time, so when RDMA live migration with postcopy > enabled, the source qemu return path thread get qemu file error. > > Those patch add the postcopy support for RDMA live migration. This description is a little misleading; it doesn't really do RDMA during the postcopy phase - what it really does is disable the RDMA page sending during the postcopy phase, relying on the RDMA codes stream emulation to send the page. That's not necessarily a bad fix; you get the nice performance of RDMA during the precopy phase, but how bad are you finding the performance during the postcopy phase - the RDMA code we have was only really designed for sending small commands over the stream? Dave > Lidong Chen (5): > migration: create a dedicated connection for rdma return path > migration: add the interface to set get_return_path > migration: implement the get_return_path for RDMA iochannel > migration: fix qemu carsh when RDMA live migration > migration: disable RDMA WRITR after postcopy started. > > migration/qemu-file-channel.c | 12 ++-- > migration/qemu-file.c | 13 +++- > migration/qemu-file.h | 2 +- > migration/rdma.c | 148 ++++++++++++++++++++++++++++++++++++++++-- > 4 files changed, 163 insertions(+), 12 deletions(-) > > -- > 1.8.3.1 > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
On Wed, Apr 11, 2018 at 8:29 PM, Dr. David Alan Gilbert
<dgilbert@redhat.com> wrote:
> * Lidong Chen (jemmy858585@gmail.com) wrote:
>> Current Qemu RDMA communication does not support send and receive
>> data at the same time, so when RDMA live migration with postcopy
>> enabled, the source qemu return path thread get qemu file error.
>>
>> Those patch add the postcopy support for RDMA live migration.
>
> This description is a little misleading; it doesn't really
> do RDMA during the postcopy phase - what it really does is disable
> the RDMA page sending during the postcopy phase, relying on the
> RDMA codes stream emulation to send the page.
Hi Dave:
I will modify the description in next version patch.
>
> That's not necessarily a bad fix; you get the nice performance of RDMA
> during the precopy phase, but how bad are you finding the performance
> during the postcopy phase - the RDMA code we have was only really
> designed for sending small commands over the stream?
I have not finished the performance test. There are three choices for RDMA
migration during the postcopy phase.
1. RDMA SEND operation from the source qemu
2. RDMA Write with Immediate from the source qemu
3. RDMA READ from the destination qemu
In theory, RDMA READ from the destination qemu is the best way.
But I think it's better to make choice base on the performance result.
I will send the performance result later.
If use another way during the postcopy phase, it will a big change for the code.
This patch just make postcopy works, and i will send another patch to
improve the performance.
Thanks.
>
> Dave
>
>> Lidong Chen (5):
>> migration: create a dedicated connection for rdma return path
>> migration: add the interface to set get_return_path
>> migration: implement the get_return_path for RDMA iochannel
>> migration: fix qemu carsh when RDMA live migration
>> migration: disable RDMA WRITR after postcopy started.
>>
>> migration/qemu-file-channel.c | 12 ++--
>> migration/qemu-file.c | 13 +++-
>> migration/qemu-file.h | 2 +-
>> migration/rdma.c | 148 ++++++++++++++++++++++++++++++++++++++++--
>> 4 files changed, 163 insertions(+), 12 deletions(-)
>>
>> --
>> 1.8.3.1
>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
* 858585 jemmy (jemmy858585@gmail.com) wrote: > On Wed, Apr 11, 2018 at 8:29 PM, Dr. David Alan Gilbert > <dgilbert@redhat.com> wrote: > > * Lidong Chen (jemmy858585@gmail.com) wrote: > >> Current Qemu RDMA communication does not support send and receive > >> data at the same time, so when RDMA live migration with postcopy > >> enabled, the source qemu return path thread get qemu file error. > >> > >> Those patch add the postcopy support for RDMA live migration. > > > > This description is a little misleading; it doesn't really > > do RDMA during the postcopy phase - what it really does is disable > > the RDMA page sending during the postcopy phase, relying on the > > RDMA codes stream emulation to send the page. > > Hi Dave: > I will modify the description in next version patch. > > > > > That's not necessarily a bad fix; you get the nice performance of RDMA > > during the precopy phase, but how bad are you finding the performance > > during the postcopy phase - the RDMA code we have was only really > > designed for sending small commands over the stream? > > I have not finished the performance test. There are three choices for RDMA > migration during the postcopy phase. > > 1. RDMA SEND operation from the source qemu > 2. RDMA Write with Immediate from the source qemu > 3. RDMA READ from the destination qemu > > In theory, RDMA READ from the destination qemu is the best way. > But I think it's better to make choice base on the performance result. > I will send the performance result later. An RDMA read certainly sounds like an interesting way for postcopy, since it means the destination would be in control; so it can RDMA into temporaries that it could then atomically place. An interesting thought. > If use another way during the postcopy phase, it will a big change for the code. > This patch just make postcopy works, and i will send another patch to > improve the performance. Sure; I just wanted to check that, because the existing code wasn't designed for sending pages, that it wasn't really terribly slow. Dave > Thanks. > > > > > Dave > > > >> Lidong Chen (5): > >> migration: create a dedicated connection for rdma return path > >> migration: add the interface to set get_return_path > >> migration: implement the get_return_path for RDMA iochannel > >> migration: fix qemu carsh when RDMA live migration > >> migration: disable RDMA WRITR after postcopy started. > >> > >> migration/qemu-file-channel.c | 12 ++-- > >> migration/qemu-file.c | 13 +++- > >> migration/qemu-file.h | 2 +- > >> migration/rdma.c | 148 ++++++++++++++++++++++++++++++++++++++++-- > >> 4 files changed, 163 insertions(+), 12 deletions(-) > >> > >> -- > >> 1.8.3.1 > >> > > -- > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
© 2016 - 2024 Red Hat, Inc.