[PATCH v3 0/7] LoongArch: KVM: VM migration enhancement

Bibo Mao posted 7 patches 1 year, 7 months ago
arch/loongarch/include/asm/kvm_host.h |  5 ++
arch/loongarch/include/asm/kvm_mmu.h  |  2 +-
arch/loongarch/kvm/main.c             |  1 +
arch/loongarch/kvm/mmu.c              | 67 ++++++++++++++++++++-------
arch/loongarch/kvm/tlb.c              |  5 +-
arch/loongarch/kvm/vcpu.c             | 29 ++++++++++++
6 files changed, 86 insertions(+), 23 deletions(-)
[PATCH v3 0/7] LoongArch: KVM: VM migration enhancement
Posted by Bibo Mao 1 year, 7 months ago
This patchset is to solve VM migration issues, the first six patches are
mmu relative, the last patch is relative with vcpu interrupt status.

It fixes potential issue about tlb flush of secondary mmu and huge page
selection etc. Also it hardens LoongArch kvm mmu module.

With this patchset, VM successfully migrates on my 3C5000 Dual-Way
machine with 32 cores.
 1. Pass to migrate when unixbench workload runs with 32 vcpus, for
some unixbench testcases there is much IPI sending.
 2. Pass to migrate with kernel compiling with 8 vcpus in VM
 3. Fail to migrate with kernel compiling with 32 vcpus in VM, since
there is to much memory writing operation, also there will be file
system inode inconsistent error after migration.

---
v2 ... v3:
 1. Merge patch 7 into this patchset since it is relative with VM
migration bugfix.
 2. Sync pending interrupt when getting ESTAT register, SW ESTAT
register is read after vcpu_put().
 3. Add notation about smp_wmb() when update pmd entry, to elimate
checkpatch warning.
 4. Remove unnecessary modification about function kvm_pte_huge()
in patch 2.
 5. Add notation about secondary mmu tlb since it is firstly used here.

v1 ... v2:
 1. Combine seperate patches into one patchset, all are relative with
migration.
 2. Mark page accessed without mmu_lock still, however with page ref
added
---
Bibo Mao (7):
  LoongArch: KVM: Delay secondary mmu tlb flush until guest entry
  LoongArch: KVM: Select huge page only if secondary mmu supports it
  LoongArch: KVM: Discard dirty page tracking on readonly memslot
  LoongArch: KVM: Add memory barrier before update pmd entry
  LoongArch: KVM: Add dirty bitmap initially all set support
  LoongArch: KVM: Mark page accessed and dirty with page ref added
  LoongArch: KVM: Sync pending interrupt when getting ESTAT from user
    mode

 arch/loongarch/include/asm/kvm_host.h |  5 ++
 arch/loongarch/include/asm/kvm_mmu.h  |  2 +-
 arch/loongarch/kvm/main.c             |  1 +
 arch/loongarch/kvm/mmu.c              | 67 ++++++++++++++++++++-------
 arch/loongarch/kvm/tlb.c              |  5 +-
 arch/loongarch/kvm/vcpu.c             | 29 ++++++++++++
 6 files changed, 86 insertions(+), 23 deletions(-)


base-commit: 50736169ecc8387247fe6a00932852ce7b057083
-- 
2.39.3
Re: [PATCH v3 0/7] LoongArch: KVM: VM migration enhancement
Posted by Huacai Chen 1 year, 7 months ago
Series applied, thanks.

Huacai

On Mon, Jun 24, 2024 at 3:14 PM Bibo Mao <maobibo@loongson.cn> wrote:
>
> This patchset is to solve VM migration issues, the first six patches are
> mmu relative, the last patch is relative with vcpu interrupt status.
>
> It fixes potential issue about tlb flush of secondary mmu and huge page
> selection etc. Also it hardens LoongArch kvm mmu module.
>
> With this patchset, VM successfully migrates on my 3C5000 Dual-Way
> machine with 32 cores.
>  1. Pass to migrate when unixbench workload runs with 32 vcpus, for
> some unixbench testcases there is much IPI sending.
>  2. Pass to migrate with kernel compiling with 8 vcpus in VM
>  3. Fail to migrate with kernel compiling with 32 vcpus in VM, since
> there is to much memory writing operation, also there will be file
> system inode inconsistent error after migration.
>
> ---
> v2 ... v3:
>  1. Merge patch 7 into this patchset since it is relative with VM
> migration bugfix.
>  2. Sync pending interrupt when getting ESTAT register, SW ESTAT
> register is read after vcpu_put().
>  3. Add notation about smp_wmb() when update pmd entry, to elimate
> checkpatch warning.
>  4. Remove unnecessary modification about function kvm_pte_huge()
> in patch 2.
>  5. Add notation about secondary mmu tlb since it is firstly used here.
>
> v1 ... v2:
>  1. Combine seperate patches into one patchset, all are relative with
> migration.
>  2. Mark page accessed without mmu_lock still, however with page ref
> added
> ---
> Bibo Mao (7):
>   LoongArch: KVM: Delay secondary mmu tlb flush until guest entry
>   LoongArch: KVM: Select huge page only if secondary mmu supports it
>   LoongArch: KVM: Discard dirty page tracking on readonly memslot
>   LoongArch: KVM: Add memory barrier before update pmd entry
>   LoongArch: KVM: Add dirty bitmap initially all set support
>   LoongArch: KVM: Mark page accessed and dirty with page ref added
>   LoongArch: KVM: Sync pending interrupt when getting ESTAT from user
>     mode
>
>  arch/loongarch/include/asm/kvm_host.h |  5 ++
>  arch/loongarch/include/asm/kvm_mmu.h  |  2 +-
>  arch/loongarch/kvm/main.c             |  1 +
>  arch/loongarch/kvm/mmu.c              | 67 ++++++++++++++++++++-------
>  arch/loongarch/kvm/tlb.c              |  5 +-
>  arch/loongarch/kvm/vcpu.c             | 29 ++++++++++++
>  6 files changed, 86 insertions(+), 23 deletions(-)
>
>
> base-commit: 50736169ecc8387247fe6a00932852ce7b057083
> --
> 2.39.3
>
>
Re: [PATCH v3 0/7] LoongArch: KVM: VM migration enhancement
Posted by maobibo 1 year, 7 months ago
Huacai,

Thanks for the carefulness and efforts.

Regards
Bibo Mao

On 2024/7/5 下午5:33, Huacai Chen wrote:
> Series applied, thanks.
> 
> Huacai
> 
> On Mon, Jun 24, 2024 at 3:14 PM Bibo Mao <maobibo@loongson.cn> wrote:
>>
>> This patchset is to solve VM migration issues, the first six patches are
>> mmu relative, the last patch is relative with vcpu interrupt status.
>>
>> It fixes potential issue about tlb flush of secondary mmu and huge page
>> selection etc. Also it hardens LoongArch kvm mmu module.
>>
>> With this patchset, VM successfully migrates on my 3C5000 Dual-Way
>> machine with 32 cores.
>>   1. Pass to migrate when unixbench workload runs with 32 vcpus, for
>> some unixbench testcases there is much IPI sending.
>>   2. Pass to migrate with kernel compiling with 8 vcpus in VM
>>   3. Fail to migrate with kernel compiling with 32 vcpus in VM, since
>> there is to much memory writing operation, also there will be file
>> system inode inconsistent error after migration.
>>
>> ---
>> v2 ... v3:
>>   1. Merge patch 7 into this patchset since it is relative with VM
>> migration bugfix.
>>   2. Sync pending interrupt when getting ESTAT register, SW ESTAT
>> register is read after vcpu_put().
>>   3. Add notation about smp_wmb() when update pmd entry, to elimate
>> checkpatch warning.
>>   4. Remove unnecessary modification about function kvm_pte_huge()
>> in patch 2.
>>   5. Add notation about secondary mmu tlb since it is firstly used here.
>>
>> v1 ... v2:
>>   1. Combine seperate patches into one patchset, all are relative with
>> migration.
>>   2. Mark page accessed without mmu_lock still, however with page ref
>> added
>> ---
>> Bibo Mao (7):
>>    LoongArch: KVM: Delay secondary mmu tlb flush until guest entry
>>    LoongArch: KVM: Select huge page only if secondary mmu supports it
>>    LoongArch: KVM: Discard dirty page tracking on readonly memslot
>>    LoongArch: KVM: Add memory barrier before update pmd entry
>>    LoongArch: KVM: Add dirty bitmap initially all set support
>>    LoongArch: KVM: Mark page accessed and dirty with page ref added
>>    LoongArch: KVM: Sync pending interrupt when getting ESTAT from user
>>      mode
>>
>>   arch/loongarch/include/asm/kvm_host.h |  5 ++
>>   arch/loongarch/include/asm/kvm_mmu.h  |  2 +-
>>   arch/loongarch/kvm/main.c             |  1 +
>>   arch/loongarch/kvm/mmu.c              | 67 ++++++++++++++++++++-------
>>   arch/loongarch/kvm/tlb.c              |  5 +-
>>   arch/loongarch/kvm/vcpu.c             | 29 ++++++++++++
>>   6 files changed, 86 insertions(+), 23 deletions(-)
>>
>>
>> base-commit: 50736169ecc8387247fe6a00932852ce7b057083
>> --
>> 2.39.3
>>
>>